| [21:17:42] | * FijiDave_ has joined #farmos |
| [21:19:44] | * FijiDave has quit (Ping timeout: 276 seconds) |
| [00:47:20] | * FijiDave has joined #farmos |
| [00:49:04] | * JustTB has quit (Quit: Leaving.) |
| [00:51:38] | * FijiDave_ has quit (Ping timeout: 276 seconds) |
| [01:29:59] | * cloaz has quit (Ping timeout: 276 seconds) |
| [01:30:32] | * cloaz has joined #farmos |
| [03:02:49] | * JustTB has joined #farmos |
| [03:06:07] | * JustTB has quit (Client Quit) |
| [03:17:32] | * JustTB has joined #farmos |
| [14:24:46] | * FijiDave has quit (Quit: Leaving...) |
| [14:29:54] | * FijiDave has joined #farmos |
| [14:46:36] | <skipper_is[m]> | I didn't see it in the API documentation, hence the question |
| [14:55:05] | <skipper_is[m]> | I am assuming it isn't just file.json?id=84 |
| [14:58:33] | <mstenta[m]> | maybe? |
| [14:58:45] | <mstenta[m]> | tbh i don't know if i've ever tested it |
| [14:59:08] | <mstenta[m]> | we've tested uploading files (because Field Kit does that)... not sure if we ever needed downloading files |
| [14:59:25] | <mstenta[m]> | ultimately, it should just give you metadata about the file |
| [14:59:38] | <mstenta[m]> | which will include the file path - then you can download that directly |
| [15:05:49] | <paul121[m]> | Good question... Does Field Kit uses the API to upload images I'm assuming? |
| [15:06:31] | <mstenta[m]> | Yup |
| [15:07:49] | <mstenta[m]> | Which is pretty simple right now... you basically have to encode the file as base64 and include it in the JSON |
| [15:08:24] | <mstenta[m]> | The restws module itself doesn't support file uploads, but the add-on restws-file module extends it to accept base64 |
| [15:08:50] | <mstenta[m]> | I'm curious to see how files are handled in the Drupal 8 API |
| [15:09:43] | <paul121[m]> | Neat |
| [15:11:25] | <paul121[m]> | Reminds me I need to document more of farmOS.py |
| [15:11:55] | <paul121[m]> | File upload/download could be included |
| [16:30:12] | <skipper_is[m]> | I'm looking at using farmOS.py to migrate everything from Postgresql to mysql, but the files are the one thing that is not making life easy at the moment |
| [16:32:52] | <skipper_is[m]> | But file.json?id=84 us returning 403, forbidden... |
| [16:33:16] | <mstenta[m]> | yikes would that be necessary for a migration? is a pgsql dump that different from a mysqldump? |
| [16:33:24] | <skipper_is[m]> | Yup |
| [16:33:39] | <skipper_is[m]> | Unfortunately |
| [16:33:56] | <mstenta[m]> | it would probably be a whole lot safer to try to import `INSERT` statements from a pgsql dump |
| [16:34:06] | <mstenta[m]> | really? what is different about it? |
| [16:34:07] | <skipper_is[m]> | different ways of variable stores maybe? |
| [16:34:27] | <skipper_is[m]> | was making a fuss about var(255) |
| [16:34:30] | <mstenta[m]> | huh - i would think the `INSERT` statements are identical though |
| [16:34:47] | <mstenta[m]> | you wouldn't need to include any `CREATE TABLE` statements... |
| [16:35:12] | <mstenta[m]> | this is what I would try: |
| [16:35:30] | <mstenta[m]> | install a brand new fresh farmOS instance on MariaDB |
| [16:35:31] | <mstenta[m]> | run `TRUNCATE` on all the tables |
| [16:35:40] | <skipper_is[m]> | The insert statements for pgsql dump are not insert tables |
| [16:35:44] | <mstenta[m]> | get a pgsql dump that is ONLY `INSERT` statements |
| [16:35:52] | <skipper_is[m]> | by default |
| [16:35:55] | <mstenta[m]> | and import that into mariadb |
| [16:36:09] | <skipper_is[m]> | COPY public.farm_sensor_data (id, "timestamp", name, value_numerator, value_denominator) FROM stdin; |
| [16:36:13] | <skipper_is[m]> | 90 1568477315 temp 2279 100 |
| [16:36:14] | <skipper_is[m]> | 90 1568477315 hum 4142 100 |
| [16:37:00] | <mstenta[m]> | try this maybe: https://stackoverflow.com/questions/2857989/using-pg-dump-to-only-get-in... |
| [16:37:09] | <skipper_is[m]> | Ah good shout |
| [16:37:18] | <mstenta[m]> | `pg_dump --column-inserts --data-only <database>` |
| [16:37:29] | <mstenta[m]> | (no idea if that will do what we want... but worth a shot!) |
| [16:37:49] | <skipper_is[m]> | Yea, will give it a spin |
| [16:38:02] | <mstenta[m]> | oh with `> my_dump.sql` at the end to dump to file |
| [16:38:26] | <mstenta[m]> | `pg_dump --column-inserts --data-only <database> > my_dump.sql` |
| [16:38:35] | <skipper_is[m]> | pg_dump --no-owner --column-inserts --data-only farmos > farmosbackup.sql |
| [16:38:35] | <mstenta[m]> | and then on the mariadb/mysql side: |
| [16:39:23] | <mstenta[m]> | `mysql -u <username> --password=<password> <database> < farmosbackup.sql ` |
| [16:39:30] | <mstenta[m]> | (i think - that's off the top of my head) |
| [16:39:54] | <skipper_is[m]> | Oh I need to truncate the cache tables first.. |
| [16:39:55] | <skipper_is[m]> | Still no idea why clear CC isn't working |
| [16:41:37] | <mstenta[m]> | if you have drush, i think `drush cc all` would do it |
| [16:42:09] | <skipper_is[m]> | Yea, drush is being a bit weird now though |
| [16:42:38] | <mstenta[m]> | :-( |
| [16:43:18] | <skipper_is[m]> | Had to get the premade release from the drupal website, rather than just building my own with drush |
| [16:43:26] | <skipper_is[m]> | Which is probably an easier way to go anyway |
| [16:46:22] | <skipper_is[m]> | I should probably set up a farmOS instance using mysql first shouldnt I.. |
| [16:46:55] | <mstenta[m]> | Yea might as well |
| [16:49:25] | <skipper_is[m]> | Does mysql have schemas? |
| [16:49:31] | <skipper_is[m]> | Might be a daft question I know... |
| [16:51:33] | <mstenta[m]> | it has tables with columns... is that what you mean? |
| [16:51:57] | <mstenta[m]> | https://stackoverflow.com/questions/11618277/difference-between-schema-d... |
| [16:52:20] | <mstenta[m]> | does postgresql use "schema" differently? |
| [16:53:51] | <skipper_is[m]> | Yea, they're a bit more like folders |
| [16:54:23] | <skipper_is[m]> | so my farmos tables are located in the public schema, so the cache table is public.cache |
| [16:54:31] | <skipper_is[m]> | My insert statements are therefore INSERT public.cache ..... |
| [16:54:46] | <skipper_is[m]> | And mysql is very unhappy about that.. |
| [16:55:09] | <mstenta[m]> | oh |
| [16:55:16] | <skipper_is[m]> | It is an easy fix |
| [16:55:19] | <mstenta[m]> | can you run a search and replace on the sql dump file? |
| [16:55:24] | <skipper_is[m]> | find replace public. with nothing |
| [16:55:27] | <mstenta[m]> | :-) |
| [16:55:46] | <mstenta[m]> | this is all good to know... i've never had to migrate from postgresql to mysql |
| [16:56:04] | <skipper_is[m]> | But because the dump doesn't have create, I can't dump it into a blank install |
| [16:56:10] | <mstenta[m]> | worth noting.... i am hoping to make postgresql the recommended database in farmOS 2.x |
| [16:56:16] | <skipper_is[m]> | Have to install with the similar tables |
| [16:56:23] | <skipper_is[m]> | well, same named tables |
| [16:56:24] | <mstenta[m]> | and the migration will be automated so you won't have to worry about all this again :-) |
| [16:56:39] | <skipper_is[m]> | :) |
| [16:56:59] | <skipper_is[m]> | I would stick with Postgresql, but it just adds that extra layer of complexity and uncertainty for deploying it to my farm workers |
| [16:57:28] | <mstenta[m]> | https://www.drupal.org/project/farm/issues/2908823 |
| [16:57:47] | <mstenta[m]> | yea... farmOS 1.x has been developed/tested thoroughly on MySQL/MariaDB... so that's definitely the safest |
| [16:58:28] | <mstenta[m]> | for farmOS 2.x, i'm going to start using PostgreSQL myself, so all dev/testing will be done with it :-) |
| [16:59:00] | <skipper_is[m]> | Ah nice :) I do like Postgres, though mainly as it was my base DB when doing GIS with the National Parks |
| [16:59:21] | <skipper_is[m]> | And the PostGIS extension is really handy for GIS |
| [16:59:43] | <skipper_is[m]> | Might be worth leveraging that for FarmOS |
| [17:00:13] | <mstenta[m]> | yes! i'm excited to explore those possibilities |
| [17:00:25] | <mstenta[m]> | although from what i've heard, mariadb has some good GIS options now too |
| [17:00:37] | <mstenta[m]> | but PostGIS has been around longer, so I'm sure it's more mature |
| [17:01:21] | <skipper_is[m]> | https://postgis.net/docs/manual-3.0/PostGIS_Special_Functions_Index.html Has a good list of functions too |
| [17:02:27] | <mstenta[m]> | it would certainly require some consideration, though: whether or not to require PostGIS... or if we still want to allow users to install farmOS on MySQL as well |
| [17:02:47] | <mstenta[m]> | that would add another barrier to self-hosting for some |
| [17:02:51] | <mstenta[m]> | (unless they use Docker) |
| [17:03:42] | <skipper_is[m]> | Indeed |
| [17:04:33] | <skipper_is[m]> | Well, I'm installing a fresh version now, and will insert all in the table when it is done |
| [17:05:54] | <mstenta[m]> | cool! eager to hear how it goes! |
| [17:06:22] | <skipper_is[m]> | I suspect it is going to go interestingly... As I haven't truncated everything created by the install process |
| [17:07:02] | <mstenta[m]> | oh you defininitely need to do that! |
| [17:07:06] | <mstenta[m]> | definitely definitely |
| [17:07:20] | <mstenta[m]> | you'll get lots of errors otherwise |
| [17:07:24] | <mstenta[m]> | duplicate entries and whatnot |
| [17:07:47] | <mstenta[m]> | not for logs/assets/areas... but for other things like Drupal variables |
| [17:07:52] | <mstenta[m]> | field definitions |
| [17:07:52] | <skipper_is[m]> | The installing process under "install optional modules" is that just creating everything in the tables? |
| [17:07:55] | <mstenta[m]> | etc etc |
| [17:08:06] | <skipper_is[m]> | or is there some other funky jazz it is up to? |
| [17:08:19] | <mstenta[m]> | oh that's another good point... make sure you install all the same modules you have installed on the pgsql site |
| [17:08:35] | <mstenta[m]> | to make sure all the same tables are there |
| [17:09:05] | <mstenta[m]> | > The installing process under "install optional modules" is that just creating everything in the tables? |
| [17:09:05] | <mstenta[m]> | that will create the database tables, yes. and also put things in some of them. but you should truncate them. you need the tables to be created, but you need them to all be empty. |
| [17:09:14] | <skipper_is[m]> | Yea, but can I interupt the install? |
| [17:09:31] | <skipper_is[m]> | It is on like 2/23, and taking forever |
| [17:11:27] | <mstenta[m]> | oh ... no don't interrupt the install |
| [17:11:31] | <mstenta[m]> | sorry... |
| [17:11:35] | <mstenta[m]> | i thought you started your import already |
| [17:12:13] | <mstenta[m]> | ACTION sent a long message: < https://matrix.org/_matrix/media/r0/download/matrix.org/GcprvvskvABviIYp... > |
| [17:13:11] | <skipper_is[m]> | Ok, I'll leave it run :( |
| [17:13:20] | <skipper_is[m]> | Just looking how to easily truncate all tables... |
| [17:13:33] | <mstenta[m]> | Do you have linux? |
| [17:13:59] | <mstenta[m]> | Or actually doesn't matter... I use MySQL Workbench, which is a client app that works on all OSes |
| [17:14:08] | <mstenta[m]> | I think you can truncate all tables with that GUI |
| [17:14:41] | <skipper_is[m]> | I do, but I'm just using webmin for most things |
| [17:18:43] | <skipper_is[m]> | I should probably use MySQL Workbench..... |
| [17:18:55] | <skipper_is[m]> | Webmin is refusing to process more than 1 sql command at once... As soon as it sees a ; it stops... |
| [17:19:27] | <mstenta[m]> | So with MySQL Workbench you can right click on a table and "Truncate table" |
| [17:19:38] | <mstenta[m]> | You may be able to select all tables at once and truncate them all at once |
| [17:19:43] | <mstenta[m]> | That's why I suggested it |
| [17:19:49] | <skipper_is[m]> | Yea, but for 215 tables... |
| [17:20:02] | <skipper_is[m]> | I just dumped the table list, and did a find/replace for a new line with "TRUNCATE TABLE " |
| [17:20:07] | <mstenta[m]> | Just an idea... whatever's easiest :-) |
| [17:20:13] | <mstenta[m]> | That works too |
| [17:20:35] | <skipper_is[m]> | I think they're all empty, as FarmOS is now very unhappy and refusing to load |
| [17:20:52] | <mstenta[m]> | haha |
| [17:20:59] | <mstenta[m]> | the install finished first I assume? |
| [17:21:47] | <skipper_is[m]> | It did |
| [17:25:05] | <skipper_is[m]> | MySQL Workbench is slowly chewing through the insert lsit |
| [17:25:07] | <skipper_is[m]> | list |
| [17:25:41] | <skipper_is[m]> | Ah nuts, not all the modules were installed |
| [17:27:21] | <mstenta[m]> | ah |
| [17:27:33] | <mstenta[m]> | Gonna need to start over probably :-/ |
| [17:27:44] | <mstenta[m]> | I also probably wouldn't recommend using Workbench for the import |
| [17:27:53] | <mstenta[m]> | Raw commandline might be faster |
| [17:28:22] | <mstenta[m]> | I gotta run... will check in on your progress later. Good luck! :-) |
| [17:31:02] | <skipper_is[m]> | It is quite handy as it flags it up nicely |
| [17:34:37] | * JustTB has quit (Ping timeout: 240 seconds) |
| [17:34:41] | * JustTB1 has joined #farmos |
| [18:09:47] | <skipper_is[m]> | So many little differences that kept flagging up, not worth it! |
| [18:10:35] | <skipper_is[m]> | Could take weeks to go through the dump and clean it up |
| [18:26:54] | <mstenta[m]> | :-( |
| [18:27:11] | <mstenta[m]> | What kinds of things came up? |
| [18:36:42] | <paul121[m]> | Quick question |
| [18:36:48] | <paul121[m]> | I thought languages were configurable per user? Is that wrong? |
| [18:45:06] | * JustTB1 has quit (Ping timeout: 268 seconds) |