IRC logs for #farmOS, 2019-08-20 (GMT)

2019-08-19
2019-08-21
TimeNickMessage
[21:50:14]* JustTB has quit (Ping timeout: 272 seconds)
[22:28:44]* JustTB has joined #farmos
[01:17:49]* JustTB has quit (Quit: Leaving.)
[03:33:23]<skipper_is[m]>Ah cheers for that, it is similar in a way to my egg one, just that my posting didn't work... I'll try again with the new library I'm writing, and figure out what is going wrong where.. if it still does, just getting cookies and auth tokens to work at the moment, it is showing success! I have aquired the cookies from the request.....
[04:03:52]* JustTB has joined #farmos
[04:11:20]* JustTB has quit (Ping timeout: 258 seconds)
[04:28:03]* JustTB has joined #farmos
[04:43:48]* JustTB has quit (Quit: Leaving.)
[05:18:41]<skipper_is[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/tSALWpKfKkkVwjZo... >
[08:46:48]<skipper_is[m]>Cannot seem to get the /user/login to work...
[08:47:42]<skipper_is[m]>posting {"name":"blablabla","pass":asdasd","form_id":"user_login"} to /user/login
[08:47:55]<skipper_is[m]>Not getting redirected to the actual login
[08:48:03]<skipper_is[m]>Just getting shunted back to the login page
[08:48:14]<mstenta[m]>Are you getting 403?
[08:48:18]<skipper_is[m]>Nope, just a boring 200
[08:48:45]<mstenta[m]>The login page is not like the other endpoints, it doesn't accept json, it is a form that needs to be submitted
[08:49:06]<skipper_is[m]>Looked like it was text/plain encoded in farmOS.py
[08:49:06]<mstenta[m]>Take a look at how farmOS.py does it
[08:49:14]<skipper_is[m]>Yea, thats what I was working off
[08:49:20]<mstenta[m]>Oh ok
[08:49:24]<mstenta[m]>Sorry not at my computer
[08:49:27]<skipper_is[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/COReIoRafXSGvLob... >
[08:49:41]<mstenta[m]>Ok I think that looks good
[08:50:02]<skipper_is[m]>is the line of code, if contains 'data' send data:{name etc etc}
[08:50:03]<mstenta[m]>If you're getting 200 then maybe it worked? Can you access other pages?
[08:50:16]<skipper_is[m]>I'm just trying to get the damned cookie XD
[08:50:21]<skipper_is[m]>All this for a biscuit
[08:50:21]<mstenta[m]>Oh
[08:51:05]<mstenta[m]>Maybe that's the problem... The form will redirect to /user if it's successful
[08:51:15]<mstenta[m]>But if you don't have the cookie stored then it would show the login again
[08:51:19]<mstenta[m]>I think
[08:52:03]<skipper_is[m]>It should... should... give me a 302 status, the Set-Cookie, and the redirect under Location
[08:52:55]<skipper_is[m]>I could probably just do it through form-urlencoded
[08:53:03]<skipper_is[m]>But I'm not so keen on that method....
[09:01:45]<mstenta[m]>Oh I thought that's what farmOS.py was doing
[09:02:26]<mstenta[m]>I think the "data" piece actually gets put into form-urlencode
[09:02:46]<mstenta[m]>Via the Python requests lovely
[09:02:52]<mstenta[m]>Library*
[09:03:02]<skipper_is[m]>XD
[09:03:06]<mstenta[m]>Double check that though I'm not sure
[09:03:30]<skipper_is[m]>I've been trying to pass some sort of JSON mash up to see if it works XD
[09:04:26]<skipper_is[m]>I think you might be right actually XD
[09:05:13]<skipper_is[m]>form-irlencoded worked first time... Why didn't I use that an hour ago..
[09:05:18]<skipper_is[m]>*urlencoded
[09:05:49]<mstenta[m]>Hooray! :-D
[09:15:37]<skipper_is[m]>I am so glad my FarmOS is self hosted, I'd have an angry admin thinking someone was trying to hack in otherwise... so many failed attempts XD
[09:16:57]<mstenta[m]>Ha
[09:51:00]<skipper_is[m]>https://github.com/Skipper-is/micropython-farmOS.py
[09:51:01]<skipper_is[m]> I'm going to make it public, at the moment, it does..literally nothing
[09:56:08]<paul121[m]>Cool!
[09:56:50]<skipper_is[m]>I've just about manage to get the authentication to work...
[09:56:55]<skipper_is[m]>...just
[09:57:02]<skipper_is[m]>With some duct tape and glue
[10:05:27]<mstenta[m]>skipper_is: awesome!
[10:59:17]* farmBOT has joined #farmos
[11:00:15]<skipper_is[m]>farmBOT needs to sort his phobia of emojis out..
[11:01:17]* farmBOT has joined #farmos
[11:02:11]<mstenta[m]>hey cut that out :-P
[11:02:16]<mstenta[m]>stop picking on farmBOT
[12:59:10]<skipper_is[m]>For getting individual logs.... Lets say a harvest log, is the request farmOS/log/23.json?
[12:59:32]<mstenta[m]>yep!
[12:59:50]<mstenta[m]>`/log/23.json`
[12:59:50]<skipper_is[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/iDuMeREvRExZpGmX... >
[12:59:57]<skipper_is[m]>farmOS/log/23.json
[13:00:16]<skipper_is[m]>I suspect it might be another PostGresql error
[13:01:17]* farmBOT has joined #farmos
[13:01:41]<mstenta[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/jpulLcgVqVmzzIdE... >
[13:01:53]<mstenta[m]>should be `/log/23.json`
[13:01:55]<skipper_is[m]>Well, my full path is https://home.skipper-iwb.co.uk/farmOS/log/69.json
[13:01:59]<mstenta[m]>oh ok
[13:02:02]<skipper_is[m]>farmOS is my base directory
[13:02:04]<mstenta[m]>gotcha
[13:02:12]<mstenta[m]>ok well....
[13:02:15]<skipper_is[m]>Other records work, just not logs
[13:02:26]<skipper_is[m]>From the error invalid input syntax for integer:    &amp;quot;69.json&amp;quot;
[13:02:41]<skipper_is[m]>It is saying that "69" isn't correct, because it is in quotes..
[13:02:46]<skipper_is[m]>Maybe?
[13:02:53]<mstenta[m]>weird
[13:03:08]<mstenta[m]>where is 69 coming from?
[13:03:13]<skipper_is[m]>Oh I tried a different log
[13:03:18]<skipper_is[m]>sorry
[13:03:21]<mstenta[m]>oh ok
[13:03:25]<skipper_is[m]>They both didnt work
[13:03:27]<skipper_is[m]>same error
[13:03:51]<mstenta[m]>same thing if you go to that path directly in your browser?
[13:03:55]<mstenta[m]>but assets work?
[13:04:41]<skipper_is[m]>Yup, leme try assets
[13:04:55]<mstenta[m]>so.... this is strange because: the other issues with postrgresql involved complex queries for finding asset location or group membership, so it made sense that we would run into issues
[13:05:02]<mstenta[m]>with logs, it should be a simple query
[13:05:18]<mstenta[m]>nothing extra is being added on to logs
[13:05:19]<mstenta[m]>as far as i can remember
[13:05:33]<mstenta[m]>can you try a few different logs with different amounts of info in them?
[13:05:34]<skipper_is[m]>farm_asset/8.json
[13:05:34]<skipper_is[m]> is fine
[13:05:46]<mstenta[m]>i wonder if it's specific to the logs you tried
[13:05:54]<skipper_is[m]>23 was quite a big one, 69 was just collected 10 eggs
[13:05:55]<mstenta[m]>eg: try making an observation log with just a title
[13:06:05]<mstenta[m]>no quantities or anything else
[13:06:10]<mstenta[m]>just to see
[13:06:15]<skipper_is[m]>I'll try one no
[13:06:16]<skipper_is[m]>now
[13:06:55]<skipper_is[m]>ERROR: invalid input syntax for integer:    &amp;quot;70.json&amp;quot;
[13:07:38]<skipper_is[m]> "id": "70", "name": "Hello World", "type": "farm_observation",
[13:07:42]<skipper_is[m]>That was very simple that one
[13:08:11]<skipper_is[m]>Supper time, I'll have a bit more of a poke later
[13:08:26]<mstenta[m]>ok thanks!
[13:08:38]<mstenta[m]>strange...
[13:16:41]<skipper_is[m]>But I do think it is trying to select by ID and putting " around an int
[13:17:05]<skipper_is[m]>Which PostGresql doesn't like, maybe mysql doesn't mind?
[13:17:24]<mstenta[m]>i'm loading up my pgsql env to test
[13:17:59]<mstenta[m]>yup. same error
[13:18:10]<mstenta[m]>alright whelp... wanna submit a bug report? :-)
[13:18:22]<mstenta[m]>you're on a roll finding the postgresql issues haha
[13:18:25]<mstenta[m]>(thank you for that)
[13:18:55]<mstenta[m]>i'll try to take a quick look...
[13:20:24]<skipper_is[m]>Yea, I'll flag it up in a minute (well, 30-45mins) putting my toddler to bed
[13:20:46]<mstenta[m]>:-)
[13:21:03]<mstenta[m]>(I'll be doing that myself in ~6 hours)
[13:21:25]<mstenta[m]>Here's the error I'm seeing in `/admin/reports/dblog`:
[13:21:29]<mstenta[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/hvPEPbdOTrXBwnOo... >
[13:21:36]<mstenta[m]>(a little more readable)
[13:21:50]<mstenta[m]>> `invalid input syntax for integer: "11.json"`
[13:22:14]<mstenta[m]>weird... seems to be passing `11.json` to the query when it should only pass `11`
[13:28:09]<skipper_is[m]>Oo, that is odd
[13:28:45]<skipper_is[m]>But not for mysql?
[13:28:53]<mstenta[m]>Very strange that it's not happening for farm assets
[13:29:12]<skipper_is[m]>Yea...
[13:29:24]<mstenta[m]>when i created the `farm_asset` module, i copied and modified the `log` module - so they are VERY nearly the same
[13:29:46]<mstenta[m]>now i'm curious...
[13:29:48]<mstenta[m]>firing up the debugger
[13:34:21]<mstenta[m]>ok i'm on the trail...
[13:36:50]<skipper_is[m]>Toddler kicked me out of the room as he wanted a poo, so I've submitted the issue
[13:37:09]<skipper_is[m]>GitHub doesn't appear to be very mobile friendly...
[13:37:14]<mstenta[m]>lol
[13:37:55]<mstenta[m]>thanks! i'll pick it up in that issue...
[13:42:51]<skipper_is[m]>Any joy with the area issue in PostGresql?
[13:43:38]<skipper_is[m]>https://github.com/farmOS/farmOS/issues/193
[13:44:53]<mstenta[m]>Haven't had a chance :-(
[13:45:17]<skipper_is[m]>No worries, your little patch worked for noe
[13:45:17]<mstenta[m]>But I did add a comment for how you can bypass it in the meantime
[13:45:17]<skipper_is[m]>Now
[13:45:22]<mstenta[m]>Oh good
[13:45:45]<mstenta[m]>Yea... that issue just requires a lot of brain power to get back into too
[13:46:14]<skipper_is[m]>I'll be rolling it out to my new team in a week and a half, so it is awesome that the FarmOS.app works
[13:46:47]<skipper_is[m]>So no more "made a cup of tea" logs for me :)
[13:47:39]<mstenta[m]>Great!!
[14:53:11]<skipper_is[m]>For additional pages of data, ( I can't test this as I don't have that much data...) do you just add &page=#?
[14:53:40]<skipper_is[m]>farm_asset.json?type=equipment&page=1
[14:53:41]<skipper_is[m]>Like this
[14:53:59]<mstenta[m]>yep
[14:54:06]<mstenta[m]>there will be 100 per page
[14:54:15]<skipper_is[m]>How many logs per page?
[14:54:17]<skipper_is[m]>....Snap
[14:56:37]<mstenta[m]>skipper_is: Interesting... I just tested the `/log/[id].json` on MySQL and it *also* passes `[id].json` into the database query.....
[14:56:47]<mstenta[m]>But for whatever reason, MySQL accepts that
[14:57:12]<mstenta[m]>So it's not a postgres specific issue... it's just the postgres refuses to work with it (rightly so!)
[14:58:19]<mstenta[m]>Now I just need to figure out why it's including ".json" in the ID for logs
[14:58:48]<skipper_is[m]>I wonder whether MySQL's int() function (or whatever it has) is like my egg logger one... Just add up all the numbers and disregard the rest...
[14:59:08]<skipper_is[m]>Can you send a request of /log/1.json2 and see what happens?
[14:59:31]<skipper_is[m]>or 1a2.json
[14:59:51]<mstenta[m]>Wow that is crazy... I can run this query in MySQL and it works:
[14:59:55]<mstenta[m]>`SELECT * FROM log WHERE id = '3.json';`
[14:59:55]<mstenta[m]>wtf
[15:00:23]<mstenta[m]>So does this:
[15:00:26]<mstenta[m]>`SELECT * FROM log WHERE id = '3.jsonfdsfsdfsdfasddfsdfsdfsdfsd';`
[15:00:34]<mstenta[m]>(both return log 3)
[15:01:07]<skipper_is[m]>PGAdmin isn't working for me at the moment...
[15:01:52]<mstenta[m]>skipper_is: you seem to have a knack for finding really deep bugs :-P
[15:01:57]<skipper_is[m]>XD
[15:02:23]<skipper_is[m]>ERROR: invalid input syntax for integer: "5.json"
[15:02:24]<skipper_is[m]>LINE 1: SELECT * FROM public.log WHERE id = '5.json'
[15:02:29]<skipper_is[m]>Nope, PG doesnt like it
[15:02:47]<mstenta[m]>yup, so that's why we see the error only on pgsql sites
[15:02:52]<mstenta[m]>pgsql is more strict
[15:03:14]<mstenta[m]>(or rather... perfectly sane... it's insane that mysql accepts "5.json" and converts it to 5)
[15:03:53]<mstenta[m]>so now i just need to figure out WHY it's being set to "5.json" and not "5" before it gets passed into the query
[15:04:22]<mstenta[m]>i think it has something to do with the way the `restws` module alters the `log` module's menu router items
[15:04:42]<skipper_is[m]>Did you try mixing numbers and letters mid stream?
[15:04:42]<mstenta[m]>but it may actually be an issue in the `log` module code
[15:04:45]<skipper_is[m]>*string
[15:05:25]<mstenta[m]>`SELECT * FROM log WHERE id = 'x3';` returns no results
[15:05:32]<skipper_is[m]>Aww :(
[15:05:36]<mstenta[m]>`SELECT * FROM log WHERE id = '3x';` returns log 3
[15:05:40]<skipper_is[m]>So it is only the first part
[15:06:15]<mstenta[m]>yea weird
[15:06:21]<mstenta[m]>anyway... i'm debugging the menu router item now
[15:08:20]<skipper_is[m]>... urequests didn't have a params function, so now I have to write a new one of those too.... Is it usually just ?bundle= and ?page= ?
[15:08:41]<skipper_is[m]>or ?bundle=&page= if you want to run them together, or are there a load more I've missed?
[15:08:47]<skipper_is[m]>Oh type as well...
[15:09:53]<skipper_is[m]>Oh.. you can select any variable as a parameter...
[15:09:59]<mstenta[m]>there's a bunch
[15:10:00]<mstenta[m]>yea
[15:10:27]<mstenta[m]>`page` might be the only one unique to the JSON endpoints
[15:10:59]<skipper_is[m]>farm_asset.json has a bunch...
[15:11:29]<skipper_is[m]>Such a shame I can't use wildcards in params though
[15:11:42]<mstenta[m]>This documents most of them I think: https://farmos.org/development/api/
[15:12:47]<skipper_is[m]>Yea, it is most of the JSON attributes
[15:13:09]<mstenta[m]>yea
[15:13:19]<skipper_is[m]>So I will have to write a param handler, rather than just implement one or two
[15:13:19]<mstenta[m]>the only ones that won't work are the field collections i believe
[15:13:21]<mstenta[m]>(for filtering)
[15:13:53]<skipper_is[m]>I can just append them to the URL...
[15:14:15]<skipper_is[m]>Well... That is where they go..
[15:31:56]<skipper_is[m]>I'll implement the Accept: application/json in micropython_farmOS
[15:43:33]<mstenta[m]>skipper_is: ok... but i think i found the core issue
[15:44:43]<mstenta[m]>Can you try something?
[15:46:27]<mstenta[m]>This will fix it:
[15:46:48]<mstenta[m]>Edit `[webroot]/profiles/farm/modules/contrib/entity/includes/entity.controller.inc`
[15:47:03]<mstenta[m]>Go to line 243
[15:47:08]<mstenta[m]>And insert this:
[15:47:15]<mstenta[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/EHLkthqURcaUUENr... >
[15:47:21]<mstenta[m]>In context it should look like:
[15:47:33]<mstenta[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/xivgdqoSOsUiWjBT... >
[15:50:28]<skipper_is[m]>..just buying a fridge..one moment
[15:50:40]<mstenta[m]>haha
[15:51:06]* JustTB has joined #farmos
[16:08:54]<skipper_is[m]>I sincerely hope it worked, because it has been a long time since I ran a backup
[16:09:12]<skipper_is[m]>Indeed it worked
[16:09:31]<skipper_is[m]>I'll still use the header for micropython-farmos
[16:09:39]<mstenta[m]>Great!
[16:12:23]<skipper_is[m]>Keep getting these notices:
[16:12:24]<skipper_is[m]>Notice: Trying to get property of non-object in restws_file_restws_request_alter() (line 30 of /var/www/html/farmOS/profiles/farm/modules/contrib/restws_file/restws_file.module).
[16:12:30]<skipper_is[m]>Rather annoying, but they're not stopping anything...
[16:12:36]<skipper_is[m]>Maybe restws is out of date..
[16:12:45]<skipper_is[m]>Last time I updated it though, it broke EVERYTHING
[16:12:50]<mstenta[m]>Nah that's an outstanding issue with the `restws_file` module
[16:13:12]<mstenta[m]>https://www.drupal.org/project/restws_file/issues/3019850
[16:13:14]<mstenta[m]>You can safely ignore it
[16:13:29]<mstenta[m]>It doesn't cause any harm, so I haven't bothered digging into it
[16:13:45]<mstenta[m]>Advice: disable logging to the screen
[16:14:21]<mstenta[m]>`/admin/config/development/logging`
[16:14:32]<skipper_is[m]>Yea, already done
[16:16:47]<skipper_is[m]>Woo, get log by ID works :)
[16:17:17]<mstenta[m]>Yay!
[16:27:37]* JustTB has quit (Ping timeout: 246 seconds)
[16:28:09]* JustTB has joined #farmos
[16:30:55]* JustTB has quit (Client Quit)
[16:46:50]* JustTB has joined #farmos
[16:58:31]<skipper_is[m]>https://github.com/Skipper-is/micropython_farmOS.py
[16:58:31]<skipper_is[m]>Well, that's it for tonight, but parts of it work well! Can send any request/post to farmOS, but you've got to set it up yourself, unless you want to get a log by ID...
[16:58:32]<skipper_is[m]>It has OAuth2 authentication, and a rewritten urequests that takes cookies and parameters
[16:59:13]<skipper_is[m]>And the readme is almost the same size as the module...
[17:02:31]* JustTB has quit (Quit: Leaving.)
[17:05:30]<mstenta[m]>Awesome!!!!
[17:05:41]<mstenta[m]>> It has OAuth2 authentication
[17:05:42]<mstenta[m]>Oh?? how does this work?
[17:05:51]<mstenta[m]>Do you have something installed on the server?
[17:08:36]<jgaehring[m]>ACTION sent a long message: < https://matrix.org/_matrix/media/v1/download/matrix.org/nDryaGVfbOMHjGIa... >
[17:09:13]<mstenta[m]>I ask because we are hoping to implement OAuth2 in farmOS in the future
[17:09:47]<mstenta[m]>As an alternative to username/password API authentication
[17:10:12]<mstenta[m]>const layers = map.getLayers().getArray()[0].getLayers();
[17:10:24]<mstenta[m]>https://www.drupal.org/project/farm/issues/3034214