| [22:39:44] | <Noaht[m]> | > <@noaht:matrix.org> What type of geometry is used for coordinate locations in the logs and how can I convert to that form from geojson?... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/f89d5d651c...) |
| [00:43:02] | <paul121[m]> | > <@noaht:matrix.org> I think implementing that would definitely be helpful. I'm having an issue getting that example to work, however. I copied that first example line... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/819b620d47...) |
| [00:48:10] | <paul121[m]> | > <@noaht:matrix.org> Where do these values:... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/b0459a9fee...) |
| [01:31:48] | * mstenta[m] has quit (*.net *.split) |
| [01:31:48] | * dazinism[m] has quit (*.net *.split) |
| [01:31:49] | * calbasi[m] has quit (*.net *.split) |
| [01:37:14] | * dazinism[m] has joined #farmos |
| [01:38:11] | * calbasi[m] has joined #farmos |
| [01:38:21] | * mstenta[m] has joined #farmos |
| [01:43:13] | * Noaht[m] has quit (*.net *.split) |
| [01:43:13] | * paul121[m] has quit (*.net *.split) |
| [01:43:13] | * skipper_is[m] has quit (*.net *.split) |
| [01:47:41] | * paul121[m] has joined #farmos |
| [01:47:43] | * skipper_is[m] has joined #farmos |
| [01:49:48] | * Noaht[m] has joined #farmos |
| [04:16:16] | * farmBOT has joined #farmos |
| [06:45:32] | * gvlx has joined #farmos |
| [07:07:37] | * gvlx has quit (Ping timeout: 248 seconds) |
| [07:30:34] | * gvlx has joined #farmos |
| [08:08:47] | * gvlx has quit (Ping timeout: 240 seconds) |
| [10:46:30] | <ludwa6[m]> | g'day room: i am starting to think seriously about the problem of having farmOS as canonical "Source of Truth" for my farm's geography, and wondering how best to go about it. |
| [10:47:02] | <symbioquine[m]> | problem = objective? |
| [10:47:28] | <symbioquine[m]> | * = objective/goal? |
| [10:47:33] | <ludwa6[m]> | Thing is: that basemap that show up by default is rather low-rez & out-of-date, while i've got a hi-rez & fairly recent aerial survey i want to use |
| [10:48:56] | <ludwa6[m]> | So: how can i swap the latter in place of the former? if not a direct swap, then what's a reasonable work-around to the same goal? |
| [10:49:55] | <ludwa6[m]> | symbioquine[m]: yes: i should stop framing goals as problems :-) |
| [10:50:29] | <mstenta[m]> | > i've got a hi-rez & fairly recent aerial survey i want to use |
| [10:50:29] | <mstenta[m]> | where is this currently? what form is it in? |
| [10:50:56] | <ludwa6[m]> | > <@mstenta:matrix.org> > i've got a hi-rez & fairly recent aerial survey i want to use |
| [10:50:56] | <ludwa6[m]> | > |
| [10:50:57] | <ludwa6[m]> | > where is this currently? what form is it in? |
| [10:50:57] | <ludwa6[m]> | GeoTIFF, on my desktop. |
| [10:51:10] | <ludwa6[m]> | pretty massive, tho |
| [10:52:04] | <mstenta[m]> | best bet is probably to get it into a self-hosted GeoServer/MapServer and serve it as a layer that OpenLayers can read - then make a simple module to add it to the farmOS map as a layer |
| [10:52:37] | <mstenta[m]> | longer-term i'm excited to see if someone will build paul121's "Imagery Asset" idea :-) https://farmos.discourse.group/t/imagery-asset-any-interest/887 |
| [10:52:50] | <mstenta[m]> | but size might be a factor in that case... |
| [10:54:12] | <ludwa6[m]> | I remember discussing this w/ you guys awhile back, but finding a reasonably good & affordable host for such layers was a stopper in the end |
| [10:54:28] | <mstenta[m]> | mm yea i have no experience with that |
| [10:54:56] | <mstenta[m]> | there must be hosting options out there? |
| [10:55:09] | <ludwa6[m]> | Then, i also remember symbioquine had some module for serving map layers OUT of farmOS... Am i recalling this wrong? |
| [10:56:20] | <ludwa6[m]> | mstenta[m]: one would think... But nothing that wouldn't cost significantly more than Farmier hosting (which for me seems like the tail wagging the dog, if you know what i mean) |
| [10:56:37] | <symbioquine[m]> | mstenta[m]: [MapProxy](https://mapproxy.org/) Might also be another option... |
| [10:58:04] | <symbioquine[m]> | ludwa6[m]: Perhaps you're thinking of farmOS_wfs? That targets a slightly different problem. Instead of imagery, it lets you view/modify farmOS asset **geometry** in other tools that support the WFS protocol. |
| [10:58:19] | <ludwa6[m]> | symbioquine[m]: MapProxy looks like a developer tool... Unless there's a hosted option |
| [10:58:55] | <paul121[m]> | COGs! Cloud optimized geotiff - only needs a file server to host them :-) Could potentially be farmOS itself or Google/Amazon S3 storage |
| [10:58:57] | <mstenta[m]> | > But nothing that wouldn't cost significantly more than Farmier hosting |
| [10:58:57] | <mstenta[m]> | keep in mind that the reason Farmier hosting is cheap is because it doesn't have to serve lots of traffic... ;-) Also you do pay extra for file storage on Farmier |
| [10:59:22] | <mstenta[m]> | So it's probably not a fair comparison :-) |
| [11:00:15] | <symbioquine[m]> | ludwa6[m]: Yeah, any of those tools GeoServer/MapServer/MapProxy/etc are going to require some admin skills to host effectively... |
| [11:00:24] | <ludwa6[m]> | > <@mstenta:matrix.org> > But nothing that wouldn't cost significantly more than Farmier hosting |
| [11:00:24] | <ludwa6[m]> | > |
| [11:00:24] | <ludwa6[m]> | > keep in mind that the reason Farmier hosting is cheap is because it doesn't have to serve lots of traffic... ;-) Also you do pay extra for file storage on Farmier |
| [11:00:24] | <ludwa6[m]> | Unlimited files up to 100mb total -is that right, mstenta ? |
| [11:00:50] | <mstenta[m]> | actually 1 gig :-) |
| [11:01:54] | <symbioquine[m]> | paul121[m]: This is probably one of the best options, but it still needs a bit of custom code to create a farmOS module that provides the new base layer using that file server. |
| [11:02:02] | <ludwa6[m]> | mstenta[m]: even better -i was just looking on the edit form, where it says 100mb- but i'll take 1gb, happily. Still, GeoTIFF is a bigger problem (er goal :-) |
| [11:02:22] | <symbioquine[m]> | symbioquine[m]: We should have a generic module for this that allows that path to be configured without code though... |
| [11:03:03] | <mstenta[m]> | oh yea it's 100mb upload limit - meaning you can upload 100mb at a time |
| [11:03:23] | <mstenta[m]> | > We should have a generic module for this that allows that path to be configured without code though... |
| [11:03:23] | <mstenta[m]> | Imagery asset? :-)_ |
| [11:03:43] | <mstenta[m]> | s/Imagery asset? :-)_/Imagery asset? :-)/ |
| [11:03:43] | <skipper_is[m]> | Tileserver is PHP based wms hosting |
| [11:03:45] | <skipper_is[m]> | Self hosted though |
| [11:03:47] | <symbioquine[m]> | > <@mstenta:matrix.org> > We should have a generic module for this that allows that path to be configured without code though... |
| [11:03:48] | <symbioquine[m]> | > |
| [11:03:48] | <symbioquine[m]> | > Imagery asset? :-) |
| [11:03:48] | <symbioquine[m]> | Maybe 🧐 |
| [11:04:13] | <ludwa6[m]> | symbioquine[m]: Is that a complex thing to write? if it can just pull from AWS by default, that would be sweet |
| [11:04:25] | <mstenta[m]> | we've also talked about a simpler module... that just asked for a URL (similar to the old MapKnitter module, but more general)_ |
| [11:04:34] | <symbioquine[m]> | > <@mstenta:matrix.org> > We should have a generic module for this that allows that path to be configured without code though... |
| [11:04:34] | <symbioquine[m]> | > |
| [11:04:34] | <symbioquine[m]> | > Imagery asset? :-) |
| [11:04:34] | <symbioquine[m]> | The details of how this actually ends up working are pretty critical. |
| [11:04:57] | <symbioquine[m]> | mstenta[m]: Yeah, I think this would be the most straight-forward path in the short term. |
| [11:05:10] | <mstenta[m]> | agreed |
| [11:05:25] | <ludwa6[m]> | I'm all for straightforward. |
| [11:05:39] | <ludwa6[m]> | TSTTCPW |
| [11:06:08] | <symbioquine[m]> | It could also trivially support WMS and x/y/z tile servers. |
| [11:07:00] | <mstenta[m]> | add TMS while you're at it! (that's what MapKnitter used) ;-) |
| [11:07:09] | <mstenta[m]> | oh or maybe it was just xyz in the module - i forget |
| [11:07:27] | <ludwa6[m]> | ACTION is lost in alphabet soup |
| [11:07:38] | <symbioquine[m]> | Yeah, any of the ones that are easily expressed as a URL :) |
| [11:07:51] | <mstenta[m]> | And that OpenLayers supports... |
| [11:08:10] | <skipper_is[m]> | XYZ and WMS for sure |
| [11:08:14] | <mstenta[m]> | just confirmed: mapknitter was xyz https://github.com/farmOS/farmOS/blob/509cfb2e928ef7537977a0f53eb93ee12a... |
| [11:09:21] | <mstenta[m]> | what do you use skipper_is to host your aerial images? |
| [11:09:28] | <skipper_is[m]> | https://github.com/Skipper-is/farm_map_vhf |
| [11:09:58] | <skipper_is[m]> | https://home.skipper-iwb.co.uk/webmapserver/ |
| [11:10:53] | <ludwa6[m]> | Setting all the backstage tech detail aside for a sec: what will the farmer need to do, to get his aerial survey map data to show up in farmOS? Once there's a module in farmOS that can digest the data and display it properly. ? |
| [11:10:58] | <skipper_is[m]> | Php based, pretty straightforward |
| [11:11:40] | <skipper_is[m]> | That first GitHub repo of mine gets my imagery into FarmOS 1.x |
| [11:11:59] | <skipper_is[m]> | Should be reasonably easy to port over too |
| [11:13:12] | <mstenta[m]> | ludwa6: I think there will ultimately need to be multiple pathways - depending on where the data is coming from and other variables |
| [11:13:15] | <mstenta[m]> | So can't really set aside the tech details yet :-) |
| [11:14:43] | <symbioquine[m]> | ludwa6[m]: I think the lowest dev-cost version involves;... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/d1df04549e...) |
| [11:14:57] | <symbioquine[m]> | Obviously with more dev work some of that could be simplified/automated |
| [11:15:29] | <symbioquine[m]> | Also it gets more complicated if the imagery needs to be private - i.e. not accessible by anyone who knows the URL |
| [11:15:55] | <mstenta[m]> | > That module existing - that allows a tile-source to be specified by URL |
| [11:15:55] | <mstenta[m]> | Overall this module would be pretty simple. Basically the same as the MapKnitter module in v1, but generalized |
| [11:16:12] | <mstenta[m]> | https://github.com/farmOS/farmOS/blob/509cfb2e928ef7537977a0f53eb93ee12a... |
| [11:16:36] | <mstenta[m]> | I think that would be a great module to include in core farmOS |
| [11:17:12] | <skipper_is[m]> | That just takes a single URL and displays it as background map? |
| [11:17:17] | <ludwa6[m]> | > <@symbioquine:matrix.org> I think the lowest dev-cost version involves; |
| [11:17:17] | <ludwa6[m]> | > * That module existing - that allows a tile-source to be specified by URL |
| [11:17:17] | <ludwa6[m]> | > * The farmer using GIS tools to convert the imagery to Cloud optimized geotiff (COG) |
| [11:17:17] | <ludwa6[m]> | ...(truncated) |
| [11:17:24] | <mstenta[m]> | > That just takes a single URL and displays it as background map? |
| [11:17:24] | <mstenta[m]> | as a first step perhaps, yes |
| [11:17:51] | <mstenta[m]> | (oh or maybe you're asking about the MapKnitter module? yes that's basically what it did) |
| [11:18:08] | <mstenta[m]> | if you wanted multiple layers, that would be more complicated |
| [11:18:08] | <skipper_is[m]> | Yea, I looked at that when making my imagery module |
| [11:18:20] | <mstenta[m]> | and maybe that's when the Imagery Asset idea would make sense |
| [11:18:53] | <symbioquine[m]> | mstenta[m]: Yeah, I'd lean towards it supporting a list of layers |
| [11:19:47] | <skipper_is[m]> | Switching? Just as a layer list? |
| [11:19:51] | <ludwa6[m]> | So it sounds like my first step would be to convert my GeoTIFF to COG format -is that it? You said symbioquine "using GIS tool" : can QGIS perform this function, do you know? |
| [11:20:15] | <symbioquine[m]> | ludwa6[m]: I don't know, but I'd guess so... |
| [11:20:49] | <symbioquine[m]> | https://www.cogeo.org/#implementations |
| [11:21:12] | <symbioquine[m]> | The COG website says QGIS supports it. Not sure exactly what the workflow would look like though. |
| [11:21:40] | <skipper_is[m]> | https://github.com/qgis/QGIS/issues/41949 |
| [11:21:43] | <ludwa6[m]> | symbioquine[m]: OK... and then it would be a matter of uploading those COG files to an AWS instance... which would all be accessible at a single URL, right? |
| [11:22:14] | <mstenta[m]> | heck you could probably host it on GitHub Pages :-) |
| [11:22:17] | <symbioquine[m]> | skipper_is[m]: Oh, yeah maybe you'd need a different tool to actually *generate* them for now then. |
| [11:22:30] | <mstenta[m]> | not that i recommend doing that ... :-P |
| [11:22:44] | <symbioquine[m]> | mstenta[m]: Maybe, depends how big it ends up I think. |
| [11:22:57] | <mstenta[m]> | true |
| [11:23:08] | <skipper_is[m]> | What resolution and area? |
| [11:23:23] | <symbioquine[m]> | https://stackoverflow.com/questions/28706454/github-pages-site-size-limits |
| [11:23:44] | <mstenta[m]> | ah good to know! |
| [11:24:27] | <symbioquine[m]> | ludwa6[m]: No, you'd probably want to upload the COG files to AWS S3 (A hosted file storage service) |
| [11:25:00] | <ludwa6[m]> | skipper_is[m]: Mine is 43ha (~100 acres: a little guy ;-), and resolution...? Pretty fine; dunno how to say, but each pixel looks like maybe 10cm? |
| [11:25:19] | <symbioquine[m]> | <ludwa6[m]> "> <@symbioquine:matrix.org> I..." <- It would probably cost about $0.0125 per GB/Mo + data transfer costs as the files are accessed. |
| [11:25:52] | <skipper_is[m]> | Ah ok, probably pushing 500mb? |
| [11:26:21] | <symbioquine[m]> | https://aws.amazon.com/s3/pricing/ (The comparable services from Azure and Google cloud are probably similar since they're all competitors) |
| [11:27:00] | <skipper_is[m]> | I've got 30ha, at 7cm coming in at 290mb, so maybe more like 400mb |
| [11:27:31] | <skipper_is[m]> | Mbtile rather than COG though, so could be smaller on a cog |
| [11:29:11] | <ludwa6[m]> | ACTION > <@skipper_is:matrix.org> Ah ok, probably pushing 500mb? |
| [11:29:11] | <ludwa6[m]> | ACTION looks for file |
| [11:30:44] | <ludwa6[m]> | > * <@ludwa6:matrix.org> looks for file |
| [11:30:44] | <ludwa6[m]> | whoa -2.58gb |
| [11:30:53] | <skipper_is[m]> | Oh wow |
| [11:31:02] | <skipper_is[m]> | Multispectral? |
| [11:31:30] | <paul121[m]> | I really want to recommend people use COGs here... WMS/XYZ should mostly be reserved for 3rd party services that have the infrastructure for it, but many are moving to COGs anyways. It's what they're designed for :-) |
| [11:31:53] | <ludwa6[m]> | skipper_is[m]: i just use this GIS, don't know much of the tech... But it's a transparent_mosaic.tiff file, sez "get info"\ |
| [11:32:15] | <skipper_is[m]> | Ah fairy nuff |
| [11:34:16] | <ludwa6[m]> | But i guess if i can just get this file "sharded" (if that's the right term) into COG format, and upload all the files to AWS, then it'll be very efficient (fast & cheap) to have farmOS pull what's needed on (low)demand. |
| [11:34:48] | <symbioquine[m]> | It doesn't even need to be multiple files per-se (as I recall) |
| [11:35:09] | <symbioquine[m]> | COG makes it so that OpenLayers (or similar) can request just the portion of the file it needs |
| [11:35:14] | <symbioquine[m]> | with HTTP range requests |
| [11:35:29] | <symbioquine[m]> | https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests |
| [11:35:45] | <symbioquine[m]> | (This is getting into the weeds of how COG actually works) |
| [11:38:04] | <ludwa6[m]> | symbioquine[m]: I aim to steer well clear of them weeds; just need to know what to do w/ this whopper file, where to sent it, what it's gonna cost -and done, i hope! |
| [11:38:50] | <symbioquine[m]> | Yeah, the first step would be to find a way to convert your file(s) into a COG file and test it with a fairly recent version of QGIS. |
| [11:40:23] | <ludwa6[m]> | symbioquine[m]: OK: there's a step i can advance on -thanks! |
| [11:40:25] | <symbioquine[m]> | Maybe by following something like https://geoexamples.com/other/2019/02/08/cog-tutorial.html/ |
| [11:41:13] | <symbioquine[m]> | symbioquine[m]: I'm not sure whether there's a more user-friendly tool/tutorial |
| [11:43:42] | <paul121[m]> | Once you have a COG file hosted with a URL you should be able to test it out here: https://geotiffjs.github.io/cog-explorer |
| [11:44:04] | <symbioquine[m]> | paul121[m]: Oh, yeah that might be better than QGIS even :) |
| [11:46:28] | <paul121[m]> | If you could run a file server locally it might be super easy to test w/o even uploading to AWS? |
| [11:48:17] | <ludwa6[m]> | paul121[m]: guess so... But first i must convert my GeoTIFF to COG format. Looks like a project for upcoming long weekend :-) |
| [12:13:50] | <FarmerEd[m]> | Somebody needs to come up with a tool to convert chats like that to forum threads! |
| [13:05:36] | * gvlx has joined #farmos |
| [15:20:25] | * gvlx has quit (Ping timeout: 248 seconds) |