| [19:01:54] | <generalredneck[m> | > Hmm I don't think it will be possible to use VBO on sensor data... :-( |
| [19:01:54] | <generalredneck[m> | So I take it you mstenta don't want to coach paul121 on how to start wring views handlers :p not that I blame ya :p |
| [19:02:08] | <generalredneck[m> | Bah quote fail |
| [19:02:33] | <generalredneck[m> | Riot phone client leaves some things to be desired |
| [19:04:23] | <generalredneck[m> | Fyi about the form handler @paul121... use the code from the `dblog` module for an example. There is a tab with a form for deleting all log entries that you could copy and past and get the confirm |
| [19:10:16] | <generalredneck[m> | I'm thinking Drupal 8 behavior... D7 is super straight forward https://git.drupalcode.org/project/drupal/blob/7.69/modules/dblog/dblog.... |
| [19:37:53] | <mstenta[m]> | @generalredneck:matrix.org: I don't think VBO can be used with Views that are not based on entities |
| [19:38:50] | <mstenta[m]> | Unless I'm mistaken. Otherwise I'd love to coach paul121 on writing a custom VBO action! :-) |
| [19:40:03] | <mstenta[m]> | (we were talking about adding a VBO button to the sensor data View, which shows rows in the {farm_sensor_data} table) |
| [19:40:46] | <mstenta[m]> | But yea in general Views handlers are a deep dive for sure ;-) |
| [19:46:10] | <generalredneck[m> | Yeah I just found the documentation around what you were saying. That said... if toy made the sensor data rows custom entities... it would work... probably more work than it's worth unless you wanna take a advantage of the hook system to be able to do things when sensor data of certain types are recorded. |
| [19:46:29] | <generalredneck[m> | This is obviously me spouting off without having looked at the code |
| [19:47:43] | <mstenta[m]> | That's true it would be possible if each row were an entity |
| [19:48:01] | <generalredneck[m> | In d8 there is benifiet (and ease) to making everything either a field type or entity type |
| [19:48:24] | <mstenta[m]> | I wonder what the overhead would be |
| [19:48:29] | <mstenta[m]> | Any? |
| [19:48:40] | <mstenta[m]> | More code to maintain... |
| [19:50:10] | <generalredneck[m> | There is a performance cost if you use the entity system to create the data. From a views perspective... of you are using the field display handler instead of rendering entities there would be no change there |
| [19:50:40] | <mstenta[m]> | It is a shame that VBO only works with entity tables |
| [19:51:04] | <generalredneck[m> | The benifiets of displays, bundles, and integration could be a big place though... depending on the outlook |
| [19:51:05] | <mstenta[m]> | Oh hmm... |
| [19:51:19] | <mstenta[m]> | So... Could the data be stored in a... field?? |
| [19:51:25] | <mstenta[m]> | 🤯 |
| [19:51:28] | <generalredneck[m> | 100% |
| [19:51:44] | <mstenta[m]> | Haha why didn't I think of that |
| [19:51:55] | <generalredneck[m> | We do that all the time |
| [19:51:56] | <mstenta[m]> | (well D7 that's why I guess) |
| [19:52:12] | <mstenta[m]> | I'm still a little skeptical... |
| [19:52:19] | <mstenta[m]> | Potentially a LOT of data |
| [19:52:29] | <generalredneck[m> | Because it's free views handling... migration handling and integration everywhere |
| [19:52:44] | <mstenta[m]> | I know folks storing one data point every minute |
| [19:52:59] | <generalredneck[m> | On one entity correct? |
| [19:53:11] | <mstenta[m]> | Fields tables might be too bulky, no? |
| [19:53:24] | <mstenta[m]> | Yes per entity |
| [19:53:31] | <generalredneck[m> | How do you store the data? Serilized text blov? |
| [19:53:37] | <mstenta[m]> | And multiple entities |
| [19:54:00] | <mstenta[m]> | Well... So the sensor asset has a "sensor type" |
| [19:54:21] | <mstenta[m]> | So modules can define types, with their own storage |
| [19:54:29] | <mstenta[m]> | farmOS comes with a simple one called listener |
| [19:54:33] | <generalredneck[m> | If the data is fixed character limits... "varchar" it will be fast for a long time |
| [19:54:55] | <mstenta[m]> | Which has a database table called {farm_sensor_data} |
| [19:54:55] | <generalredneck[m> | Ah I see... so it really does act like an entity on its own |
| [19:55:11] | <mstenta[m]> | Columns: timestamp, sensor_id, name, value |
| [19:55:15] | <mstenta[m]> | (iirc) |
| [19:56:48] | <generalredneck[m> | I think you would have a less performant query with one table like that unless you are heavily indexed than you would with an entity that allowed you to attach fields to it |
| [19:57:23] | <mstenta[m]> | Yes agreed. That was a trade-off we made in the beginning to keep the listener relatively flexible |
| [19:57:50] | <mstenta[m]> | And also why we made it possible to have other sensor types |
| [19:59:00] | <mstenta[m]> | So I think I need to understand what you're proposing a little better. I want to make sure I'm not misunderstanding |
| [19:59:24] | <generalredneck[m> | Let me get to a computer and draw something up |
| [19:59:26] | <mstenta[m]> | Right now we have an asset type (bundle) called sensor |
| [19:59:51] | <mstenta[m]> | Haha ok well no pressure :-) |
| [20:00:05] | <mstenta[m]> | You got me thinking now though :-) |
| [20:01:02] | <mstenta[m]> | You're proposing making the sensor data table into a field on that bundle, right? |
| [20:02:59] | <generalredneck[m> | The main thing is when a table gets large it has to go into ram at some point and I think farm_sensor_data... if done a certain way would start having issues... but that's only if you used text or blog fields like drupal likes to do |
| [20:04:20] | <generalredneck[m> | so what i'm proposing is that the sensor data itself be an entity in addition to the asset... and you use an entity reference to link the 2 |
| [20:05:37] | <generalredneck[m> | This would allow you to make bundles of sensor data |
| [20:05:42] | <mstenta[m]> | So one idea I proposed (as a next step that can be updated with update.php) is to auto create tables for each sensor+name combination (which certainly adds complexity to maintain) |
| [20:05:55] | <mstenta[m]> | Ah ok (sensor data as entity)... |
| [20:06:10] | <mstenta[m]> | Interesting |
| [20:06:40] | <symbioquine> | Why not use some sort of time-series database that is optimized for that sort of data? |
| [20:07:14] | <mstenta[m]> | symbioquine: yes! That's a good example of where a new sensor type i makes sense |
| [20:07:27] | <mstenta[m]> | You could create a type that connects to the time series database |
| [20:07:54] | <mstenta[m]> | Or heck, a CSV file |
| [20:08:11] | <mstenta[m]> | Point being: you can have different storage systems if you want |
| [20:08:37] | <generalredneck[m> | which in D8... storage systems on an entity is a thing too |
| [20:09:22] | <generalredneck[m> | it doesn't have to be the database, but I've not seen someone take real good advantage of it. |
| [20:09:28] | <mstenta[m]> | Oh yea?? |
| [20:10:17] | <mstenta[m]> | Here's some relevant forum discussions: |
| [20:10:17] | <generalredneck[m> | by default it's "sql" storage |
| [20:10:29] | <mstenta[m]> | https://farmos.discourse.group/t/integrating-an-existing-automated-farmi... |
| [20:10:39] | <mstenta[m]> | https://farmos.discourse.group/t/sensor-data-storage-query/71 |
| [20:10:56] | <mstenta[m]> | https://farmos.discourse.group/t/controlling-remote-devices-from-within-... |
| [20:11:07] | <mstenta[m]> | Ooh yeah |
| [20:11:35] | <mstenta[m]> | I wonder if we could leverage that |
| [20:12:53] | <generalredneck[m> | this was a protype... showcasing such https://www.drupal.org/project/external_entities |
| [20:13:15] | <mstenta[m]> | My thought is: the "listener" type is a simple, flexible, up and running quick type... And other advanced types can fill in more use cases |
| [20:13:19] | <generalredneck[m> | * this was a prototype... showcasing such https://www.drupal.org/project/external_entities |
| [20:13:38] | <mstenta[m]> | Oh very neat |
| [20:14:22] | <generalredneck[m> | When Ron Northcutt demoed it to us at Texas Camp a long time ago... he was getting his external entities from the Wikipedia API |
| [20:14:47] | <generalredneck[m> | and so his entities used THAT as the store |
| [20:15:35] | <generalredneck[m> | so when you saved an entity it did a Put or Patch against the api... and did a GET to retrieve the entity from the api... |
| [20:16:35] | <mstenta[m]> | Wow |
| [20:16:39] | <generalredneck[m> | I know that's a little off topic from our original discussion, but theoretically you could do something simular with sensor data in D8 |
| [20:16:50] | <mstenta[m]> | Oh yea totally |
| [20:17:43] | <mstenta[m]> | So writing an integration with a time series database (for example) would be done via standard Drupal sub system |
| [20:18:10] | <generalredneck[m> | but given what symbioquine said... yeah your way of handling it is best... obviously whomever creates intergration would have to account for the "Delete all option" paul was talkign about earlier |
| [20:18:24] | <mstenta[m]> | Mmm true |
| [20:19:15] | <mstenta[m]> | Maybe we need a delete function that hooks into each sensor type, to handle that |
| [20:19:26] | <generalredneck[m> | if you fire hooks for the standard CRUD things and not just make a quick query like we recomended to paul... then that would work out |
| [20:20:47] | <generalredneck[m> | you are displaying the sensor data via views right now though |
| [20:20:59] | <generalredneck[m> | so you are kinda limited to a sql db anyway |
| [20:21:20] | <mstenta[m]> | symbioquine: ps saw your replies on the GitHub issues. Great details! I will respond when soon! |
| [20:21:50] | <mstenta[m]> | @generalredneck:matrix.org: yea |
| [20:22:08] | <mstenta[m]> | Which is really all we need |
| [20:22:17] | <mstenta[m]> | For simple sensor data anyway |
| [20:23:24] | <mstenta[m]> | So custom db table + hook_views_data() was a simple first step |
| [20:23:33] | <mstenta[m]> | As we head into D8, though, it's an opportunity to reassess |
| [20:23:41] | <generalredneck[m> | yeah I hear ya. so we can test this.... we can have a script create millions of records and see what happens. I had a client just the other day who had a 2GB dblog table... that was a fun performance thing to find :P |
| [20:23:54] | <mstenta[m]> | Haha cool |
| [20:24:10] | <mstenta[m]> | Yea stress testing ++ :-) |
| [20:24:15] | <mstenta[m]> | Let's do it |
| [20:24:24] | <mstenta[m]> | Be good to have some numbers |
| [20:36:01] | <generalredneck[m> | so farm_sensor_data is literally 5 fields... all of the fields sized... keep it. |
| [20:37:17] | <generalredneck[m> | so sensor data can never be a string... which means you would have to convert values to a numerical representation... |
| [20:37:42] | <generalredneck[m> | example... S == South == 180 |
| [20:38:09] | <mstenta[m]> | True only numerical data can be stored currently |
| [20:38:51] | <generalredneck[m> | but no... you have a multifield key on this table... so it's indexed by that key |
| [20:39:10] | <generalredneck[m> | so you've pretty much did all the performance tuning on that one table you can with the current setup |
| [20:39:54] | <generalredneck[m> | and it SHOULD be pretty damn fast... and the only time you will ever have a headache is if you are returning say all the data for one sensor. |
| [20:40:06] | <mstenta[m]> | Ok cool |
| [20:40:10] | <mstenta[m]> | Yea that makes sense |
| [20:41:05] | <mstenta[m]> | And a potential next step, if we hit limits with this approach, is dynamically generate tables for each data name |
| [20:41:26] | <mstenta[m]> | So we have separate tables for eg temperature, humidity, etc |
| [20:41:56] | <generalredneck[m> | that would be just like the fields table implementation |
| [20:42:27] | <generalredneck[m> | which was kinda why I was pitching the idea... |
| [20:42:40] | <mstenta[m]> | Gotcha gotcha |
| [20:42:59] | <mstenta[m]> | Makes perfect sense |
| [20:43:32] | <generalredneck[m> | drupal's data structure is pretty performant... it's all the other shit to make sure you can be extensible that's not... |
| [20:44:40] | <generalredneck[m> | like as a site developer.. one of the biggest performance favors you can do yourself is use nodes as nodes but if you got other custom objects that isn't specifically "content", create a custom entity type... even if you have to use ECK to do it :P |
| [20:45:37] | <generalredneck[m> | fun module if you want to prototype something... or don't want to bother with doing the leg work of hooking up all the wiring... https://www.drupal.org/project/eck |
| [20:48:27] | <mstenta[m]> | Oh yea I've seen that... Haven't tried it though |
| [20:48:54] | <mstenta[m]> | ACTION is nostalgic for CCK |
| [20:49:01] | <mstenta[m]> | Wait no I'm not |
| [20:49:22] | <generalredneck[m> | the Four Kitchens team uses it pretty heavily in conjunction with paragraphs to make content represent "things" |
| [20:50:12] | <mstenta[m]> | Ah yes paragraphs |
| [20:50:14] | <mstenta[m]> | Another I've yet to try |
| [20:51:03] | <mstenta[m]> | I was involved in the issue thread about deprecating Field Collection in favor of Paragraphs |
| [20:51:07] | <generalredneck[m> | well for what you are doing here... you don't have to try a whole lot. as a drupal consulting agency... (and in site support & maitenice no less) I get exposed to a lot |
| [20:51:23] | <mstenta[m]> | Yea I can imagine! |
| [20:51:37] | <mstenta[m]> | I've tried to keep farmOS as "simple" as possible |
| [20:51:44] | <generalredneck[m> | > I was involved in the issue thread about deprecating Field Collection in favor of Paragraphs |
| [20:51:44] | <generalredneck[m> | Or... how about this noval idea... a totally different entity because that's all a field collection is... |
| [20:51:59] | <mstenta[m]> | To minimize dependencies etc |
| [20:52:05] | <generalredneck[m> | cept it went totally wrong somewhere :P |
| [20:52:31] | <mstenta[m]> | Yes! Well I'm all in to just create custom field types in farmOS D8 instead of field collections :-) |
| [20:52:45] | <generalredneck[m> | I was involved in creating the migration handlers for Field Collections for D7 to D8 |
| [20:53:32] | <generalredneck[m> | If you need some help migrating data... I can help you there... I was thinking about that the other day actually |
| [20:53:39] | <mstenta[m]> | Oh wow! |
| [20:53:51] | <generalredneck[m> | you are going to have to write handlers to go from d7 to d8 entities |
| [20:53:59] | <generalredneck[m> | assuming you want people to have an upgrade path |
| [20:54:10] | <mstenta[m]> | Yes yes |
| [20:54:47] | <mstenta[m]> | Migration is a huge part of the upgrade |
| [20:55:22] | <generalredneck[m> | thank god MIgrate is so much more intuitive in D8 IMO... since it's mostly configuration now |
| [20:55:31] | <mstenta[m]> | Oh man I'm excited |
| [20:56:38] | <mstenta[m]> | As excited as you can be for data migration :-) |
| [20:57:12] | <generalredneck[m> | Well take a look at this! |
| [20:57:12] | <generalredneck[m> | https://github.com/NCAR/drupal_8_migration_example/blob/master/config/in... |
| [20:57:25] | <generalredneck[m> | this is a whole migration... |
| [20:57:50] | <generalredneck[m> | this one does a d7 file to a d8 file entity |
| [20:58:20] | <generalredneck[m> | it's using all built in functionality |
| [20:58:24] | <mstenta[m]> | 😍 |
| [20:59:08] | <generalredneck[m> | like process plugins do 1 thing... they take an input and change it so that it achives what you want |
| [20:59:23] | <generalredneck[m> | and that's the real meat of the action |
| [20:59:55] | <generalredneck[m> | here's the full list of core ones... https://www.drupal.org/docs/8/api/migrate-api/migrate-process-plugins/li... |
| [21:00:57] | <generalredneck[m> | so example... you have 2 fields you want to stick together to make the title you are migrating into... you used to have to like putz around with prepare_row() and all that jazz... now you have |
| [21:02:10] | <generalredneck[m> | ACTION sent a long message: < https://matrix.org/_matrix/media/r0/download/matrix.org/LjpTbaoWFAmfqstH... > |
| [21:02:30] | <generalredneck[m> | tada... title = field1 + field2 |
| [21:04:57] | <generalredneck[m> | that's a basic thing but... there are some that are not so basic such as migration_lookup and flatten... |
| [21:04:57] | <generalredneck[m> | and it's super simple to add your own bits of reusible functionality... |
| [21:12:04] | <mstenta[m]> | Cool! |
| [21:12:32] | <mstenta[m]> | I'm curious about using Migrate instead of Feeds for CSV importers |
| [21:13:45] | <generalredneck[m> | Do it |
| [21:14:21] | <generalredneck[m> | I use migrate instead of feeds to pull in json data from cron jobs. It's much more performat |
| [21:16:02] | <mstenta[m]> | Is it end user friendly? |
| [21:16:31] | <mstenta[m]> | Eg a rancher importing animal birth/weight logs |
| [21:19:37] | <generalredneck[m> | Naw... totally still a use case for that |
| [21:20:27] | <generalredneck[m> | Theres no user interface that I know of currently that allows you to manipulate a migration via the ui |
| [21:21:09] | <mstenta[m]> | Womp :-( |
| [21:22:49] | <generalredneck[m> | Pulling information from a fix api is pretty easy though. Example this is part of evercurrent |
| [21:23:04] | <generalredneck[m> | ACTION posted a file: migrate_plus.migration.evercurrent_server_sa_json.txt (3KB) < https://matrix.org/_matrix/media/r0/download/matrix.org/EFTYAvANMIJZFNxC... > |
| [22:11:10] | <generalredneck[m> | So question... I see you last did anything for d8 7 months ago... are you looking for pretty much a straight translation of entities and data to start with? Or do you have the grand plan somewhere? |
| [22:21:31] | <mstenta[m]> | A mix of those two :-) |
| [22:25:54] | <mstenta[m]> | For the most part, the entities will be a straight translation |
| [22:26:16] | <mstenta[m]> | With some exceptions like field collections |
| [22:26:36] | <mstenta[m]> | And some things moving out to other places |
| [22:27:06] | <mstenta[m]> | Like the map stuff to farmOS-map (my current endeavor) |
| [22:27:16] | <mstenta[m]> | And quick forms to field modules |
| [22:36:03] | <generalredneck[m> | Well if we get the data model down first (regardless of what the field widgets look like) we can start doing the migration work and massage it out as things change. Quick forms for example wouldn't have any bearing on migration except for some configuration settings. |
| [22:36:03] | <generalredneck[m> | Something to keep in mind... configuration is now entitles too and thus migratable. In fact you can migrate a core D7 site (configuration and all) using the core migrate_updaye module... and you can hook into that with your own modules to help migrate over your own configuration and data. |
| [22:38:56] | <mstenta[m]> | Yes very true! And the quick form stuff can happen in parallel in Field Kit |
| [22:39:15] | <mstenta[m]> | I need to familiarize myself with all the possibilities in migrate |
| [22:40:01] | <mstenta[m]> | I've used the contrib migrate long ago |
| [22:40:41] | <mstenta[m]> | And have seen some details of the core d8 migrate. Very exciting to see how far it has come |
| [22:41:07] | <mstenta[m]> | I'm hoping we can make the process pretty painless |
| [22:41:44] | <mstenta[m]> | And then hit the ground running in d8 :-) |
| [22:42:19] | <mstenta[m]> | I have a draft roadmap coming together, hoping to publish soon |
| [22:42:45] | <mstenta[m]> | The broad strokes are there |
| [22:42:56] | <mstenta[m]> | But the devil is in the details always |
| [22:43:19] | <mstenta[m]> | First steps are underway |
| [22:44:01] | <mstenta[m]> | The Openlayers module was the biggest dependency without an upgrade path |
| [22:45:33] | <mstenta[m]> | The new farmOS-map library will replace that in D7 and be reusable in D8 as well |
| [22:46:17] | <mstenta[m]> | Almost done. Planning on including it in 7.x-1.3 |
| [23:08:15] | <generalredneck[m> | For migration we will likely have to start here https://www.drupal.org/docs/8/api/migrate-api/writing-migrations-for-con... |
| [23:10:56] | <generalredneck[m> | Because we have to do like nodes and create a migration for every asset and log type. |
| [01:29:15] | <symbioquine> | Awesome, mstenta[m] no rush on the issues, but it's cool to get to the bottom of them :) |
| [01:35:26] | <symbioquine> | I might be stating the obvious, but the big "selling point" of time series databases is how they can perform special kinds of rolling aggregations and do things like age-out the fine-granularity datapoints while keeping the aggregations. e.g. feed in per-minute/second temperatures, but query average/percentiles/etc at the hour or day level. I don't think dropping subsets of the datapoints tends to be one of their strengths |
| [01:35:26] | <symbioquine> | actually. |
| [06:16:25] | * symbioquine_ has joined #farmos |
| [06:19:00] | * symbioquine has quit (Ping timeout: 258 seconds) |
| [06:29:47] | * symbioquine__ has joined #farmos |
| [06:32:02] | * symbioquine_ has quit (Ping timeout: 240 seconds) |
| [07:26:35] | * JustTB has quit (Quit: Leaving.) |
| [08:59:03] | * JustTB has joined #farmos |
| [09:37:29] | <mstenta[m]> | generalredneck (@generalredneck:matrix.org): interesting |
| [11:57:32] | <paul121[m]> | It would be fairly easy to add a time series DB container to the docker-compose file, yea? |
| [11:57:42] | <paul121[m]> | That's probably not the best for production though? |
| [12:29:38] | * JustTB has quit (Ping timeout: 240 seconds) |
| [12:31:04] | <mstenta[m]> | Definitely! |
| [12:31:31] | <mstenta[m]> | Prod would work |
| [12:43:05] | * JustTB has joined #farmos |
| [15:28:54] | <mstenta[m]> | Question for all: is there ever a need for a log to have BOTH a "movement geometry" and a more general "log geometry"? Or is it redundant? |
| [15:29:26] | <mstenta[m]> | Currently it's possible to have two geometries on some logs, and I'm considering simplifying to one geometry in the future. |
| [15:30:23] | <mstenta[m]> | Can you think of cases where it would be useful to have both? Imagine you are recording the movement of an asset (animal, planting, equipment, etc). Is there ever a time when you might want to have an additional (non-movement) geometry on the same log? |
| [15:38:39] | <symbioquine__> | From the verbiage in FarmOS, it sounds like they were intended to be mutually exclusive, but it looks like one could use the current implementation to describe their garden/farm in terms of coarse-grained areas, then use the geometry to track specific locations within those areas... |
| [15:41:46] | <symbioquine__> | I would imagine that would tend to make more sense for larger perennials. |
| [15:45:17] | <symbioquine__> | It would also be useful for small trial plantings where you might only have a few rows of something in a larger bed. |
| [15:56:13] | <mstenta[m]> | Yes that's correct - the "movement geometry" field allows you to define a more precise movement geometry. |
| [15:56:31] | <mstenta[m]> | https://farmos.org/guide/location/#movement-fields |
| [15:57:00] | <mstenta[m]> | The question is: many logs have a "Movement Geometry" field (for defining asset location) AND a general purpose "Geometry" field. Do we need both? |
| [15:57:47] | <mstenta[m]> | Or, could we just have a single "Geometry" field on logs, and a simple checkbox that says "This log is a movement" - which then tells farmOS to use that geometry field for asset location. |
| [15:58:01] | <mstenta[m]> | Or are there cases where you need BOTH a movement geometry AND a general geometry on a log? |
| [15:58:41] | <mstenta[m]> | (This is a very nuanced question I admit - I think most users don't even realize there are two fields - and I think that can also cause confusion) |
| [15:58:43] | <symbioquine__> | So this is separate from the association of an asset with an area? |
| [15:59:36] | <mstenta[m]> | If you want to see what I mean, go to Logs > Activities and create a new activity log |
| [16:00:10] | <mstenta[m]> | You'll see there is a "Location" group on the left, with a "Geometry" field in it. |
| [16:00:32] | <mstenta[m]> | And there is ANOTHER separate Geometry field in the Assets > Movement group. |
| [16:01:34] | <mstenta[m]> | I'm thinking about simplifying to just one. |
| [16:01:41] | <symbioquine__> | I see |
| [16:01:48] | <mstenta[m]> | With a simple checkbox to say "Treat this log as a movement" |
| [16:01:52] | <paul121[m]> | > Currently it's possible to have two geometries on some logs, and I'm considering simplifying to one geometry in the future. |
| [16:01:52] | <paul121[m]> | For an observation log, you might want to simply record multiple points on a map for that observation. i.e Weeds in field, or rocks in field |
| [16:02:14] | <mstenta[m]> | paul121: Right. So in that case you would leave "This is a movement" unchecked. |
| [16:02:31] | <mstenta[m]> | Question is: are there cases where you need/want both? |
| [16:02:44] | <mstenta[m]> | Both a movement AND just a regular area reference |
| [16:03:19] | <paul121[m]> | Hmm |
| [16:04:08] | <paul121[m]> | For a movement, the `activity` log would have to have an `asset` associated with it, correct? |
| [16:04:16] | <mstenta[m]> | Correct |
| [16:04:38] | <paul121[m]> | (or an `observation` log) |
| [16:04:51] | <mstenta[m]> | Yea, or any log type that's defining a movement |
| [16:07:04] | <paul121[m]> | And if, in the `geometry` group you selected an area instead of custom geometry, that would auto check "treat this as a movement" |
| [16:07:43] | <mstenta[m]> | No I think you would need to manually click "treat this as a movement" no matter what... that should be an intentional choice IMO |
| [16:07:50] | <paul121[m]> | Seems like if you ever did a movement and needed to reference a custom geometry, the custom geometry reference would be better off in a separate log |
| [16:08:01] | <mstenta[m]> | Yea that's sort of my thinking as well |
| [16:08:17] | <mstenta[m]> | I'm trying to remember my thinking from a long time ago :-) |
| [16:08:25] | <paul121[m]> | Otherwise you're doing a movement and observation in the the same log |
| [16:08:29] | <mstenta[m]> | I feel like I had some use-cases in mind where having both would be useful |
| [16:08:36] | <mstenta[m]> | But now I can't think of them |
| [16:08:55] | <mstenta[m]> | And even if I did have ideas... it might be better to be more explicit and force the user to create separate logs |
| [16:09:09] | <paul121[m]> | Question: what was the intention of allowing movements in other `log` types? |
| [16:09:24] | <paul121[m]> | Why not force people to use `movement` logs? |
| [16:09:42] | <mstenta[m]> | There is no `movement` log by itself |
| [16:10:06] | <mstenta[m]> | Movements are really just a field on a log |
| [16:10:18] | <paul121[m]> | Got it |
| [16:10:33] | <paul121[m]> | Similar to `inventory` and `group` |
| [16:10:39] | <mstenta[m]> | This allows you to define asset location with an `activity`, `observation`, `seeding`, `transplanting`, etc |
| [16:10:42] | <mstenta[m]> | Yea exactly |
| [16:11:18] | <mstenta[m]> | (although little known fact: movement logs USED to be their own type :-) ) |
| [16:11:57] | <mstenta[m]> | https://www.drupal.org/project/farm/issues/2871165 |
| [16:12:30] | <paul121[m]> | Ohh so it's a "Field Collection" ?! |
| [16:12:36] | <paul121[m]> | (starting to figure this out) |
| [16:12:51] | <mstenta[m]> | Yea... and I want to move away from Field Collections entirely |
| [16:12:59] | <mstenta[m]> | They are unneeded complexity |
| [16:13:03] | <mstenta[m]> | And deprecated in D8 |
| [16:13:31] | <paul121[m]> | Ah ok. What does D8 use? |
| [16:14:05] | <paul121[m]> | Is there a visual map of the hierarchy of nodes/entities in Drupal? Something that expands to include Views + VBO + Fields + Field Collections? |
| [16:14:35] | <mstenta[m]> | The Paragraphs module has taken over... both are contrib modules... not in Drupal core. So the Field Collection maintainers are saying to use Paragraphs instead. But Paragraphs is not a great fit for us, nor do we really need the complixity of either. |
| [16:14:37] | <paul121[m]> | (I'm thinking similar to that JS Function Pyramid - https://staltz.com/javascript-getter-setter-pyramid.html) |
| [16:14:48] | <mstenta[m]> | We should make one for farmOS |
| [16:15:38] | <paul121[m]> | Especially going into D8.. that would help a lot |
| [16:16:02] | <paul121[m]> | Then I wouldn't ask if Movements were a log type :-) |
| [16:16:16] | <mstenta[m]> | Yea it would be nice to have a visual |
| [16:16:58] | <mstenta[m]> | But for the sake of simplicity, just think of "movements" as a field on logs |
| [16:17:22] | <mstenta[m]> | (Technically they are a Field Collection right now... which is a type of field... but I want to make them just a plain field type) |
| [16:17:47] | <mstenta[m]> | https://www.drupal.org/project/farm/issues/2991383 |
| [16:18:12] | <paul121[m]> | So back to the original question.. |
| [16:18:32] | <paul121[m]> | Is the current `Location` a field? |
| [16:18:35] | <mstenta[m]> | In that issue you can see "Potential hurdles" where I describe part of the reason why I'm asking the original question |
| [16:19:00] | <paul121[m]> | Or is Location -> Geometry a `field` ? |
| [16:19:02] | <mstenta[m]> | > The two fields contained in the Movement Field Collection already exist on many of the Log entities, so there would be a conflict moving them there. They would need to be renamed. |
| [16:19:19] | <mstenta[m]> | Yea, "Location" is a "field group" with two fields in it: "Areas" and "Geometry" |
| [16:19:25] | <mstenta[m]> | Areas is an area reference field |
| [16:19:29] | <mstenta[m]> | Geometry is a geofield |
| [16:19:53] | <mstenta[m]> | The "Movement" Field Collection has those same two fields in it |
| [16:20:43] | <paul121[m]> | Gotcha |
| [16:23:36] | <mstenta[m]> | The biggest consideration is: if there ARE folks using both sets of fields on the same log, and we remove one of the fields in future version, we need to decide how the automatic migration code should handle that |
| [16:23:44] | <mstenta[m]> | I wouldn't want to discard that data |
| [16:23:54] | <mstenta[m]> | So maybe it would split the log into two |
| [16:24:00] | <mstenta[m]> | One movement, and one non movement |
| [16:24:14] | <mstenta[m]> | But... there may NOT be any cases where both are used in the wild... I don't know! |
| [16:25:10] | <mstenta[m]> | Because in simplifying, we would essentially be taking away existing functionality |
| [16:26:31] | <mstenta[m]> | (oh just had an idea... this could be an interesting use-case for the aggregator... as a way to report usage of certain features in a way we could use for development decisions) :-) |
| [16:38:17] | <mstenta[m]> | Get ready for a lot of nitty gritty questions like these as we work towards farmOS 2.x :-) |
| [16:38:36] | <mstenta[m]> | It's my chance to refactor things that have been on my mind for a while |
| [16:45:44] | <paul121[m]> | > (oh just had an idea... this could be an interesting use-case for the aggregator... as a way to report usage of certain features in a way we could use for development decisions) :-) |
| [16:45:44] | <paul121[m]> | haha yea just thought about that! |
| [16:46:31] | <paul121[m]> | mstenta: did you get a chance to read my thoughts here? https://github.com/farmOS/farmOS-aggregator/issues/62 |
| [16:46:40] | <paul121[m]> | No worries, its kinda a lot |
| [16:46:40] | <mstenta[m]> | no sorry! i'm meaning to! |
| [16:46:59] | <mstenta[m]> | i have it in my inbox, so i will i promise... |
| [16:47:49] | <paul121[m]> | *but* .... I think tracking something like "are there any logs that define a movement and additional geometries?" would be pretty easy to implement! |
| [16:49:14] | <symbioquine__> | Hard to retroactively query older versions for new kinds of information though right? |
| [16:50:16] | <mstenta[m]> | Right - it would have to be something we do in advance |
| [16:50:27] | <mstenta[m]> | But I don't think that will be necessary... |
| [16:50:40] | <mstenta[m]> | We can just write migration code to handle those cases if they exist |
| [16:50:46] | <mstenta[m]> | I think that's the right approach |
| [16:50:59] | <mstenta[m]> | So we don't really need to know if it's used or not... we just need to provide a migration |
| [16:51:33] | <paul121[m]> | Makes me curious if we could start collecting some of this kind of use data |
| [16:51:55] | <paul121[m]> | And the migration to D8 would be a good time, in a way |
| [16:52:34] | <paul121[m]> | re: the aggregator collecting that info - it would require getting all `logs` and checking if they define a `movement` and `geometries` |
| [16:52:40] | <mstenta[m]> | This particular one is very very edge-case I think - I don't know if it's worth going out of our way for something like this |
| [16:52:56] | <paul121[m]> | which requires a permission of "reading all logs" |
| [16:53:52] | <paul121[m]> | BUT the farmOS server "metrics module" could have something that checks if anyone has a movement + geometry, and could provide a `yes` or `no` to the aggregator, without providing the data in all those logs |
| [16:54:09] | <mstenta[m]> | true |
| [16:54:16] | <paul121[m]> | > This particular one is very very edge-case I think - I don't know if it's worth going out of our way for something like this |
| [16:54:17] | <paul121[m]> | probably not! haha but in general, it would be good to proactively collect some kinds of info? |
| [16:54:32] | <mstenta[m]> | yea the metrics approach is a good idea |
| [16:54:32] | <paul121[m]> | If people opt in :-) |
| [16:54:40] | <mstenta[m]> | yes - this ^ |
| [16:55:55] | <paul121[m]> | Having a `metrics` module would make it a lot easier. "Calculating" metrics on the farmOS server makes more sense than from the Aggregator backend |
| [16:56:09] | <mstenta[m]> | Yea agreed |
| [16:56:18] | <mstenta[m]> | And it's like a pre-anonymization step |
| [16:56:34] | <paul121[m]> | exactly |
| [16:56:37] | <mstenta[m]> | It would be really easy to expose the existing dashboard metrics |
| [16:56:53] | <paul121[m]> | and in the module certain metrics could be disabled, even |
| [16:57:17] | <mstenta[m]> | oh yea that's a good idea |
| [16:57:29] | <paul121[m]> | > It would be really easy to expose the existing dashboard metrics |
| [16:57:29] | <mstenta[m]> | gotta run! dinner time |
| [16:58:03] | <paul121[m]> | ^^ The community aggregator needs a running count of `Acres managed` :D |
| [16:58:20] | <mstenta[m]> | that would be awesome |
| [16:58:47] | <mstenta[m]> | although easy to exploit... i could just make a polyon in test.farmos.net that encompasses the whole US ;-) |
| [16:59:16] | <paul121[m]> | ugh true |
| [16:59:19] | <mstenta[m]> | (and then you could block me) ;-) |
| [16:59:35] | <paul121[m]> | we could put a max size limit in the calculation |
| [16:59:48] | <paul121[m]> | it'll never be perfect |
| [17:00:00] | <mstenta[m]> | nope - and it doesn't have to be |
| [17:00:12] | <mstenta[m]> | ttyl! |
| [17:06:23] | <symbioquine__> | Yeah, would be pretty easy to ignore unreasonably large acreage values (Maybe > 1000 acres) unless the FarmOS install in question is whitelisted. Of course the advantage of doing the calculation on the aggregator-side would be that the logic could eventually be even smarter and detect likely bogus data e.g. Separate installations with overlapping areas, intersections with open ocean, etc |
| [17:12:23] | <paul121[m]> | Good points. especially ocean |
| [17:31:18] | <mstenta[m]> | Haha true |
| [17:31:33] | <mstenta[m]> | Unless you're a floating farm ;-) |
| [17:51:50] | <symbioquine__> | Well, in that case you could be whitelisted :) |