| [01:07:01] | * jonasbits has quit (Ping timeout: 244 seconds) |
| [01:07:56] | * jonasbits has joined #farmos |
| [13:33:17] | * farmB173 has joined #farmos |
| [13:33:21] | * farmBOT has quit (Remote host closed the connection) |
| [13:35:17] | * farmBOT has joined #farmos |
| [13:47:17] | * farmBOT has joined #farmos |
| [14:07:44] | * MarcosCarballal[ has joined #farmos |
| [14:07:45] | <MarcosCarballal[> | ACTION uploaded an image: (43KiB) < https://libera.ems.host/_matrix/media/r0/download/matrix.org/iHFgFUYFJnX... > |
| [14:09:30] | <MarcosCarballal[> | Hi all, we are looking for support on an issue we are having with our build/deploy process.... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/66912f246b...) |
| [16:33:14] | <mstenta[m]> | Marcos Carballal: Sorry can't stay for long, but I would be curious to see your `composer.json` file. The part that is responsible for populating Drupal's "scaffolding" (eg the `index.php` file) is `drupal/core-composer-scaffold` package: https://github.com/farmOS/farmOS/blob/e434715e7483162cfe09fb479ad2321fb3... |
| [16:33:49] | <mstenta[m]> | You should make sure that your `composer.json` file is using `composer-merge-plugin` to merge in the `composer.project.json` file provided by farmOS core |
| [16:34:21] | <MarcosCarballal[> | ACTION sent a code block: https://libera.ems.host/_matrix/media/r0/download/libera.chat/7a22934e8c... |
| [16:34:24] | <mstenta[m]> | The farmOS `composer-project` template does that here: https://github.com/farmOS/composer-project/blob/5214879c44c421e5160a0091... |
| [16:35:02] | <mstenta[m]> | ok, i think that looks right |
| [16:35:27] | <mstenta[m]> | (looks like you started with the `composer-project` template 👍️) |
| [16:35:56] | <mstenta[m]> | > This was working just fine on Friday, and as of yesterday we are facing this issue. |
| [16:35:56] | <mstenta[m]> | I would be curious about what changed between friday and yesterday, anything? |
| [16:36:08] | <MarcosCarballal[> | Nothing at all. |
| [16:36:53] | <MarcosCarballal[> | When I run the `composer install` with this composer.json, I get lots of messages saying it is downloading/installing various packages, but it doesn't say where it put those files and that directory is appearing empty |
| [16:37:34] | <MarcosCarballal[> | composer.json hasn't changed for weeks |
| [16:39:19] | <mstenta[m]> | So just to give an overview of how it (should) work... normally `composer` will put packages into the `vendor` directory. In our case, we want it to put some things (Drupal and modules) into the `web` directory. That's done via the `oomphinc/composer-installers-extender`, along with these lines: |
| [16:39:33] | <MarcosCarballal[> | That is the most confusing bit to me. This issue appeared in all of our local dev environments and is reproducible on all of our branches. I'm thinking maybe it is a dependency that was updated (Since we pull the dependencies on build) and broke something? |
| [16:39:40] | <mstenta[m]> | ACTION sent a code block: https://libera.ems.host/_matrix/media/r0/download/libera.chat/0d9fb3070b... |
| [16:40:41] | <mstenta[m]> | It looks like it is working in the core farmOS repo... we run a build nightly via GitHub Actions to catch that sort of thing: https://github.com/farmOS/farmOS/actions |
| [16:41:19] | <mstenta[m]> | (The most recent one failed for a different known issue - but the build step itself worked fine) |
| [16:41:43] | <mstenta[m]> | You can see all the logs from those builds if you want something to compare against that you know is working as intended |
| [16:42:08] | <MarcosCarballal[> | Yeah, we are not facing any error messages in our build step, it just doesn't populate the web folder like expected |
| [16:43:10] | <mstenta[m]> | Do you have a `core` folder with most of the Drupal core code? |
| [16:43:29] | <mstenta[m]> | The scaffolding piece is just for `index.php` and a few other things so it *sounds* like that's the issue... but let's be sure |
| [16:43:29] | <MarcosCarballal[> | No |
| [16:44:03] | <mstenta[m]> | OK so nothing is in `web` except for what you have in your repo itself? |
| [16:44:49] | <MarcosCarballal[> | Yes, and it only gets there because I mount the folder in my docker-compose file |
| [16:44:50] | <MarcosCarballal[> | ` - './web/modules/custom/cig_pods:/opt/drupal/PODS/web/modules/custom/cig_pods' |
| [16:44:50] | <MarcosCarballal[> | ` |
| [16:45:08] | <mstenta[m]> | Can you check the `vendor` directory to see if perhaps they are ending up in there instead? eg: maybe the `composer-installers-extender` isn't working and they aren't being put where they should (so they are going to the default `vendor/[org]/[package]` dir |
| [16:45:27] | <MarcosCarballal[> | Sure, I'll try that |
| [16:45:35] | <mstenta[m]> | Oh is it possible that you changed your volume mounting or Docker config recently? |
| [16:46:33] | <mstenta[m]> | (Long shot, but curious... I know that if you bind-mount a volume into a container it REPLACES that directory in the container - thereby making it empty - so if you were mounting `web`, then it would replace anything that was built IN the container itself, if that makes sense) |
| [16:47:13] | <MarcosCarballal[> | Interesting, I'll try launching the docker image without any of the mounts and see what happens. |
| [16:48:50] | <mstenta[m]> | eg: if you have the following in `docker-compose.yml`, then I think it would cause what you're seeing:... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/e717947b75...) |
| [16:48:51] | <MarcosCarballal[> | ACTION sent a code block: https://libera.ems.host/_matrix/media/r0/download/libera.chat/a582457c6b... |
| [16:48:54] | <mstenta[m]> | oh ok |
| [16:49:02] | <mstenta[m]> | yea that looks ok |
| [16:49:45] | <mstenta[m]> | so yea i would check the `vendor` dir... see if that's there at all! |
| [16:49:57] | <mstenta[m]> | maybe `composer install` is actually failing, and nothing is being populated? |
| [16:50:26] | <mstenta[m]> | But the fact that you say nothing changed is very odd |
| [16:51:44] | <MarcosCarballal[> | I see a whole bunch of stuff under vendor |
| [16:52:11] | <MarcosCarballal[> | ACTION uploaded an image: (67KiB) < https://libera.ems.host/_matrix/media/r0/download/matrix.org/fgslwyqmodI... > |
| [16:52:56] | <MarcosCarballal[> | ACTION posted a file: (34KiB) < https://libera.ems.host/_matrix/media/r0/download/matrix.org/lqzEtnDydql... > |
| [16:53:24] | <MarcosCarballal[> | It doesnt look like the composer install is failing. Log attached |
| [16:55:27] | <mstenta[m]> | ok hmm... it does looks like everything is going into `vendor`, when some should be going into `web` |
| [16:55:41] | <mstenta[m]> | open `vendor/drupal` - is there a `core` dir in there? |
| [16:56:38] | <MarcosCarballal[> | Yup I see the core dir in there |
| [16:57:28] | <mstenta[m]> | alright so we know what the problem is :-) |
| [16:57:35] | <mstenta[m]> | now to figure out the solution... |
| [16:57:41] | <MarcosCarballal[> | 🙂 that's always progress |
| [16:57:58] | <mstenta[m]> | not really sure why that wouldn't be working |
| [16:58:04] | <mstenta[m]> | could you be using Composer v1 by any chance? |
| [16:58:40] | <MarcosCarballal[> | 2.3.7 |
| [16:58:48] | <mstenta[m]> | ok |
| [16:59:10] | <mstenta[m]> | Oh... hmm maybe you need this: |
| [16:59:23] | <mstenta[m]> | https://github.com/farmOS/farmOS/blob/e434715e7483162cfe09fb479ad2321fb3... |
| [16:59:58] | <mstenta[m]> | Oh I bet this is it, actually! paul121 said this was going to change upstream in July :-) |
| [17:00:12] | <mstenta[m]> | Here is the PR that he made a while back to prepare for it: https://github.com/farmOS/farmOS/pull/467 |
| [17:00:31] | <mstenta[m]> | Actually this explains it better: https://github.com/farmOS/composer-project/pull/6 |
| [17:00:45] | <mstenta[m]> | > For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins |
| [17:00:45] | <mstenta[m]> | > You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins. |
| [17:01:29] | <MarcosCarballal[> | The `allow-plugins` config goes in the composer.json, or elsewhere? |
| [17:01:40] | <mstenta[m]> | I think if you were running `composer install` manually you would be given a prompt - but because you are running it via Docker build you don't have the option (maybe? @paul121 might have better sense) |
| [17:02:20] | <mstenta[m]> | Yes I think you can add it to your `composer.json` |
| [17:02:30] | <mstenta[m]> | We do it slightly differently in our Dockerfile - I forget why exactly |
| [17:02:38] | <mstenta[m]> | But yes I think you can do that |
| [17:03:06] | <mstenta[m]> | We can take a look at the built farmOS composer.json file to see what it needs to look like... one sec... |
| [17:04:32] | <mstenta[m]> | ACTION sent a code block: https://libera.ems.host/_matrix/media/r0/download/libera.chat/c4faa62bc3... |
| [17:04:41] | <mstenta[m]> | Add that to your `composer.json` and try again |
| [17:04:57] | <MarcosCarballal[> | Trying it now! |
| [17:07:32] | <MarcosCarballal[> | It worked! |
| [17:07:35] | <MarcosCarballal[> | You are a wizard Mike |
| [17:07:38] | <mstenta[m]> | Huzzah! |
| [17:08:09] | <mstenta[m]> | That was a sneaky one... paul121 was right - he said this might cause folks trouble |
| [17:08:41] | <mstenta[m]> | It is a result of an upstream change in the way Composer handles plugins, for security reasons |
| [17:09:54] | <MarcosCarballal[> | I'll let you return to your vacation 😅. That is definitely a sneaky one I would not have got there on my own. |
| [17:09:54] | <MarcosCarballal[> | So it is that the composer-installers-extender and composer-merge-plugin were not able to do their thing because we didn't explicitly allow them? |
| [17:09:54] | <MarcosCarballal[> | Is this just for plugins that need to run when composer is run? For example, why do I need to add every single module that the project depends on into this list? |
| [17:09:55] | <MarcosCarballal[> | * do I NOT need to |
| [17:10:28] | <mstenta[m]> | Exactly. Some packages are composer plugins - and those have a bit more power (for good or evil) |
| [17:10:47] | <mstenta[m]> | So Composer decided to change the way they work by default, to make it harder to exploit |
| [17:11:02] | <mstenta[m]> | By forcing you to explicitly allow them, rather than allowing them by default |
| [17:12:05] | <MarcosCarballal[> | Makes sense why they did that. Adding a blurb to the composer output explaining that wouldn't have killed them though. It just doesn't run them and is very silent about it. |
| [17:12:24] | <mstenta[m]> | Yea agreed! I'm surprised it doesn't give any clues! |
| [17:12:45] | <mstenta[m]> | I wonder if there's a more verbose `composer install` flag that does |
| [17:13:08] | <MarcosCarballal[> | Yeah, probably worth adding to the build process if there is |
| [17:13:10] | <MarcosCarballal[> | Anyways |
| [17:13:23] | <MarcosCarballal[> | I'll let you get back to what you were doing. Thanks so much Mike |
| [17:13:41] | <mstenta[m]> | No problem - glad we could figure that out :-) |
| [17:13:53] | <MarcosCarballal[> | What day are you back? |
| [17:14:09] | <mstenta[m]> | I'll be back Monday |
| [17:27:23] | <MarcosCarballal[> | Cool, I think you will be excited by the progress we have made |
