Problem/Motivation
Currently, we are using yarn package manager to manage node dependencies.
This works fine but the problem is that if you have multiple projects each project and your custom theme will have separate node_modules. This is a big waste of disk space because each node_module can be around 500MB of the size!
Additionally, for each new custom theme, you need to download again all these dependencies - not a big problem if you are using yarn which is cashing libraries locally.
There is relatively new package manager pnpm which claims to fix this and also it should be drop in replacement for mpn or yarn.
Proposed resolution
Evaluate pnpm and switch to it if is better.
Comments
Comment #2
ytsurkWould be great to save disk space, vote for this - yes !
Comment #3
pivica commentedJust tested this and it's quite trivial to implement. First, you install pnpn
npm add -g pnpmAnd then install all node modules with
pnpm installInstead of `yarn install` or `npm install`.
To execute a gulp task you then do
pnpx gulp sass.The only thing we should add is a new lock file pnpm-lock.yaml and update the documentation so people can use any method they prefer, pnpn, yarn or npm.
Comment #4
pivica commentedComment #5
primsi commentedInitial patch. As per chat we would need to update https://www.drupal.org/docs/8/themes/bs-base/quick-start#s-usualtasks too
Comment #6
pivica commentedTook some time to think about this and how can we solve this in a better way so we allow custom projects to pick their own package system and also we allow easier changes when next better package management system is developed and we want to implement it.
We could do two things.
1. Instead of hardcoding package install and build commands in code we could move this to package.json scripts section. For example, we could add to our package.json next section
This is for old yarn and for a new pnpm it could be (note using of npx which is bundled in node from 5.2 version):
Not sure should we have all in one command or maybe in two (one for install and another for build). I guess having just one is enough?
Then we can change drush to:
We should also remove yarn.lock if we are switching by default to pnpm lock.
Comment #7
primsi commentedImplemented the above + what discussed.
Comment #8
pivica commentedSome minor text fixes, let's test this now a bit more.
Comment #9
pivica commentedHere is a patch updated against patches in other issues, suitable for composer.
Comment #10
pivica commentedIt would be nice that we fallback to npm in build-css script so we don't fail hard if pnpm is not installed on the system.
Comment #11
sasanikolic commentedAgree, this is currently failing when trying to execute the theme update. What would be also helpful to have is a better error message and aliases for compiling sass, instead of pnpx gulp sass, yarn gulp sass etc., we could just write compile-css or something similar.
Comment #12
pivica commented> Agree, this is currently failing when trying to execute the theme update.
We could try something like this maybe:
> we could just write compile-css or something similar.
Yes that is the whole idea of having package.json script for this so you can do
Instead of build-css we can do compile-css but it's pretty much the same.
Comment #13
pivica commentedHere is a new patch, added fallback from pnpm to npm.
Comment #14
pivica commentedWe need to update documentation before closing this :)
Comment #15
pivica commentedDocumentation is updated.
Comment #16
sasanikolic commentedWith the latest patch applied, it tries to run npm install (I think) on every theme compilation. In the case of multiple subthemes, this slows the compilation time significantly. Could we improve this somehow? Maybe run this only once at the start - save it in a local variable or so?
Comment #17
pivica commented> With the latest patch applied, it tries to run npm install (I think) on every theme compilation.
I don't think that this is the case. The compilation is slower because of internal color contrast algorithm we are using now - it requires more time. If that script is triggering npm instead of pnpm then you would see package-lock.json in theme folders and you don't see them - just tried that.
To additionally verify that is going on you can go to the theme folder and execute `npm run build-css`:
Drush is using the same script run. Note that 1.1s for sass is not correct value, try to execute `time npx gulp sass` and you will see what i mean:
So it says 1.17s but in reality it needs 11.29s.
Comment #19
pivica commentedCommitted.