Problem/Motivation

This module depends on having a D2 binary available. But this can be tricky (see: #3566478: Error installing D2 package via script)

Recently we have considered and tested different approaches to be able to pull in and locate the D2 binary from this module's code and did some research:

  • Originally we were hoping to be able to bring the binary in via a custom package declared in the root composer.json and composer's "bin" property, but this was not successful.
  • We also tried using composer-installers to move this custom package elsewhere, but it seems that composer-installers does not work with packages that are custom-declared in composer.json.
  • Status quo (where we are as of this writing), using a custom composer script handler which via the root composer.json is configured to run on the composer post-install-cmd and post-update-cmd events. This is combined with a custom repository in the root composer.json which brings in the D2 binary via tarball (GitHub URL). We also need to deal with the v0.7.1 directory layer in our script handler, and we ultimately depend on specific environment variables ($HOME/$COMPOSER_HOME). It seems a bit fragile overall and to get all this running on production we had to do some hackery, hence why we are revisiting all of this.
  • Considered "vendoring" (including) the d2 binary into this Drupal module, but hesitated because even targeting one platform like Linux-AMD64 would add ~45MiB, and over time the Git repository for this project would balloon with each different d2 binary included.
  • Looked into third-party composer plugins that might be able to help with bringing in binaries, came up short.
  • Considered creating a custom Drush command that would download, untar, put the binary into place somewhere. While this is an option, it is an extra step and dependency, more code to write, and doesn't seem to be a compelling solution at this time.
  • Reviewed composer documentation to see if the schema supports anything that would allow us to create something like a metapackage to bring in the binary without shipping it in the package. Came up short.
  • Examined composer packages of the 'binary' type. Based on the composer docs this is not a supported package type and this wasn't very instructive.
  • Examined existing composer packages that just bring in a binary such as https://github.com/h4cc/wkhtmltoimage-amd64. This seems like a viable approach for us, and is simple.

Proposed resolution

Creating a separate composer package that just holds the binary, similar to https://github.com/h4cc/wkhtmltoimage-amd64. We can also build a separate PHP library that would provide a process wrapper and would know where to find the D2 binary but that can happen later.

This module can then depend on the newly-created composer package in its composer.json. The composer package would only handle Linux AMD64 but that should be the 80-90%+ use case anyway. Down the line, we or others can fork the composer package for other platforms, and use composer's "replace" functionality. Users of our D2 Drupal module on other platforms would then need to require this module as well as the platform-specific composer package if the target is not Linux-AMD64.

Setting aside the multi-platform bit for a second, with this new composer package we can rely on the binary being available via our newly-created composer package and don't need to do any extra handling in our module for our use case, which is Linux-AMD64. Multi-platform logic or more complex d2 binary location logic could live in the separate PHP library package if need be.

Remaining tasks

Note: Since this is a Plan ticket, the actual changes to this module can be moved to a separate ticket if desired.

Note also: The first three steps can be tested and iterated on locally by adding a local repo/path to the root composer.json "repositories" config.

  1. Create new GitLab repository to hold the new composer package. It needs to include: D2 Bin (Linux AMD64 platform), LICENSE, and README, and ensure there are instructions on how to obtain the D2 source code (MPL 3.2a). Version it to match the included d2 binary. If need be, we can include an extra digit in the version/tag in case of mistakes, such as 0.7.1.1 (which would provide the 0.7.1 d2 binary). Composer internally stores all versions with 4 digits anyway so we know this is compatible.
  2. Submit this composer package to packagist.org
  3. Wait a bit
  4. Update our module to add the new package as a dependency
  5. Refactor our code that locates the binary. Hardcode to the newly-created Linux AMD64 package for now, see notes under Proposed resolution about multi-platform.
  6. Remove now-unused script handler from our module. This is a breaking change but we are in alpha.
  7. Review code for other potential cleanup

Comments

ergonlogic created an issue. See original summary.

star-szr’s picture

Title: Consider depending on terrastruct/d2 pseudo-package » Create separate D2 binary composer package, refactor to depend on it and call the d2 binary directly from that package
Issue summary: View changes

Updating based on recent work and discussions.

star-szr’s picture

Issue summary: View changes
star-szr’s picture

Issue summary: View changes
star-szr’s picture

Issue summary: View changes
star-szr’s picture

Issue summary: View changes
star-szr’s picture

Issue summary: View changes
star-szr’s picture

Issue summary: View changes
ergonlogic’s picture

  • ergonlogic committed 5d468cb2 on 1.0.x
    feat: #3566503 Split D2 wrapper into stand-alone package.
    

ergonlogic’s picture

Status: Active » Needs review

All done. I'm going to test this in Synchrony before closing this ticket.

ergonlogic’s picture

Status: Needs review » Fixed

Works very nicely. I'm going to cut a BETA release.

I also added the ability to set D2 CLI options directly when generating an SVG. That way we don't need to set global options via our hook, when those options may be specific to a single diagram.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • ergonlogic committed 214bc793 on 1.0.x
    docs: #3566503 Update README to remove custom installation steps.
    

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.