By Rounder on

I've set up a test site to do some experimentation before I try to convert over my working site. But I have a question about modules that require other modules to work. I was under the impression from some of the documentation that this is done automatically, but my JSON file makes it look like not everything is registered.

For example, the Pathauto module requires Token as well as CTools. When I entered the command:

composer require drupal/pathauto

I got these responses back:

Using version ^1.0 for drupal/pathauto
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing drupal/token (1.0.0)
    Downloading: 100%

> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
  - Installing drupal/ctools (3.0.0)
    Downloading: 100%

> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
  - Installing drupal/pathauto (1.0.0)
    Downloading: 100%

> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
Writing lock file
Generating autoload files
> Drupal\Core\Composer\Composer::preAutoloadDump
> Drupal\Core\Composer\Composer::ensureHtaccess

Based on what I see above, both Token and CTools were downloaded and installed. Yet when I look at the updated JSON file, I see this:

    "require": {
        "composer/installers": "^1.0.24",
        "wikimedia/composer-merge-plugin": "~1.4",
        "drupal/pathauto": "^1.0"
    },

But when I look in the LOCK file, I do see entries for Pathauto, Token and CTools.

Why are Token and CTools not shown in the JSON file? Since Pathauto is required, and Token and CTools are necessary for it to work, shouldn't they be listed as "required" as well (along with showing their current level)? What if there were an update to Token or CTools - wouldn't the current level be listed in JSON?

⋅ Categories: Drupal 8.x

Comments

sprite’s picture

sprite commented

When I run "composer.phar require drupal/{somemodule}", the composer utility updates/modifies the composer.json file.

You may want to ask the creator of the composer utility what configuration problem might generate the symptoms you have experienced.

I am running multiple D8 dev sites and have not seen the problem described above.

You should post your entire composer.json, not just a snippet, to provide fore better analysis.

The composer.lock file seems to be less important, and is easily regenerated. With a properly configured composer.json file you can run composer and it will regenerate the composer.lock automatically.

The composer utility documentation is written in a terse style that is able to describe "everything" while at the same time producing the result that people reading it have the impression that it hasn't communicated anything.

https://getcomposer.org/doc/

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

Rounder commented

>> You should post your entire composer.json, not just a snippet, to provide for better analysis.

I know everyone says that Drupal should be installed manually. But since the website that I will need to install Composer on (so that I can install Commerce 2.x) had Drupal installed via Softaculous, I created the test site using Softaculous as well.

Here is the JSON file that was on the test site's root directory after the vanilla Softaculous Drupal install. All I did to modify it was check the existing statements in the installer-paths section and then add the few that were missing per the recommendations on https://www.drupal.org/node/2718229. My install of Pathauto as described in the OP is reflected in the require section:

{
    "name": "drupal/drupal",
    "description": "Drupal is an open source content management platform powering millions of websites and applications.",
    "type": "project",
    "license": "GPL-2.0+",
    "require": {
        "composer/installers": "^1.0.24",
        "wikimedia/composer-merge-plugin": "~1.4",
        "drupal/pathauto": "^1.0"
    },
    "replace": {
        "drupal/core": "~8.3"
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "config": {
        "preferred-install": "dist",
        "autoloader-suffix": "Drupal8"
    },
    "extra": {
        "_readme": [
            "By default Drupal loads the autoloader from ./vendor/autoload.php.",
            "To change the autoloader you can edit ./autoload.php.",
            "This file specifies the packages.drupal.org repository.",
            "You can read more about this composer repository at:",
            "https://www.drupal.org/node/2718229"
        ],
        "merge-plugin": {
            "include": [
                "core/composer.json"
            ],
            "recurse": false,
            "replace": false,
            "merge-extra": false
        },
        "installer-paths": {
            "core": ["type:drupal-core"],
            "libraries/{$name}": ["type:drupal-library"],
            "drush/{$name}": ["type:drupal-drush"],
            "modules/contrib/{$name}": ["type:drupal-module"],
            "profiles/contrib/{$name}": ["type:drupal-profile"],
            "themes/contrib/{$name}": ["type:drupal-theme"],
            "drush/contrib/{$name}": ["type:drupal-drush"],
            "modules/custom/{$name}": ["type:drupal-custom-module"],
            "themes/custom/{$name}": ["type:drupal-custom-theme"]
        }
    },
    "autoload": {
        "psr-4": {
            "Drupal\\Core\\Composer\\": "core/lib/Drupal/Core/Composer"
        }
    },
    "scripts": {
        "pre-autoload-dump": "Drupal\\Core\\Composer\\Composer::preAutoloadDump",
        "post-autoload-dump": [
          "Drupal\\Core\\Composer\\Composer::ensureHtaccess"
        ],
        "post-package-install": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "post-package-update": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup"
    },
    "repositories": [
        {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    ]
}
sprite’s picture

sprite commented

When it comes to the composer utility (whose conception functionality I consider useful generally, while detesting its implementation specifically), most of us are still in learning mode.

Some detailed comparison work may be helpful, with the composer docs pages open alongside for reference.

(example of working composer.json)


{
    "name": "drupal/drupal",
    "description": "Drupal is an open source content management platform powering millions of websites and applications.",
    "type": "project",
    "license": "GPL-2.0+",
    "require": {
    	"drush/drush": "8.1.12",
        "phpdocumentor/reflection-docblock": "2.0.5",
        
        "composer/installers": "^1.0.21",
        "wikimedia/composer-merge-plugin": "~1.3",
        "drupal/core": "^8.3",

        "drupal/admin_toolbar": "^1.19",        
        "drupal/module_filter": "3.0.0",
        "drupal/metatag": "^1.1",                                      
        "drupal/password_policy": "^3.0@alpha",
        "drupal/honeypot": "^1.25",                     
        "drupal/token": "1.0.0",
        "drupal/ultimate_cron": "^2.0@alpha",
        "drupal/pathauto": "^1.0",
        "drupal/xmlsitemap": "~1.0"
    },
    "replace": {
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "config": {
        "preferred-install": "dist",
        "autoloader-suffix": "Drupal8"
    },
    "extra": {
        "_readme": [
            "By default Drupal loads the autoloader from ./vendor/autoload.php.",
            "To change the autoloader you can edit ./autoload.php."
        ],
        "merge-plugin": {
            "include": [
                "core/composer.json"
            ],
            "recurse": false,
            "replace": false,
            "merge-extra": false
        }
    },
    "autoload": {
        "psr-4": {
            "Drupal\\Core\\Composer\\": "core/lib/Drupal/Core/Composer",
            "Drupal\\composer_manager\\Composer\\": "modules/composer_manager/src/Composer"
        }
    },
    "scripts": {
        "pre-autoload-dump": "Drupal\\Core\\Composer\\Composer::preAutoloadDump",
        "post-autoload-dump": "Drupal\\Core\\Composer\\Composer::ensureHtaccess",
        "post-package-install": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "post-package-update": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "drupal-rebuild": "Drupal\\composer_manager\\Composer\\Command::rebuild",
        "drupal-update": "Drupal\\composer_manager\\Composer\\Command::update"
    },
    "repositories": {
        "drupal": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    }
}

One, possibly irrelevant, suggestion I can make is to move your "drupal/core" directive from "replace" to require, and have it use the ^ option so that composer knows all higher versions are acceptable ("drupal/core": "^8.3").

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

Rounder commented

@sprite

Thank you for posting your JSON file for me. I'm going to thoroughly compare it to mine.

But right away I noticed that yours has the same "problem" I wrote about in my OP. If Pathauto requires Token and CTools, and both of those dependencies are listed in the LOCK file, why aren't both in the "require" section?

Even though your JSON file has Pathauto in it, there is no mention of CTools anywhere in that file. I see Token is there, but did that get there because of Pathauto or had you manually run the command to install it?

sprite’s picture

sprite commented

Frankly, I got to a stage where the composer utility has been generally "working", and actually saving a lot of code configuration management time, and have left any further investigation of it to others.

As my other posts indicate, IMO, while the composer utility - kind of sort of works - I don't think it should exist at all in the context of a "modern" website CMS, in which the entire point of the software is the development of inherently GUI websites.

For example, I write PHP with GUI phpstorm for the same reason I haven't used VI or emacs in eons (although there is an entire universe of hard core webdev code geeks who think it's real "macho" and look down on anyone and everyone who doesn't master the CLI universe (been there done that before they were even born). I am old, with my professional software engineering experience dating back to command line Unix and DOS days (days when compsci didn't exist as a major - had to major in math) , (when Linus Torvald (Linux originator) was still in elementary school), and the invention of the Xerox Star, the MacOS GUI, Unix X11 GUI, and finally Windows 3.0 in 1990, ushered in a computing in which the archaic nature of the command line interface was supposed to have faded into ancient history.

Meanwhile, a new generation, that wasn't alive to survive the early days of Unix and DOS, now thinks command line is just "so cool" and so "macho". On my VPS, I have WHM and cPanel installed, so that I can run the server with a few clicks, and even build automated cron driven tasks from the GUI by filling in a couple fields with commands to save, and manage email accounts, DNS, and all that sort of think with GUI confirmation and ease, a work style the macho command line code geeks sneer at.

These are the clashing "code cultures" that compete for attention and dominance, even while working with something like Drupal, to build GUI websites, repeat graphical user-interface websites.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

John Pitcairn’s picture

John Pitcairn commented

Composer doesn't add a package's dependencies to the composer.json file, since they are already explicitly listed in the dependent package's composer.json file, and it examines that when building the project.

The dependencies ARE added to the composer.lock file when you run composer update, and that also includes the exact versions used after all dependencies are resolved.

The .lock file is what gets used when you run something like composer install --no-dev on production, and this ensures you are getting exactly what you have on any other installations.

That's why you generally only use composer update in your dev environment - it will check for updates to any versions specified as ~x.x or ^x.x - and composer install everywhere else.

sprite’s picture

sprite commented

Even on a production site, core and module updates are needed, especially when security patches are released.

But yes, removal all of "dev" branch directives from a production composer.json is best practice.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

John Pitcairn’s picture

John Pitcairn commented

Even on a production site, core and module updates are needed, especially when security patches are released.

Of course. But I wouldn't run composer update directly on production, what if a new version of some other dependency (there are lots, not just Drupal contrib module dependencies) is released in between the testing you've done on dev/staging and the update on production? Then you won't have an identical system. The workflow I've been using for any updates is:

On dev:
composer update
drush updatedb
Test everything still behaves.
drush cex
git commit
(optional) git tag [tagname]
git push

On production:
git pull
(optional) git checkout [tagname]
composer install --no-dev
drush updatedb
drush cim

sprite’s picture

sprite commented

ahh ... the Drupal 8 perspective ...

Meanwhile the average D8 beginner has neither composer, drush, nor git, installed - - hence the existence of this thread and related threads.

The existence of and need for composer, drush, and git, in the context of site building with a CMS, severely limits its audience, and is why WordPress has a user base ten times the size. The command line oriented approach above leaves behind a HUGE section/demographic of the potential audience for Drupal, or another CMS for that matter.

--

So ...

I'm not nearly so paranoid about problems with updates, even on production.
From a test copy of production, on a dev URL, I run the update at the dev copy URL and test.
I've learned to tune/tool the composer.json by hand, to use require directives to control what updates and what doesn't.
For updating the production version, an alert block gets scheduled telling visitors of overnight maintenance.
When ready, I use a module that logs off all users who don't have admin role.
I use WHM/cPanel to set the site's docroot directory to password protected, making the site not accessible to the public.
Run the updates needed, using require directives to prevent updates to problematic modules (just set require to a specific version - no fuzzy parameter).
Run the composer update, clear all caches, double check that things are working.
In WHM/cPanel, get rid of directory privacy, and allow logins again.
Site is updated.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

John Pitcairn’s picture

John Pitcairn commented

Sure, there are other ways to skin this particular cat. The workflow I use is very fast and simple for me, and I'm not sorry I started using git and drush in D7, then composer in D8 (I'm also using it for D7 sites now). The hard thing is getting started, as you note. I totally agree there could/should be a GUI wrapper around composer inside Drupal, but that's perhaps rather non-trivial to implement and maintain.

Run the updates needed, using require directives to prevent updates to problematic modules (just set require to a specific version - no fuzzy parameter).

That's what composer install is for, reading the composer.lock file which does contain the specific versions. So you can have fuzzy version parameters on dev for modules that aren't problematic and should get ongoing upgrades, require a specific version or dev commit for problematic modules (I do this too), and be assured you will get exactly the same when you install (not update) on production from those composer.json and composer.lock files.

sprite’s picture

sprite commented

Again, the primary point/reason this thread exists in the first place is because a HUGE swath of the demographic that is interested in CMS tools isn't going to take well to using command line tools. That isn't the reason they explored CMS tools in the first place.

Requiring the average potential Drupal CMS user to:
- have/acquire command line SSH access to their site
- - setting up SSH would probably huge ask for many
- learn shell commands in general
- learn composer
- learn drush
- learn git

wherein the above is all way outside what ought to be the product audience for website CMS tools, and their required knowledge set.

Just look at the average sorts of questions asked by people who find their way to the drupal.org forums looking for answers. I've answered questions from people who couldn't differentiate from the components in their LAMP stack and Drupal itself. Just got to my accounts post list and see what sorts of questions I answer for people:

https://www.drupal.org/user/47774/track

While experience code geeks may take well to the complexities of Drupal 8, the scenario above is alienating potential Drupal 8 user (potential drupal user in general) by the thousands every day.

Drupal is losing "market" share every day because it has evolved to appeal primarily, first and foremost, to professional wedev code geeks, and has left the rest of the potential audience for website CMS to wrestle with the limitations and security risks of WordPress.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

John Pitcairn’s picture

John Pitcairn commented

Drupal is losing "market" share every day because it has evolved to appeal primarily, first and foremost, to professional wedev code geeks, and has left the rest of the potential audience for website CMS to wrestle with the limitations and security risks of WordPress.

Well, yes, but that was also a complaint when I first started using it back in 2009 (and decided I didn't like Wordpress much then either). Griping on the forum about how Drupal isn't Wordpress isn't going to help make it so.

But my comments about what composer.json and composer.lock are for, and how they get used in a composer deployment workflow, are definitely relevant to this thread, good practice, and were posted with good intentions. Hopefully they are useful to somebody, if not you.

sprite’s picture

sprite commented

You appear not to have taken the time to understand that varying audiences exist, with regard to Drupal in general, and with regard to questions posed in these very forums.

As most of my posts in this thread make clear, I am experienced, a systems software engineer, in addition to being a Drupal developer of some years, a PHP, JS, C++, C#, Java, etc., programmer with decades of experience, and so on. For longer than you, I've been developing with Drupal since 2004/2005, and my profile here is 11.5 years old:

https://www.drupal.org/u/sprite

My comments in my posts just above were not about my personal needs, but were about the needs of that other, more general, CMS audience.

Griping on the forum about how Drupal isn't Wordpress isn't going to help make it so.

In response to the above, my opinions have been expressed in the relevant "issues queues", where the discussions about Drupal by the people in control of Drupal exist. My comments about their perspectives and attitudes come from reading and participating in those discussion threads. It seems to take the average newbie quite a long time to discover the relevant discussions in the "issues queues" by the people in actual control. Meanwhile though, outlining such topics in these forums does not diminish bringing up where the people in control seem to interact exclusively. With eyes wide, those of us in the know, know that Drupal is essentially an Acquia controlled project, and that Drupal Commerce is run and controlled by the owners of platform.sh and commerceguys.com, and that they control the future of that code as they see fit. That is a reality that newbies probably can't/don't perceive directly.

To clarify again, I use composer, drush, git, and so on. I've been using version control systems since before the creators of git were even in elementary school. My comments were not about me. My comments were about potential Drupal users generally.

Unlike many of the people who restrict their drupal.org interactions to the "issues queues", I see the perspective on a regular basis of the average Drupal newbie who comes to the forums. Surely if many of the average Drupal newbies can't even find their way to the drupal.org /docs/* pages, they are even less likely to grapple with many of the complexities that Drupal 8 imposes (SSH, composer, drush, git), or have patience with the various bugs, missing features, and other general problems that impact Drupal 8 currently.

I also do not believe that making distinctions about a CMS like WordPress amounts to "griping" either. It is my impression that you haven't taken the time to measure the audience that is the original poster (OP) who is grappling with just getting composer to work at all (for the first time ) to manage his Drupal installation, nor to differentiate that I represent a different audience, an audience with significant experience and expertise. When in the forums, it remains important to tailor suggestions to and with measure to varying audiences, and to know the differences between them.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

John Pitcairn’s picture

John Pitcairn commented

Yeah alright. I'll bugger off back to the issue queues then. Job well done mate.

sprite’s picture

sprite commented

(actual)


{
    "name": "drupal/drupal",
    "description": "Drupal is an open source content management platform powering millions of websites and applications.",
    "type": "project",
    "license": "GPL-2.0+",
    "require": {
    	"drush/drush": "8.1.12",
        "phpdocumentor/reflection-docblock": "2.0.5",
        
        "composer/installers": "^1.0.21",
        "wikimedia/composer-merge-plugin": "~1.3",
        "drupal/core": "^8.3",

        "drupal/address": "1.x-dev",
        "drupal/commerce": "2.x-dev",
        "commerceguys/intl": "v0.7.4",

        "drupal/fontyourface": "3.1.0",
        "drupal/admin_toolbar": "1.19.0",        
        "drupal/better_exposed_filters": "^3.0@alpha",
        "drupal/captcha": "^1.0@beta", 
        "drupal/ctools": "^3.0",
        "drupal/imce": "^1.6", 
        "drupal/metatag": "~1.1",                                      
        "drupal/password_policy": "^3.0@alpha",
        "drupal/paragraphs": "^1.1",
        "drupal/honeypot": "^1.25",                     
        "drupal/token": "^1.0",
        "drupal/superfish": "~1.1",        
        "drupal/entity_reference_revisions": "~1.3",
        "drupal/profile": "~1.0",
        "drupal/state_machine": "~1.0",
        "drupal/video_embed_field": "^1.5",   
        "drupal/scheduler": "^1.0@RC",
        "drupal/ultimate_cron": "2.0.0-alpha1",
        "drupal/inline_entity_form": "~1.0",
        "drupal/pathauto": "^1.0",
        "drupal/views_slideshow": "4.5.0",
        "drupal/module_filter": "3.0.0",
        "drupal/pathologic": "1.x-dev",
        "drupal/rules": "^3.0@alpha",
        "drupal/typed_data": "^1.0@alpha",
        "drupal/xmlsitemap": "~1.0",
        "drupal/tvi": "^1.0@beta",
        "drupal/devel": "^1.0@RC",
        "drupal/webform": "^5.0@beta",
        "drupal/popup_message": "^1.0@beta",
        "drupal/easy_breadcrumb": "^1.6",
        "drupal/contact_storage": "^1.0@beta",
        "drupal/contact_emails": "^1.13",
        "drupal/node_clone": "1.x-dev",
        "drupal/super_login": "^1.0@beta",
        "drupal/login_security": "^1.2",
        "drupal/search404": "^1.0@beta",
        "drupal/link_click_count": "^1.1",
        "drupal/empty_page": "^1.0",
        "drupal/block_scheduler": "^1.0",
        "drupal/persistent_login": "^1.0@alpha",
        "drupal/private_message": "^1.0@beta",
        "drupal/fullscreen_gallery": "^1.0",
        "drupal/simple_timeline": "^1.1",
        "drupal/flippy": "1.x-dev",
        "drupal/field_collection": "^1.0@alpha",
        "drupal/search_api": "^1.1",
        "drupal/we_megamenu": "1.x-dev",
        "drupal/asset_injector": "^2.0",
        "drupal/custom_tokens": "^1.0@RC",
        "drupal/views_argument_token": "^1.0@alpha",
        "drupal/views_tools": "^1.0",
        "drupal/views_tree": "2.x-dev",
        "drupal/fpa": "2.x-dev",
        "drupal/feeds": "3.x-dev",
        "drupal/fieldblock": "^2.0@alpha",
        "drupal/field_paywall": "^2.0@beta",
        "drupal/field_timer": "^1.0",
        "drupal/field_validation": "^1.0@alpha",
        "drupal/file_download_tracker": "^1.0",
        "drupal/filebrowser": "^2.0@RC",
        "drupal/flexslider": "^2.0@beta",
        "drupal/flipclock": "^1.0",
        "drupal/accordion_blocks": "^1.0@beta",
        "drupal/bootstrap_site_alert": "^1.1",
        "objectivehtml/flipclock": "^0.7.7",
        "drupal/jquery_countdown_timer": "^1.1",
        "drupal/footable": "^1.0@alpha",
        "drupal/username_validation": "^1.0",
        "drupal/realname": "^1.0@RC",
        "drupal/blockgroup": "^1.0",
        "drupal/token_filter": "^1.0@beta",
        "drupal/views_blogspot_archive": "1.x-dev",
        "drupal/registration_role": "^1.0@alpha",
        "drupal/photos": "^4.0@RC",
        "drupal/juicebox": "^2.0@beta",
        "drupal/redirect": "^1.0@alpha",
        "drupal/seo_checklist": "^4.0",
        "drupal/workbench": "^1.0@alpha",
        "drupal/message": "^1.0@beta",
        "drupal/message_notify": "^1.0@beta",
        "drupal/commerce_shipping": "^2.0@beta",
        "drupal/color_field": "^1.0",
        "drupal/commerce_authnet": "^1.0@alpha",
        "drupal/commerce_paypal": "1.x-dev",
        "drupal/commerce_stripe": "1.x-dev",
        "drupal/commerce_wishlist": "^3.0@alpha",
        "drupal/fraction": "^1.0",
        "drupal/entityqueue": "^1.0@alpha",
        "drupal/oauth": "^2.0",
        "drupal/twitter": "1.x-dev",
        "drupal/menu_export": "^1.0",
        "drupal/libraries": "3.x-dev",
        "drupal/lightgallery": "^1.0",
        "drupal/tagclouds": "1.x-dev",
        "drupal/eck": "^1.0@alpha",
        "drupal/entity_browser": "^1.0",
        "drupal/entity_clone": "1.x-dev",
        "drupal/replicate": "1.x-dev",
        "drupal/block_visibility_groups": "^1.1",
        "drupal/commerce_order_number": "^1.0@alpha",
        "drupal/replicate_ui": "1.x-dev",
        "drupal/field_group": "^1.0@RC",
        "drupal/views_data_export": "^1.0@alpha",
        "drupal/features": "^3.5",
        "drupal/commerce_avatax": "1.x-dev",
        "drupal/migrate_tools": "^4.0@beta",
        "drupal/migrate_plus": "^4.0@beta",
        "drupal/media_entity_browser": "^1.0@beta",
        "drupal/entity_browser_enhanced": "^1.0@beta",
        "drupal/media_entity": "^1.6",
        "drupal/entity_embed": "^1.0@beta",
        "drupal/embed": "^1.0",
        "drupal/file_entity": "^2.0@beta",
        "drupal/media_entity_audio": "^1.0@beta",
        "drupal/media_entity_image": "^1.2",
        "drupal/file_browser": "^1.1",
        "drupal/focal_point": "^1.0@beta",
        "drupal/views_bootstrap": "3.x-dev",
        "drupal/currency": "^3.1",
        "drupal/filefield_paths": "^1.0@beta",
        "drupal/field_tools": "^1.0@alpha",
        "drupal/ajaxify_add_cart": "1.x-dev",
        "drupal/commerce_cart_blocks": "^1.0@alpha",
        "drupal/name": "^1.0@beta",
        "drupal/sitemap": "^1.3",
        "drupal/alinks": "^1.0",
        "drupal/block_titlelink": "1.x-dev",
        "drupal/address_phonenumber": "^1.1",
        "drupal/codefilter": "1.x-dev",
        "drupal/content_access": "1.x-dev",
        "drupal/views_secondary_row": "^1.0",
        "drupal/url_redirect": "^2.1"
    },
    "replace": {
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "config": {
        "preferred-install": "dist",
        "autoloader-suffix": "Drupal8"
    },
    "extra": {
        "_readme": [
            "By default Drupal loads the autoloader from ./vendor/autoload.php.",
            "To change the autoloader you can edit ./autoload.php."
        ],
        "merge-plugin": {
            "include": [
                "core/composer.json"
            ],
            "recurse": false,
            "replace": false,
            "merge-extra": false
        }
    },
    "autoload": {
        "psr-4": {
            "Drupal\\Core\\Composer\\": "core/lib/Drupal/Core/Composer",
            "Drupal\\composer_manager\\Composer\\": "modules/composer_manager/src/Composer"
        }
    },
    "scripts": {
        "pre-autoload-dump": "Drupal\\Core\\Composer\\Composer::preAutoloadDump",
        "post-autoload-dump": "Drupal\\Core\\Composer\\Composer::ensureHtaccess",
        "post-package-install": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "post-package-update": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "drupal-rebuild": "Drupal\\composer_manager\\Composer\\Command::rebuild",
        "drupal-update": "Drupal\\composer_manager\\Composer\\Command::update"
    },
    "repositories": {
        "drupal": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        },
        "desandro/imagesloaded": {
            "type": "package",
            "package": {
                 "name": "desandro/imagesloaded",
                 "type": "drupal-library",
                 "version": "4.1.3",
                 "dist": {
                     "url": "https://github.com/desandro/imagesloaded/archive/v4.1.3.zip",
                     "type": "zip"
                 }
            }
        },
        "desandro/masonry": {
            "type": "package",
            "package": {
                 "name": "desandro/masonry",
                 "type": "drupal-library",
                 "version": "4.1.1",
                 "dist": {
                     "url": "https://github.com/desandro/masonry/archive/v4.1.1.zip",
                     "type": "zip"
                 }
            }
        },
        "objectivehtml/flipclock": {
            "type": "package",
            "package": {
                 "name": "objectivehtml/flipclock",
                 "type": "drupal-library",
                 "version": "0.7.7",
                 "dist": {
                     "url": "https://github.com/objectivehtml/FlipClock/archive/0.7.7.zip",
                     "type": "zip"
                 }
            }
        },
	"sachinchoolur/lightgallery": {
	       "type": "package",
	       "package": {
	           "name": "sachinchoolur/lightgallery",
	           "type": "drupal-library",
	           "version": "1.2.21",
	           "dist": {
	               "url": "https://github.com/sachinchoolur/lightGallery/archive/1.2.21.zip",
	               "type": "zip"
           	   }
               }
        }                
    }
}

Your questions are excellent, probably best asked of the guys who develop the composer utility (see github).

https://github.com/composer
https://github.com/orgs/composer/people
https://github.com/naderman
https://github.com/Seldaek

I actually maintain some aspects of the composer.json files I use, by hand.
I have explicitly added modules to the require section: like token and ctools that are dependencies of pathauto, and other modules.

As you point out, if one runs "composer require drupal/pathauto", then token and ctools get added to the composer.lock file, but they don't appear in the require section even though the composer utility will behind scene load them onto the server and update them as needed. This phenomenon is also true of many other "packages" that get installed by the composer utility. For example running "composer require drupal/core" or "composer update drupal/core" runs dependencies for dozens of php and javascript libraries/packages that its installs into the /vendor directory, and which don't have any explicit inclusion in the composer.json file.

The composer docs pages do not contain explicit Drupal specific configuration documentation, from what I can find. In fact, I have yet to find any detailed documentation that explains how all the ancillary directives related to Drupal D8 site configuration work, and might be customized (1).

I would welcome link(s) to truly comprehensive documentation about this topic myself.

(1) as noted elsewhere, my opinion is that the functionality composer implements ought to be implemented as a "headless" tool that is fully integrated into the Drupal 8 admin GUI, to completely eliminate the need for command line dependency management. The current state of affairs really hurts the potential audience for Drupal 8. The need for the command line tool composer (which has proven very necessary), limits the audience for Drupal 8 to a very narrow demographic of code geeks, and has pushed a lot people to WordPress and Backdrop CMS. The dependency management functionality that the composer utility attempts to implement is great, but their should be an API only (headless) version of this functionality that Drupal could/should have integrated directly into Drupal core itself. The fact that this did not happen (despite many years of "planning and "design") indicates some serious judgment problems (market understanding problems)(2) on the part of the people in control of Drupal Core and the future of Drupal.

(2) Regarding judgment and perspective among geeks, consider the recent global blow up over the following:
https://gizmodo.com/exclusive-heres-the-full-10-page-anti-diversity-scre...
which reflects an arrogance problem in much of the software development universe ...

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

Rounder commented

@sprite

Thank you very much for all the time and effort you spent posting detailed responses to me. It is greatly appreciated.

I've spent the past few days screaming at my computer due to the documentation and videos I have found on aspects of Drupal, Composer and now Commerce 2.x (although I am just starting to get into Commerce - maybe its material will be better than the others). What I am seeing is that a lot of the material is basically "preaching to the choir": people with years and years of Drupal experience, writing for people with the same amount of experience who all praise its substance.

Don't these people realize that there are many people coming into the platform as completely new users? Can't they write their material with those people in mind, while still writing it to satisfy the experienced developers?

I know I did in my past job in the enterprise mainframe environment, both in written documentation and in giving live training sessions to white collar, blue collar and pink collar employees.

When I was younger people used to say that I acted like a grumpy old man. Now I am one. (vbg)

sprite’s picture

sprite commented

@marrer

It seems like even since decades ago, the whole computing thing has been about separating those who can simply figure things out themselves, from the rest of the world (by the way, rarely reveal your demographics (age/sex/ethnicity/politics), since all will be held against you (the rest of the world)) ...

However, there are other layers, and sociopolitical/demographic phenomena, at play ...

Drupal 8 Commerce 2.x (abbreviated here as D8DC) ...

The short answer to your core question is - NO - the people in actual control of projects like Drupal Core and Drupal Commerce do not care much at all about beginners these days. Explaining the layers of that is beyond the scope here though. However, an important aspect of Drupal Core and Drupal Commerce is that in both cases they are the means to an end that support commercial businesses that use Drupal as a foundation for paid services. For example the people who create Drupal Commerce, are primarily interested in it as a platform for paid commerce site building and "hosted" commerce websites. Their commercial companies are: platform.sh and commerceguys.com. The reality is that the more difficult their tools are to use, the more likely corporate interests are to hire them to write all the additional code and do all the additional configuration to make them work, that they don't provide out of the box. This is why Drupal Core ships with the ugly Bartik theme, and why even the new D8DC implements the backend for complex commerce, while nearly ignoring many of the basics for small businesses to be able to use D8DC to integrate commerce quickly into an existing D8 website (.. and its riddled with all sorts of bugs that wouldn't never be tolerated by "traditional" commercial/packaged software businesses and their customers. The developers of commerce (D7 and D8) have the direct conflict of interest that what they want is a platform for commercial paid services referrals to their commercial businesses (platform.sh and commerceguys.com). The reality is though, that without the commercial support of money making businesses, projects like Drupal Commerce probably wouldn't exist at all, and Drupal Core itself probably wouldn't exist either, taking note that Drupal Core code commits are largely controlled by Acquia, the $100 million Drupal services business run by Drupal's founder.

The creators of D8DC are a prime example of the phenomena you are experiencing in a widespread way. The need for accoutrements like "documents" and "support", are treated like dissonance within much of the open source culture, where the next "release" is a mere github commit away, running on "internet time".

BTW, if you want your head to start spinning around in circles, check out the planning threads for the Drupal 8 "media initiative". IMO, the Drupal 8 media initiative's results will guarantee that new users will be completely perplexed by their massive suite of low level modules, all of which require intensive configuration, do not come pre-configured, and which create abstractions that will require others to write full scale doctoral dissertations explaining how they work:

https://www.drupal.org/node/2825215
https://www.drupal.org/node/2860796
https://www.drupal.org/node/2825215

Meanwhile, Drupal is in major version 8, and it still doesn't integrate image, video, and audio, management very well, and where it does, highly detailed configuration and setup are required to get any of it to work. Meanwhile on D7 I use a custom patched version of the audiofield module and the soundmanager2 module to host mp3 player fields on D7 websites (not part of the new media initiative but working). On D8 there is a version of the audiofield module, not connected with the "media initiative" and it currently does an easier to work with job than trying to use the "media initiative" modules, although it all requires using patched modules to achieve anything reasonably workable. Meanwhile, it's Drupal 8, and Drupal still just doesn't make image/video/audio integration at all easy, and makes it completely daunting for beginners I'm sure.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

Rounder commented

@sprite

Looks like I am going to be having fun with this platform. (vbg) Although I've pretty much had to learn everything on my own in my career anyway (and the best form of teaching I ever had was having to fix other people's poor code - that'll teach you best practices in a hurry!).

I figure if I could teach myself APL, I can figure D8 out. Heck, in 1976 for my bachelor's project I wrote a chess program in APL. Our college had a copy of the Greenblatt chess program and my program even beat that in a game! (I didn't press my luck by having it try a second game)

If you've never seen what APL looks like (many Greek-looking characters), here is a sample image I found online. APL is read and executed from right to left. It's still my all-time favorite language to code in.

Example of APL coding

As for the commerce guys, I was wonder how and why they were willing to spend so much time and effort on open source code. They are obviously following a variation of the business model developed by King Gillette: give away the razor but make your profit on the blades. Only in this case, give away the code but make it so complicated, you have to hire them to implement it. (g)