The books and materials I have used to date (such as Drupal 8 Explained) have only covered modules being installed using the Extend function. I'm now in the position of having to install a shopping cart system on one of my sites. That lead me Drupal Commerce 2.x, which requires the use of Composer.

I'm just now digging into how that facility works, and starting to look at some documentation. To try to get some background material, I was listening today to some podcasts on DrupalEasy and Talking Drupal. Despite the name of the first show, these are definitely not for beginners, but they assume the listener has many, many years of experience working with the platform.

One thing I did get out of it is that you cannot change horses in mid-stream: it's pretty difficult if not impossible to get Composer to work if to date you simply installed and enabled modules from the Administration panels for a website.

So how best to set up Composer, if you are not manually installing Drupal on a remote server? I am using InMotion Hosting for my service, and have used Softaculous to install Drupal 8 on specific folders or subdomains.

Softaculous only allows you to specify the following during a new installation:

  1. Version level
  2. Protocol
  3. Domain name
  4. Directory (optional)
  5. CRON timings
  6. Site name
  7. Admin userid & password
  8. Admin email
  9. Database name
  10. Table prefix
  11. Backup location

So can I go ahead and use Softaculous to create a site, and then manually do what is needed to get Composer up and running? Or are there other specific things that need to be done, so that I really need to do the entire Drupal manually and have to skip using Softaculous completely?

Comments

sprite’s picture

Just install D8 by hand, and do not use softaculous.
It seems like you must be using cPanel.
If need be, create an addon domain or subdomain.
- you may need to wait for DNS to resolve if you do that.
- make the directory /public_html/{domain-prefix}.{domain}.{tld-suffix}/
- - such as /public_html/test.mydomain.com/

In cPanel, upload the D8 core release to the site docroot (directory described just above) and then right-click and use the GUI option to extract the archive.

Then in cPanel, create a database at the "MySQL Databases" page, add a user, and give the user all privileges.
Store the database name and password in a text or word document for safe keeping.
Confirm the database exists by entering the phpmyadmin utility from cPanel to see the empty database.
Then from a browser run {site}/install.php.
The install will check prereqs and ask for database credentials.
The install will complete and D8 core will be ready to use.
Add a free theme like magazine_lite or something so your site doesn't look like Bartik hell.

-

COMPOSER UTILITY
As a prerequisite to installing composer, you need to setup SSH command line access.
In cPanel there is an "SSH Access" page where you can create a public/private key pair.
You will also need putty.exe for a Windows local machine, or some similar terminal program for MacOS.
You will need to figure out and execute all the details of setting up SSH and putty and so on.
Depending on your hosting service, if you are using a VPS or dedicated server, the hosting service may have other SSH setup requirements, including their explicit enabling of your SSH key pair.
You may also need to create a local php.ini and include increased memory limit settings in it.
- - such as: memory_limit = 1024M
Once you can log in via SSH, you want to go to your cPanel main root directory and run:

php -r "readfile('https://getcomposer.org/installer');"

It will install composer in your user root directory and create a .composer directory where it caches files it uses (cache will get big).

The composer install will create a composer.phar file.
Place a copy of that composer.phar in your website's docroot directory.

Then, from the SSH command line - run:

php composer.phar {arguments}

To perform whatever composer operations you need.

The composer utility relies on the contents of a configuration file called composer.json in the website docroot directory that describes the Drupal installation configuration (see example below). You'll want to learn how it works. A well configured composer.json makes updating D8 code and modules very easy and reliable.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

fkelly12054@gmail.com’s picture

Depending on your hosting service, if you are using a VPS or dedicated server, the hosting service may have other SSH setup requirements,

Very good instructions generally. Thanks. But what if you are not using a VPS or dedicated server? Does anyone actually have Composer running on a shared service (I use Inmotionhosting too, but shared).

sprite’s picture

Yes people do.

Hosting services are coming around to providing "jailed" SSH access to shared hosting customers, since there are now multiple CMS and webdev platforms that pretty much require it, and can't live without it.

Some have only done so reluctantly, and only after customers have made SSH access a deal breaker.

Hosting companies that don't, will get the "doesn't do Drupal 8 well" mark of death, and will lose business.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

sprite’s picture

Here is an example composer.json, with basic modules added that nearly every D8 site ought to have, that you can use to get your use of composer to maintain you D8 site started:


{
    "name": "drupal/drupal",
    "description": "Drupal is an open source content management platform powering millions of websites and applications.",
    "type": "project",
    "license": "GPL-2.0+",
    "require": {
    	"drush/drush": "8.1.12",
        "phpdocumentor/reflection-docblock": "2.0.5",
        
        "composer/installers": "^1.0.21",
        "wikimedia/composer-merge-plugin": "~1.3",
        "drupal/core": "^8.3",

        "drupal/admin_toolbar": "^1.19",        
        "drupal/module_filter": "3.0.0",
        "drupal/metatag": "^1.1",                                      
        "drupal/password_policy": "^3.0@alpha",
        "drupal/honeypot": "^1.25",                     
        "drupal/token": "1.0.0",
        "drupal/ultimate_cron": "^2.0@alpha",
        "drupal/pathauto": "^1.0",
        "drupal/xmlsitemap": "~1.0"
    },
    "replace": {
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "config": {
        "preferred-install": "dist",
        "autoloader-suffix": "Drupal8"
    },
    "extra": {
        "_readme": [
            "By default Drupal loads the autoloader from ./vendor/autoload.php.",
            "To change the autoloader you can edit ./autoload.php."
        ],
        "merge-plugin": {
            "include": [
                "core/composer.json"
            ],
            "recurse": false,
            "replace": false,
            "merge-extra": false
        }
    },
    "autoload": {
        "psr-4": {
            "Drupal\\Core\\Composer\\": "core/lib/Drupal/Core/Composer",
            "Drupal\\composer_manager\\Composer\\": "modules/composer_manager/src/Composer"
        }
    },
    "scripts": {
        "pre-autoload-dump": "Drupal\\Core\\Composer\\Composer::preAutoloadDump",
        "post-autoload-dump": "Drupal\\Core\\Composer\\Composer::ensureHtaccess",
        "post-package-install": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "post-package-update": "Drupal\\Core\\Composer\\Composer::vendorTestCodeCleanup",
        "drupal-rebuild": "Drupal\\composer_manager\\Composer\\Command::rebuild",
        "drupal-update": "Drupal\\composer_manager\\Composer\\Command::update"
    },
    "repositories": {
        "drupal": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    }
}


spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

@sprite

Thank you for all of the information. It'll help get me started in the right direction.

So that I have the information at hand, and also make sure that I don't mess up an installation, are there any books or video documentation that I can use as a reference that also give detailed examples (so that I always have the necessary steps at the tips of my fingers)? I don't mean various websites but actual published material.

I just ordered Chaz Chumley's new book Mastering Drupal 8 (released this week), but I have been unable to find a detailed table of contents for it online, so I do not know if it contains material on Composer (along with SSH, etc). How about the books from Packt and Apress that have the term "Enterprise" in their titles?

I come from the mainframe environment, where I am used to having several binders' worth of 8.5"x11" manuals on my desk that documented all aspects of the platform and the languages (in my case z/VM with CP and CMS commands, XEDIT, REXX and Pipes, etc), so I am looking for something similar with Druapl.

sprite’s picture

Sure buy some books.

However, with Drupal, IMO the easiest and simplest way is to use online resources and to learn by doing.

Read the online drupal.org docs very thorough, multiple times.

https://www.drupal.org/docs/8
https://www.drupal.org/docs/develop
https://api.drupal.org/api/drupal/8.3.x

Learn to use google search to search drupal.org.

{parameters} site:drupal.org

There are also THOUSANDS of really good youtube videos on every aspect of Drupal, from running installs to theming and building modules.

On youtube, checkout the channels: leveluptuts, webwash, codekarate, buildamodule, "watch and learn", jacob rockowitz (creator of D8 webforms), and many more.

https://www.youtube.com/user/codekarate
https://www.youtube.com/user/watcAndLearnTuts/videos
https://www.youtube.com/user/LevelUpTuts/videos
https://www.youtube.com/user/webwashnet/videos
https://www.youtube.com/channel/UCuNY72GurNzdzXMsffs4VZw/videos

(many more)

Learn to make BACKUPS daily of development websites, which can even be automated with shell scripts.

Keep in mind that plain Drupal core out of the box (ugly Bartik theme) doesn't give you ANY idea of what Drupal can do and what Drupal can look like. Check out some free themes like the following:

https://www.drupal.org/project/magazine_lite

https://github.com/morethanthemes/magazine-lite

https://www.drupal.org/project/showcase_lite

https://github.com/morethanthemes/showcase-lite

https://www.drupal.org/project/newsplus_lite

https://github.com/morethanthemes/newsplus-lite

https://www.drupal.org/project/scholarly_lite

https://github.com/morethanthemes/scholarly-lite

Use the distributions from github to install the complete D8 distribution with theme and sample content for the best way to get started with those.

Keep in mind that the same guys create commercial versions of the themes above that are in themselves like a Drupal university. They sell their entire package of distributions for a flat fee too. Rather than books on Drupal, I've learned as much about Drupal from studying their theme distributions, as from many other sources. Check out the demos of their commercial theme distributions which they sell with all the demo content alright installed and ready to work with, such as the follow, and more:

http://demo.morethanthemes.com/enterpriseplus8/default/

http://demo.morethanthemes.com/corporateplus8/default/

http://demo.morethanthemes.com/magazineplus8/default/

The above should keep you busy for the next couple years.

====

In addition to the drupal forums - get to know the "issues queues"(1) which is where the hardcore Drupal geeks hang out. Click follow on interesting threads in the issues queues. For example, following the threads at the following link to keep up with where Drupal core may be going in the future.

https://www.drupal.org/project/issues/ideas

Every module, distribution, theme, component, etc., has an "issues queue"(1), so follow them.

====================

(1) I have a pet peeve with the epidemic western cultural colloquial misuse of the word "issue" as an incorrect substitute for the word "problem". The proper term is "software problem", but they're incorrectly called "issues", these days, in a terrible misuse of the English language, even to the extent it seems a genocide of the word problem, an attempt to obliterate the word "problem" from the English language. The only proper use of the word "issue" that even comes close is use of the word "issue" as a substitute for the word dispute, as in the disputes in a lawsuit are called the "issues", i.e. the disputes between the parties.

In any event follow the "software problem lists" euphemistically known as "issues queues" on drupal.org.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

sprite’s picture

For doing Drupal work, and webdev in general, get yourself some modern code editors. Many are open source. Many have all sorts of plugins and such available to do things like check syntax, highlight syntax, perform API code completion (yeah they know the Drupal API and can complete code phrases).

ALL FREE AND ALL CROSS-PLATFORM
- - - don't forget to study and install the plugins which can be installed from each of the following:

https://electron.atom.io/
- very good

https://code.visualstudio.com/
- also good

https://www.sublimetext.com/
- also good

https://notepad-plus-plus.org/
- this lasst doesn't have the advanced UI of the others, but it is very lightweight, and great for perusing large *.SQL database export files.

====

For module development and debugging the following IDE is very good and probably most popular.

https://www.jetbrains.com/phpstorm/

====

Ditch the mainframe book paperweights and go digital.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

adminMN2023’s picture

Sprite - maybe I missed it in the explanation - but are you alluding to the fact that composer needs a fresh install of drupal to work? Or can it be added after a site has been built and live already?

You taking the time to explain all this is excellent.

Thanks for helping all us beginners out.

sprite’s picture

The composer utility is easily added to an existing drupal installation using the instructions above.

There isn't any implication in the instructions above that a fresh Drupal install needs to be created in order to also install the composer utility.

To some extent, Drupal, from the perspective, of the Drupal GUI Drupal admin GUI environment, it doesn't really know that the composer is behind the scenes maintaining the code sitting on the server.

===

I can say that once I started using composer, I've found it really helpful, even essential for D8 site maintenance. When composer gets confused though, composer can be quite frustrating and has a learning curve to figure out how to deal with its idiosyncrasies.

For example, yesterday when 8.3.6 came out, I have a single shell script on each server that automatically updates ALL the Drupal instances on the server, with one single shell script command (which takes a while to run though, because composer can be slow).

===

I have also been one of many people who believe that the functionality of the composer utility ought to be embedded into Drupal itself via a "headless composer" version that Drupal can access with APIs so that composer style updates can all be performed from the Drupal admin GUI without any need for command line shell access. That functionality is one of the most important features I think Drupal needs going forward, in order to appeal to a wider sitebuilder audience, and the corporate IT departments that want that sort of thing.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

@sprite

You said above:

"The composer utility is easily added to an existing drupal installation using the instructions above.

"There isn't any implication in the instructions above that a fresh Drupal install needs to be created in order to also install the composer utility."

So when you do install Composer after installing and enabling several modules, what happens when in the Administration panels you see the pop-up message that says "There are updates available for one or more of your modules or themes. To ensure the proper functioning of your site, you should update as soon as possible."

Can the two co-exist? When going to Composer and doing any updates, will Drupal Admin be happy and recognize the updates and not warn me anymore (at least until the next new version is available)?

Oh yeah - in an earlier post you said "Ditch the mainframe book paperweights and go digital.". I've often run into situations where how I am describing the problem I need to solve is not worded the same as the authors do in the Table of Contents or index. So I like paper because it is easier to thumb though the book or manual rather than the digital version looking for what I need.

I know I am a conundrum. I have a MS in Computer Engineering, have worked for 40 years in first engineering (test equipment development) and then I.T., but am still a bit of a Luddite! (grin) I still like paper.

sprite’s picture

The notices in the Drupal admin GUI and the composer utility co-exist just fine.
Once you've installed composer, when you see one of those notices on a D8 site, open a SSH command line window and run:

"composer update"

You can also use composer to update a single module, for example:

"composer update drupal/metatag"

will update just that module and its dependencies. Once the update occurs, when you return to the Drupal admin GUI and view the "available updates" report, the module will appear updated. Don't forget to run {site}/update.php after every module update, and clik on the "clear cache" button in the Drupal admin GUI.

The documentation for the composer utility is tersely/poorly written:

https://getcomposer.org/doc/

but if you go over it a few times and just try out commands, it gets familiar.

---

So, once you install composer only use the admin GUI for enabling modules and to "uninstall" modules, keeping in mind that Drupal admin GUI module "uninstall" removes its configuration data from the SQL database, but it does NOT remove physical files from the server.

To remove physical components files from the server with composer, use the:

composer remove {some-component}

----

Go get your server setup and try out all of this.
It will become obvious.

p.s. note that I wrote "sure buy some books" ...

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

sprite’s picture

Just learn by doing it.
Install Drupal core a few times over and over, just to fully get the hand of doing.
Get familiar with using phpmyadmin to create and manage databases, to perform import/export on databases, and so on.

----

Also try installing one of those distributions from github, where the procedure just slightly different from I have described.
When installing one of the distributions I linked to you:
- import the provided database into the database for your site/url
- extract the code contents into your site's docroot.
- run {site}/install.php and enter database credentials, and it will respond that the site is installed already.
- visit the {site} in browser to check.
- put a composer.phar file into the docroot, swap out the composer.json with one configured like my example.
- run php composer.phar update, and it will update the distributions core

Done and ready to rock with one of the free distributions I've linked to.
Seriously, if you are new to Drupal, try out one of those free distributions.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

sprite’s picture

Even with the most powerful CMS ever created, there is still a LOT of repetitive grunt work involved in site building. In addition, to have a site, somebody has to write all the text , and create all images, for a site's content. Plus a site needs to have an overall plan.

Trust me, it takes years to feel like even scratching the surface of Drupal.

Get a server with multiple Drupal test installations running, and just try out site features, contrib modules, and so on, see what they do.

Watch youtube videos on Drupal (see links above), of which there are literally thousands. It can be helpful to have a Drupal test site running to follow along with many of the youtube Drupal videos, to replicate what they do in their screencasts. It all takes time, so allocate plenty of time, to just site and do it.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

@sprite

"Even with the most powerful CMS ever created, there is still a LOT of repetitive grunt work involved in site building. In addition, to have a site, somebody has to write all the text , and create all images, for a site's content. Plus a site needs to have an overall plan."

That's exactly where I am at now. Having reached the twilight years of my work career, I've decided to go off on my own as a B2B copywriter. I figured it made sense to combine both my experience in engineering/I.T. with my previous (late '80s) freelance work as a newspaper and magazine sports writer, where I won regional and national awards (including one that is not given out every year, but only when an entry meets the specific criteria of the award).

Although it wasn't an official part of my I.T. career, I did a lot of writing and documenting in my jobs. So maybe you can understand my frustration in the lack of proper centralized documentation for aspects of Drupal.

Even published books have problems. Drupal 8 Explained was released March 2017, and I purchased the 1.0.2 version from Amazon right when it was released the first week of May. But that book was unfortunately full of all sorts of mistakes and incorrect instructions, not to mention the typos & misspelled words and incorrect punctuation. But with a paying client as my focus - plus having to also be a rainmaker to get more - I did not have the time (or inclination) to document the myriad of mistakes I found. (Sorry Stephen & Cindy).

Thanks for the warning that "the documentation for the composer utility is tersely/poorly written". To paraphrase Bing Crosby who said to Frank Sinatra during their late '50s Christmas TV special, "I'll read it privately while I gnash my teeth." (grin)

Although I was starting to learn Drupal as a CMS for my own interests, I hadn't intended to get into website development for my business (wanted to get away from coding). But this particular client was not happy with their existing site and especially the quality of support (or rather the lack of it) they were getting from that company, so one thing led to another and they realized that they could hire two services - content and website development - from one person.

Thanks again for your detailed instructions. Another day of experimenting and learning ahead for me.

sprite’s picture

Contrary to what you wrote just above, IMO, there is a lot of Drupal documentation online, and in a quite "central" location - the drupal.org website.

As suggested above, the drupal.org documentation is a reasonably good source, and it is wiki oriented, so that you too can edit it if you think something is missing!!!!:

https://www.drupal.org/docs/8

also learn to search specific topics using the following syntax which will search only a given website:


{search keywords} site:drupal.org 

Also, youtube videos are free and provide some of the best information around as people screencast and step by step work through doing things drupal, so you can see that they are correct as they go.

Meanwhile, it can take years to even think about mastering Drupal.

Just before posting, and when I go back, in a moment, digging through all the backend basic configuration things that are different between D7 and D8 takes hours/days/months/years. There are tens of thousands of D7 and D8 modules, and they are different between D7 and D8, which now requires knowing the two different versions of modules. Even essential contrib modules like metatag and xmlsitemap have different GUIs between D7 and D8.

Drupal is an entire universe it and of itself.

... again, go digital, to learn drupal efficiently ...

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

Rounder’s picture

Removed and made a new topic: https://www.drupal.org/node/2900487

Rounder’s picture

Removed and made a new topic: https://www.drupal.org/node/2900484

sprite’s picture

IMO, you were astute to begin separate topics ....

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

riverrat’s picture

Found this thread and thought to add the results of a long discussion with inmotion hosting support about composer.

It is possible to install composer but it is not possible to run it. To do this you have to have a VPS account

Concretely support increased the memory limits but still had the problem as per their response

"It appears you are hitting the limits of your memory for the server. If you want to use composer you would have to upgrade your plan."

Hope this saves some others wasted time