First thanks for the great work on Aegir.

But I think it is a little harder than it should to install because of confusing documentation.

The main page oa AEgir says prominently that the install.txt is the reference.
Except it seems not to be up to date

for instance :

the line

/var/aegir/drush/drush.php dl hosting install_profile_api

will have drush install the hosting module in sites/all/modules instead of the profiles/hostmaster/modules where it should be.
later on this will cause the Call to undefined function hosting_get_queues() error. especially if Drush does not bring in the latest version...

The documentation never explictly state that the DEFAULT site MUST BE the aegir system, it even goes to suggest otherwise in the section "Install the Drupal" where you have lines like

mkdir drupal-6.*/sites/aegir.example.com
chmod a+w drupal-6.*/sites/aegir.example.com/settings.php
....

witch leads you to think you should or can multi host AEgir under the sites/ directory (in that case a "cp" command would be missing for the settings.php file anyway)

I see lots of efforts put into documentation on various fronts (install, release doc, wiki, bug reports ...) and even an "automated script", I understand the logic but rather than some scripting that will lead many people to ask why the script don't work on their distrib, and be of little use for users of other Unix or distribs, .... maybe we should have ONE body of documentation, an INSTALL.txt up to date AND including the troubleshooting face and the Bug submission guidelines.

The video is cool and useful but when I see the efforts put into it, I am sad it is "already" outdated in a way ....

Plain text rules I think :)

Many "mandatory to succeed" things should be expressed up front in the doc.
Never run drush as root
The AEgir directory MUST be the home dir of the aegir user (so counter intuitive to me :) )
etc ....

Those are, for instance, in the bug submission guidelines, witch I, as of now can't get my hand on on the website again ...

Thanks a lot for your time and understanding, and thanks for the work on AEgir, very useful and impressive.

Comments

Anonymous’s picture

It looks like hostmaster/INSTALL.txt was not updated for the rc3 release, which is tripping some people up who are not upgrading from rc2.

The AEgir directory MUST be the home dir of the aegir user (so counter intuitive to me :) )

Mine is actually in /opt/www/deploy and i run it as the web user, whose home is /home/web :) the only thing that goes in here is .drush/provision. So it's not impossible.

web@aegir:/opt/www/deploy$ ls -al
drwx------  2 web web    4096 2009-08-15 16:08 backups
drwx------  3 web web    4096 2009-08-10 19:33 config
drwxr-xr-x 10 web web    4096 2009-08-15 18:51 drupal-5.19
drwxr-xr-x 10 web web    4096 2009-08-16 18:23 drupal-6.13
drwxr-xr-x  9 web web    4096 2009-08-15 16:04 drupal-7.x-dev
drwxr-xr-x 10 web web    4096 2009-08-10 20:44 openatrium

But I agree there are discrepancies in the docs, and fortunately I've been using it long enough to get around it on fresh installs.

For the purposes of the ticket also note that this step:

cp drupal-6.*/profiles/hostmaster/apache2.conf.txt config/vhost.d/aegir.example.com

note that apache2.conf.txt contains similar 'drupal-6.*' wildcards in the Directory or DocumentRoot tags (I forget which), which should be edited. This is clearly an example vhost config, and it didn't trip me up, but it should be made more obvious in the documentation that edits are required.

Perhaps a job for the ninja of Aegir documentation steveparks?! :)

Or, actually, even better, consider a rewrite/improvement of INSTALL.txt and submit it as a patch! I'm sure the developers would love you for it :)

Annakan’s picture

I thought about it.

As soon as my install works I want to give it a shot, rewriting the install docs based of the great work already done..

but unrfotunately I am stuck for now at the verify step and a provision module error

Anonymous’s picture

Priority: Normal » Critical

Marking critical per anarcat's comment http://groups.drupal.org/aegir/0.3-RC3#comment-87651

Annakan’s picture

That's really great to see the issue bumped up and as I said I would love to snitch up some documentation as soon as I can get Aegir working for myself (I have another issue opened about that)

I saw some comments in http://groups.drupal.org/aegir/0.3-RC3#comment-87651
that makes sense about the chosen path, I think we should build a "reference tree" clearing all this path issues and planing for the various platforms, including the hosting platform.

something like

\aegirHome\www\platforms\hosting
----------------------------------\drupal6.13
----------------------------------\drupal5.19
----------------------------------\pressflow
----------------------------------\aquia
----------------------------------\Whatever else
\aegirHome\Drush
\aegirHome\.drush
\aegirHome\backup
\aegirHome\config\
------------------------\vhost.d
\aegirHome\Databases
etc...

or something like that, making clear what should be web published (\aegirHome\www) and what should be backup-ed everyday (\aegirHome\Databases), what should be versionned (all the rest ? ) etc ...

Again :) I would love to help as soon as ... :)

Anonymous’s picture

A lot of stuff like versioning and backups etc ought to (in my opinion) be up to the user. I think the documentation should (and more or less already does) define a 'supported' or 'recommended' tree of where things should be, or a list of 'known configurations to work'

I have

/opt/www/deploy/drupal-6-13 (and other platforms at this level as per my earlier comment)
/opt/www/deploy/drupal-6.13/profiles/hostmaster
/opt/www/deploy/drupal-6.13/profiles/hostmaster/modules/hosting
/opt/www/deploy/drupal-6.13/profiles/hostmaster/modules/install_profile_api
/opt/www/deploy/drupal-6.13/profiles/hostmaster/themes/eldir
/opt/www/deploy/drupal-6.13/sites/deploy.mig5.net (my actual aegir site, and I symlink the 'default' dir to it)
/opt/www/deploy/config/vhost.d (note how it is outside the public-served drupal-6.13)
/opt/www/deploy/backups
/home/web/drush/drush.php
/home/web/.drush/provision

my aegir user is my 'web' user. (it happens to be the user I use to clone my git trees on this live server)

However I can see the most common use is running as user 'aegir', and:

/var/aegir/drupal-6-13
/var/aegir/drupal-6.13/profiles/hostmaster
/var/aegir/drupal-6.13/profiles/hostmaster/modules/hosting
/var/aegir/drupal-6.13/profiles/hostmaster/modules/install_profile_api
/var/aegir/drupal-6.13/profiles/hostmaster/themes/eldir
/var/aegir/drupal-6.13/sites/my.main.aegir.deployment.site.com
/var/aegir/config/vhost.d
/var/aegir/backups
/var/aegir/drush/drush.php
/var/aegir/.drush/provision

And with aegir user's home dir set to /var/aegir. Sure, one could set a 'www' directory under /var/aegir and I see others using /var/aegir/platforms. There's no reason why other configurations shouldn't work (like mine), but yes there should be (and is) a recommended approach, just needs to be laid out crystal clear in the docs.

Annakan’s picture

thanks for the input
First , I agree that backup and versioning should be at the user discretion BUT the layout should support this and help it by cleanly separating the stuff with varied lifetimes/changetimes.

Second, but completely about something else, I am very interested by your experience staging drupal properly. I was often refrained to recommend drupal or hesitant about its use because honestly there seemed to be no simple way to have a dev environment, a test one and a production one with an "automatic/reproducible" delivery mecanism, since the "parameters/configuration" of the website are completely intermixed with the content. To me it is the most important point that could keep drupal from becoming a "fully recommendable" tool. I see that you seem to use deploy and I should look into it..., would be interesting to take such things into consideration while defining a "reference directory layout" for AEgir.

Third : I think we need several things :
* a "reference directory layout", preferably separating the HOME dir of the AEgir user from the AEgir deployment directories (because I hate to have .ssh, whatever was downloaded and such into a potential production dir). The only thing that makes it more complicated that is should is the use of the .drush directory for drush specific plugins ... maybe there is another drush mechanism that would allow us to fully gather in one place all that is AEgir related. And thus even allow various AEgir version on a server for test purposes.

* a clear explanation of the dependencies and search paths to allow people to understand the implication of NOT following the recommended layout, sometime as a user pointed it, they do not have the choice and must do so because of hosting constraints. This is not something we could fully support but documenting it will probably in fact lower the troubles induced by such changes

* clear policies about what role the hosting platform have, it is better to leave the hosting drupal alone with only that site, or is it ok, or recommend to have various sites under the same platform engine ? Where to put new platforms ? etc .. How is the reference platform upgraded ? classic migration of the hosting platform toward the new version ? special treatment ? As far as I know this is not documented.

I personally and just following my guts would prefer to have a separate hosting platform put under \platform and (\platform\Drupal_AEGIR_XXX for instance) and the websites under separate platforms under platform also \platform\drupal6-x, platform\drupal5-x etc ...). I like symetry and I love the same things, sames life cycles, same places principle.

* Last I think it is better to have separate user for AEgir and the web server, It is a better security policy for people that are not UNIX security gurus and will avoid having undue privileges given to the AEgir-running-user, because these priviledges are needed for another reason, being the webserver or a VCS or whatever...

I, of course talk in a reference point of view, whoever knows better and know how to do it can do however he wants but for people new to AEgir having a reference template would be a boon and also a big reduction in support requests.. if my personal experience is of any significance :) LOL

Thanks a lot again for the input.

Anonymous’s picture

preferably separating the HOME dir of the AEgir user from the AEgir deployment directories (because I hate to have .ssh, whatever was downloaded and such into a potential production dir)

Well, there is the aegir home dir, and the DocumentRoot in the Apache configs. The stuff in the home dir outside of the platform roots should not be served by Apache.. so I don't think anything is 'wrong' about how it's currently advised to set up. I may have misunderstood

And .ssh dir should have certain permissions set so that only the aegir user (or root obviously) can view. But since it's outside the document root for apache, I don't think it's really that insecure

* clear policies about what role the hosting platform have, it is better to leave the hosting drupal alone with only that site, or is it ok, or recommend to have various sites under the same platform engine ?

I'm running my Drupal 6 sites under the same platform that the aegir site itself runs on. I guess it can't hurt to say 'yes this is possible' in the documentation, but also why not just try it and see if it works :) or rather, there is no reason for it not to work, so perhaps the choice has been to not point out where things work fine, and have people just file a ticket if it doesn't work.

* Last I think it is better to have separate user for AEgir and the web server, It is a better security policy for people that are not UNIX security gurus and will avoid having undue privileges given to the AEgir-running-user, because these priviledges are needed for another reason, being the webserver or a VCS or whatever...

Well I think this is already the way it is, Apache runs as www-data (or'apache' depending on your OS) user and you have an 'aegir' user that has extra sudo permissions to restart Apache but otherwise performs all the php cli commands when requested via the web frontend, but not as the www-data user. But perhaps again I misunderstood your concern here

All good discussion anyway - it's extremely quiet at the moment after an rc3 release that seems to be going wrong in several areas, hopefully we can help fix some stuff like documentation. The best we can do is either keep up the chatter, or (I still think this is a good idea once we have some solid ideas) patch the INSTALL.txt and then improve some of the documentation accordingly over here.

anarcat’s picture

Status: Active » Fixed

So I could have sworn I already closed and commented on this issue, but it seems i lost it.

I have committed a few changes in the documentation (INSTALL.txt) that fix most of the comments you have raised here. I mostly ignored the last two comments as I think they are site-specific policies that don't need to be in INSTALL.txt, but feel free to open new issues regarding that.

In general, I do think that INSTALL.txt is the canonical source of information on host to install Aegir. The screencast, the release nodes, the faq, the wiki, all the rest is just candy on top. The INSTALL.txt (and UPGRADE.txt) files *always* need to be in sync with the code and critical issues must be opened if it's not possible to install aegir by following the instructions in there.

I consider the discussion here closed, open new issues if there are specific problems that need to be addressed. Thanks a lot for the feedback!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

  • Commit 6e15663 on 599758_edit_main_site, 640952_client_preview, dev-dns, dev-features, dev-headless_install, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newsiteform, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-relationships, dev-restore, dev-server_nodetype, dev-services, dev-site_rename, dev-ssl, dev_dns, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-588728-views-integration, dev-1403208-new_roles, dev-helmo-3.x authored by anarcat:
    #550352 - make sure we install the modules and themes in the right...
  • Commit f02229e on 599758_edit_main_site, 640952_client_preview, dev-dns, dev-features, dev-headless_install, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newsiteform, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-relationships, dev-restore, dev-server_nodetype, dev-services, dev-site_rename, dev-ssl, dev_dns, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-588728-views-integration, dev-1403208-new_roles, dev-helmo-3.x authored by anarcat:
    #550352 - clarify some tricky non-intuitive parts of the install process...
  • Commit 699e76d on 599758_edit_main_site, 640952_client_preview, dev-dns, dev-features, dev-headless_install, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newsiteform, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-relationships, dev-restore, dev-server_nodetype, dev-services, dev-site_rename, dev-ssl, dev_dns, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-588728-views-integration, dev-1403208-new_roles, dev-helmo-3.x authored by anarcat:
    #550352 - add a sample database layout, thanks mig5!
    
    

  • Commit 6e15663 on 599758_edit_main_site, 640952_client_preview, dev-dns, dev-features, dev-headless_install, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newsiteform, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-relationships, dev-restore, dev-server_nodetype, dev-services, dev-site_rename, dev-ssl, dev_dns, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-588728-views-integration, dev-1403208-new_roles, dev-helmo-3.x authored by anarcat:
    #550352 - make sure we install the modules and themes in the right...
  • Commit f02229e on 599758_edit_main_site, 640952_client_preview, dev-dns, dev-features, dev-headless_install, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newsiteform, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-relationships, dev-restore, dev-server_nodetype, dev-services, dev-site_rename, dev-ssl, dev_dns, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-588728-views-integration, dev-1403208-new_roles, dev-helmo-3.x authored by anarcat:
    #550352 - clarify some tricky non-intuitive parts of the install process...
  • Commit 699e76d on 599758_edit_main_site, 640952_client_preview, dev-dns, dev-features, dev-headless_install, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newsiteform, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-relationships, dev-restore, dev-server_nodetype, dev-services, dev-site_rename, dev-ssl, dev_dns, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-588728-views-integration, dev-1403208-new_roles, dev-helmo-3.x authored by anarcat:
    #550352 - add a sample database layout, thanks mig5!