[09:56] <isntall> jthorson: how do i test drupalci to use sqlite for the Drupal DB?
[10:03] <jthorson> isntall: Not coded.  
[10:04] <jthorson> We'll need to add a check at the beginning of DrupalCI/Plugin/BuildSteps/environment/DbEnvironment->run(); and if the db parameter is 'sqlite3', add a new variable to the $job->buildVars array that we can use later in processing to ensure we set up the actual sqlite database.
[10:04] <jthorson> And then bail without setting a service container.
[10:06] <jthorson> Then we'll need to play around with the definition/variable plugins to i) create the database file, and ii) ensure the run-tests.sh arguments are constructed properly to support it.
CommentFileSizeAuthor
#4 2484103-4.patch3.82 KBamateescu
#2 2484103.patch1.37 KBamateescu

Comments

jthorson’s picture

Issue summary: View changes
amateescu’s picture

StatusFileSize
new1.37 KB

Started to look into this a bit but I got lost at "Then we'll need to play around with the definition/variable plugins to i) create the database file, and ii) ensure the run-tests.sh arguments are constructed properly to support it." :/

jthorson’s picture

Okay ... taking a closer look at the code ... This gets into both DCI_DBURL and DCI_RunScript mangling, which is by far the most complex flow within this code. :)

1) Location within DBEnvironment->run() is correct, but we won't need to set a build variable ... We can key off of DBVersion, so just return from here.

2) The existence of a DCI_DBVersion parameter will trigger src/DrupalCI/Plugin/Preprocess/definition/DBVersion.php to add a 'dbcreate: sqlite' line in the job definition. There could be a problem here with the existing code expecting a database value (which it extracts with an explode() command), when sqlite won't have one.

3) We'll then need to create a src/DrupalCI/Plugin/BuildSteps/dbcreate/SQLite.php plugin, similar to the MySQL.php plugin in the same directory. Output of this plugin should set $cmd to whatever you need to do to create the sqlite database (Is this just 'touch filename'?) and then parent::run($job, $cmd). The DBUrlBase.php, DBUser.php, and DBPassword.php plugins may interfere in the code path here; as they will try to mangle the DCI_DBURL value, which is most likely where we would set the sqlite filename we want to use ... so we may need some special case logic here too. This mangling also gets passed into the --db-url parameter on the run-tests.sh command; so we need to ensure that we end up with a sqlite-compatible db-url that run-tests.sh is happy with after this mangling.

amateescu’s picture

Status: Active » Needs review
Issue tags: +D8 Accelerate
StatusFileSize
new3.82 KB

Managed to get this fully working! Thanks for the detailed explanation, it helped a lot in understanding this part of DrupalCI :)

Here's the output of a test run:

» sudo ./drupalci run simpletest
Calculating job definition
Loading DrupalCI platform default arguments:
DCI_CodeBase
Loading job type default arguments:
DCI_DBVersion,DCI_PHPVersion,DCI_CoreRepository,DCI_CoreBranch,DCI_GitCheckoutDepth,DCI_RunScript,DCI_DBUser,DCI_DBPassword,DCI_DBURL,DCI_TESTGROUPS,DCI_SQLite,DCI_Concurrency,DCI_XMLOutput,DCI_PHPInterpreter,DCI_Color
Loading local DrupalCI environment config override arguments.
DCI_TESTGROUPS,DCI_DBTYPE,DCI_DBVersion
Creating codebase data volume directory
Checkout directory created at /tmp/simpletest-10TEES
Parsing required Web container image names ...
Adding image: drupalci/5.4
Validating container images exist
Found image drupalci/web-5.4 with ID b626800f
Populating container codebase data volume.
Entering setup_checkout_git().
Performing git checkout of git://drupalcode.org/project/drupal.git 8.0.x branch to /tmp/simpletest-10TEES.
Git Command: git clone -b 8.0.x git://drupalcode.org/project/drupal.git /tmp/simpletest-10TEES --depth=1
Cloning into '/tmp/simpletest-10TEES'...
remote: Counting objects: 15753, done.
remote: Compressing objects: 100% (13214/13214), done.
remote: Total 15753 (delta 2811), reused 8448 (delta 1468)
Receiving objects: 100% (15753/15753), 12.98 MiB | 253.00 KiB/s, done.
Resolving deltas: 100% (2811/2811), done.
Checking connectivity... done.
Checkout complete.
Container /trusting_brown created from image drupalci/web-5.4 with ID 018f9229
Executing on container instance 018f9229:
mkdir -p /var/www/html/results
Command created as exec id 7c1db27d
HTTP/1.1 200 OK
Content-Type: application/vnd.docker.raw-stream


Executing on container instance 018f9229:
mkdir -p /var/www/html/results/xml
Command created as exec id eb5196a5
HTTP/1.1 200 OK
Content-Type: application/vnd.docker.raw-stream


Executing on container instance 018f9229:
chown -fR www-data:www-data /var/www/html/sites /var/www/html/results
Command created as exec id 98fbf56d
HTTP/1.1 200 OK
Content-Type: application/vnd.docker.raw-stream


Executing on container instance 018f9229:
cd /var/www/html && sudo -u www-data php /var/www/html/core/scripts/run-tests.sh  --sqlite /var/www/html/results/simpletest.sqlite --concurrency 4 --xml /var/www/html/results/xml --php /opt/phpenv/shims/php --color --dburl sqlite://localhost/sites/default/files/db.sqlite action
Command created as exec id c6d8ab92

Drupal test run
---------------

Tests to be run:
  - Drupal\action\Tests\ActionUninstallTest
  - Drupal\action\Tests\BulkFormTest
  - Drupal\action\Tests\ConfigurationTest

Test run started:
Drupal\action\Tests\ConfigurationTest                         51 passes                                      
Drupal\action\Tests\ActionUninstallTest                       13 passes                                      
Drupal\action\Tests\BulkFormTest                              72 passes                                      

Test run duration: 13 sec

HTTP/1.1 200 OK
Content-Type: application/vnd.docker.raw-stream
jthorson’s picture

Status: Needs review » Needs work

I'm currently still getting mysql dbcreate commands in the test run with the latest code ... what were your local override values for DCI_DBTYPE and DCI_DBVersion?

jthorson’s picture

Status: Needs work » Active

Ah ... I see.

I was trying to avoid having to set both DBType and DBVersion ... which feels redundant. So I was simply overriding the DBVersion value in the DBEnvironment class, which wasn't actually taking effect.

So I've verified this runs successfully with the proper local overrides ... I'm going to poke a bit more and see if I can render the DBVersion override redundant.

jthorson’s picture

Issue summary: View changes
Status: Active » Fixed

Looks like the SQLite preprocess plugin is unnecessary ... committed the rest.

I'd still like to get to the point where we don't need both DBType and DBVersion ... but that became a rathole of dependency calculation when running through the variable preprocessing plugins; and this is good enough for the time being (i.e. acceptable for the alpha target).

Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.