Problem/Motivation
No reason to fetch a whole drupal history, it will help
- to transfer less data from Gitlab
- speeds-up each CI job (fetching 50 commits takes ~10seconds comparing to 1minute for a whole repo)
Steps to reproduce
See https://git.drupalcode.org/project/drupal/-/jobs/82685
Getting source from Git repository 01:05
Proposed resolution
- add GIT_DEPTH=5to fetch only limited amount of of commits from MR branch
- fix 📔 Spell-checking job - it should fetch only branch to compare with
The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines.
Remaining tasks
- review/comit
User interface changes
API changes
Data model changes
Release notes snippet
Issue fork drupal-3387400
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3387400-gitlabci-should-fetch
changes, plain diff MR !4790
Comments
Comment #3
andypostComment #4
andypostComment #5
catchWe might be able to reduce the depth further once we've done #3387058: [PP-1] Add caching to cspell job in GitLabCI assuming that can take advantage of cloud caching.
This looks good to me and think we should try it.
Comment #6
andypostHere's example when fetch failed https://git.drupalcode.org/issue/drupal-3387400/-/jobs/88976
Comment #7
andypostCloud caching has downsides
- more data transfered from/to runner
- zipping cache before sending takes time and brings high load to caching server
Meantime I see #3386566: Add support for 'test only' changes to gitlab CI which also gonna diff changed files so this functionality can be extracted to before script for such jobs
Moreover the job should fail when it unable to find a lit of changed files
Comment #8
andypostAdded to IS about
${CI_COMMIT_BRANCH}which probably should not be used in scriptsComment #9
andypostit was added in #3386680-8: Run jobs on GitLab CI branch tests
Comment #10
andypostChanged to fetch less to compare and fixed IS as
TARGET_BRANCH=${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}${CI_COMMIT_BRANCH}which works both for MRs and commitsComment #11
catchI think we should see how this affects job performance, RTBCing and will fish for a +1 in slack.
This Unit test run finished in 1m45s https://git.drupalcode.org/issue/drupal-3387400/-/jobs/89382 vs 2m48s for HEAD: https://git.drupalcode.org/project/drupal/-/jobs/90216
So that is consisent with the ~50s that andypost reported above.
Comment #12
longwaveComment #13
catchDiscussed with @longwave in slack - let's move the cspell change to its own issue since it's harder, and we might just try full runs at least on branch tests anyway. We'll get the benefit from the rest of the changes here without that, especially in terms of load on the git hosting.
Comment #14
andypostThe job fixes can't be moved to separate issue because it will fail
I did test it in https://git.drupalcode.org/issue/drupal-3252386/-/jobs/88908
Comment #15
andypostSorry the valid link is https://git.drupalcode.org/issue/drupal-3252386/-/jobs/88870
Job does not fail but doing nothing, it smells a new issue
Comment #16
andypostit needs work for
CI_MERGE_REQUEST_PROJECT_URLwhich is missing for commit/tag runsComment #17
longwaveAdded a suggestion that might work.
Comment #18
andypostLooks failure was random, on restart it passed
Comment #19
andypostComment #20
longwaveSpellcheck job passed, code looks OK, the only way to branch test is to merge it and see.
Comment #21
andypost+1 RTBC and using the variable in job, suggested the same to backport to D7 #3387052: [D7] GitLab CI integration for core
Some tests still running but what we fixing here is https://git.drupalcode.org/project/drupal/-/jobs/91306
Comment #24
catchCommitted/pushed to 11.x and cherry-picked to 10.1.x, thanks!