This is the fifth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:
- Where we stand now, and our evaluation criteria.
- The options we considered.
- An illustration of what Drupal.org integration with a 3rd party tooling provider could look like.
- A possible implementation plan for this initiative.
- You are here: Announcing our migration
In this post we are announcing our migration to a new tooling provider, and outlining the phases of that migration process to take place over the next several months.
Announcing our partnership with GitLab
Wait, what?
Yes, in our four part series from December of last year it certainly looked like we were going in a different direction for the future of Drupal's developer stack.
So what changed?
Last year we laid out a model for integrating Drupal.org with a third party tooling provider, which we described as "Drupal Flow". This model was deliberately laid out to be agnostic to the provider we chose, so long as certain requirements were met. We worked with representatives from three potential providers at the time: GitHub, GitLab, and BitBucket, and each one had pros and cons. Once we had completed our evaluation, BitBucket was the only provider without hard blockers to the integration we wanted to build.
However, following our blog series, the GitLab team reached out directly to the Drupal Association team, and asked us to give them the chance to resolve the blockers and close the gaps in our integration.
At the same time, we saw an outpouring of feedback from our community asking us to see if we could find a way to make GitLab work.
And so we did.
The Agreement
For the past six months we've been working closely with Eliran Mesika, the Director of Partnerships at GitLab, in addition to CEO Sid Sijbrandij and members of GitLab's engineering team. They've escalated the internal priority of issues that blocked our adoption of GitLab, offered technical and financial support for the migration, and made a commitment to ongoing support for the Drupal project.
And so we're happy to announce that Drupal.org is going to be moving our code collaboration tools for our forty-five thousand projects to GitLab over the course of the coming months.
Three Phases to the Migration
Phase 1: Replacing Drupal.org's Git backend
The first phase of the Drupal.org migration
- Transparently replace Drupal’s current underlying Git infrastructure (for repository hosting, maintainer permissions, code viewing) with GitLab repositories, GitLab roles and permissions for maintainers, and the GitLab code viewing UI.
- Enable inline code editing (only for maintainers for this phase).
- During this phase, Drupal.org will remain the primary source of information. SSH keys, new projects, etc. will be created on Drupal.org.
This first phase, while modest, will bring some concrete benefits to the project:
- Maintainers will be able to begin familiarizing themselves with GitLab's code collaboration tools.
- Code viewing will receive a significant upgrade from CGIT to GitLab's built-in code viewer.
- And Drupal.org's old Git server will be phased out.
Phase 2: Enabling Merge Requests, Inline Code Editing, and Web-based Code Review
- The timeline for Phase 2 is dependent on GitLab’s resolution of a diskspace deduplication issue, which they have committed to on our behalf: https://gitlab.com/gitlab-org/gitlab-ce/issues/23029
- Enable GitLab Merge Requests, GitLab inline code editing in the web UI, and GitLab web-based code review.
- During this phase, Drupal.org will handle any 'create branch/merge request' integrations from the Drupal.org Issue queues, and related call-backs from GitLab into the Drupal.org issue comment stream.
Phase 2 is where we realize some tremendous benefits to developer velocity and collaboration:
- By adding merge requests, contributing to Drupal will become much more familiar to the broad audience of open source contributors who learned their skills in the post-patch era.
- By adding inline editing and web-based code review, it will be much easier to make quick contributions. This not only lowers the barrier to contribution for people new to our community, it also saves significant effort for our existing community members, as they'll no longer need to clone work locally and generate patches.
- Finally, by creating a tight integration between the Drupal.org issue queues and GitLab's development tools, we'll be able to transition to this new toolset without disrupting the community's existing way of collaborating.
Phase 3: Evaluating Additional Features
Phase 3 has no strict timeline, but will be dependent on feedback from the community as they get up to speed on using the new GitLab-based contribution workflow for Drupal.
-
Evaluate additional features such as:
- Integrating or replacing DrupalCI with GitLab CI
- Enabling GitLab issues for a sub-set of projects
- Enabling GitLab confidential issues for specific use-cases (security releases)
- Possible MatterMost integration, etc.
These additional features may allow us to further improve the velocity of the Drupal project, or realize additional cost savings for the association. For example, we may be able to use GitLab's test runner integration to orchestrate tests across a wider variety of cloud platforms, helping us find the best pricing. We may be able to entirely replace security.drupal.org with a private issue tracker, eliminating an entire sub-site for the Drupal.org team to maintain. We may even be able to enhance existing community services like SimplyTest.me by integrating features like GitLab's AutoDevops tools to automatically create review environments for issues or branches.
We won't really know what's possible within the scope of our resources until the first two phases are completed, but this helps to show that by hitching our toolset to a partner that specializes in collaboration, we may be able to realize even more benefits for our community.
Changes to Git Remotes
- Password authentication for Git is deprecated. We recommend that all users switch to ssh key authentication.
- Git remote urls for pushes to full projects have changed:
-
If you have an established Git remote in the format
<username>@git.drupal.org:project/<yourproject>.git
the format should be changed to:
git@git.drupal.org:project/<yourproject>.git
-
- HTTPS clone urls for full projects are unchanged.
- HTTPS clone urls and Git remote urls for sandbox projects have changed:
- For remotes of the format:
<username>@git.drupal.org:sandbox/<username>/<node-id>.git
the format should be changed to:
git@git.drupal.org:sandbox/<username>-<nodeid>.git
- Clone urls will be changing from:
https://git.drupal.org/sandbox/<username>/<nodeid>.git
to the format:
https://git.drupal.org/sandbox/<username>-<nodeid>.git
- For remotes of the format:
Important: If you have any automated systems which authenticate to Git, such as CI pipelines or repo mirroring, ensure they are updated as well.
For more detailed information about these changes, as well as instructions for changing your Git remotes or setting ssh keys, please consult these instructions: https://drupal.org/gitauth
How to follow our progress
Issues for the Drupal.org migration to GitLab will be opened in the Drupal.org Infrastructure queue and tagged 'GitLab'.
For questions or concerns, please create an issue at https://www.drupal.org/node/add/project-issue/infrastructure
Learn more on the GitLab Blog
You can learn more about the decisions that brought us to this point on the GitLab blog.
Comments
YAY :D
YAY :D
We love our own gitlab installation at Acro Media. I think this will be a huge win for the community and the Drupal ecosystem. SUPER EXCITED. GREAT WORK. GO DA!
I'm really excited about this
I'm really excited about this change. Thank you to the DA and and the folks at GitLab!
Wow!!
Thats a great news.
Good choice
I just migrated my repos from GitHub to GitLab, and it went flawlessly. Little things, like GitLab calling it a merge request in stead of pull request also makes more sense to me.
Woo hoo! Excited to hear this
Woo hoo! Excited to hear this
+1
Glad to hear, GitLab team clearly have a great philosophy, have been adding some great features, and I've been enjoying using their platform for my own projects for a while now.
A big coup for GitLab. A leap forward for Drupal.org
I know I was one of the commenters when drupal.org started discussing moving Git providers and I instantly saw GitLab as both the ideal fit, and ideal partner. This represents not only a big coup for GitLab, as so many developers and projects give them a much larger user base, but will enable Drupal.org to get some great tooling and partnerships. I actually could be more pleased with the decision. Wonderful news.
Great news indeed.
Great news indeed.
Just what is needed!
This is a great choice for the Drupal community and beyond.
Give a Drupal
What a great move!
This is great news... I can not imagine how many new contributors we will finally get onboard when we'll be able to open PRs to fix issues!
This is super exciting! :+1000:
Keep up the good work.
Keep up the good work.
Live long ... and prosper!
Gitlab CI
I'm looking forward to seeing sometime in the future some guidance on how to get a Drupal project working with Gitlab's CI process, which is very dependent on Docker. Seeing some good documentation on best practices and setup would save me a ton of time.
GitLab CI does not have a
GitLab CI in the community edition does not have a dependency on Docker. You can utilize job runners using your favorite approach - VMs, Bare metal, Docker, K8s.
DrupalCI has been using
DrupalCI has been using Docker for quite awhile and will continue being used on Drupal.org issues in the first 2 phases of the migration. When we look into integrating GitLab CI in phase 3, it may continue using DrupalCI for getting tests running, using GitLab to replace DrupalCI’s Jenkins instance which queues test runs.
Big WIN!
GitLab folks are very quick to adopt, change and accommodate. It is a HUGE win for them! The Drupal projects are one of a kind with the active community and sheer number of projects, it would certainly elevate GitLab's traffic permanently.
I look forward to seeing how everything fits with all the bells and whistles that GitLab provides.
Great News!
Great news, thanks for sharing. It will help Drupal to accelerate developer velocity and contributors to the project.
At long last
Excited to see this effort moving forward. Congratulations GitLab!
Great news
This is so exciting, not only because I'm glad that GitLab is going to be our new platform for collaboration. What's even more exciting is the fact that the DA followed through a process where listeing to the community has been a major component and the feedback really had significant impact on the decision making process. Thanks to the team and the DA - well done!
Awesome
To me this is one of the most important things to the project in a long time. I can't tell you the amount of times old patch files uploaded to a comment thread has detured me. FINALLY
Sounds cool! It looks like
Sounds cool! It looks like this integration will solve this issue lil' sooner: https://gitlab.com/gitlab-org/gitlab-ce/issues/46108
Even better :)
I tend to get suspicious when
I tend to get suspicious when there is so much fulsome support and not a single doubt about the move. Just saying. I have to say though that it is an area of development I am not familiar with.
Exciting news!
I think this is very exciting for Drupal - I have been extremely frustrated with Drupal.org's workflow - it has been a barrier to my further involvement with contrib/core development. Hopefully there will also be some improvements in process, especially around getting modules into contrib.
Great decision
Many Thanks to the DA and GitLab for this successful collaboration.
Twitter: @ismaeil_
This is the right way to go
And Drutopia joins the projects welcoming Drupal to GitLab— we host all our projects and project management on GitLab.com. Still working on the ideal multi-repository project setup for easy development of a Drupal distribution, though. Maybe Drupal moving to GitLab will ultimately help make that possible!
benjamin, Agaric
Excellent news !
We also use Gitlab in my company and it's great to hear that Drupal will switch to it ! With all the integrated tools, it makes the life of a developer easy ! Maybe in the end, GitlabCI would be used for CIT/CD, and even review apps could be created automatically using Kubernetes ;)
SW or SaaS
Question: is this about GitLab (self-hosted SW) or gitlab.com (SaaS) ?
Self-hosted. The Git remote
Self-hosted. The Git remote URLs will stay at git.drupal.org, although with changes in some cases, as noted in the post.
This is great news and the
This is great news and the two projects are an awesome fit. Very smart but also admirable for GitLab to have stepped up.
--
gbyte.dev
Excellent!
Excellent!
Awwwww :)
This is just amazing. We moved our projects to gitlab a while ago. Good decision.
This is GREAT news!
Thank you very much for that right decision. We're so happy with it. I think it's a big step for Drupal!
http://www.DROWL.de || Professionelle Drupal Lösungen aus Ostwestfalen-Lippe (OWL)
http://www.webks.de || webks: websolutions kept simple - Webbasierte Lösungen die einfach überzeugen!
http://www.drupal-theming.com || Individuelle Responsive Themes
H#$Y F#$K!!! This is AMAZING
H#$Y F#$K!!! This is AMAZING NEWS! Very well done team, really excited about this!
-----------------------------------------------
The Future is Open!
Such exciting news! I love
Such exciting news! I love GitLab, can't wait! Huge thank you to all who made this happen!
This is the best news ever!
:-)
Yes! Gitlab
I happy you are going with the self hosted solution. I've been using the free version for a while now and overall I LOVE it! However, it seems like it is a live dev environment so it ends up going offline pretty frequently as updates are pushed.
Excited to see this roll out!
Cheers!
Dustin Harrell
Avid New Media
https://avidnewmedia.io
Yay GitLab!
We've loved working in GitLab at Texas Creative. This is an exciting prospect for the future of Drupal!
Really great news
This will go a long way to removing friction in contribution and accelerating Drupal's velocity. Great decision and well done!
This is great news
This is great news
Great news ! Very promising
Great news ! Very promising change for our development practices and the contributed modules ecosystem. That ought to occupy many of our discussions in Frankfurt in 2 weeks #DrupalEurope2018.
Can't wait ! :-D
Great Choice
We have been using Gitlab CE for the past year for our business and it has been great for our team and our clients.
I was hoping Drupal would go this direction.
Any rough ETA?
I hate to be one of "those" people, but I need to ask: do we have any rough ETA? ...or at least any update of where we're at?
Sorry, but having gotten used to GitHub and PRs has made me too lazy to be bothered with applying patches :P
Progress report
We recently posted a progress report in this issue: https://www.drupal.org/i/3002827#comment-12876000
Making great headway!
--
Tim L
Drupal Association - CTO
Thanks :) GitLab CE is Open Source.
Thanks Engineering Team and all contributors for choosing GitLab :) Wise choice.
For those not familiar with GitLab, the code of the Community Edition (CE) version is available in open source format. Joy :) This means it is more secure than other non-open source versioning software. Because like Drupal, the code of GitLab CE is available for public reviews and contributions. With an attractive MIT license at https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE
Sandbox url is not correct
Discard comment