This is the fifth post in our series about integrating Drupal.org with a 3rd party developer tooling provider:

In this post we are announcing our migration to a new tooling provider, and outlining the phases of that migration process to take place over the next several months.

Announcing our partnership with GitLab

Wait, what?

Drupal Association + GitLab

Yes, in our four part series from December of last year it certainly looked like we were going in a different direction for the future of Drupal's developer stack.

So what changed?

Last year we laid out a model for integrating Drupal.org with a third party tooling provider, which we described as "Drupal Flow". This model was deliberately laid out to be agnostic to the provider we chose, so long as certain requirements were met. We worked with representatives from three potential providers at the time: GitHub, GitLab, and BitBucket, and each one had pros and cons. Once we had completed our evaluation, BitBucket was the only provider without hard blockers to the integration we wanted to build.

However, following our blog series, the GitLab team reached out directly to the Drupal Association team, and asked us to give them the chance to resolve the blockers and close the gaps in our integration.

At the same time, we saw an outpouring of feedback from our community asking us to see if we could find a way to make GitLab work.

And so we did.

The Agreement

For the past six months we've been working closely with Eliran Mesika, the Director of Partnerships at GitLab, in addition to CEO Sid Sijbrandij and members of GitLab's engineering team. They've escalated the internal priority of issues that blocked our adoption of GitLab, offered technical and financial support for the migration, and made a commitment to ongoing support for the Drupal project.

And so we're happy to announce that Drupal.org is going to be moving our code collaboration tools for our forty-five thousand projects to GitLab over the course of the coming months.

Three Phases to the Migration

Phase 1: Replacing Drupal.org's Git backend

The first phase of the Drupal.org migration

  • Transparently replace Drupal’s current underlying Git infrastructure (for repository hosting, maintainer permissions, code viewing) with GitLab repositories, GitLab roles and permissions for maintainers, and the GitLab code viewing UI.
  • Enable inline code editing (only for maintainers for this phase).
  • During this phase, Drupal.org will remain the primary source of information.  SSH keys, new projects, etc. will be created on Drupal.org.

This first phase, while modest, will bring some concrete benefits to the project:

  • Maintainers will be able to begin familiarizing themselves with GitLab's code collaboration tools.
  • Code viewing will receive a significant upgrade from CGIT to GitLab's built-in code viewer.
  • And Drupal.org's old Git server will be phased out.

Phase 2: Enabling Merge Requests, Inline Code Editing, and Web-based Code Review

  • The timeline for Phase 2 is dependent on GitLab’s resolution of a diskspace deduplication issue, which they have committed to on our behalf: https://gitlab.com/gitlab-org/gitlab-ce/issues/23029
  • Enable GitLab Merge Requests, GitLab inline code editing in the web UI, and GitLab web-based code review.
  • During this phase, Drupal.org will handle any 'create branch/merge request' integrations from the Drupal.org Issue queues, and related call-backs from GitLab into the Drupal.org issue comment stream.

Phase 2 is where we realize some tremendous benefits to developer velocity and collaboration:

  • By adding merge requests, contributing to Drupal will become much more familiar to the broad audience of open source contributors who learned their skills in the post-patch era.
  • By adding inline editing and web-based code review, it will be much easier to make quick contributions. This not only lowers the barrier to contribution for people new to our community, it also saves significant effort for our existing community members, as they'll no longer need to clone work locally and generate patches.
  • Finally, by creating a tight integration between the Drupal.org issue queues and GitLab's development tools, we'll be able to transition to this new toolset without disrupting the community's existing way of collaborating.

Phase 3: Evaluating Additional Features

Phase 3 has no strict timeline, but will be dependent on feedback from the community as they get up to speed on using the new GitLab-based contribution workflow for Drupal.

  • Evaluate additional features such as:

    • Integrating or replacing DrupalCI with GitLab CI
    • Enabling GitLab issues for a sub-set of projects
    • Enabling GitLab confidential issues for specific use-cases (security releases)
    • Possible MatterMost integration, etc.

These additional features may allow us to further improve the velocity of the Drupal project, or realize additional cost savings for the association. For example, we may be able to use GitLab's test runner integration to orchestrate tests across a wider variety of cloud platforms, helping us find the best pricing. We may be able to entirely replace security.drupal.org with a private issue tracker, eliminating an entire sub-site for the Drupal.org team to maintain. We may even be able to enhance existing community services like SimplyTest.me by integrating features like GitLab's AutoDevops tools to automatically create review environments for issues or branches.

We won't really know what's possible within the scope of our resources until the first two phases are completed, but this helps to show that by hitching our toolset to a partner that specializes in collaboration, we may be able to realize even more benefits for our community.

Changes to Git Remotes

  • Git remote urls for pushes to full projects have changed:
    • If you have an established Git remote in the format
      <username>@git.drupal.org:project/<yourproject>.git
      the format should be changed to:
      git@git.drupal.org:project/<yourproject>.git

  • HTTPS clone urls for full projects are unchanged.
  • HTTPS clone urls and Git remote urls for sandbox projects have changed:
    • For remotes of the format:
      <username>@git.drupal.org:sandbox/<username>/<node-id>.git
      the format should be changed to:
      git@git.drupal.org:sandbox/<username>-<nodeid>.git
    • Clone urls will be changing from:
      https://git.drupal.org/sandbox/<username>/<nodeid>.git
      to the format:
      https://git.drupal.org/sandbox/<username>-<nodeid>.git

Important: If you have any automated systems which authenticate to Git, such as CI pipelines or repo mirroring, ensure they are updated as well.

For more detailed information about these changes, as well as instructions for changing your Git remotes or setting ssh keys, please consult these instructions: https://drupal.org/gitauth

How to follow our progress

Issues for the Drupal.org migration to GitLab will be opened in the Drupal.org Infrastructure queue and tagged 'GitLab'.

For questions or concerns, please create an issue at https://www.drupal.org/node/add/project-issue/infrastructure

 

Learn more on the GitLab Blog

You can learn more about the decisions that brought us to this point on the GitLab blog.

Comments

joshmiller’s picture

YAY :D

We love our own gitlab installation at Acro Media. I think this will be a huge win for the community and the Drupal ecosystem. SUPER EXCITED. GREAT WORK. GO DA!

merauluka’s picture

I'm really excited about this change. Thank you to the DA and and the folks at GitLab!

santhoshj’s picture

Thats a great news.

ressa’s picture

I just migrated my repos from GitHub to GitLab, and it went flawlessly. Little things, like GitLab calling it a merge request in stead of pull request also makes more sense to me.

jonathanshaw’s picture

Woo hoo! Excited to hear this 

MustangGB’s picture

Glad to hear, GitLab team clearly have a great philosophy, have been adding some great features, and I've been enjoying using their platform for my own projects for a while now.

willhallonline’s picture

I know I was one of the commenters when drupal.org started discussing moving Git providers and I instantly saw GitLab as both the ideal fit, and ideal partner. This represents not only a big coup for GitLab, as so many developers and projects give them a much larger user base, but will enable Drupal.org to get some great tooling and partnerships. I actually could be more pleased with the decision. Wonderful news.

Gogowitsch’s picture

Great news indeed. 

freescholar’s picture

This is a great choice for the Drupal community and beyond.

Give a Drupal

matthieuscarset’s picture

This is great news... I can not imagine how many new contributors we will finally get onboard when we'll be able to open PRs to fix issues!

This is super exciting! :+1000:

asiby’s picture

Keep up the good work.

Live long ... and prosper!

cosmicdreams’s picture

I'm looking forward to seeing sometime in the future some guidance on how to get a Drupal project working with Gitlab's CI process, which is very dependent on Docker.  Seeing some good documentation on best practices and setup would save me a ton of time.

Bcwald’s picture

GitLab CI in the community edition does not have a dependency on Docker. You can utilize job runners using your favorite approach -  VMs, Bare metal, Docker, K8s. 

drumm’s picture

DrupalCI has been using Docker for quite awhile and will continue being used on Drupal.org issues in the first 2 phases of the migration. When we look into integrating GitLab CI in phase 3, it may continue using DrupalCI for getting tests running, using GitLab to replace DrupalCI’s Jenkins instance which queues test runs.

tsega’s picture

GitLab folks are very quick to adopt, change and accommodate. It is a HUGE win for them! The Drupal projects are one of a kind with the active community and sheer number of projects, it would certainly elevate GitLab's traffic permanently.

I look forward to seeing how everything fits with all the bells and whistles that GitLab provides.

rajeevgole’s picture

Great news, thanks for sharing. It will help Drupal to accelerate developer velocity and contributors to the project.

suzanne.aldrich’s picture

Excited to see this effort moving forward. Congratulations GitLab!

jurgenhaas’s picture

This is so exciting, not only because I'm glad that GitLab is going to be our new platform for collaboration. What's even more exciting is the fact that the DA followed through a process where listeing to the community has been a major component and the feedback really had significant impact on the decision making process. Thanks to the team and the DA - well done!

NicholasS’s picture

To me this is one of the most important things to the project in a long time. I can't tell you the amount of times old patch files uploaded to a comment thread has detured me. FINALLY

Pene’s picture

Sounds cool! It looks like this integration will solve this issue lil' sooner: https://gitlab.com/gitlab-org/gitlab-ce/issues/46108

Even better :)

voipfc’s picture

I tend to get suspicious when there is so much fulsome support and not a single doubt about the move. Just saying. I have to say though that it is an area of development I am not familiar with.

peterhebert’s picture

I think this is very exciting for Drupal - I have been extremely frustrated with Drupal.org's workflow - it has been a barrier to my further involvement with contrib/core development. Hopefully there will also be some improvements in process, especially around getting modules into contrib.

izus’s picture

Many Thanks to the DA and GitLab for this successful collaboration.

Twitter: @ismaeil_

mlncn’s picture

And Drutopia joins the projects welcoming Drupal to GitLab— we host all our projects and project management on GitLab.com.  Still working on the ideal multi-repository project setup for easy development of a Drupal distribution, though.  Maybe Drupal moving to GitLab will ultimately help make that possible!

benjamin, Agaric

Renrhaf’s picture

We also use Gitlab in my company and it's great to hear that Drupal will switch to it ! With all the integrated tools, it makes the life of a developer easy ! Maybe in the end, GitlabCI would be used for CIT/CD, and even review apps could be created automatically using Kubernetes ;)

kmajzlik’s picture

Question: is this about GitLab (self-hosted SW) or gitlab.com (SaaS) ?

drumm’s picture

Self-hosted. The Git remote URLs will stay at git.drupal.org, although with changes in some cases, as noted in the post.

gbyte’s picture

This is great news and the two projects are an awesome fit. Very smart but also admirable for GitLab to have stepped up.

ath.galanis’s picture

Excellent!

thomas.frobieter’s picture

This is just amazing. We moved our projects to gitlab a while ago. Good decision.

Anybody’s picture

Thank you very much for that right decision. We're so happy with it. I think it's a big step for Drupal!

http://www.DROWL.de || Professionelle Drupal Lösungen aus Ostwestfalen-Lippe (OWL)
http://www.webks.de || webks: websolutions kept simple - Webbasierte Lösungen die einfach überzeugen!
http://www.drupal-theming.com || Individuelle Responsive Themes

Elijah Lynn’s picture

H#$Y F#$K!!! This is AMAZING NEWS! Very well done team, really excited about this!

-----------------------------------------------
The Future is Open!

portulaca’s picture

Such exciting news! I love GitLab, can't wait! Huge thank you to all who made this happen!

dsnopek’s picture

:-)

doostinharrell’s picture

I happy you are going with the self hosted solution.  I've been using the free version for a while now and overall I LOVE it!  However, it seems like it is a live dev environment so it ends up going offline pretty frequently as updates are pushed.

Excited to see this roll out!

Cheers!
Dustin Harrell
Avid New Media
https://avidnewmedia.io

artis’s picture

We've loved working in GitLab at Texas Creative.  This is an exciting prospect for the future of Drupal!

Cameron Tod’s picture

This will go a long way to removing friction in contribution and accelerating Drupal's velocity. Great decision and well done!

jaiiali’s picture

This is great news

DYdave’s picture

Great news ! Very promising change for our development practices and the contributed modules ecosystem. That ought to occupy many of our discussions in Frankfurt in 2 weeks #DrupalEurope2018.

Can't wait ! :-D

larsdesigns’s picture

We have been using Gitlab CE for the past year for our business and it has been great for our team and our clients.

I was hoping Drupal would go this direction.

klonos’s picture

I hate to be one of "those" people, but I need to ask: do we have any rough ETA? ...or at least any update of where we're at?

Sorry, but having gotten used to GitHub and PRs has made me too lazy to be bothered with applying patches :P

hestenet’s picture

We recently posted a progress report in this issue: https://www.drupal.org/i/3002827#comment-12876000

Making great headway!

--
Tim L
Drupal Association - CTO

Francewhoa’s picture

Thanks Engineering Team and all contributors for choosing GitLab :) Wise choice.

For those not familiar with GitLab, the code of the Community Edition (CE) version is available in open source format. Joy :) This means it is more secure than other non-open source versioning software. Because like Drupal, the code of GitLab CE is available for public reviews and contributions. With an attractive MIT license at https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

Loving back your Drupal community result in multiple benefits for you  
xSDx’s picture

Discard comment