Once #2488266: [META] Improve Git workflow on Drupal.org by implementing issue workspaces allows per-issue forks of Drupal.org projects, and branches on those forks, we should be able to open GitLab merge requests on them.

Original issue summary

I'm sure that this has already been requested somewhere, but wasn't able to find it.

Developers around the world are flocking to Github. Let's learn from this and emulate patterns that developers are starting to expect for managing their projects.

The most popular element that folks seem to be requesting is pull requests. We should be able to do this so that we can take some of the work off of the shoulders of maintainers.

As @wwalc said here #2183843-3: Roadmap for a New Stable Release

Github + pull requests ;-) I love the situation when I simply get an email when there is a pull request and it takes me seconds to approve it if the change makes sense. The less time some operation takes, the more things you will do in the same amount of time.

CommentFileSizeAuthor
#30 Untitled.png38.21 KBdrumm
#29 Screenshot from 2020-11-11.png71.42 KBvoleger

Issue fork drupalorg-2205815

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

    Support from Acquia helps fund testing for Drupal Acquia logo

    Comments

    drumm’s picture

    Title: Emulate Github & Start Using Pull Requests » Implement pull requests
    Status: Active » Postponed (maintainer needs more info)

    I wasn't able to find an issue this would be duplicating, but I wouldn't be

    @mgifford - Again, we should not just throw features at Drupal.org because they seem to work on other Drupal sites.

    What specifically would the change look like on an issue page, repositories, and other areas of Drupal.org affected?

    mgifford’s picture

    @drumm As usual, good questions and cautions.

    It's something that a number of people like @wwalc have called for when I have been asking what would make maintaining a module or theme easier.

    I will dig through the links and find a few other references and see if I can get folks to detail specifically what their needs are.

    I don't use Github this way but enough other folks have brought this up in passing that I think it is worth exploring what it could look like.

    You've got enough other high priority items I've got no problem with this being marked Postponed. I mostly wanted somewhere to start exploring this.

    EDIT: Adding prior discussions around pull requests:

    mgifford’s picture

    Some other Github related issues:

    Definitely bigger than just pull requests, but we should keep this focused.

    mgifford’s picture

    From a Prairie Initiative discussion https://groups.drupal.org/node/138859

    kenahoo’s picture

    +1 from me for pull requests. Without pull requests much of the power of git is lost, particularly in the area of collaboration and making maintenance easy. Since many Drupal projects seem to go dormant or become orphaned, easing maintenance seems like a really big win.

    Obvious question: why "compete" with Github though? Why not take advantage of their excellent platform for code hosting?

    mgifford’s picture

    Lots of folks have looked at the Github option. It's slick, but there's lots we can't control.

    • Their interface isn't very accessible.
    • It doesn't have Drupal's branding.
    • Forces everyone to sign onto Github's Terms of Service (which isn't great)
    • It also takes away the control over how we manage the community.

    But not sure those were the reasons that it hasn't gone further.

    joshuami’s picture

    Issue tags: +d.o issue workflow
    donquixote’s picture

    Title: Implement pull requests » [meta] Pull requests in d.o. issue queues?
    Status: Postponed (maintainer needs more info) » Active

    I really really want this.

    Yes, moving to Github seems like an easy solution, but it also would mean that we lose all the communication history of the issue queues.
    Much better would it be to have pull requests as part of the issue work flow.

    This is probably a lot of work. And the first thing here would be to find out which tools we can use to make this easier. I have no idea myself atm.

    I think the "maintainer needs more info" should be answered by the community. There *will* be discussion, and there needs to be a place for it. So I am going to reopen this as a "meta" issue.

    YesCT’s picture

    The staff at the DA has an excellent proposal that I think will be the good things about git workflow, and keeping our collaborativeness kind of workflow. I'll keeping asking that they publish their plan.

    joshuami’s picture

    @YesCT, more details are coming soon. We are focused on some updates to account creation and profiles first, but the ideas for improving the issue workflow, Git integrations specifically, are exciting.

    YesCT’s picture

    @joshuami oh! thanks so much for that https://www.drupal.org/roadmap/issue-workflow link! :)

    donquixote’s picture

    Yes, this is all very nice to hear :) Looking forward to it.

    Mixologic’s picture

    #2488266: [META] Improve Git workflow on Drupal.org by implementing issue workspaces Is our shiny new issue about issue workspaces. We've got a mound of stories that I'll be making into some issues soon. There has even been some improvements to what is proposed there as a result of many discussions at drupalcon LA.

    Speaking of, I gave a presentation on this at DC LA: https://events.drupal.org/losangeles2015/sessions/issue-workspaces-bette...

    matthieuscarset’s picture

    Hi community member!
    How are we doing with this issue?

    Me - and many other developers I've been talking to lately - really think that working with Patch and Interdiff is a huge blocker to our development.

    Having the option to open Pull Requests will certainly ease a lot contributions.

    How can we help to move this forward?

    voleger’s picture

    amjad1233’s picture

    Hi,

    Is there any progress in this one ?

    As we have moved GitLab but there is no option for Merge Requests ?

    Regards,
    Amjad

    drumm’s picture

    Title: [meta] Pull requests in d.o. issue queues? » Merge requests in Drupal.org issue queues?
    Project: Drupal.org infrastructure » Drupal.org customizations
    Version: » 7.x-3.x-dev
    Component: Git » Code
    Category: Feature request » Plan
    Issue summary: View changes
    Status: Active » Postponed

    Once #2488266: [META] Improve Git workflow on Drupal.org by implementing issue workspaces allows per-issue forks of Drupal.org projects, and branches on those forks, we should be able to open GitLab merge requests on them.

    drumm’s picture

    Issue summary: View changes
    drumm’s picture

    Status: Postponed » Active

    #2488266: [META] Improve Git workflow on Drupal.org by implementing issue workspaces is now in a good place. Everyone can create issue forks on some projects, #3152637: Opt-in to the Drupal.org Issue Forks and Merge Requests beta. And we’ve started looking into the requirements for enabling merge requests.

    drumm’s picture

    Assigned: Unassigned » drumm
    Issue tags: -

    • drumm committed 1900004 on 7.x-3.x
      Issue #2205815: Enable merge requests for new projects (does not include...

    • drumm committed 48708a3 on 7.x-3.x
      Issue #2205815: Add a method to get a merge request IID by branch name
      
    • drumm committed 49a9120 on 7.x-3.x
      Issue #2205815: Move merge request URL generation into a resuable method
      
    • drumm committed a495610 on 7.x-3.x
      Issue #2205815: Simpler and more correct link to the merge request’s...
    • drumm committed d5945b3 on 7.x-3.x
      Issue #2205815: Add a method to check if a merge request can be merged
      
    • drumm committed f54db32 on 7.x-3.x
      Issue #2205815: Specify visibility on hasPushAccessHtml() method
      

    • drumm committed 6d1f0cb on 7.x-3.x
      Issue #2205815: Replace compare link with link to merge request diff,...

    • drumm committed 2deef68 on 7.x-3.x
      Issue #2205815: Hide branches when their merge request closes, show when...
    • drumm committed be32cc0 on 7.x-3.x
      Issue #2205815: Less emphasis on creating a new branch
      

    • drumm committed b83b1e2 on 7.x-3.x
      Issue #2205815: Insert the note at the end of the series of notes if it...

    • drumm committed 229a0bb on 7.x-3.x
      Issue #2205815: Only process merge request events initiated from an...

    • drumm committed a90065b on 7.x-3.x
      Issue #2205815: Visually distinguish compare & plain diff links
      

    • drumm committed 6182f07 on 7.x-3.x
      Issue #2205815: When a merge request is open, name the compare/changes...
    voleger’s picture

    Merge requests now enabled for all projects.

    Anyway, can we add some change to the merge request message on the issue page? See the screenshot:

    drumm’s picture

    FileSize
    38.21 KB

    We have a link to the plain diff in the issue fork summary:

    Screen shot of issue fork summary highlighting “plain diff” link

    I used the label “plain diff” to match GitLab’s UI. This is in the summary, so it isn't confused with the initial diff when the merge request was opened.

    colan’s picture

    I couldn't find a child issue for this so I thought I'd ask about it here.

    How do maintainers enable no-fast-forward (--no-ff or merge) commits? When I went to merge an MR recently, it said it was going to be a fast-forward commit, and there didn't appear to be a way to change it, which is actually the default Gitlab behaviour.

    Merge commits are good because the provide contextual metadata:

    1. they package the associated commits together, and allow folks to reference the issue and MR from looking at the tree (if, say, committers forgot to mention the ticket number in the message), and
    2. the entire MR can be reverted at once without having to deal with finding the individual commits and reverting those (which is very time consuming).

    If it can't be the default behaviour, can there at least be an option to change it?

    Okay to open a sub-issue about this?

    drumm’s picture

    Yes, please do open a child issue for this. To minimize disruption, the default merge strategy is set to mirror the patch workflow as much as possible. There isn’t going to be one single best merge strategy for every project, so we should provide the options. #3072047: Handle merging of merge requests has some details on the current implementation.

    colan’s picture

    Wim Leers’s picture

    I cannot find pre-existing issues about either improving performance or documenting expected performance for GitLab comments being loaded on d.o.

    I keep missing GitLab activity because it routinely takes >10 seconds to load the page due to the half a dozen XHR requests, which then also causes the "jumping page" behavior. For example at #3201637: Figure out how to prevent data loss during upgrade/migration path.

    mradcliffe’s picture

    Some upstream GitLab issues regarding discussions endpoint performance. There are a lot more front-end proposals in GitLab for filtering not listed here. The last two seem to be actively worked on so hopefully performance improves steadily.

    - Meta: 26809: Improve discussion loading performance
    - Merged: 217673: Streamline discussions requests and rendering. Some of the UI in 2021 releases seem to use this now: Merge Request 52089: Updated diffs notes app to use paginated notes, Merge Request 34628: Paginate the notes incremental fetch endpoint
    - 233926: Handle streamlined discussions requests and rendering on the frontend closed in favor of 209784: Improve performance of discussions.json action for Projects::MergeRequestsController under load into next tier
    - 332967: Introduce caching on Projects::MergeRequestsController#discussions

    Theoretically we might already get some improvement enabling :paginated_notes.

    drumm’s picture

    We did have one request to just remove the GitLab notes being inserted into the Drupal.org issue page, #3218920: Merge request comments add noise to Drupal.org issues, which would certainly speed up loading.

    Issue pages use GitLab’s GraphQL & Rest APIs. GitLab’s frontend performance improvements aren’t directly related, although I’m sure there’s some overlap. Some possible improvements:

    • Take another look to make sure we’re still doing the minimum requests. I don’t expect there to be too much that can be done here.
    • Improve loading indicators on issue pages. Currently there isn’t much showing that requests are happening in the background.
    • Evaluate CDN caching options for GitLab. Currently we only have raw files cached by our CDN, since those are always safe to cache. Some API responses may be cacheable. GraphQL is incompatible with CDN caching.
    mradcliffe’s picture

    I don't see any jumping at all on drupal.org (see below). Only from drupal.org to drupalcode.org.

    From #34,

    This drupal.org issue url, https://www.drupal.org/project/ckeditor5/issues/3201637#mr58-note32098, doesn't actually take that long to load without cache (graphql requests are cache miss). There is an additional issue of the fragment not working at all.

    This drupalcode.org GitLab Url, https://git.drupalcode.org/project/ckeditor5/-/merge_requests/58#note_30238, takes much longer to load in GitLab UI and shows "jumping" behavior (due to the upstream issues in #35).

    I assumed that the main issue in #34 is with the latter (drupal.org to drupalcode.org) not the former (drupal.org only).

    drumm’s picture

    Status: Active » Fixed
    Issue tags: -

    I think we can call this fixed at this point. Merge requests are now in regular use on Drupal.org.

    Status: Fixed » Closed (fixed)

    Automatically closed - issue fixed for 2 weeks with no activity.