As part of the move of git services to the new servers, cgit will need to be deployed there. See #1914722: Evaluate alternatives for gitweb for reference.

See the parent issue at #2099277: Move git services to new git servers

Comments

webchick’s picture

Here's an example so people can see what this looks like: http://cgit.freedesktop.org/

neclimdul’s picture

marvil07’s picture

Another example, a little more 'themed', from GNOME https://git.gnome.org/browse/

marvil07’s picture

Issue summary: View changes

added parent issue

Stevel’s picture

Added the parent issue as a proper parent issue.

Other than that, it would be great if we could get it styled in Bluecheese.

nnewton’s picture

Test instance up: cgit.drupalcode.org

With cgit it is likely we will not support viewing all sandboxes, but instead a whitelist of 'important' sandboxes. We are still considering this.

greggles’s picture

I asked about motivation for not supporting sandboxes. There are two problems. One is that sandboxes are basically untrusted content and putting untrusted content onto the web is risky. We've had a few xss issues in this realm. I think that issue is moderately solved by putting it on drupalcode.org.

The second reason is that every single repository we add to cgit slows it down even for a single client browsing the content. Add to that the fact that more repositories means more pages for spiders to crawl and the effect is probably even worse. So...we'll need to figure out something to mitigate that issue.

nnewton’s picture

Greggles pointed out that the sandbox view is used a lot for reviews. Due to this, even if cgit gets launched without sandbox support, it'll likely be added via another vhost or another solution to mitigate the performance issues of the sandbox repositories.

-N

andypost’s picture

Suppose spiders should not index this pages, maybe it makes sense to provide a robots.txt

greggles’s picture

I could see people using external search engines to find content on drupalcode.org. That seems like a pretty good idea to me, actually, so a robots.txt to disallow all doesn't seem like a great solution.

jhodgdon’s picture

I think all sandboxes need to have some sort of repository viewing, and if deploying cgit means we don't have that any more, that seems like a regression of d.o functionality.

webchick’s picture

Indeed. :\

Sandbox repo viewing is pretty critical for use cases like:

- Approving new Drupal module/theme contributors from https://drupal.org/project/issues/projectapplications. Would-be contributors are instructed to put their code into a sandbox and both maintainers and other applicants use the repo viewer extensively to provide code reviews to help the process along.

- Rallying contributors around crowd-sourced efforts like Migrate in Core: http://drupalcode.org/sandbox/chx/2105305.git The repo viewer is basically a view into what's done / left to do for a given initiative. We generally don't want Drupal core forks to be first-class projects, as that's very confusing.

killes@www.drop.org’s picture

Narayan, can you summarize what the issue is with sandboxes and why we shouldn't make all them visible anymore?

Maybe a better solution would be to retire sandboxes that haven't been updated in a while.

nnewton’s picture

This was actually already semi-resolved via IRC.

@webchick: yes, those were the two issues pointed out. The core forks one is solvable via a whitelist, but the approval of new themes/modules is not easy to work around without sandbox viewing and is obviously very important.

@killes: In its easiest to roll out form, it simply doesn't work. The issue being how cgit scans for git repos. The load time for any page goes from seconds to 15-20 seconds. This was what lead to the discussion about just removing viewing and then delaying viewing (rolling it out without sandbox support and then adding support).

As it stands at the moment, I have delayed the roll out and am working on a script to fix this.

nnewton’s picture

The sandbox issue is now fixed, as are most of the redirects. This is likely ready to go. Will coordinate testing.

-N

nnewton’s picture

Status: Active » Needs review

cgit is now active. Drupalcode is redirecting to it. Most common URI's _should_ redirect.

-

greggles’s picture

Awesome. I just tested a few urls and they redirected fine. It feels very fast and it seems great to me that it's got an overview page at drupalcode.org rather than the sparse page that was there before.

Are we purposefully not using google analytics on this site? It seems like a good idea to get at least some metrics about it.

nnewton’s picture

In testing, it seemed like it actually has a much nicer interface. In particular for patches/diffs.

I don't think we have ever looked at GA on drupalcode. I don't really feel strongly either way. Could do awstats as well/instead of GA. (might be easier since its CGI based)

-N

xjm’s picture

Unfortunately, this appears to lack a git blame feature presently. The UI for git blame is pretty much the only reason I've ever used drupalcode.org over my local git repository (other than occasionally linking someone a particular diff), and it has been an important tool for helping contributors understand the development of Drupal core and how to navigate the core community and codebase. We have https://drupal.org/documentation/git/blame explaining how to use the feature that was provided by the previous drupalcode.org application (and probably other references in other documentation).

I'm concerned about this change being deployed right before DrupalCon Austin (with its enormous sprint for new contributors), and was wondering what the implications of rolling it back and deploying after Austin instead would be.

nnewton’s picture

While its definitely possible to roll back. I am very against it. The version of gitweb we were running is actively dangerous and I really do no want to be running it during a drupalcon. This was part of the reason we have been trying to get this done before Austin.

greggles’s picture

In addition to being insecure, gitweb was broken on certain tasks and certain files. Viewing the source and blame seemed to be the 2 things that most frequently triggered errors for me.

https://dev.haiku-os.org/ticket/8510 seems to be the issue about adding annotate/blame to cgit and its stale since 2012 (but gitweb was stale since longer than that, afaik).

drumm’s picture

Yep, https://drupal.org/project/issues/infrastructure?text=server+error&statu... has a few of the issues with breaking on certain files. I'm sure there are more that weren't categorized. Gitweb also had a habit of using up all the resources on our util server frequently.

For more details on the problems with Gitweb and deciding on a replacement, see #1914722: Evaluate alternatives for gitweb, which is also linked from this issue's summary.

zerolab’s picture

Just a heads up, the switch breaks any snapshots used in make files (e.g. http://drupalcode.org/project/contextual_view_modes.git/snapshot/d5d3ced...)

Is there a way to bring those back?

On the upside, cgit does feel a LOT faster.

Thanks

greggles’s picture

What were the snapshots? Is it a tarball from a specific commit hash?

I don't think that was really a supported feature. Can't you specify in the make file getting the files via git clone of a specific hash?

zerolab’s picture

They were indeed tarballs from a specific commit hash.
I've since updated the make file(s) to clone the specific commit hash, thanks.

Though I'd bring that up in case people stumble upon it.

Cheers,
Dan

neclimdul’s picture

jamsilver’s picture

Actually it looks like snapshots are supported/switched on, it's just that the links are hidden in the UI. You just need to craft the right URL:

http://cgit.drupalcode.org/PROJECTNAME/snapshot/COMMITSHA1.tar.gz

For example, the very first commit in the master branch of the Brightcove module is downloadable here.

nnewton’s picture

I am honestly unsure what I think about snapshots, but ya they are supported in cgit.

drumm’s picture

I wouldn't recommend using the snapshot tarballs from either Cgit, or previously Gitweb, in drush make files. While they may work, drush make isn't designed to support them well.

basic’s picture

Syntax highlighting has been enabled with the use of a custom python+pygments script.

markcarver’s picture

I've noticed that http://cgit.drupalcode.org/bootstrap doesn't work for https://drupal.org/project/bootstrap.

Talked with @basic and he did a search: http://cgit.drupalcode.org/?q=bootstrap

There's two "bootstrap" repos.

nnewton’s picture

Fascinating. From a quick look I think there is something 'abnormal' with that repo. Will have to look in detail.

-N

nnewton’s picture

Yes indeed there is something weird with this repo. It looks like a REF was either renamed or failed to save. It looks like what happens in this case is undocumented, which is great. We may have to call Mr. Boyer into this to look at this repo.

EDIT: Or perhaps not...looks like this is the case for some other repos that work....may still need Boyer.

-N

mattlt’s picture

This project doesn't look to be working…

http://cgit.drupalcode.org/commerce

Search shows duplicate repositories…

http://cgit.drupalcode.org/?q=commerce

Just an FYI.

Thanks,

•• matt

andypost’s picture

I think snapshots should be reverted back because this is a only way to get tarball for sandbox project

greggles’s picture

@andypost see #26. What scenario do you need to download tarballs of sandbox?

nnewton’s picture

Since its not really disabled, just hidden. I'm going to just turn snapshots on unless anyone thinks thats a really bad idea.

-N

nnewton’s picture

@mattIt Thanks for the report, we are tracking this down. It is happening on what seems to be a very few repos and looks like it has to do with naming.

-N

greggles’s picture

re #36: Historically, we have purposefully not packaged sandboxes as tarballs to avoid a casual user who might think "tar.gz exists = stable/supported". Sandboxes are often not "production ready" code. In this case, making it "hard" to download (i.e. git only) is a feature.

nnewton’s picture

@mattIt/Mark

This issue should be resolved. Thanks for the reports.

killes@www.drop.org’s picture

I agree with greggles, I don't think we want to promote sandboxes more than neccessary.

andypost’s picture

@greggles the scenario for tarballs is:
1) from sandbox project visit "Repository viewer" link
2) copy a link to tarball
3) paste the link into 'admin/modules/install'

drumm’s picture

Not packaging sandbox projects is by design. They are intended for collaboration among developers, not for site builders to easily install. See https://drupal.org/node/1294804.

killes@www.drop.org’s picture

I think this issue can be marked as "fixed" now. Any follow-ups should get their own issue.

marvil07’s picture

FTR I have configured cgit versioncontrol webviewer url handler plugin at git7site for testing.

AFIAK there is not a cgit instance in git7site, so I pointed it to production cgit. Should be good enough to try ;-) i.e. see the links generated at https://git7site.devdrupal.org/node/148303/commits or https://git7site.devdrupal.org/commitlog/commit/3428/2b9b962066c6e9331b7...

killes@www.drop.org’s picture

Status: Needs review » Fixed

Marking as fixed. Please open new issues as follow ups.

xjm’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

YesCT’s picture

YesCT’s picture