While people are discussing the mockup and prototype for first iteration of changes to improve the issue queue workflow at https://groups.drupal.org/node/390268, this meta-issue exists to start identifying the practical steps needed to implement those changes.

Who will coordinate the effort?
Contact Mike Crittenden: mcrittenden via e-mail or IRC. We're hanging out in #drupal-contribute.
Where are we working?
Dev site: http://form-drupal.redesign.devdrupal.org/ (username: drupal/drupal, then bacon/bacon)
Staging site: http://page-drupal.redesign.devdrupal.org/ (username: drupal/drupal, then bacon/bacon)
How can I help?
Follow the instructions at https://drupal.org/node/1018084 to get your SSH key added to devwww.
Who is available to work on implementation?
sanchiz, webchick, japerry and Mark Carver, drumm, so far... add your name here!
When?
webchick: hackish and bug-ridden development the week of Dec 23, QA anytime after that
sanchiz: ?
japerry: Dec 27 & 28 (Friday and Saturday)
Mark Carver: various days throughout the week of the 23rd (not 25th, but here and there). Just ping me.
drumm: ongoing
Can some fundamental work begin while discussion is still taking place? If so, what?
All the child issues of this issue can be started now. Hop in!

Comments

webchick’s picture

Probably can't help much with implementation unless this is ready to hack on by the holidays (aka, "next week") when I'll have some downtime. However, happy to help with QA. I'm very, very good at breaking Drupal.org. ;)

tvn’s picture

I created a few implementation issues for specific tasks. They are listed under Child issues here in the sidebar. We might split off some more tasks later.

mcrittenden will do general coordination. So people interested in helping out should talk to him. Next step for them will be requesting their SSH key added to our server (Instructions: https://drupal.org/node/1018084), we then will figure out which dev site they could work on.

tvn’s picture

Issue tags: +D.o UX
drumm’s picture

Are the changes to fieldset styles part of the design, or an artifact of wireframing? I'm assuming they are part of the design, and should be applied everywhere we have fieldsets on Drupal.org.

eliza411’s picture

Assigned: Unassigned » mcrittenden
Issue summary: View changes

Updating the issue summary with answers to the questions and assigning to mcrittenden per our discussion

webchick’s picture

Issue summary: View changes
webchick’s picture

Issue summary: View changes
webchick’s picture

Issue summary: View changes

Sorry for all the edits. Trying to get the finer points of what people need to know to work on this in one spot.

webchick’s picture

Ok, tried to get this bootstrapped today, but here's where I'm stuck:

#1: I don't have access to write to anything under /var/www/dev/form-drupal.redesign.devdrupal.org, which is uh. Sort of a blocker. ;) According to japerry, my "webchick" user is not in /etc/groups.

#2: I also don't know for sure where the latest version of the in-progress code sanchiz was working on is. What I am going to assume is that page-drupal is it, but this may not be correct and if it's not, it'd be *great* to know before we start hacking in form-drupal so we don't inadvertently throw out a bunch of his work. :(

#3: I am sort of stuck on how to organize the Git repo because I don't really know Git outside of the version control tab. :D What Mike and Jakob and I discussed on IRC though was was deploying the "master" branch to form-drupal (dev) for hacking, and a "staging" branch to page-drupal (stg). We merge things over when they seem to work and tag anytime there's something worth deploying/git pulling to the page-drupal (stg) server for people to look at. Unless anyone has any huge concerns, that's what we're going to do. Hopefully this repo only lives 2-3 weeks.

In terms of next steps, Mike set up a private bitbucket repo (because Bluecheese) at https://bitbucket.org/mcrittenden/issue-queue-redesign. My plan was to initialize it with the code from from-drupal (dev), then copy in the code from page-drupal (stg) and commit that as a starting point to form-drupal (dev again) we could work from. Whew!

If you want to hack on this too, ping Mike for access (his contact form or IRC is best). It's limited in the # of people though so please only ask for access if you actually intend to do something with it. :)

webchick’s picture

Issue summary: View changes

Moar instructions.

webchick’s picture

Issue summary: View changes
webchick’s picture

Also, since I am leading neither the implementation nor coordination of this effort, I will only say this once, and then live with whatever is decided:

I would highly recommend we de-scope #2159819: Update text format display on comment and node/edit forms and #2159823: Use Libricon icons instead of default BUEditor buttons from this initiative. They're great suggestions, and it's wonderful if they happen, but they're pre-existing issues affecting the site in D6 and which we could fix at any time.

And just fair warning that I'm going to be extremely cheesed off (to put it mildly) if we get to a point where #2159813: Display parts of the issue edit form instead of the comment form on issue pages (which, by contrast, does directly fix the workflow problems with the issue queue post-D7 upgrade) is RTBC and those issues are not and we're blocked on deployment because of them. ;)

markhalliwell’s picture

Issue summary: View changes

Willing to help out when I can (probably theme related), created https://drupal.org/node/2160903 to add my key

Bojhan’s picture

@webchick I understand your point of view, I put those parts in. I am also here to defend them, its always very easy to descope parts that look like polish. But improving a design is about the big picture, and occasionally this requires fine tuning of details for it to feel coherent. It is not a "nice to have" part, it is a part that makes it a whole. The fact is d.o hasn't had a whole lot of visual design, since the redesign of mark and leisa - because it is so easy to descope.

Obviously it should in no way hold up improving the key part! Just keep in mind its probably like 30 lines of code, easily do-able for more inexperienced people, not tied to releasing the key improvement and possibly to be released independently. Since the other issue requires quite some knowledge about all the different parts, these issues might be a nice way to get involved.

I know you are scared this might in some way, hold it up. I will do my very best to make sure, that it doesn't.

eliza411’s picture

I opened a ticket for write access, which I'm linking as a related issue.

I'm not totally sure when code will get pushed to the dev site, but if it's generally going to be coordinated by mcrittenden, then as long as folks can commit to the repo and he can pull, we should be good.

webchick’s picture

Yeah, I've blown another several hours now today trying to get a copy of the code and db on localhost in order to work around the server permissions issues and get something into Git. I have a 1G .gz db dump that's currently importing, so sometime tonight when that finishes I'll try and instantiate something into the Git repo if no one beats me to it.

webchick’s picture

Incidentally, joachim's approach at #2159813-10: Display parts of the issue edit form instead of the comment form on issue pages would do this as a patch against project_issue module proper rather than a d.o customizations, which sounds both a) much more sustainable long-term and b) means we can avoid having to mess around with anything Drupal.org-specific in order to get started.

So part of the several hours mentioned above was searching in vain for a D7 version of https://drupal.org/project/drupalorg_testing for Project module (I swear we made one around the time of the D.o sprint last year but I definitely couldn't find it). Any hints?

(And yeah, I know d.o server environments are preferred, but obviously the permissions issue is more pervasive than just me, so we need a workaround. Also, if we do this against project_issue itself (which I think we should) it's likely preferable to make sure it works in a non-d.o environment first.)

eliza411’s picture

For now, here's what I'm suggesting:

* Fork project_issue (just for development) into a d.o. sandbox
* Do the work there. Setting up locally isn't really feasible for most people from what I'm told, so that means a bunch of blind work getting pushed, then pulled to dev

Ultimately, the code can be moved to drupalorg_customizations if needed.

eliza411’s picture

Looks like sanchiz and mcrittenden are working something out in IRC continuing down the private bitbucket route anyway. I'll just stay out of the way.

webchick’s picture

Yeah, unfortunately the changes are going to be outside of just project_issue (for example, the new fieldset styling is Bluecheese, and there are some other aspects of the design that are D.o-specific like "issue summaries" and whatnot that will indeed go into drupalorg module). However, hopefully the bulk of #2159813: Display parts of the issue edit form instead of the comment form on issue pages can happen upstream.

This is where a tech lead for the project portion of developer tools would come in super handy, so they could weigh in on questions like this! ;)

If sanchiz and mcrittenden are able to instantiate the repo, that'd be great! The only reason I'm messing with all of this in the first place is to get people a concrete place to get rolling. But it's hard to do during 25-40 minute baby naptime chunks while high on DayQuil. :)

webchick’s picture

Side note: opened #2161199: The partial DB on d.o dev environments could probably stand to be a lot more partial. ;) for my findings in analyzing the devwww db.

I couldn't get ahold of anyone in IRC at this late hour about the current state of the bitbucket repo, but it is still empty for me. If I can get a working site going on localhost, I'm going to give populating that repo a shot. If you see a loud cloud of impenetrable smoke above the Pacific Northwest, you know it was me trying to figure out Git. ;)

webchick’s picture

Ok, here's what I did. I will warn you that it has been several years now since I worked on a Drupal site "for real" so these steps are probably both hilarious and wrong. :) But I have a few hours to spend on something Drupal programming-related tomorrow, providing my health holds up, so I wanted to get *something* working here.

1) I grabbed a tarball of form-drupal (dev) development environment files into my home directory, since I can write there.
2) I SFTPed them down to my localhost like it was 1996. \m/
3) I un-tarred form-drupal (dev) and added/committed that to Git, minus:
- The .bzr directory, because that could probably break things.
- sites/default/settings.local.php, since that has DB connection info, etc. in it.
(There's probably other things I should never have added to the Git repo but oh well I guess we'll learn together. ;))
4) Applied/committed sanchiz's patch at #2159813-11: Display parts of the issue edit form instead of the comment form on issue pages as a starting point. Note that joachim has inadvertently duplicated efforts in #10, since this code wasn't available to start from originally. :\
5) Then I created a branch "staging" for us to use as the reference implementation. (Originally I tried to take the page-drupal dev environment files and copy/paste them over the form-drupal ones, but sadly that dev environment is from Dec. 8 and so is missing many subsequent deployments, making the changes relevant to the issue form hard to suss out.)
6) Finally, went to go push all of that, and....

remote: abort: the account owner has exceeded their user limit - write access is disabled to the repository.
fatal: unable to access 'https://webchickenator@bitbucket.org/mcrittenden/issue-queue-redesign.git/': The requested URL returned error: 402

Wah. Wah. Wahhhhhhh. :)

However, the good news is I have a fully-working local environment now that I can hack on, despite the fact that it's done in a truly terrible way, and despite no one else can actually see what I'm doing since I can neither push it to the Bitbucket repo nor can I edit the files on the dev site. ;) I'll post patches to #2159813: Display parts of the issue edit form instead of the comment form on issue pages tomorrow if I manage to make any progress. First order of business is probably to try and reconcile joachim and sanchiz's work into a single patch.

webchick’s picture

Btw, should someone else who's not blocked by access issues get to this before those things are resolved for me, here were going to be my next steps:

1) git push ;)
2) Carefully copy the .bzr directory and sites/default/settings.local.php out of /var/www/dev/form-drupal.redesign.devdrupal.org, then blow away the contents, then replace the contents with a git clone from master branch of the bitbucket repo.
3) Carefully copy .bzr and settings.local.php back in.
4) Do that same thing for /var/www/dev/page-drupal.redesign.devdrupal.org, only from the "staging" branch (this might be harrier since the DB is 11 days behind, may want to either take a backup of the DB/files first if there's room on the server to do so.)

webchick’s picture

Hm. And that might actually be an argument for keeping .bzr in Git. (Ow, my head.) Anyway, signing off for the night now. :)

webchick’s picture

Update for today:

- Reviewed both sanchiz and joachim's patches and made the determination that sanchiz's is the best way to go for now. Detailed review is at #2159813-14: Display parts of the issue edit form instead of the comment form on issue pages.
- Did some research nevertheless to determine if it was possible to use existing contrib modules to do this rather than custom code, in order to help make the result more likely to be accepted into project_issue "proper."
- While doing that, accidentally hosed my local so am re-importing the DB overnight. ;)
- Opened #2161601: Create styling that matches issue edit form mockup and #2161619: Remember last "collapsed state" of collapsible fieldsets as related sub-issues.

It felt good to get some real, actual work done on this today (yay!), but I hasten to point out that the only reason I made any progress whatsoever is because I cheated and downloaded a copy of the devsite files + db to my local and was hacking on it there. #2161151: Give write permissions for form-drupal dev site is still a critical blocker for doing this the "preferred" way on devwww.

EDIT: Yay, I'm unblocked there now. Sorry, didn't see the update since I wasn't subscribed to that for some reason.

mcrittenden’s picture

Issue summary: View changes
markhalliwell’s picture

Issue summary: View changes

Added better development workflow instructions.

markhalliwell’s picture

Issue summary: View changes

Separated setup from workflow

webchick’s picture

Issue summary: View changes
markhalliwell’s picture

Issue summary: View changes
webchick’s picture

Issue summary: View changes

Adding an example of a local alias code block since I had no idea what that meant. :)

webchick’s picture

Some big updates today:

1) Switched from bitbucket to a private Github repo on markcarver's plan since we were hitting a ceiling on contribs @ bitbucket.
2) Got that repo populated and both the dev and staging servers pointing to it (after a lot of permission futzing)
3) Made a firm decision to go the drupalorg / form_alter route (aka "the sanchiz approach") on implementation so we can stop hand-wringing about that.
4) I did a code review of the latest code in the staging server and left comments in #2159813-24: Display parts of the issue edit form instead of the comment form on issue pages.
5) markcarver made some awesome headway on the fieldset styling in #2161601: Create styling that matches issue edit form mockup which should be visible any time on http://page-drupal.redesign.devdrupal.org/node/2130811#comment-form
6) Also fleshed out the documentation at the top to enable other contributors to jump in and help.

webchick’s picture

Today, made some improvements to the patch at #2159813: Display parts of the issue edit form instead of the comment form on issue pages and also solved this core bug #1525176: Using array_diff_assoc() for multilevel arrays in DrupalDefaultEntityController->cacheGet() which was causing annoyance on my local machine.

Next steps are to really drill down into the files processing and see if I can get to the bottom of the "delete all the files when you upload one" bugs I experienced on the prototype. None of the file stuff is working on localhost at all, so need to do some debugging there.

dww’s picture

A) Unloved for a while, but https://drupal.org/project/projectinstall is a quasi-working D7 install profile with the Project* suite sort of configured for use. ;)

B) Yes, improving things "upstream" is fine with me when appropriate. I'll try to look at the patches and weigh in later today.

C) I'm still sad that we're fixing this in the proposed way - back to the D6 approach of hacking the comment form into an issue edit form. I wish we were improving the UI/UX of the edit form along these lines, and injecting that into the bottom of the issue via AJAX when you click "Update this issue", plus 10 lines of JS to copy over the value of the comment field into the right text area on the edit form. I don't understand the objection about needing to tab back and forth -- there would be no reason to ever go back to just the comment form once you decided you actually wanted to update the issue -- and that form would be the only thing swapped out via the new tab. Seems like then we could just use any of a number of existing off-the-shelf solutions to inject a real edit form, instead of all the partial form altering hacks. And we'll have less x-posting conflicts since we'd only inject the edit form once you started updating, not as soon as you load the page. But whatever, I've been over-ruled on this, and everyone seems keen on doing it this way, so I'll do what I can to try to help make it happen. But for all sorts of reasons I think this is the worse approach of the two main options we were considering...

markhalliwell’s picture

Added #2163197: Allow fieldsets to be persistent and save focus when AJAXing
Added the bzr dir back to the repo (@drumm couldn't do a diff against the theme)

webchick’s picture

I think one of the concerns was the comment getting lost when flipping between tabs. There may have been others.

It's worth pointing out though that the patch doesn't alter the comment form to make it a node form. It merely embeds the node form itself at the bottom of the comment stream, and then does some form alter stuff on it to make it look nicer. So still far less complex than comment_upload module, although the files stuff is being tricky.

dww’s picture

Right -- that's the (approx) 10 lines of JS (maybe less). And again, I just see "upgrading", not flipping. There'd be no reason to flip back. But whatever, I'm sure the same 10 lines of JS would handle that case, too. ;)

Perhaps I should just RTFP, but if we're just injecting the node edit form at the bottom of the comment stream, what happened to the comment form itself?

webchick’s picture

Ok, japerry and DyanneNova and I had a hackfest tonight on this at Chez Webchïck and a new, semi-final patch has been uploaded to #2159813: Display parts of the issue edit form instead of the comment form on issue pages. I think you'll be happy with the results; it's only 6.47K, about half of which is comments. ;)

joachim’s picture

I'm trying to follow the workflow in the summary, but I'm having a few problems:

> cp /var/www/dev/form-drupal.redesign.devdrupal.org/.ssh/id_rsa* ~/.ssh/; chmod -R 700 ~/.ssh/id_rsa*

This scares me. It looks like it's going to overwrite my own local SSH keys. Not good! Also, I don't understand why I need to do that. Have I missed something?

> Add your local settings.php file.

This should say that you need to set Drupal up locally -- or at least have a DB and a settings.php that points to it. Installing with 'drush si' is the simplest way to do that, even if the DB data you create is about to get wiped in the next steps.

> Use drush sql-sync @drupalorg.dev @drupalorg.local or drush pipe @drupalorg.dev @drupalorg.local

> $aliases['sandbox'] = array(

Shouldn't these two match? Otherwise, the drush command says that it doesn't know about @drupalorg.local.

joachim’s picture

One other thing: roughly how long should the 'drush sql-sync' take? How much data is there to download?

I was hoping to quickly get this set up before going to bed, and my terminal is just sitting there doing nothing...

drumm’s picture

We've done our best to sanitize the DBs, but local copies still make me very uncomfortable. Encouraging them is not good. Do go ahead and make one-off local copies if you really need them, treat them as confidential, and delete when done. (Once #636340: Provide generally available, regular exports of already public data from the drupal.org database is deployed, this will change.)

joachim’s picture

> Do go ahead and make one-off local copies if you really need them,

I've been told on IRC that the DB is 4Gb! It makes me rather uncomfortable too, and it's going to take ages!

Is there another way I can work on this issue?

drumm’s picture

I really recommend doing dev work on devwww directly. The DB is large and local installation issues inevitably happen for local installations. If something on devwww is getting in the way, we can work on improving it.

webchick’s picture

The biggest thing in the way on devwww is that sandboxes assume only a single developer working on changes. So editing files directly on the server with zero version control behind you ala 1994 and FTP "works" (for some value of works) because there's no one to conflict with in those cases.

In this case, however, we have front-end devs, back-end devs, ux folks, and several other people spread out over multiple timezones with totally unpredictable availability collaborating on this change. It's literally impossible to do this if we don't have a version control system to track who did what, when, why, and how. We also have been forced to change implementation plans a few times as more architectural feedback came in, which required branching off (to keep the existing dev/stg sites functional) and then merging back in later. This would not have been possible without Git.

joachim’s picture

I'm not sure doing a DB copy is going to be terribly feasible either.

I just started it going and left my laptop for an hour+ and got back to find:

> mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `field_revision_field_issue_category` at row: 661919

joachim’s picture

Tried again, got

> mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `field_revision_field_issue_category` at row: 631380

:(

webchick’s picture

Yeah, I ended up gzipping it in my home directory and FTPing it down. Like a boss. :P

eliza411’s picture

@joachim - if you're still able to lend a hand, I've created an alternative to the large DB download. You can work directly on the remote server - see #2169889: Site Availability - Assign this issue to yourself while you're working for the protocol, which is basically to assign the issue to yourself while working and unassign when you're done.

You still have to be okay working from the command line, etc. but hopefully this space can supplement the collaborative space above.

joachim’s picture

Working a text editor from the command line is going to be rather painful...

I've come up with what I think is a workaround for the sql-sync problem:

$ nohup drush sql-dump --result-file=~/dorg.sql

That can be left to its own devices thanks to the nohup command -- you can disconnect and leave it to get on with it, though it actually only took about 5 minutes.

Next step is going to be downloading the sql dump, same way webchick did it.

Any update / comments on #39?

drumm’s picture

Many editors handle remote files well. Or you can mount via SSH with sshfs. Subpages of https://drupal.org/node/1018084 have some platform-specific help. As always, feel free to edit those pages if you have good additions.

drumm’s picture

I'm doing more review this week, starting with cleaning up the CSS for #2161601: Create styling that matches issue edit form mockup.

drumm’s picture

Issue summary: View changes

Updating the issue summary to remove Git branching and local installation instructions.

Until #636340: Provide generally available, regular exports of already public data from the drupal.org database is done, making copies of the database off devwww should not be encouraged. We believe all personal data is sanitized out, but have not ensured we have a good system to keep that being true ongoing. If you do want a local copy, go ahead, know it will be large and cumbersome, treat is as confidential, and delete when you are done.

For branching: as issues move toward deployment, we need patches against projects on Drupal.org (now even including bluecheese). I find the best way to work is to use Git checkout of the relevant projects in sites/default/{modules,themes}. This gets the advantages of Git, and commits are ready for deployment in the end. For me, I always work on a feature branch that I just name "drumm" since I usually try to stick to working on one deployment at a time; I rebase it and merge to projects I have commit access to, and post patches when I want review. Working with more people probably needs a more complex branching strategy.

Over the past couple weeks, I've been working on restws-drupal.redesign.devdrupal.org, since it was there. I'll start a fresh dev site when the regressions settle down and/or when any other volunteers want to jump in on a sub-issue.

sun’s picture

I just filed #2193857: Issue node gets unpublished in case of a cross-post conflict, because it occurs very often for me in the meantime.

mcrittenden’s picture

Assigned: mcrittenden » Unassigned
drumm’s picture

drumm’s picture

This has improved enough to no longer be a priority.

tvn’s picture

Status: Active » Fixed

The first iteration of the issue queue improvements was deployed a few weeks ago. We can probably close this issue now.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.