First pass: migrate "Companies worked for"

  • Create a field collection (exported in a Feature) containing:
    • String company name
    • Current/past toggle
  • Migrate "Companies worked for" profile data to these fields.

Second pass: migrate "Current company or organization"

  • Migrate "Current company or organization" to the fields.
  • Update marketplace and integration code to use the fields.

Testing

Testing is done at https://companies-drupal.redesign.devdrupal.org

Deployment

  1. Deploy drupalorg 7.x-3.x-dev
  2. Confirm "Drupal.org user customizations" feature is not overridden
  3. Set profile field to hidden at /admin/config/people/profile/edit/72
  4. Run updatedb, to migrate the old "Current company" profile field values to the new core field.
  5. Delete the old profile field from /admin/config/people/profile.

Original report

In http://drupal.org/node/35543 the field for the current company was added. The field "companys worked for" is also available. Both are parsed so that all employees of a company can be listed on one page like http://drupal.org/profile/profile_current_company_organization/undpaul

The field "current company" is not parsed correctly (or it currently isn't wanted) that you can add several current companies seperated by commas. The field "companys worked for" is created correctly. You can see this behaviour in my profile => http://drupal.org/user/87891

Please enable multiple (automatically parsed values) for the field "current company or organization"

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

greggles’s picture

I think this is somewhat by design. The idea is to list the one main company where you currently work.

McGo’s picture

Because of the naming of that field it's not ment to hold the _main_ company one is working for, it is about to hold the _current_ company. According to http://www.census.gov/newsroom/releases/archives/facts_for_features_spec... 5 percent off all fulltime elpoyees have a part time job. It is obvious that our web business is a great place to start as a part time freelancer. So it is not only possible to work for more than one comany at the same time.

+1 for changing the field settings.

christefano’s picture

I'm a founder of two Drupal organizations (Exaltation of Larks and Droplabs) and would like to list both on my profile.

Techivist’s picture

+1 here also.

I'm the founder of one (small) Drupal shop (the OpenMindz Group) & am working closely on another Drupal project that is working towards putting forth a mobile Drupal distro which will be contributed back to the community under GPL (VozMob). I, like Christefano, would like to list both on my profile.

I don't like adding work to others' plate, greggles, but I think it would be a benefit to the community. It seems there are many freelancers in the community, or folks who split their time evenly between a couple of Drupal shops, so I think it'd be a feature embraced & used by the majority of the Drupal community.

greggles’s picture

Since #4 is addressed to me I want to chime in to say: this isn't really my call. I understand the desire here.

If this goes ahead:
We'll need clearer labels on the items to denote how they are different.
We may need the new marketplace that Drumm is working on to be coded differently somehow to include this. If you look at Current company or organization on my profile you can see my current comopany/organization is replaced with the image from the marketplace.

drumm’s picture

The code for the logo is at http://drupalcode.org/project/drupalorg.git/blame/refs/heads/6.x-3.x:/dr.... It will need a patch for this. See http://drupal.org/node/1006562 for help on coordinating and testing these changes.

jbrauer’s picture

Another +1.

This field is also being pulled to describe one's Drupal Association membership. As such the organizational membership for my family-owned company is now being attributed to my employer who is also already an organizational member. It would be much better to allow proper descriptions of these things.

larowlan’s picture

Another +1

Happy to help work on this if someone can nudge me in the right direction.
As with @jbrauer I have an organisational membership for my family-owned company but this is interfering with my employer's membership.

Lizz Trudeau from the DA said I should chime in here.

Lee

drumm’s picture

Project: Drupal.org site moderators » Drupal.org customizations
Version: » 7.x-3.x-dev
Component: User account » User interface
Status: Active » Postponed

Now that we are in the process of moving to Drupal 7, we can look into making it a multi-value field. This should be a lot more doable than cramming it into the profile module's storage. I recommend waiting for the D7 upgrade to settle down. We want to minimize what's changing on launch day, so this will have to wait for another deployment.

christefano’s picture

Title: Multiple Values for current companies » Multiple Values for listings of current companies and organizations

Updating title to make this easier to find.

mgifford’s picture

Issue summary: View changes
Status: Postponed » Active
mgifford’s picture

So changing this field to a multi-value field is done in the DB. This will affect the user profile page mostly.

Will it affect the marketplace?

I'm coming at it from #2177459: Highlight Supporting Organizations in the Issue Queue in that individuals often have more than one entity that they do Drupal work with.

Also good to keep an eye on this related issue #1968480: Keep historical tracking of users' employers

fizk’s picture

I'm also interested in this.

@mgifford @drumm What are the next steps?

drumm’s picture

Issue summary: View changes

I think it makes sense to implement this in 2 steps, as I've added in the issue summary. This allows us to first get the field structure right, without changing integration code in the marketplace section. Then the actual current company field, and marketplace integration can be migrated.

There are a couple decisions to be made in field structure.

I think the company/organization name should be a string, rather than entity reference to organization nodes. A string makes migration straightforward, and lets people keep entering whatever they want, without needing an organization page. An entity reference would be easier to work with for future development, such as creating Views using the relationship. We can convert it later if we need to; or even make a hidden computed reference field.

For tracking whether a position is current or past, I think we should stick to a simple toggle, not start/end dates. We don't have the data to migrate in to start/end dates, and it would be nice to keep the UX straightforward for filling out your profile.

drumm’s picture

Issue summary: View changes

Cleaning up language in issue summary. After typing out alternatives, I settled on what I think is a good field structure for now.

drumm’s picture

@fizk - let me know if you are ready to start on the first pass and we can build out a dev site for this.

fizk’s picture

@drumm Thanks, I'm ready to start on the first pass.

I think having two separate field collections - "Companies worked for" and "Current company/companies or organization(s)" - would be even more straight forward than using a checkbox on each company. Thoughts?

tvn’s picture

I started dev site build, will be ready in ~1,5 hours.
Url will be https://companies-drupal.redesign.devdrupal.org/
Some useful info on how to work on our dev server: https://drupal.org/node/1018084

drumm’s picture

Keeping the fields separate, and skipping field collections, may be a good way to go. The UI difference is that you would have to cut & paste an organization from one field to the other when switching jobs, instead of unchecking a checkbox.

joachim’s picture

drumm’s picture

multifield is worth considering. Project releases use field collections. Staying with the same modules could help maintainability. However, the entity abstraction of field collection can be cumbersome. I don't think multifield existed when the decision to use field collection was made.

fizk’s picture

Status: Active » Needs review
FileSize
19.27 KB

This seems to work now: https://companies-drupal.redesign.devdrupal.org/user/1

  • The "Current company or organization(s)" is now a multi-line textfield.
  • If you choose to separate companies using commas instead of newlines, it will automatically be converted to a newline list on submission.
  • The logo of each current company will appear on your profile, if available.
  • The membership badge of each current company will appear on your profile.
  • You will appear in the marketplace company profile page for all of your current companies.

The conversion to a multi-line textfield was done using SQL: UPDATE profile_field SET type = 'list' WHERE fid = 72;

fizk’s picture

I'm guessing there's also some translation work needed if we change the field name from "Current company or organization" to something like "Current company or organization(s)".

drumm’s picture

Status: Needs review » Needs work

The data needs to be migrated to a core multi-valued text field, with the field configuration exported to a Feature. Since core profile module is deprecated, its use should be removed any time we work with profile data. See also #2212631: Pages matching the URL /profile/.* disabled.

fizk’s picture

Status: Needs work » Needs review
FileSize
26.72 KB

This patch:

  • uses a new multi-valued core text field
  • contains the exported feature for the new field
  • can migrate all users to the new field via Batch API on /admin/config/system/drupalorg
  • does not delete the existing profile field: this should be done manually when ready.
fizk’s picture

FileSize
29.67 KB

Previous patch was missing a new tpl.php file.

Shyamala’s picture

Issue tags: +Needs DSWG Dev Tools Team feedback

Adding tags

mgifford’s picture

@fizk - What are the steps required to test this?

Apply patch, run update.php, view profile?

fizk’s picture

@mgifford The steps are:

1. Apply patch.
2. Recreate the "Drupal.org user customizations" feature, which should create the new "field_current_company_org" field.
3. Go to /admin/config/system/drupalorg and click "Begin migration".

After verifying everything's in place, the the old user profile field can be deleted.

mgifford’s picture

I don't have the file permissions to recreate the feature...

I think this would have done it:

[mgifford@devwww default]$ drush features-update drupalorg_user
Module appears to already exist in sites/default/modules/drupalorg/features/drupalorg_user
Do you really want to continue? (y/n): y
file_put_contents(sites/default/modules/drupalorg/features/drupalorg_user/drupalorg_user.features.field.inc): failed to open stream: Permission denied drush.inc:792 [warning]
file_put_contents(sites/default/modules/drupalorg/features/drupalorg_user/drupalorg_user.info): failed to open stream: Permission denied drush.inc:792 [warning]
file_put_contents(sites/default/modules/drupalorg/features/drupalorg_user/drupalorg_user.module): failed to open stream: Permission denied drush.inc:792 [warning]
Created module: drupalorg_user in sites/default/modules/drupalorg/features/drupalorg_user

But other attempts failed too. I'll reach out to drumm for permissions access as he owns the files.

drumm’s picture

Issue summary: View changes
Issue tags: -Needs DSWG Dev Tools Team feedback

@Shyamala - this isn't a dev tool. Untagging.

@mgifford - test on the dev site for this issue, https://companies-drupal.redesign.devdrupal.org.

drumm’s picture

Status: Needs review » Needs work

Overall, this is great progress!

Instead of the admin UI, batch_update_current_company should be in drupalorg_update_N(). It doesn't need to be batched, we always run updatedb via drush with a high memory limit, and the time limit can be turned off. Since there is some batching already, it doesn't hurt to keep it, if convenient when moving to hook_update_N().

Let's skip the custom code for #user_category. As fields move from profile to core fields, we can start organizing them with field groups, such as vertical tabs or basic fieldsets. This will export nicely without custom code. We don't want to perpetuate profile module behavior as custom code.

The logo listing on profile pages

      $nid = db_query("SELECT nid FROM {node} WHERE status = :status AND type = :type AND title = :title", array(':status' => 1, ':type' => 'organization', ':title' => $company))->fetchField();

      if ($nid != 0) {
        $organization = node_load($nid);

Should be an EntityFieldQuery and node_load_multiple() to cut down on queries, and modernize the code. It could even be changed to an embedded View, or that could be saved for a followup issue.

fizk’s picture

Status: Needs work » Needs review
FileSize
28.08 KB

@drumm Without the custom code for #user_category, the "Current company or organization(s)" field will show up on the Account tab instead of the Work tab until we've migrated all of the profile fields to core fields and organized them with field groups (unless you know of another way?). I don't think we should delay pushing this fix for that to happen since this issue is more than three years in waiting.

@mgifford I've updated the patch to use hook_update_N() and EntityFieldQuery.

fizk’s picture

@tvn To test the hook_update_N() code, we'll need a new DB dump into "companies_drupal", since I've already done the migration and deleted the old field.

mgifford’s picture

FileSize
24.5 KB

@fizk you're developing your code here I assume https://companies-drupal.redesign.devdrupal.org

If so I'm not sure if there is any way I can help test your patch. I'll need some clarification on how I can help out with this initiative if at all.

Other than by posting a screenshot of where I can see it in action here /user/%/edit/Work

Option to show multiple companies in profile page.

mgifford’s picture

FileSize
86.22 KB

This is just an example of course, but figured a screenshot of this is worth illustrating that it is working as expected in /user/%

Other logos in the profile page.

fizk’s picture

@mgifford If we can reset the DB on https://companies-drupal.redesign.devdrupal.org to a fresh install of D.org, you could try running the hook_update_N() and recreating the "Drupal.org user customizations" feature, and then making sure multiple companies show up on the profile view and edit page, which you've already done in #35 and #36.

@drumm If you think the hook_update_N() code looks simple enough, we can just go ahead with deploying the patch.

drumm’s picture

Status: Needs review » Needs work

I've scheduled a rebuild of https://companies-drupal.redesign.devdrupal.org. These are queued up to Dev Days Szeged, so it will be done in 8-10 hours.

This is a full site reset, including the codebase. Please take notes on deployment and add to the issue summary; even if it is just apply patch, run updatedb. If there are any configuration changes, we should have them written down.

@drumm Without the custom code for #user_category, the "Current company or organization(s)" field will show up on the Account tab instead of the Work tab until we've migrated all of the profile fields to core fields and organized them with field groups (unless you know of another way?). I don't think we should delay pushing this fix for that to happen since this issue is more than three years in waiting.

Let's at least put a todo comment in there to remove that custom code in favor of field groups. I do want to be aggressive in moving fields to the main edit page, but I guess that might be too aggressive.

I spotted a small code style thing, else should be on the next line:

} else if ($form['#user_category'] == 'Work') {

On EntityFieldQuery, please move that out of the $companies loop. There should be a single EntityFieldQuery which uses IN to check the titles. If there are results, a single node_load_multiple() can load the organization nodes. Then the looping over the loaded nodes can happen as needed. Then the number of queries won't grow as the the number of companies grows.

fizk’s picture

Issue summary: View changes
FileSize
28.63 KB

@drumm Thanks, this patch has all of your suggestions:

  • added TODO comments
  • fixed the else if position
  • moved EntityFieldQuery out of the $companies loop
  • uses node_load_multiple()
fizk’s picture

Status: Needs work » Needs review
fizk’s picture

@drumm @mgifford Has the latest patch been tested? Let's mark this RTBC and get it in production.

opdavies’s picture

Status: Needs review » Needs work
FileSize
2.96 KB
25.15 KB

I've had to split the patch into two separate patches, one for drupalorg and one for Bluecheese. Both are attached.

The Bluecheese one is applies correctly, but the drupalorg patch does not.

This is what I get when attempting to apply the patch to drupal.org.

error: patch failed: drupalorg/drupalorg.module:2190
error: drupalorg/drupalorg.module: patch does not apply

error: patch failed: features/drupalorg_user/drupalorg_user.info:8
error: features/drupalorg_user/drupalorg_user.info: patch does not apply

This will need a re-roll against the latest 7.x-3.x release of drupalorg.

opdavies’s picture

Assigned: Unassigned » opdavies

I'll be doing the re-roll of the drupalorg patch.

opdavies’s picture

FileSize
24.92 KB

Here's a re-roll of the original patch just for the drupalorg module against the original commit (5ab8334f2b3e12c0d4150d156020e6c7afe8593e).

Now to re-roll it against the current 7.x-3.x version.

opdavies’s picture

Status: Needs work » Needs review
FileSize
17.97 KB
362.23 KB
233.56 KB

Re-rolled against the latest 7.x-3.x-dev.

I've created a new dev site at https://companies2-drupal.redesign.devdrupal.org, applied it there and everything including the migration seems to be working OK. I'm getting some Varnish errors but the update seems to have run OK.

drumm’s picture

Status: Needs review » Needs work

Thanks, this is great progress! @fizk - sorry the membership badge changes had to go in first, derailing this.

As we've added organization/partner membership types, the code querying those has blown up a bit, currently one query per membership type. This will make it one per membership type and badge. I think it should be optimized to do one query per-company, or even one for the whole user page.

In drupalorg_civimembership_data(), remove :type from the query. Store all records, or lack of records, returned in a static variable, so the query happens once per company, and no other code changes should be needed. I'm tempted to want to refactor it more, but that should wait for a followup issue.

opdavies’s picture

Assigned: opdavies » Unassigned
drumm’s picture

Assigned: Unassigned » drumm
drumm’s picture

Issue summary: View changes

  • drumm committed 3dc4420 on 7.x-3.x authored by fizk
    #968994 Multiple Values for listings of current companies and...
  • drumm committed 8851a60 on 7.x-3.x
    #968994 Static caching for organization memberships.
    
  • drumm committed c0c8411 on 7.x-3.x
    #968994 Remove dead code.
    
drumm’s picture

Status: Needs work » Fixed
Issue tags: +needs drupal.org deployment

I believe this is ready to be deployed, planning for tomorrow.

  • drumm committed 5273907 on 7.x-3.x
    #968994 Speed up update.
    
drumm’s picture

The DB update was on track to take ~2 days on our staging servers. The last commit should make it go up to twice as fast.

drumm’s picture

Actually, it is quite a bit faster. 8 saved in the first minute vs. 92.

drumm’s picture

The update is now running on production. Feel free to fill in your organization(s) if you get to it before the update does.

drumm’s picture

Component: User interface » User profiles
Parent issue: » #2322245: [meta] Migrate profile fields to core fields

The migration is 4.4% done, so it will take a few hours to complete. I think the speedup on git7site was because it doesn't have bakery configured.

drumm’s picture

Issue tags: -needs drupal.org deployment

The migration completed.

tim.plunkett’s picture

Now my profile is missing the nice big Acquia logo for my current employer. That was nice to have, because it was easy to see the company out of the list of info.
I realize this overlaps with the work to redesign the profiles, but is there a way to fix this regression.

drumm’s picture

I think the logos may have been working before in this issue; but were not, as far as I could tell, in #45. Combined with logos not being in #2281763: Make Drupal.org user profiles more robust, I chose to remove the code rather than debug it.

tvn’s picture

The logos are definitely present in #2281763: Make Drupal.org user profiles more robust. There are two mockups there, in the first one company does not have org page so there is no logo, in the second one there is a page (Acquia) and therefore there is logo. Should we open a separate issue to bring logos back?

drumm’s picture

Ah I missed the second mockup. Yes, let's do a separate follow up.

webchick’s picture

An entity reference would be easier to work with for future development, such as creating Views using the relationship. We can convert it later if we need to; or even make a hidden computed reference field.

Yes, please! #2323705: Add user's organization computed reference field

tvn’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

fizk’s picture

Thanks for committing @drumm! I missed the update emails for some reason.

joshuami’s picture