Turn Mapillary organizations into groups with members, roles and content

Problem/Motivation

Mapillary Organization (mapillary_organization) provides an entity that mirrors an organization record of the Mapillary Graph API: organization_id, slug, name and description. Nothing references it, it has no members, and nothing fills it from the API. The Data Tuning tab next to it (TuningForm, route mapillary.tuning) saves restrict_to_registered_orgs, which no code reads, so its label ("Publish only registered organizations' street views") promises a behaviour the module does not have.

Sites need organizations to say who belongs to which team, which Mapillary content belongs to which team, who manages the team, and who did what where. Four kinds of sites:

• Institution A has one GIS officer and one Mapillary organization. It does not need organizations in Drupal and should not have to install or configure anything for them.
• Institution B has several departments, each with its own Mapillary organization. The staff of each department are members of that department's organization, and reports show which department captured or imported what, and where.
• Institution C hires contractors in several countries. Each contractor puts the people who do Mapillary work for Institution C into its own organization and removes them when they leave. A contractor may have no Mapillary organization of its own and upload under Institution C's.
• Institution D works with service partners. Each partner is a separate organization with its own permissions.

The Group module already provides memberships, group roles, group permissions and relationships between groups and content. The Graph API gives what is needed to match content to organizations: an image has an organization object (id, name, slug), absent for images without one, and the sequence of an image has the creator's username, which the module stores as creator_username on mapillary_sequence.

Steps to reproduce

1. Enable mapillary_organization and add an organization.
2. Open the Data Tuning tab, check "Publish only registered organizations' street views" and save.
3. Nothing changes in imports or listings, and there is no way to add users or content to the organization.

Proposed resolution

Organization as a group type

• mapillary_organization depends on group:group and provides a group type mapillary_organization. The group label is the organization name; the group has the fields organization_id (optional, unique when set), slug and description. An organization with no Mapillary organization, such as a contractor, leaves organization_id empty.
• Supported Group versions: ^3.3 || ^4.0. Group 4 renamed content_plugin to relation_type in relationship type config and moved to core's Access Policy API, so relationship types are created in code, not shipped in config/install. A first compatibility check runs one relation plugin and one permission check against both versions; if one code base cannot serve both without many version branches, the module requires ^3.3 only.
• Sites that do not need organizations do not enable mapillary_organization; the base module and the other submodules do not depend on Group.

Content and roles

• Relation plugins for mapillary_image, mapillary_sequence, mapillary_map_feature and mapillary_detection, each available only when its entity type exists. Content may belong to more than one organization, and a user may be a member of more than one organization.
• Four default group roles: Admin (manages members and the organization's settings), Editor (edits the organization's content, adds content by hand, sees conflicts), Importer (imports and refreshes the organization's own content) and Member (views the organization's content and reports).
• An Importer imports in the context of one organization. After the API response arrives, the import is rejected unless the image's Mapillary organization is the organization's organization_id or its creator is a member under the rules below.
• Site administrators are never restricted: they can create any organization, add any user, set any value below, import any ID and add any content to any organization. Automatic rules never undo what was added by hand.

Matching people and images

• A user field mapillary_username, matched against creator_username. It is unique and set by site administrators, or by an organization's Admins for that organization's members; users cannot set their own. organization_id on an organization is set only by site administrators.
• The image importer requests organization and stores its ID in a new organization_id field on mapillary_image. The thumbnail refresh requests it too, so existing images get the value over time.

Assignment rules, for one image with Mapillary organization O and creator C

• An organization counts as the creator's only if the creator's membership started on or before the image's capture date, so past work of a user who moved to another organization is not a conflict there.
• O empty: the image goes to every organization of the creator.
• O set and an organization has organization_id O: the image goes to that organization and to every organization of the creator that has no organization_id (contractors and partners). In every other organization of the creator that has an organization_id, the image is a conflict.
• O set and no organization has it: the image goes to every organization of the creator without an organization_id, and is a conflict in the creator's organizations that have one.
• Sequences and detections follow their images; a map feature follows the images it was detected in.
• Assignments are made when the data first allows it (import, refresh, a member's mapillary_username set, a membership added) and are never removed automatically, so a worker who leaves keeps their past work in the old organization's reports. Wrong assignments are removed by hand.

Conflicts

• On an organization's page, content in conflict is listed separately as mismatched content: read only, and not counted in that organization's numbers or reports.
• A notice is shown to the creator on their user page and to the Admins of every organization involved. It explains how to resolve the case at hand: update the membership, correct the image's organization on Mapillary and import or refresh it again, or ask a site administrator to register the Mapillary organization.
• Notices are computed when the page is built, not stored, so they disappear as soon as the data agrees. They respect the visibility setting: a viewer who cannot see the content gets the count and the organization it belongs to, not the items.

Organization settings, replacing the Data Tuning tab

• Visibility: public, or members only. Members-only content is hidden from non-members even when they have the site-wide view permission; content in several organizations is visible if one of them is public or the viewer is a member of one of them.
• Import filter: whether creator matching may add content to this organization, so an organization can accept only the images of its own Mapillary organization.
• Scheduled fetch: an enabled flag and an interval, stored for a follow-up that fetches an organization's new images on cron. Nothing reads them yet.
• Removed: TuningForm, the mapillary.tuning route, its menu link and local task, the mapillary_organization.settings config object and its schema.

Logs and reports

• mapillary_logs records, for each API call, the organizations the acting user belonged to at that moment. Calls from cron or Drush without such a user record none.
• The views of images, sequences, map features, detections and API calls get an organization filter and column.
• Nodes with Mapillary fields join organizations through Group's own gnode module; the documentation describes this, no code is added for it.

Upgrade path from 1.0.0-alpha6

• If there are no mapillary_organization entities, the update does not install Group: it uninstalls mapillary_organization and says so. The site can enable it again later.
• If other configuration references the old entity type (entity reference fields targeting mapillary_organization, site-built views on its tables), the update stops with a list of them and changes nothing.
• Otherwise it installs Group, creates the group type, fields, roles and relationship types, creates one group per organization with its values, translations and any fields a site builder added, replaces the mapillary_organizations view, removes the old entity type's permissions from roles, and removes the old entity type and its tables only after every row has been copied. On any failure it throws and leaves the old data in place.
• A post update deletes mapillary_organization.settings.
• Kernel update tests from a 1.0.0-alpha6 fixture for the three cases.

Remaining tasks

• Compatibility check against Group 3.3 and Group 4.0.
• Decide whether the work is split into child issues.
• Merge request.
• Review.
• Update rehearsal from 1.0.0-alpha6.
• Documentation, including the gnode section.

User interface changes

• Organizations are managed as groups: members, roles and content tabs from Group.
• Per-organization settings (visibility, import filter, scheduled fetch) instead of the Data Tuning tab, which is removed.
• A "Mapillary username" field on user accounts.
• A mismatched content list on organization pages, and conflict notices on organization and user pages.
• Import forms take an organization for users with the Importer role.
• An organization filter and column in the Mapillary listings and in the report of API calls.

API changes

• The mapillary_organization content entity type and its class MapillaryOrganization are removed; organizations are group entities of the mapillary_organization group type.
• New Group relation plugins for the four Mapillary entity types.
• The route mapillary.tuning and TuningForm are removed.
• New dependency: group:group (^3.3 || ^4.0) for mapillary_organization.

Data model changes

• Group type mapillary_organization with the fields organization_id, slug and description, four group roles and four relationship types.
• New field organization_id on mapillary_image.
• New field mapillary_username on user.
• Organization settings stored on the group.
• The organizations of the acting user stored with each API call in mapillary_logs.
• Removed: the mapillary_organization tables and the mapillary_organization.settings config object.