Problem/Motivation

A merging join (wait all, matching, threshold, quorum, timeout) can gather one variable from each joined branch into a list: the collect, into and scope settings. Only one: a review where each branch records a vote and a comment needs two lists, and today that takes a second join or a script step. A list of values that should appear once (the departments a request went through, tags set by several reviewers) also comes out with every repeat, and the steps after the join have to remove them.

Two smaller points in the same code:

  • The collected list starts with the token whose arrival fired the join, then the ones that were waiting (array_merge([$token], $waiting) in JoinCoordinator::evaluate()), so the first value is the last arrival's, not the first one's.
  • The quorum join reuses collect as the variable it votes on and makes it required there, so the one collect line is not free for anything else on a quorum.

Proposed resolution

Replace collect and into with a list of collect lines, typed in a textarea, one per line:

vote|votes
comment|comments
department|departments_seen|distinct
  • Each line reads source from each joined branch, through that branch's own lineage as today, and writes the values as a list to target. Each line is its own list.
  • The optional third part is the keyword distinct: a value already in the list is not added again. Values are compared strictly, after the unwrapping collect already does (a {outcome, comment} payload is collected as its outcome), so "1" and 1 are two values and a structured value is compared as a whole. The first occurrence is kept.
  • Lists follow the order the branches arrived in, the order of their token IDs, as the Branch variables setting does (#3629938: Let a synchronizing join keep the branches' token-scoped variables, from the first or the last arrival): the first value is the first arrival's, which is also the occurrence distinct keeps.
  • The lists are not aligned: a branch that never set a source adds nothing to that list, and distinct drops repeats, so position 2 of one list and of another may come from different branches. To pair a branch's vote with its comment, use the Branch variables setting or a variable name per branch. The documentation says so.
  • scope stays one setting, for every list.
  • The quorum join counts the branches a quorum_condition holds on: a flow condition, picked from the same plugins as on flows (comparison, count, any, all) and tested on each arrived branch with that branch's own variables, as a flow tests the token leaving on it. It is required, and the form starts on a comparison with equals. A quorum can then count a value other than one, one of several values, an amount over a limit. It is no longer read from the collect lines, which are optional on a quorum as on the other joins. A distinct list would make a count condition such as "at least 2 approved" meaningless, which is one more reason not to count a collect line.
  • How many branches make the quorum is set by quorum_unit: count (the default, quorum_value branches; quorum_value replaces count), percent (quorum_value percent of the branches, rounded up: 50% of 3 is 2) or majority (more than half: 3 of 4, 2 of 2; no value). A share is always taken of the branches the join waits for, never of those arrived so far, which the first branch to count would reach on its own. A count is never lowered to the branches that run, as #3629951 made it on 1.x: when fewer run than the count, the quorum is out of reach from the start and the first arrival settles it as not reached.
  • An optional quorum_value_variable reads the value from a variable instead, in the arriving branch's view, so one set before the split is the same for every branch. A variable that is not set, or holds anything but a whole number from 1 (to 100 for a percentage), raises an incident rather than deciding on a number nobody chose.
  • An optional quorum_result names a variable the quorum join writes whenever it fires, whichever way it went: the number of branches the condition held on, in the join's scope, in the same write as the collect lines. A comparison condition such as approvals >= 2 then routes the outcome without collecting the values and counting them again. An optional quorum_reached receives TRUE or FALSE, whether the quorum was reached: the variable to route on with a percentage or a majority, where the number of branches it takes depends on how many ran (reached is not empty on one flow, is empty on the other).

Validation, in the form:

  • Each line has two or three parts separated by |; the third, when present, is distinct. Anything else names the line and refuses the form.
  • Source and target follow the variable name rules, reserved __ names included, as the variable name field refuses them today.
  • A target used by two lines is refused: the second list would overwrite the first.
  • A quorum_result or quorum_reached that a collect line also writes, or the two naming the same variable, is refused: they are written together when the join fires, so one would overwrite the other.
  • A percentage over 100 is refused.
  • The save check #3629951 added (a join requiring more branches than flows enter its node is refused, by the workflow and in the Complete modeler) asks the join plugin through a new FixedCountJoinInterface instead of naming the threshold and quorum joins: only a quorum count written in the settings is checked, never a percentage, a majority or a value read from a variable.

Storage: the textarea is only how the lines are typed. Config stores them as a sequence of mappings, with a schema:

collect:
  - from: vote
    into: votes
    distinct: false
  - from: department
    into: departments_seen
    distinct: true

No update hook: the project has no beta yet, so the old collect and into keys are replaced, and the shipped example_quorum workflow is updated.

Cost: none added per arrival. The join already reads each arrival's variables once to collect; several lines read the same values.

Remaining tasks

  • Settings, schema and the textarea on MergingJoinBase, with the validation above.
  • The quorum_condition, quorum_unit, quorum_value, quorum_value_variable, quorum_result and quorum_reached settings on the quorum join, the condition tested by the coordinator on each branch.
  • Gather every line in JoinCoordinator, in arrival order, with distinct; one merge write per line.
  • Update example_quorum, docs/joins-splits.md (merge policy) and the French translations.
  • Kernel tests: two lines from two branches; distinct keeping the first occurrence; the list order following the arrival order; the quorum counting the branches its condition holds on, including an any condition, its result and reached variables routing the outcome, a percentage, a majority and a value read from a variable; a form refusing a bad line and a target used twice.

User interface changes

On the merging joins, the Collect variable and Into list variable fields become one Collect textarea. The quorum join's fields become Quorum condition (a flow condition picker starting on a comparison with equals, showing the chosen condition's fields), Quorum (a number, a percentage or a majority of the branches), Quorum value and an optional Quorum value variable (both hidden for a majority), and the optional Result variable and Reached variable.

API changes

New BranchConditionJoinInterface (getBranchCondition()) and JoinResultInterface (getResultVariables()), both implemented by the quorum join; FixedCountJoinInterface (getFixedCount(), getFixedCountKey()), implemented by the threshold and quorum joins; JoinState::$variables, the arriving branch's variables; and FlowEvaluator::evaluateCondition(). The threshold and quorum joins' DEFAULT_COUNT constants become private.

Data model changes

The join settings collect (a string) and into become collect, a sequence of {from, into, distinct}. The quorum join's approve_value and count become quorum_condition (a flow condition, {plugin, settings}), quorum_unit and quorum_value, and it gains quorum_value_variable, quorum_result and quorum_reached.

AI-Generated: Yes (Claude Code was used to help draft this issue summary. I reviewed it.)

Issue fork orchestra-3629948

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mably created an issue. See original summary.

mably’s picture

Status: Active » Needs review
mably’s picture

Issue summary: View changes
mably’s picture

Issue summary: View changes
mably’s picture

Title: Let a join collect several branch variables, each into its own list, optionally without repeats » Let a join collect several branch variables into lists, and let a quorum count a condition, as a number, a percentage or a majority
Issue summary: View changes

  • mably committed b264d696 on 1.x
    feat: #3629948 Let a join collect several branch variables into lists,...
mably’s picture

Status: Needs review » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.