RFC-016: Collection Operations

AcceptedImplemented
Depends on
RFC-004 (Uniform Operation Syntax), RFC-007 (Cross-Law Execution Model), RFC-012 (Untranslatables), RFC-013 (Execution Provenance), RFC-023 (Quantities in Law YAML (Money, Percentages, and Dimensioned Values))

Context

Dutch law reasons about groups of people and things whose size is not known in advance: the medebewoners in a household, the children a benefit is calculated for, the registrations in a register, the events in a case file. The schema has no way to iterate over such a group, so every law that needs one is either unmodeled or flattened until it fits.

This is not a hypothetical gap. It shows up in four places in this repository, each independently.

The engine asked for it by name. Until this RFC, corpus/regulation/nl/wet/test_untranslatables/2025-01-01.yaml carried an accepted untranslatable whose suggestion field was the title of this one:

- construct: som van alle deeltoeslagen reason: Aggregation over variable-length collections is not supported suggestion: Add SUM/FOREACH operation to engine accepted: true

The article it belonged to output a hardcoded 0. Four lines above it in the same file sat the untranslatable for statutory rounding, which was the seed of RFC-024 and is now the ROUND operation. The aggregation one was still open, and this RFC closes it: that entry now records an open norm, which is what RFC-012 is actually for.

A count has been flattened to a yes-or-no. The Participatiewet declares this input for the kostendelersnorm:

- name: heeft_kostendelende_medebewoners type: boolean required: false description: 'Heeft kostendelende medebewoners (bron: BRP)'

Article 22a lid 1, quoted verbatim in the same file, does not ask whether there are any. It computes with how many:

Indien de belanghebbende van 21 jaar of ouder een of meer kostendelende medebewoners heeft, is de norm per kalendermaand voor de belanghebbende: Hierbij staat: A voor het aantal kostendelende medebewoners plus de belanghebbende en zijn echtgenoot van 21 jaar of ouder indien hij gehuwd is.

A is a number, and the boolean cannot carry it, so the norm cannot be computed at all and article 22a has no machine_readable section as a result. Whoever supplies that boolean also has to decide what a kostendelende medebewoner is before the law gets a say, and the two limits of 21 in the same sentence make that easy to get wrong: they govern the belanghebbende and the echtgenoot, not the medebewoners. A reading like that belongs in the law, where a jurist can check it against the text, and not in the query that fetches the household.

Articles are quoted and then skipped. AWIR article 7 lid 2 brings the toetsingsinkomen of the medebewoners into the draagkracht, and lid 5 exempts the first €4.100 for a medebewoner who is a first-degree relative under 23. The article sits in the corpus with its full text and no machine_readable. Article 8 beside it is modeled, and is a two-term ADD over the applicant alone. Lid 5 is a filter, a per-element subtraction, and a sum. There is no scalar to pre-aggregate it into that does not also carry the age limit and the relationship test.

Even two people is too many. The zorgtoeslag model carries this comment:

# Art. 2 lid 2: "wordt het gezamenlijk toetsingsinkomen van de verzekerde en # de toeslagpartner in aanmerking genomen." # TODO #377: Bij partners moet het gezamenlijke toetsingsinkomen worden gebruikt.

Household aggregation fails at N=2.

This is a restoration

FOREACH was in the schema. It appears in the operationType enum of v0.2.0 through v0.4.0 and was dropped in v0.5.0, when the operation set was cut back. The corpus has grown into the gap since.

What was never removed is the machinery. RuleContext still carries a local scope, set_local(), and create_child(), and the module documentation still explains all three as existing for FOREACH loops. Value::Array and Value::Object are variants of the value type. MAX_ARRAY_SIZE is set to 1000. The schema still accepts type: array and type: object on an input field, so a law may declare a collection today and then find no operation that will read it.

Two published pages already describe the capability as present: the engine component page lists FOREACH as the source of local-scope variables, and the conformance reference puts collection operations in the core level. This RFC makes those true.

Decision

Add a FOREACH operation. It iterates over a collection, evaluates an expression once per element with that element bound to a named variable, and optionally reduces the results to a single value.

Syntax

operation: FOREACH collection: $kinderen # expression evaluating to an array as: kind # local variable name, defaults to "item" filter: # optional: skip elements where this is false operation: GREATER_THAN_OR_EQUAL subject: $kind.leeftijd value: 12 body: $kind.bedrag # evaluated once per surviving element combine: ADD # optional aggregation

Counting, with body: 1:

operation: FOREACH collection: $medebewoners as: medebewoner body: 1 combine: ADD

That is the kostendelersnorm case, and it needs no filter: article 22a counts every kostendelende medebewoner. The two age limits of 21 in its first paragraph apply to the belanghebbende and to the echtgenoot, not to the medebewoners, so they are ordinary conditions elsewhere in the article rather than a filter here. Getting that wrong is easy, which is the point: the reading belongs in the law, where a jurist can check it, and not in the query that fetches the household.

Property names

FOREACH introduces operands no other operation has. The names are chosen so that none of them collides with an existing meaning (RFC-004).

PropertyWhy this name
collectionDistinct from subject (comparisons) and values (arithmetic)
asIteration binding, familiar from SQL and template languages
filterDistinct from conditions (AND/OR); says what it does
bodyDistinct from value (comparison target, action assignment)
combineSays what it does: combine per-element results into one

These names are the whole vocabulary. Earlier drafts of this RFC used subject, value and where, and the engine could accept those as aliases, but no law was ever written with them: in v0.2.0 through v0.4.0 FOREACH was a catch-all with additionalProperties: true and no fixed field names at all. An alias that the schema rejects would only let the engine execute a document that just validate refuses, which is exactly the soundness gap the conformance suite exists to close.

Semantics

  1. Evaluate collection in the scope where the FOREACH appears. An array iterates as-is; null is an empty collection; any other value iterates as a single-element collection.
  2. For each element: create a child context with an empty local scope, bind the element to as, evaluate filter if present, and evaluate body.
  3. Apply combine to the collected results, or return them as an array when combine is omitted.

Scope

FOREACH is the only operation that defines a variable rather than reading one. The binding lives in the child context and is gone when the iteration ends. It shadows an outer variable of the same name.

Nesting follows from step 1. collection is evaluated where its FOREACH is written, before the child context exists, so an inner collection can see the outer binding. The inner body cannot: it runs in a child context whose local scope starts empty.

operation: FOREACH collection: $huishoudens as: huishouden body: operation: FOREACH collection: $huishouden.leden # outer scope: $huishouden is bound here as: lid body: $lid.inkomen # inner scope: $lid only combine: ADD combine: ADD

Passing an outer variable into an inner body means routing it through collection. There is no other path across that boundary, and that is deliberate: a child context that inherited its parent’s locals would let one iteration’s variable leak into the next.

Elements that are objects also have their fields injected as locals, so $status works where $reg.status does. Existing YAML uses that form. Dot notation is clearer and is what new laws should use, because a flattened field name can shadow something in an outer scope.

Combine

combineResultEmpty collection
ADDSum; concatenates strings and flattens arrays per RFC-0070
ORTrue if any result is truthyfalse
ANDTrue if every result is truthytrue
MINLowest valuenull
MAXHighest valuenull
omittedThe results as an array[]

These five are the aggregations that Dutch legal text actually performs: het totaal van, indien ten minste een van, indien aan alle, het laagste of hoogste van. SUBTRACT, MULTIPLY, and DIVIDE are excluded because they do not associate over a collection in any way a law asks for. Subtracting a list leaves open what it is subtracted from. A law needing such an aggregation can omit combine and apply the arithmetic to the resulting array.

MIN and MAX return null on an empty collection because there is no lowest value of nothing, and the caller has to handle that. AND returning true is vacuous truth: a law that must not read “no items” as “all conditions met” has to check the collection is non-empty itself.

ADD on an empty collection returns integer 0 whatever the body would have produced, because no body runs and the type cannot be inferred. A law expecting a string or an array from a possibly-empty collection should guard with an IF.

Unknown and untranslatable

An error in body or filter aborts the whole operation. Partial results are not returned: a sum over some of the children is not a legal determination.

Value::Untranslatable (RFC-012) taints the whole result, from collection, from filter, or from any element’s body. Dropping the untranslatable elements and combining the rest would produce a number that looks complete and is not.

A filter that evaluates to null propagates null. The engine cannot tell whether the element belongs in the collection, so it cannot tell what the total is. An element whose filter is definitively false is skipped and contributes nothing.

A body that evaluates to null is the same situation one step later: the element is definitely in the collection and its contribution is not known. ADD, MIN and MAX therefore return null rather than combining the rest, which would report a confident total that is short by an unknown amount. OR and AND follow the standalone operations: a definitive true (for OR) or false (for AND) settles the answer whatever the unknown turns out to be, and otherwise the result is null.

An empty collection is not the same case. Nothing is missing there, so each combine returns its identity per the table above.

Units

Under RFC-023, a FOREACH carries the unit of its body. ADD, MIN, and MAX preserve it, the way IF preserves the shared unit of its branches. AND and OR produce a boolean and therefore no unit. Without combine the result is an array, which carries no unit. collection and filter are checked for internal consistency and do not contribute one.

Bounds

A collection longer than MAX_ARRAY_SIZE (1000) is an error. filter and body are evaluated one level deeper than the FOREACH itself, so nesting is bounded by the shared MAX_OPERATION_DEPTH (100); collection is evaluated at the FOREACH’s own depth, because it belongs to the scope the FOREACH appears in. Collections come from finite data sources; the schema has no generators.

Note that the two bounds multiply. A FOREACH nested in a FOREACH, each over a thousand elements, is a million body evaluations, and both limits are satisfied.

Trace

Each iteration is a node in the execution trace (RFC-013), carrying the element index and the value the body produced, or the element itself when the filter skipped it. The FOREACH node above them reports how many elements were evaluated and how many were skipped. A reduction that shows only its total is not an explanation of how it was reached.

Why

Benefits

The legal test stays in the law. A register supplies the people and their attributes; the law decides who counts and on what condition. Nobody has to encode a threshold into a query to make an article computable, and when the reading is contested it is contested against the wettekst rather than against a data pipeline.

The YAML matches the sentence. Legal text says voor elk kind dat 12 jaar of ouder is in one clause, combining selection, transformation, and totalling. FOREACH with filter and combine is that clause. Separate MAP, FILTER, and REDUCE operations would force intermediate outputs that appear nowhere in the law.

The engine is already built for it. Child contexts, local bindings, and scoped resolution have been in RuleContext since before v0.5.0, unused. Only the dispatch was missing.

Tradeoffs

Variable binding is new. Every other operation reads variables and none defines one, so as introduces a definition point and, with it, shadowing and scope rules that the rest of the language does not have.

Pre-aggregation remains simpler when it works. Where the pattern really is “count the items in fixed categories” and no legal condition is involved, a counted input is less machinery. FOREACH earns its place when the per-element logic is a legal test, or when the result is a collection rather than a total.

Nesting can run long. The bounds cap it, but a nested FOREACH over two collections of a few hundred elements is real work in an engine that is otherwise close to constant-time per article.

Alternatives considered

Pre-aggregate in the data source. Works for simple sums, and it is what the corpus does today. Rejected because it moves age limits and status definitions out of the law, which is the coupling this project exists to remove. The kostendelersnorm boolean is what it looks like in practice.

A fixed maximum with unrolled operations. Generate branches for up to N items. Rejected: the limit is arbitrary, the YAML is unreadable, and it breaks on the first household larger than N.

Separate MAP, FILTER, and REDUCE. Conventional in functional languages. Rejected because Dutch legal text does not decompose that way. De som van de bedragen voor elk kind dat 12 jaar of ouder is is one clause, and modeling it as three operations invents two named intermediates the law never mentions.

Leave it as an untranslatable. Defensible for a genuinely open norm, and RFC-012 exists for that. But an aggregation is not an open norm. It is arithmetic the engine can do, and recording it as untranslatable while the scaffolding sits unused is a decision to keep laws unmodeled that need not be.

Implementation notes

  • ActionOperation::Foreach in packages/law-model/src/model.rs, with CombineOp as a typed enum so an invalid combine fails at deserialization rather than at execution.
  • execute_foreach in packages/engine/src/operations.rs, reached through a resolver method that only RuleContext implements, because a simple resolver has no child scopes to create.
  • FOREACH is nested-only. Like IF and LIST it is in the operationType enum but rejected at action level, where the fixed field set has nowhere to put collection or as.
  • Schema v0.5.7 adds foreachOperation to definitions and to the operation union. Released schema versions are immutable, so this is a new directory.

References

RegelRecht

An exploration by Bureau Architectuur of the Dutch Ministry of Economic Affairs and Climate Policy into the possibilities of transparent, executable legislation.

Links

GitHub repository
How it works
Stay informed
Documentation
Research

Contact

regelrecht@minbzk.nl

Part of

Bureau Architectuur
Ministry of Economic Affairs and Climate Policy