config.triggers and are authored declaratively
alongside the rest of the graph.
The load-bearing idea is identity: an unattended run that touches an external
system authorizes as a service principal you bind to it — never as you, and
never as an anonymous system actor. The authoring surface enforces this, so an
unsafe automation is rejected when you write it, not at 06:00 when it fires.
The shape
Every trigger shares a small envelope and carries exactly one per-kind block:kind—cron,change, orthreshold. Exactly the matching per-kind block must be present.enabled— omit ortrueto arm the trigger; only an explicitfalsedisables it (a disabled trigger is inert but preserved, handy for rollback).runs_as— the service principal the fired run authorizes as. Required for any unattended side effect (see Firing identity).acknowledged— your explicit sign-off that an auto-firing writeback is intended (see Acknowledgement).id— server-minted and content-addressed; leave it unset when authoring. It stays stable across re-saves and only changes when the trigger’s definition changes.
Cron — fire on a schedule
expression— a standard 5-field cron (minute hour day-of-month month day-of-week).timezone— an IANA name (e.g.Europe/Dublin); omit for UTC.catchup(on the envelope, cron-only) —none(default) orone: run a single catch-up pass for fire windows the scheduler missed while it was down.
Change — fire when upstream data moves
upstream_node_ids— the upstream nodes to watch; an empty list watches all direct upstreams.debounce_ms— coalesce rapid upstream changes into a single fire within this window.predicate— optional. A serialized boolean expression (same shape as a threshold predicate) that gates whether a change actually fires; omit to fire on any change.
runs_as principal.
Threshold — fire a writeback when a predicate holds
A threshold trigger sits on the node whose output it watches and fires a named writeback only when a predicate over that output is true:fires_writeback_node_id— the writeback this threshold invokes. Required, and must resolve to a writeback node.predicate— required. A boolean expression over thetrigger.thresholdscope, which exposes the watched output’s summary stats:artifact.row_count, and per columnartifact.<col>__null_count,artifact.<col>__value_count,artifact.<col>__min,artifact.<col>__max. Example source strings:artifact.row_count = 0,artifact.amount__null_count > 10. It’s authored as an ExpressionSpec; the compiler pins it to a canonical AST hash, so author it through the tooling rather than hand-writing the hash.cooldown_ms— suppress re-firing within this window after a fire.input_mappings— optional map of writeback input name → a template expression resolved from the firing context.
core_threshold_dry_run
MCP tool. It reports would_fire — the verdict — without firing anything, and
without writing anything: it is gated at Project#view_graph, so every probe it
makes is read-only, including the authority probes below.
matched alone is not that verdict: a predicate can hold, and the runs_as
identity can resolve, while the fire is still refused because that identity
holds none of the grants the writeback’s own gates check. So the report also
carries:
required_grants— every authority a fire needs, one entry per{securable, privilege, gate, held, advisory, undetermined, pending_hierarchy, reason, remedy}.gatenames the enforcement site, so you can check the claim rather than take it.missing_grants— the blocking subset: definite denials only. Each entry’sremedyis thecore_edit_scopecall that fixes it, already addressed at the right securable, which is also how you learn the pipeline securable id when a writeback’srequires_permissionbinds to one.arm—{armed, reason, remedy}, and not a grant. A writeback node is born paused (pause_reason: awaiting_arm) because authoring a destination never arms delivery; while it is held the fire is refused before any gate runs, soarmed: falsesuppresseswould_fireon its own, with every grant in place. See Arming below.
Each entry’s
securable is spelled the way the grant surface takes it, so the
remedy is a call you can follow. For writeback_source_read that is
DucklakeTable:<schema.table> — the qualified table the bound producer port
writes. The gate keys its own Keto question on that table’s internal DuckLake
catalog id; the report does not publish the id, because nothing on the surface
consumes it and the grant it would address is refused anyway (PAN-4500, below).
writeback_source_read is the one most often missing, because it is the only
one that is not satisfied by a USE grant. DucklakeTable#read traverses
project.permits.read, so an identity granted only USE passes the floor and
is then refused with “writeback invocation denied: invoker cannot read the
bound source table”. Grant the runs_as principal READ on the project —
the traversal is what satisfies the gate. A narrower grant directly on the
ducklake_table securable is refused today even for the project owner
(PAN-4500), so the project grant is the route to use.
run_nodes is the other one a project grant does not satisfy, and it is the
last gate in the chain rather than the first. A threshold fire’s run cause is
pipeline_run, so the worker rechecks Pipeline#run_nodes on the runs_as
principal immediately before the delivery run executes — after the fire was
accepted, the invocation recorded and the run dispatched. Without it the
delivery fails with “permission revoked between dispatch and execute”, which
is deliberately generic — a deny on that path must not leak secret material — so
the run error does not name the grant you are missing. Grant RUN on the
pipeline securable the report’s remedy names; RUN on the project is a
different grant and does not cover it.
An entry marked advisory: true is a documented requirement that no gate on
this particular fire path rechecks. This is now rare: the dispatch-time secret
gate enumerates a writeback’s credentials for every kind, so a database or HTTP
writeback’s stored credential is blocking. An advisory requirement sits
outside every gate’s own enumeration — grant it anyway (the delivery fails at
the destination without it), but it does not hold would_fire down.
An entry marked undetermined: true is one the dry-run could not decide at all —
no authorization checker, no project session, a failed check — and it is reported
rather than guessed at. It carries no grant advice, because the grant may already
exist, and it never holds would_fire down: calling an unverifiable requirement
blocking would refuse an alert that does in fact fire. Every probe applies that
one rule, and a definite denial still suppresses the verdict — including a grant
suppressed by an unapplied revocation in keto_sync_outbox, which refuses a live
fire exactly as it refused the probe. The undetermined cause you can act on is
pending_hierarchy: true on
writeback_source_read: the direct table read was denied only because the
structural table→project tuple the gate traverses has not been written yet. A
real run writes that tuple and re-checks; the dry-run is read-only and
deliberately does not, so it cannot run that re-check. Re-run the dry-run after
the node next runs — and if the runs_as identity already holds READ on the
project, no grant is needed.
would_fire: false with an empty missing_grants means one of four things, and
the report says which: the writeback is not armed (arm.armed: false — check
this first on a new alert), the predicate did not match, the cooldown is
suppressing, or evaluated is false because the node has no live output yet.
Arming
A writeback node is born paused.pause_reason is unconfigured_target
while it has no delivery target, and becomes awaiting_arm once the target is
valid — valid target, still held. Arming is the resume:
- While the node is held,
core_invoke_writebackand every trigger fire are refused with “writeback node … is paused and cannot be invoked” — before any authority gate runs. No grant fixes it, and the gates in the table above are never reached. paused: falseis refused whilepause_reasonisunconfigured_target: there is nothing to deliver to, and answering the request with success would be a lie. Configure the target first; the latch then relabels itselfawaiting_armfor a human to arm.
core_threshold_dry_run reports this as arm, with the exact call as its
remedy.
Firing identity
A credential plus an unattended trigger equals unattended external access, and must name a durable principal.A scheduled or reactive run authorizes as
system:scheduler, not as you.
Your own grants never apply to it. That is deliberate: the run outlives your
authoring session, you may leave, and your grants can change without anyone
touching the pipeline. So any unattended run that reaches an external system must
name a service principal — a durable, grantable identity — via the trigger’s
runs_as (or the node’s config.runs_as as a fallback).
Binding one is a three-step path an agent or the API can drive end to end. Every
step below is a real MCP call; follow them in order.
1. Create a service principal. Needs the operator scope (tick it on the
consent screen).
created_ids[]; it is also listed by
core_read_scope tenant.dsl/service_principals. Do not issue a credential —
a trigger fire authorizes as the principal server-side and needs none.
2. Grant it what the run needs. Each grant is one additive grant op on the
securable’s own permissions path. It carries no document, so you never have to
re-send (or invent) the document’s owner. You must hold MANAGE_GRANTS on the
securable and the privilege you are handing out.
Find the secret’s key with core_list_secrets — the listing returns it as
key (and, for compatibility, also as name):
USE,
plus whatever the writeback node’s own requires_permission names:
RUN on the pipeline, which is a separate grant and is needed for
every fired delivery — not only when the writeback’s requires_permission
happens to name a pipeline:
Pipeline#run_nodes on the runs_as principal immediately before executing it.
RUN on the project does not satisfy that check, and the resulting failure
names no grant (see run_nodes above). This is the grant most often missing
after everything else is in place, because every other authority in the chain is
satisfied project-side.
You do not have to guess any of them, or the pipeline’s securable id: run
core_threshold_dry_run and read missing_grants — every entry names its
securable, its privilege, and the exact call that grants it.
Removing a grant is deliberately not an op. Read the document with
core_read_scope, then use core_edit_scope with a non-empty old on the same
path, so the removal is checked against the document you actually read.
3. Set runs_as on the trigger (act-as-gated at bind time, so you can only
bind a principal you’re allowed to act as), arm the writeback node (see
Arming — a newly authored one is paused, and no grant lifts that),
then dry-run it and confirm would_fire: true.
A run that dispatches with no resolvable identity is refused loudly
(TRIGGER_IDENTITY_UNRESOLVED) rather than silently downgraded to a system
actor.
Acknowledgement
Identity says who fires; acknowledgement says you meant it to. An auto-firing (cron or change) trigger on a writeback node must setacknowledged: true —
an explicit confirmation that you intend an unattended external side effect.
Without it the write is rejected at authoring time.
What the authoring surface enforces
Three rules reject an unsafe automation the moment you write it, not when it fires:
A resolvable identity is not a granted one: these three rules check that a
principal is named, never that it holds anything. What it holds is checked at
fire time, which is why
core_threshold_dry_run’s missing_grants is the
authoring-time signal to look at before you arm anything.
The last rule is why a scheduled Python node that calls to_sql with a stored
secret can’t ship without an identity: a credential plus a schedule is
unattended external access. The rejection names the remedy directly — “a
scheduled run authorizes as system:scheduler, not as you — your own grants do
not apply.”
Related
- Database writebacks — the governed delivery a trigger fires.
- Outcomes and notifications — the derived health, freshness and assertion rules that watch what your triggers produce, and where to be told when one needs you.