Skip to content

Loom Design Invariants — Technical Reference

Purpose: This document describes the non-obvious design decisions, deliberate constraints, and architectural invariants in the Loom framework. It exists because well-intentioned contributors — human or LLM — routinely propose "improvements" that would break these invariants.

Read this before proposing structural changes to Loom or any application built on it. Every section explains what the invariant is, why it exists, and how it fails if violated.

Part I — Framework Invariants

1. Worker statelessness is enforced, not optional

Workers process one task, then reset(). The reset() call is unconditional — it executes even if the task raised an exception. There is no mechanism to carry state between tasks because workers are deployed as NATS queue group replicas.

Why: If replica A processes task 1 and replica B processes task 2, any state accumulated during task 1 is invisible to replica B. Stateful workers silently diverge when horizontally scaled.

How it fails: Instance variables that persist across tasks produce correct results in single-replica testing and corrupt results in multi-replica production. The failure is silent and data-dependent — the hardest kind to diagnose.

2. The router is deterministic — no LLM in the routing path

The TaskRouter dispatches by worker_type and model_tier using rules from router_rules.yaml. It never calls an LLM.

Why: Routing must be fast (sub-millisecond), predictable, and auditable. An LLM in the routing path would add latency, cost, and non-determinism to every single task dispatch. The decomposer already chose the worker_type — the router just delivers it.

How it fails: Adding "smart routing" (e.g., LLM-based worker selection) creates a recursive dependency: the router needs an LLM call, which needs routing, which needs an LLM call. It also makes dispatch latency unpredictable and adds cost proportional to total task volume.

3. Rate limiting is dispatch-side only

The token bucket in the router tracks dispatched tasks, not completed tasks. When a task is published to a worker queue, a token is consumed immediately. The bucket does not know when (or whether) the worker finishes.

Why: True backpressure would require completion callbacks from every worker, adding a round-trip per task. The current design is a simple dispatch throttle that prevents flooding worker queues. It is explicitly not a concurrency limiter.

How it fails: If you assume the rate limiter caps concurrent in-flight tasks, you will over-provision workers. N dispatched tasks can all be in-flight simultaneously if workers are slow. The bucket only prevents new dispatches from exceeding the configured rate.

4. Config validation returns error lists, not exceptions

Every validate_* function in config.py returns list[str]. An empty list means valid. A non-empty list contains all errors found.

Why: Different callers need different error handling. The CLI aborts on the first error. The Workshop collects all errors and displays them together. Eval runs might log and continue. Exceptions force a single handling strategy.

How it fails: If validation raises exceptions, you can only ever report the first error. Users must fix one error, re-run, discover the next error, and repeat — a frustrating cycle that compound validation avoids.

5. JSON Schema validation is intentionally shallow

contracts.py validates required fields and shallow types. It does not validate nested objects, $ref, allOf/oneOf, or string format constraints. It does not use the jsonschema library.

Why: Every worker has I/O schemas. The 90% case is "does this dict have the right top-level keys with the right types?" Full JSON Schema validation would add a dependency, increase per-message overhead, and encourage schema complexity that LLMs struggle to satisfy. Shallow validation catches misconfigured workers; deep validation is the LLM's job via system prompt instructions.

How it fails: If you add complex schemas (nested required fields, conditional subschemas), the validator silently accepts invalid data. The contract is: keep schemas shallow, and this validator is sufficient.

Note on schema_ref: The input_schema_ref/output_schema_ref feature (v0.7.0) resolves Pydantic models to JSON Schema at config load time. It does not change the validation depth — the resolved schema is still validated shallowly by contracts.py. schema_ref is about where schemas are defined (Python models vs. inline YAML), not how deeply they are checked.

Critical detail: Boolean checks come before integer checks because Python's bool is a subclass of int. Without this ordering, True validates as an integer, and workers receive wrong types.

6. Dependency inference from input_mapping is the parallelism mechanism

PipelineOrchestrator parses input_mapping paths to determine which stages depend on which. A path like stages.source_process.output.claims creates a dependency on the source_process stage. Paths starting with goal.* have no inter-stage dependency. Kahn's topological sort groups independent stages into execution levels that run concurrently via asyncio.gather.

Why: Explicit depends_on annotations are error-prone and redundant — the data flow already encodes the dependency graph. Auto-inference means pipeline authors get parallelism for free when they design independent stages.

How it fails: If you modify input_mapping without understanding that it defines the execution graph, you can accidentally serialize previously parallel stages (performance regression) or parallelize stages that need sequencing (data race, missing inputs).

7. Per-goal state isolation enables concurrency without locks

OrchestratorActor stores all mutable state inside per-goal GoalState containers. There is no global mutable state, no shared counters, no locks. When max_concurrent_goals > 1, goals run concurrently with zero synchronization overhead.

Why: Locks serialize execution and create deadlock risk. Per-goal isolation means concurrent goals are as independent as separate processes, but cheaper.

How it fails: Adding any shared mutable state (even an innocent counter or metric accumulator) between goals re-introduces the need for synchronization. A single shared list or dict without a lock will corrupt under concurrent access. With a lock, you've created a serialization bottleneck that defeats the purpose of concurrent goals.

8. Malformed NATS messages are skipped, not crashed

NATSBus catches json.JSONDecodeError and UnicodeDecodeError on incoming messages, logs a warning, and continues processing the subscription.

Why: A single corrupted message must not halt an entire worker. In production with high message volume, transient corruption (network glitches, partial writes) would repeatedly crash workers if treated as fatal.

How it fails: If you change this to raise an exception, one bad message kills the subscription loop, and all subsequent valid messages go unprocessed until the worker is restarted. The bad message remains in NATS, so the worker crashes again on restart.

9. OpenTelemetry is optional via runtime feature detection

The tracing/otel.py module uses contextlib.suppress(ImportError) to conditionally import OTel SDK. If not installed, a _HAS_OTEL flag stays False and all public functions become no-ops. The module is always importable.

Why: Tracing is valuable but not required. Production code calls tracing functions unconditionally without conditional imports. This keeps instrumentation code clean while allowing bare-metal deployments without OTel.

How it fails: If you remove the suppress(ImportError), any deployment without uv sync --extra otel crashes at import time.

10. Condition evaluation: malformed → TRUE, missing path → FALSE

Pipeline stage condition evaluation has two distinct failure modes with different defaults:

  • Malformed condition (wrong format, not three tokens): defaults to TRUE (run the stage) and logs a warning. This prevents a typo in the condition syntax from silently dropping a stage.
  • Missing path (path references a context key that doesn't exist): defaults to FALSE (skip the stage) and logs a warning. This is the expected behavior for conditional stages like extract.output.needs_ocr == true — if the upstream stage didn't produce the field, the condition is not met.

Summary: malformed condition → TRUE (run), missing path → FALSE (skip).

Why the split: A structural error in the condition expression is almost certainly a mistake that should not silently skip work. A missing path, however, is the normal outcome when an upstream stage doesn't produce an optional field — skipping is the expected semantics. Both cases log a warning so typos are discoverable.

How it fails: If you change missing-path to True, every conditional stage runs unconditionally when the upstream field is absent, which defeats the purpose of conditions. If you change malformed to False, a syntax typo silently removes a stage from execution.

11. ProcessorWorker serialize_writes is per-instance only

SyncProcessingBackend with serialize_writes=True uses an asyncio.Lock to serialize calls within a single process. It does NOT protect against concurrent writes from multiple instances.

Why: Cross-process locking requires external coordination (file locks, Valkey locks) which adds infrastructure dependencies. The design contract is: run exactly one processor instance for single-writer backends like DuckDB.

How it fails: Running two processor instances with serialize_writes=True against the same DuckDB file causes database corruption. The per-instance lock is useless — each instance has its own lock.

12. Dead-letter store is bounded with FIFO eviction

DeadLetterConsumer stores at most max_size entries (default 1000), discarding oldest entries when full. Entries are inserted most-recent-first.

Why: In a system producing many errors, unbounded dead-letter storage becomes a memory leak. Operators care about recent failures; ancient failures are diagnosable from logs.

How it fails: Removing the size limit turns dead-letter storage into runaway memory consumption. Under sustained error conditions (e.g., a misconfigured worker), the store grows indefinitely until OOM.

13. Path traversal protection uses resolved absolute paths

WorkspaceManager canonicalizes both the workspace root and the requested file path with .resolve() before comparing. This catches ../ traversal and symlink escapes.

Why: Workers can resolve file:// references in their payloads. Without canonicalization, ../../etc/passwd or a symlink pointing outside the workspace would grant arbitrary filesystem access.

How it fails: Removing .resolve() allows symlinks that point outside the workspace to be read. Comparing un-resolved paths allows ../ traversal.

14. InMemoryBus exists for testing, not as a feature

InMemoryBus is a synchronous in-process message bus with no network dependency. It exists so that the full test suite runs without NATS.

Why: Tests must be fast and infrastructure-free. InMemoryBus has the same interface as NATSBus but delivers messages within the process.

How it fails: If someone uses InMemoryBus in production, they lose: queue group load balancing, multi-process scaling, persistence, and failure isolation. Everything runs in one process with one failure domain.

15. ResultStream is single-use and subscription-scoped

ResultStream owns a bus subscription for its lifetime. It can be iterated exactly once — calling collect_all() or async for a second time raises RuntimeError. This is not a limitation; it prevents the subtle bug where two consumers compete for messages from the same subscription.

Why: NATS subscriptions are stateful — messages are consumed destructively. If two iterators shared a subscription, each would see a random subset of results. Single-use enforcement makes this impossible.

How it fails: Allowing reuse would produce "missing result" bugs that only manifest under concurrent load (when the second iteration races the first).

16. Pipeline parallel levels use FIRST_COMPLETED, not gather

Within a parallel level, PipelineOrchestrator uses asyncio.wait(FIRST_COMPLETED) in a loop rather than asyncio.gather. This enables incremental progress reporting — each stage's result is logged and stored in context as soon as it completes, rather than waiting for the entire level.

Why: In baft's audit pipeline (LA, PA, RT parallel), the slowest auditor previously blocked progress reporting for all three. With FIRST_COMPLETED, the Workshop and MCP bridge see each stage complete in real time.

How it fails: Using gather is functionally correct but observationally opaque — all three stages appear to complete simultaneously at the moment the slowest one finishes. The latency is the same; only the observability differs.

Part II — Application Design Patterns

These are architectural patterns that Loom applications should follow when building pipelines with epistemic constraints, blind audits, or information barriers. They are not Loom framework code — they are design principles that emerge from how the framework is meant to be used.

17. Knowledge silo isolation is an epistemic quarantine, not a convenience grouping

When an application uses knowledge silos to implement information barriers (e.g., between analytical workers and audit workers), silos marked as isolated enforce epistemic quarantine. This is what makes blind audits actually blind.

Why: If audit nodes can access the analytical framework they are supposed to evaluate, they will pattern-match to existing conclusions and produce pseudo-confirmatory "independent" judgments. The audit becomes epistemically worthless — it tells you what you already believe, not whether what you believe is correct.

How it fails: Adding domain knowledge to any audit node's knowledge_sources breaks audit independence. The failure is invisible: audits still produce professional-looking output, but their conclusions are contaminated by the framework they were supposed to evaluate. There is no runtime error, no warning, no indication that the audit is compromised.

The trap: It is tempting to "help" audit nodes by giving them more context so they can be more informed. This is precisely the wrong thing to do. Blind auditors must be knowledge-deprived by design.

18. Neutralization stages are audit firewalls, not text processors

When a pipeline implements blind auditing, a neutralization stage strips domain-specific vocabulary before blind auditors see the text. The neutralizer must have minimal knowledge — only the vocabulary mapping and procedural rules.

Why: If the neutralizer receives domain knowledge, it leaks domain-specific framing into the "neutral" text. Audit nodes then receive text that, while superficially generic, carries the structural fingerprint of domain conclusions.

How it fails:

  • Adding entity registries to the neutralizer shifts it from lexical to semantic processing — a different epistemic role.
  • Opaque identifiers (entity codes, reference IDs) must pass through unchanged. The neutralizer transforms vocabulary, not references.
  • Removing the neutralizer entirely and sending raw text to auditors exposes every domain-specific term as a vector for framework contamination.

19. Blind auditors should have tiered knowledge deprivation

Not all audit nodes in a blind audit pipeline should be equally blind. Different audit functions require different levels of knowledge deprivation:

  • Adversarial challengers should be maximally blind — only procedural rules. Giving them evaluation rubrics shifts them from adversarial challenge to structured critique, which is a different cognitive function.
  • Structured auditors (logic, perspective, methodology) need rubrics to know what to evaluate without knowing what the domain framework says.
  • Synthesis nodes need audit outputs and decision logs to detect blind spots, but must not see the domain framework to avoid contamination.

How it fails: Adding any knowledge source to an adversarial challenger makes it less adversarial. Adding domain content to structured auditors makes them confirmatory. Adding domain content to synthesis nodes makes blind-spot detection useless (it validates decisions against the framework that produced them).

20. Neutralization maps must be computed per-run, not pre-cached

When a neutralizer produces a reverse map (neutral term -> original domain term), that map must be computed per document, not pre-cached globally.

Why: Terminology usage varies by document. A document about one topic uses different domain terms than a document about another. A global reverse map produces incorrect de-neutralization.

How it fails: Pre-computing a global reverse map produces wrong results for documents that don't use all terms. Worse, it maps neutral terms back to wrong domain terms when there are many-to-one mappings.

21. Quality gates should be content-driven, not intent-driven

When a pipeline uses flags to trigger escalation (e.g., routing to a more expensive audit tier), the flag should be set based on the analytical worker's assessment of content quality, not based on whether the user requested escalation.

Why: Users may not recognize when their work has crossed a quality threshold that warrants peer review. The flag is a content-quality signal, not a workflow button.

How it fails: If the flag is set based on user intent ("publish this"), users can bypass the audit pipeline by not requesting escalation. Conversely, important analytical shifts skip audit because the user didn't ask for review.

22. Escalation thresholds gate expensive operations

When an adversarial challenge node produces a high-strength challenge that triggers escalation (e.g., to manual review with an alternate LLM provider), the threshold must be calibrated carefully.

Why: Escalation targets are expensive (different provider, human review, longer cycles). The threshold must ensure genuine threats escalate while routine challenges don't.

How it fails: Too low: false-positive escalations waste resources and erode trust ("the system always escalates"). Too high: genuine analytical failures pass through to publication.

23. Context flows through messages, not worker state

Session identifiers, request context, and inter-stage metadata must flow through input_mapping template references, not through worker instance state.

Why: Workers are stateless (invariant #1). They cannot track sessions internally. The pipeline's input_mapping is the mechanism for passing context through a stateless execution chain.

How it fails:

  • If an intermediate worker filters context fields from its output, downstream workers lose access and cross-cutting concerns (governance audits, session tracking) break silently.
  • If context is added to worker instance state instead of flowing through messages, multi-replica deployments lose context tracking.

24. Behavioral monitors must be isolated from analytical content

When a pipeline includes a behavioral monitoring worker (e.g., monitoring analyst fatigue, tunnel vision, or cognitive bias), that worker must NOT have access to the analytical framework or domain database.

Why: If a behavioral monitor sees domain content, it evaluates whether the user's analysis is correct rather than whether their behavior is healthy. It becomes a second analytical worker with worse prompting instead of a cognitive monitor.

How it fails: Adding domain content to a behavioral monitor means it flags analytical disagreements as behavioral anomalies. "User spent 40 minutes on entity X" gets flagged as tunnel vision even if entity X genuinely requires deep analysis.

25. Universal silos must never contain domain-specific analytical content

If a silo is shared across all workers — including blind auditors — it must contain only procedural and epistemic discipline rules (source evaluation standards, neutrality requirements, anti-bias framing). Never analytical conclusions, domain assessments, or entity evaluations.

Why: If a universal silo contains analytical content, blind auditors receive domain context through the one channel that bypasses all isolation checks.

How it fails: Someone adds "current high-priority findings" to a universal silo as "standing guidance." Now every blind auditor knows what the framework considers important. Audit independence is destroyed through the one silo everyone trusts.

26. There is no "improve the audit by giving auditors more information"

This is the most frequently proposed and most damaging class of "improvement" to blind audit pipelines. The information asymmetry between sighted and blind nodes is the mechanism, not the bug.

The principle: Audit quality comes from independence, not from information. A well-informed auditor who has read the conclusions will confirm them. A blind auditor who has not read the conclusions will challenge them on logical and perspectival grounds. Both are necessary. Merging them destroys the one you can't get any other way.

27. YAML configs are the right medium for what they describe

Loom application configs are typically 80% natural-language system prompts and 20% structural configuration (schemas, mappings, silo assignments). A Python DSL would help with the structural 20% but would make the prompt 80% harder to read and edit.

The right approach: Generate the structural parts (I/O schemas, silo references, pipeline topology) from typed Python models. Keep the natural language in YAML where it's readable without a Python interpreter.

How it fails: Replacing all YAML with Python forces system prompts into Python strings (escaping hell, no syntax highlighting, harder to diff). It also removes the ability for non-developer domain experts to review and suggest changes to worker behavior.

28. Single-writer processors must be truly single-instance

When a processor worker uses serialize_writes=True for a single-writer store (DuckDB, file-based databases), the application must ensure exactly one instance runs. The asyncio lock only serializes within one process (invariant #11).

Additionally, no other worker should bypass the designated writer to access the store directly. The writer typically enforces validation, cross-referencing, and governance triggers that direct access would skip.

How it fails: Running two instances causes write races. Bypassing the writer skips validation and governance triggers (e.g., operation count thresholds that fire audit escalations).

Summary — Red Lines

These are things that must never happen, regardless of how reasonable they sound:

  1. Never add domain content to blind auditor knowledge sources. Not even "just the entity names" or "just the high-level findings." Any domain leakage contaminates the audit.

  2. Never put LLM calls in the router. Routing is deterministic and fast. Smart routing belongs in the decomposer.

  3. Never carry state between worker tasks. If you need context, pass it through messages. Workers are stateless replicas.

  4. Never skip contract validation. It's the only type-safe boundary between actors. Removing it for "performance" removes the only safety net.

  5. Never put analytical content in universal silos. They are the one channel that reaches every node, including blind auditors.

  6. Never pre-compute neutralization reverse maps. They must be per-document.

  7. Never change condition-evaluation defaults from TRUE. Silent stage skips are worse than unnecessary stage runs.

  8. Never run multiple instances of a single-writer processor. The per-instance lock does not protect across processes.