When someone asks me what Orazaka is, technically, I often answer with a list of repositories. It is a bad answer. A list does not say what we are building, or why.
On 26 October, I had the architecture reviewed by an outside audit: a language model, given the context. It suggested a hybrid architecture, keeping some parts and replacing others with well-known frameworks. Some of it was right. Some of it described as existing things we have not built. We checked it against the code, line by line, then a counter-review tightened what was left. The outcome is one decision, ADR-092, and one sentence:
We own the core, we integrate the edges.
This chapter explains that sentence. It also says, without rounding, where we are today.
What we own, what we plug in
An AI engine is a lot of parts. Most of them already exist elsewhere, and are well made: a protocol to talk to tools, a message broker, libraries to call a model. Rewriting them would teach us nothing and cost us years. But a few parts are what make Orazaka Orazaka. Those we keep.
What we own:
- The orchestration. A request becomes a graph of typed steps. Each step has an output contract (what it must return) and a control policy (how we judge what it returned).
- The control loop. What decides whether an output is good, and what to do when it is not.
- The GPU lease. Who may use a machine's graphics card, and for how long.
- Live steering. Changing a step while a run moves on, without redoing everything.
- Access. Who may read what, checked when the assistant searches and when it calls a tool (ADR-091).
What we plug in as it is:
- MCP, the tools protocol, through the official SDK, by way of Spring AI. We do not implement the protocol.
- RabbitMQ and the outbox, so that work leaves reliably, in the same transaction as the run's state.
- The Python media worker, for images and video. It stays in Python. It does not become Java.
In between sits Spring AI 2, the library that talks to model providers. We use it, but behind our ports: one adapter module knows it. The domain itself must be plain Java 21.
What we refuse: LangChain4j, LangGraph4j, a second workflow engine, and StructuredTaskScope while it is a preview API. The reason is the same each time: two ways of doing one thing means two upgrade trains and two places where a run's state can drift apart. We already paid for the move to Spring AI 2. We already have a graph of steps, with its saga, its approvals and its credit holds. We do not want a second one.
We own
- The orchestration: a graph of typed steps
- The control loop
- The GPU lease
- Live steering
- Access, checked at retrieval and at the tool
We plug in
- MCP, through the official SDK
- RabbitMQ and the outbox
- The Python media worker
- Spring AI 2, behind our ports
We refuse
- LangChain4j
- LangGraph4j
- A second workflow engine
- StructuredTaskScope while it is a preview
One request, end to end
The orazaka workspace brings together 20 repositories: reusable Krizaka building blocks (users, notifications, billing), the foundations (contracts, edge), the AI engine, the clients (web, mobile, command line), the media worker and the packs.
A correction first, because I used to say it wrong. The "AI engine" is not a service. It is a library, orazaka-ai-engine, that the services embed: the AI client, the interceptor chain, the tools. The graph of steps lives in the Studio service.
Here is the path of a request, in order.
- The client (web, mobile or a product)The web app goes through its backend-for-frontend (BFF), mobile through a token, a product such as Yomizaka through a service key. Nobody calls a service directly.
- The edge (the single door)It exchanges an API key for a session, applies CORS and tracing, and sends the route to the right service.
- The Studio (the graph of steps)It holds the packs, the installations and the runs. It moves a Studio's plan forward step by step, with compensation when a step fails.
- RabbitMQ and the outbox (interactive or batch)Work is written in the same transaction as the run's state, then published to a queue. A database row says which one.
- The job service and the worker (code or a GPU)It drains the queue and hands the work to the executor the same row names: code inside the service, or the media worker.
- Billing (hold, settle, release)Every run held its credits before it started. At the end, we settle what ran and release the rest.
Routing lives in the database. A capability is a row: routing_key says where the work goes, handler_key says which code runs it. Before, the code guessed the queue with a chain of contains() on a name, and an unknown capability went silently to the wrong queue. Today, a governance test stops the build if a row has no executor.
The control loop
This is the part that matters most to me, and the one with the most work ahead of it.
The idea is simple. A model produces an output. Before we hand it to anyone, we judge it. And we do not ask a model what a rule can check. Here is the intended order, for every step:
- Deterministic checks (first, no model)The schema holds, the file exists and can be read, the duration and the proportions are right. No token. Target: under 10 ms.
- A semantic judge (a drift score)A model reads the output and says how far it strays from what was asked. It never runs on an output the checks rejected.
- A correction (at most K tries)Above the threshold, we regenerate, telling the model what is wrong. An error memory refuses a correction that repeats a failure already seen.
- A human (last)After K tries, or from the start when the policy says so, the decision goes to a person.
Every judgement is a typed verdict: Validated, DriftAdjustment or HumanEscalation. That is not a code detail. A typed verdict is written to the run's log, can be counted, can be read back. The thresholds, K and the policies are database rows that an administrator sets, not ifs in the Java.
Law 25. Quebec's Act respecting the protection of personal information in the private sector, in section 12.1, governs decisions based exclusively on automated processing. The person concerned must be able to have the decision reviewed by someone. The HumanEscalation verdict is that path. It is recorded with the run, the reason, the person who decided, the time and the action taken, in the same transaction as the run's state. A policy can make escalation mandatory for a step that decides something about a person. An obligation in the law becomes a feature of the engine.
A machine's GPU
On the Mac, and on a server with a single card, two things want the GPU's memory at the same time: Ollama, which keeps text models loaded, and the media worker, which loads a video model of several tens of gigabytes. When the two fight, the machine crawls or one of them falls over.
The intended answer is a lease per device. Whoever wants the GPU asks for a lease, with a time to live and a heartbeat. A lease that stops beating is taken back. There are two profiles: SHARED_LIGHT, where several small jobs share the card, and EXCLUSIVE_HEAVY, for a heavy render. Before it grants an exclusive lease, the arbiter asks Ollama to free its memory (keep_alive: 0) and checks on /api/ps that it really is free. We do not trust, we check. A small model stays outside the heavy lock, so the interface keeps answering.
Live steering
A long run, say a video in several shots, takes time. Meanwhile, you change your mind. Redoing the whole run for one detail is expensive.
The intended steering rests on typed edges between steps. An edge says what a change upstream means downstream:
CONTENT: the next step depends on the content, so it is regenerated;ASSEMBLY: only the assembly, which is deterministic, is redone;CONTINUITY: only the control loop runs again on the neighbouring step. It regenerates only if the drift is above the threshold; otherwise it flags "impact detected" and leaves the decision to the person.
We cancel only the part of the graph that is touched, not the whole run.
Who may read what
The question that made me reread everything, I asked like this: if an employee asks the assistant for the CEO's salary, and that salary is in an indexed document, what happens?
There are two ways to get it wrong. Trusting the model to be careful: it will not be, a text a model reads can carry orders, and an instruction in the prompt is only advice. Or filtering the answer afterwards, once the passage has already been read. The check has to come before.
ADR-091 sets three rules:
- The caller travels. The verified identity (the organisation, the person, their roles, their groups) goes with every search and every tool call, in a signed token. Never in a request body.
- The right is in the query. Every indexed passage carries the rights of its source. The filter is part of the search itself, before ranking. A passage you may not read is never read, so it cannot leak through a snippet, a count or a cache.
- Every tool declares its roles. A tool that declares nothing is for administrators only. Without an identity, nothing runs.
The fix is in progress, before we index anything sensitive. I will not go into more detail until it ships.
Packs, and the proof that the core stays generic
A pack is a folder: a manifest, its translations, its Studios (plans of steps), sometimes its own worker. Installing it writes data: capabilities, a price, a catalogue entry. It changes no code. The engine does not know what a lease is. It knows how to start a plan, hold credits, call a tool, judge an output and keep a trace.
How do we know that is true, and that it stays true? A rule of ADR-092 says it: the proof is a pack. The document verification pack (orazaka-packs), the one with the rental-lease rules, must evolve without a single change in the core. Those rules come from the law: a document is judged only by a promoted ruleset, after its reference cases and a review (ADR-089). If one day the core has to change for a product, the pull request is refused, and the change goes into a port or a pack.
The same goes for media. There is no video module in the core. There is the media worker, and packs that use it.
Products go through the API, and only the API. Yomizaka never talks to a model. It calls the pack's Studios on the edge with a service key, and keeps the run id of every analysis: that is the trace of what ran. Akinaza follows the same rule. The shared client, @krizaka/orazaka-client, starts a run with an idempotency key, reads its state and waits for its end. Consent, budget, audit and idempotency are written once, in the engine. A product that went around it would have to write them again, and one day it would forget one.
Where we are
That is the direction. Here is what the code says on 26 October. The audit and the counter-review found it, and I would rather write it here than let someone discover it.
- The core still imports Spring AI. 52 engine files import it: 36 in the core, 14 in the interceptors, 2 in the tools (measured in ADR-092). An ArchUnit test will count those files, and the count will only be allowed to go down. No big bang: we move one slice at a time to the adapter module, down to zero.
- The core's
DagCompileris a stub. Two task names written by hand, no executor, and nothing calls it. The real graph is the Studio's plan: it has its dependencies, its parallelism, its output contracts and its approval steps. The stub will be deleted. - The control loop has a flaw. On each new try, the interceptor checked the same text again instead of generating a new one, and after the last try it served the answer without it being validated. The fix has been merged since (sovkora-engine#117). Typed verdicts, the drift score and escalation come next.
- The GPU lease does not exist yet. Nothing calls
/api/pstoday, and nothing setskeep_alive. The worker manages its own memory, and that is all. - Live steering is designed, not built (ADR-088). We can approve a step, resume from a step and cancel a whole run. Not yet cancel a single branch.
- The cache works per run, not per step. An identical run is served from the cache. One changed step redoes the whole run.
- The access control of the previous section is not finished.
And what it costs, measured:
- A run is expensive in statements, not in time. A single-step run takes 37 SQL statements, about 9 of them redundant. The saga's overhead stays small: a p95 of 19.2 ms after dispatch moved through the outbox, far from the 250 ms threshold set before measuring (ADR-067). What dispatch really waits for is the outbox relay: at most 500 ms, 250 on average.
- A rented GPU costs money even asleep. On 16 October, a pod ran for five hours with the GPU at 0 %, for about $20. A guard stops it now (ADR-078).
- A component never builds alone. With 20 repositories, CI clones the workspace and builds the dependencies first. It is the right way to catch a break, and it is slow.
In incubation: Recipes
One more idea, not built. It sits in incubation, at the top of the column.
When you type a good prompt in the chat, you lose it. The idea: keep it as a recipe. A recipe is private by default. The engine spots what changes from one time to the next (a document, a date, an amount, a name) and turns it into typed variables. You press ▶ Play, a short form asks for the document or the date, and the run starts.
What I like is what sits underneath. A recipe does not break the intent down again each time. It runs a plan that is already compiled and versioned, the same graph of steps as the rest of the engine. It is faster, the cost is predictable, and the result can be reproduced. No second workflow engine there either.
Later, a recipe will be able to take a trigger (every Monday at 8 a.m., a new document in a folder) and become a Routine. It will then run under the identity of the person who switched it on, never under a system account. Its rights will be checked again at every run, and if the control loop asks for a human, the Routine stops and waits.
And v3?
v3 has no branch, no repository and no date. But it now has a shape: it is where the slices of ADR-092 lead. We will not build it next to v2. We will get there by turning the slices green, one after the other, on the engine that already runs.
Its goal stays the same: do the same work faster, for less, with outputs we can prove.
The slices, in order
- The boundary. Spring AI in one adapter module, the domain in plain Java 21, the ArchUnit count at zero, the graph stub deleted.
- The verdict. First the test that reproduces the loop's flaw, then the typed verdicts, the judge, the K corrections with the diff, and escalation.
- The GPU arbiter. The lease per device, checked eviction, jobs grouped by model so we do not reload tens of gigabytes between two clips. The worker reports its memory and its loaded models.
- Steering. Commands between two steps, typed edges, cancelling a single branch.
- The per-step cache. One key per step, computed from its inputs, so a change only redoes the branches it touches.
The last three answer the measured pains directly. A warm GPU and the arbiter go after model loading. Steering, and events instead of re-reading state, go after waiting. The per-step cache goes after work redone for nothing.
How we will know it is better
"Faster" without a figure is a feeling. The method already exists: the M0 measurement set its threshold before measuring, and the decision followed the figure. v3 is those figures turned into budgets, checked in CI.
| Budget | Today | Target |
|---|---|---|
| Files importing Spring AI outside the adapter | 52, measured | 0 |
| SQL statements, single-step run | 37, measured | 28 or fewer, without the duplicate reads |
| Saga overhead, p95 | 19.2 ms, measured | no more, and always under 250 ms |
| Dispatch wait | 250 ms on average, 500 at most, measured | no periodic polling on this path |
| A step's deterministic checks | not measured | under 10 ms |
| Loading Wan 2.2 TI2V 5B on an H100 | 32.1 s, measured | under 12 s |
| Rented GPU at rest | five hours at 0 % on 16 October, measured | $0 |
| One step changed in a run | the whole run redone | only the branches it touches |
Every budget is a ratchet: it starts from today's figure, and it can only go down. It is the same mechanism as for Spring AI in the core.
What we refuse
- A rewrite. I know the temptation. You see 52 misplaced files, a useless stub, a loop that does not regenerate, and you want to start over clean. In July, we split Orazaka's monolith in stages, each one green before the next (ADR-032). v3 follows the same path.
- Optimising without measuring. The 9 redundant statements have been known since the first measurement. We left them, because the figure held its threshold. They will go with the budget that counts them.
- Calling something intellectual property before it is built. The GPU arbiter is planned. Until it is there, I do not sell it.
What I take from it: an architecture is not a diagram you draw once. It is a line you hold. What gives the engine its value, we own and we prove. The rest, we plug in, and we keep it behind a door we can change.
