At first, Orazaka was a discovery. A personal project, on my MacBook, to understand what you can do with AI models running at home. Ollama for text, the Mac's GPU (Metal) for images, a database and a few services started by hand. If someone had taken the laptop, there would have been no Orazaka left.
That Mac is the first server of Orazaka, the AI engine I'm building. The idea behind Orazaka is simple: run AI on a machine you control. For me, that machine is my laptop first. Here's what it taught me.
On 12 October, the Mac hit the wall
On 12 October my Mac hit the wall. The system load swung between 100 and 260, and tools started failing with ENFILE: the system refused to open one more file. Several agent sessions were working in parallel, each with its own native builds, Docker containers or mobile simulator. All of it on a MacBook Pro M1 Max with 64 GB of memory.
Even without AI, a platform needs several things at once: a database, a message broker, the Java services themselves. Add a language-model server, an image engine, a worker for video. They all share the same memory. When two big dishes cook at once, the oven gives out.
My answer on 12 October was simple, almost dumb: a lock. Before a heavy task (a native build, a Docker stack, a generation), a session creates a .heavy.lock folder. If the folder already exists, it waits a minute and tries again. Like the single bathroom key in a café: one person at a time, the others wait. Everyone slows down a little, but nothing collapses any more.
The Java services take their share too. Orazaka has up to eleven Spring Boot services. An open proposal estimates 4 to 6 GB at rest for the lot, and suggests a single-JVM development profile. Those are estimates, not measurements, and it hasn't shipped.
Models become data
In June I moved the platform to Spring Boot 4.0 and Spring AI 2.0 (ADR-030). That brings Spring Framework 7 and Jackson 3, which moves com.fasterxml.jackson to tools.jackson. The version bump interested me less than what it made possible.
Before, the choice of model lived in configuration: a spring.ai.* block in application.yml, and starters that wire clients for you. Switching models meant redeploying. I removed all of it. Today an ai_providers table holds each provider's address and key, and an orazaka_models table links a model name to its provider. A DynamicModelRouter reads those rows on every call and caches the client.
Think of a phone book. Before, every number was carved into the wall. Now it's written in the book: change one line and the next call goes to the right place, no renovation needed. An admin edits a row, and the platform follows. That's the decision that made everything else possible: changing machines no longer means changing code.
One detail makes this work: Ollama and LocalAI speak the same protocol as OpenAI. A single OpenAI-compatible client, pointed at an address read from the database, covers every local provider.
You aim at the lowest common denominator: a feature specific to one provider is useless if the others don't have it. You have to evaluate each model on your own tasks, because a model name guarantees no quality.
The ClassCastException that killed the chat
Of course it broke. Everything goes through the OpenAI-compatible client, which expects OpenAiChatOptions. But a step earlier in the pipeline tagged local providers with OllamaChatOptions. The result: a ClassCastException, and no chat at all.
The fix, in RoutedChatModel, converts the options before the call:
static Prompt toOpenAiPrompt(Prompt prompt) {
ChatOptions options = prompt.getOptions();
if (options == null || options instanceof OpenAiChatOptions) {
return prompt;
}
OpenAiChatOptions.Builder builder =
OpenAiChatOptions.builder()
.model(options.getModel())
.temperature(options.getTemperature())
.maxTokens(options.getMaxTokens());
// ...tool callbacks, when present
return new Prompt(prompt.getInstructions(), builder.build());
}There was a second, quieter trap. A step that asked for qwen2.5:7b sometimes went to hosted inference instead, because the model name was ignored except for two providers. Since orazaka-ai-engine#13, routing follows the catalogue's link between a model and its provider. And a model whose provider doesn't answer now fails with an error naming both, instead of quietly going somewhere else. For sovereign AI, that kind of silence is the worst bug there is.
Two more problems came from mixing the Mac and the cloud. The dev platform runs on DigitalOcean, and the Mac joins it through a tunnel, via a relay. First: generated media were written to the container's disk, which every deploy wipes. The database rows survived; the files didn't. We now go through durable storage, with a Bunny adapter (orazaka-ai-engine#21).
Second: every deploy restarts the relay and drops the tunnel. A clip halfway through rendering was lost, then started again from zero. The worker now reconnects and never processes the same job twice (orazaka-worker-media#24).
From a monolith to services
Then the project took shape, and I decided to share it as open source. From there, one question took all the room: how do you run elsewhere what only ran on my machine?
The first Orazaka was a single Java program that did everything: chat, long tasks, media, identity. It was already tidy inside, and architecture rules checked on every build (ArchUnit) stopped any layer from calling any other. But one application owned all the data and all the ways in.
In July, I split the monolith into services (ADR-032). Not all at once: with the "strangler fig" pattern. You first put a facade in front, then move one piece at a time behind it, and each step has to pass the end-to-end tests before the next. One database per service, each with its own role, and no foreign key between two services.
At that point everything stayed 100% local: no CI, no cloud, it was written in the decision. The Mac still ran everything, but it ran it the way a server would.
Then came splitting the code into public repositories, about twenty, one per component. Clean on paper, costly day to day: a change that crosses three repositories is three pull requests. I'm now bringing back together what belongs together.
Containers and a command line
For someone else to start Orazaka, there had to be one way in. That's the command line:
npx orazaka install --server # on a Linux server
orazaka start
orazaka doctor
orazaka models image listThe services and databases run in containers. The models run as close to the hardware as possible: on a Mac, directly on Metal, because virtualisation doesn't give access to Apple's GPU.
The delivery documentation weighs the options for containers on a Mac. Docker Desktop isn't redistributable and becomes paid for large organisations. OrbStack is excellent, but needs a paid licence per user. The default is Colima, free and scriptable, and if the machine already has one of the other two, we use it. For large organisations, Kubernetes and Helm will come later, when a client needs high availability. Not before.
None of this would hold without tests. Integration tests start real databases and a real broker in containers (Testcontainers), from one shared base class. Architecture rules are checked on every build. An automatic rewriting tool (OpenRewrite) runs in CI and fails if it finds code to modernise. And the most useful criterion I've written is simple: a pack living in a folder outside the repository must install and run without any change to the repository. The first time, it failed four times, on four defects nobody would have seen from inside (ADR-049).
What the Mac does well, and where it gives up
For everything else, the Mac is a great place to start. It runs chat with Ollama, voice with Piper, music with ACE-Step and images with stable-diffusion.cpp. A FLUX.2 [klein] 4B image at 1024×1024 takes 62 seconds at the median, command to file. No bill, no cloud account, no driver to install. And every new developer already has a laptop.
Three pictures from that machine. Under each, "How this was made" gives the prompt, the seed, the time and the Orazaka run that produced it.
How this was made
- Model
- FLUX.2 [klein] 4B (Q8_0 GGUF) · Apache-2.0
- Engine
- stable-diffusion.cpp on Metal, 4 steps, guidance 1.0
- Machine
- MacBook Pro, Apple M1 Max, 64 GB (Metal)
- Through
- Orazaka Studio run on dev.orazaka.com
- Studio
art-miniature- Prompt
A tiny winter park in Montreal: figurines skating on a frozen pond, a small plain wooden cabin with a glowing window and no signboard, snow-covered spruce trees and warm string lights at dusk. Tilt-shift miniature diorama photograph: handcrafted figurines and props at 1:24 scale, macro lens, extremely shallow depth of field with a thin plane of focus, soft window light, ultra-detailed textures of felt, embroidered thread, painted resin and carved wood. Warm ochre, saffron and indigo palette, playful and tender. No text, no lettering, no logos.
- Seed
1643- Size
- 1360×768
- Time
- 75 s, as recorded by the run
- Cost
- no cash cost (own hardware; electricity not metered)
- Sovkora run
- not recorded
How this was made
- Model
- FLUX.2 [klein] 4B (Q8_0 GGUF) · Apache-2.0
- Engine
- stable-diffusion.cpp on Metal, 4 steps, guidance 1.0
- Machine
- MacBook Pro, Apple M1 Max, 64 GB (Metal)
- Through
- Orazaka Studio run on dev.orazaka.com
- Studio
art-miniature- Prompt
A tiny souk alley: figurine spice merchants behind pyramids of saffron, paprika and cumin, hanging pierced-brass lanterns, woven baskets, a cat asleep on a blue doorstep. Tilt-shift miniature diorama photograph: handcrafted figurines and props at 1:24 scale, macro lens, extremely shallow depth of field with a thin plane of focus, soft window light, ultra-detailed textures of felt, embroidered thread, painted resin and carved wood. Warm ochre, saffron and indigo palette, playful and tender. No text, no lettering, no logos.
- Seed
1212- Size
- 1024×1280
- Time
- 100 s, as recorded by the run
- Cost
- no cash cost (own hardware; electricity not metered)
- Sovkora run
- not recorded
How this was made
- Model
- FLUX.2 [klein] 4B (Q8_0 GGUF) · Apache-2.0
- Engine
- stable-diffusion.cpp on Metal, 4 steps, guidance 1.0
- Machine
- MacBook Pro, Apple M1 Max, 64 GB (Metal)
- Through
- Orazaka Studio run on dev.orazaka.com
- Studio
art-montreal-night- Prompt
A narrow back alley of the Plateau between plain brick walls with no signs, exterior spiral staircases, wooden balconies with potted plants and parked bicycles, a lone cyclist far away. Night street photograph in Montreal after rain: wet cobblestones and asphalt mirroring warm window light and a neon glow of magenta and teal, light mist, 35mm lens at f/1.8, cinematic colour grading, deep blacks, rich textures of old brick, wrought-iron spiral staircases and wet pavement. No readable signs, no lettering, no logos.
- Seed
12- Size
- 1360×768
- Time
- 109 s, as recorded by the run
- Cost
- no cash cost (own hardware; electricity not metered)
- Sovkora run
- not recorded
They're bigger than 1024×1024, hence 75 to 109 seconds each.
I tried generating video locally with Wan 2.2 TI2V 5B, vertical, 480×832 and 25 frames. The model ran out of memory while decoding: it asked for 26 GiB while 62 GiB were already taken on a 64 GB machine. At 704×1280, same story. The Mac's unified memory is generous for images, not for video.
Video goes elsewhere. Hosted, a five-second Wan 2.2 A14B clip at 832×480 takes 131 seconds at the median. On a rented H100 we get to 720p. So video goes to a GPU when I choose to send it, and the rest of the work stays on the laptop. The numbers, the bills and the mistakes behind that choice are in the post about costs.
What I haven't measured: the Mac's memory and swap over a working day, the real memory of the JVMs at rest, and local chat speed. I use them every day, but I don't have a recorded number to show yet.
