At first, Orazaka was a discovery. A personal project, on my MacBook, to understand what you can do with AI models running at home. Ollama for text, the Mac's GPU (Metal) for images, a database and a few services started by hand. If someone had taken the laptop, there would have been no Orazaka left.
Then the project took shape, and I decided to share it as open source. From there, one question took all the room: how do you run elsewhere what only ran on my machine? This post tells that industrialisation, step by step, with the options I weighed. I only talk here about choices written down in our architecture decisions or our documentation.
Step 1: a monolith on a laptop
The first Orazaka was a single Java program that did everything: chat, long tasks, media, identity. It was already tidy inside, and architecture rules checked on every build (ArchUnit) stopped any layer from calling any other. But one application owned all the data and all the ways in.
The first move towards portability was taking the models out of the code. With the move to Spring Boot 4 and Spring AI 2, the choice of model became a row in a database, read on every call by a router. A local model and a hosted model go through the same API. I tell it in the post about the Mac. That's the decision that made everything else possible: changing machines no longer means changing code.
Step 2: services, without breaking everything
In July, I split the monolith into services (ADR-032). Not all at once: with the "strangler fig" pattern. You first put a facade in front, then move one piece at a time behind it, and each step has to pass the end-to-end tests before the next. One database per service, each with its own role, and no foreign key between two services.
At that point everything stayed 100% local: no CI, no cloud, it was written in the decision. The Mac still ran everything, but it ran it the way a server would.
Then came splitting the code into public repositories, about twenty, one per component. Clean on paper, costly day to day: a change that crosses three repositories is three pull requests. I'm now bringing back together what belongs together.
Step 3: containers and a command line
For someone else to start Orazaka, there had to be one way in. That's the command line:
npx orazaka install --server # on a Linux server
orazaka start
orazaka doctor
orazaka models image listThe services and databases run in containers. The models run as close to the hardware as possible: on a Mac, directly on Metal, because virtualisation doesn't give access to Apple's GPU.
The delivery documentation weighs the options for containers on a Mac. Docker Desktop isn't redistributable and becomes paid for large organisations. OrbStack is excellent, but needs a paid licence per user. The default is Colima, free and scriptable, and if the machine already has one of the other two, we use it. For large organisations, Kubernetes and Helm will come later, when a client needs high availability. Not before.
Step 4: GPUs that only exist while they work
The development environment now runs in the cloud, without a GPU. But it had to generate for real. The decision (ADR-077): a GPU rented by the hour, that only exists while it works. A small one for development, a big one for demos, never one left on for nothing.
The options are written down plainly. An H100 renders a video two to three times faster, for about the same cost per clip, but every idle minute costs almost three times more. For the network, the GPU dials out, through a relay: we refused exposing the message broker on the Internet, a managed broker and a VPN. And for models, we set aside those whose licence caps revenue or commercial use, even when they were good.
Since then, the pods have a watchdog, and a proposal is on the table: serverless workers that cost nothing at rest. The mistakes that led there are in the post about costs.
The Mac doesn't disappear. It goes back to what it was at the start, but by choice: a local runner. The desktop app, a Tauri window of a few megabytes that drives the command line, is meant to run automations offline. It's in progress; I explain it in the post about agents.
What keeps it all honest
None of this would hold without tests. Integration tests start real databases and a real broker in containers (Testcontainers), from one shared base class. Architecture rules are checked on every build. An automatic rewriting tool (OpenRewrite) runs in CI and fails if it finds code to modernise. And the most useful criterion I've written is simple: a pack living in a folder outside the repository must install and run without any change to the repository. The first time, it failed four times, on four defects nobody would have seen from inside (ADR-049).
What stays hard
CUDA and Apple Silicon. The same image model runs on Metal on the Mac and on CUDA on a rented GPU, with the same engine and the same recipe. But that's two paths to build, two images to test, and video simply doesn't fit on the Mac.
The size of the weights. A 14-billion-parameter video model weighs tens of gigabytes. You have to download them once, keep them somewhere, and not download them again at every start.
Cold starts. A GPU that only exists while it works has to start, load its models, then work. That's what makes on-demand cheaper, and slower for the first job.
The cost of a warm GPU. Keeping it on makes everything faster, and every idle minute is paid at the price of a minute of compute.
Installing at other people's. A signed and notarised Mac app, a package for Linux, one version for the whole set, updates that a small firm without an IT person can run alone. It's written down. It isn't shipped yet.
What you can take from it
If your project only runs on your machine, start by taking out of your code everything that depends on it: model names, addresses, paths. Split little by little, behind a facade. Keep models close to the hardware and the rest in containers. Pay for GPUs by use, with a watchdog. And write a test that checks, from outside, that someone else can really use it.
