Skip to main content

Architecture

If you're building on top of Osaurus — writing plugins, scripts, or integrations — this page is the orientation. It maps the user-facing surfaces (chat window, Settings, HTTP API) to the components underneath, and points to the deeper pages for each layer.

The harness​

Osaurus presents three entry points:

  • The chat window (⌘;) — the daily driver
  • Settings… (⌘ ,) — Orchestrator, agents, models, images, tools, memory, themes, workspaces, automation
  • The HTTP API (on :1337) — OpenAI / Anthropic / Open Responses / Ollama / MCP, media, and loopback configuration

The built-in Orchestrator can configure these surfaces and delegate to custom agents; agent runs funnel into the same agent loop, which talks to your memory, skills/methods, and the automation surface (schedules, watchers). Inference goes out to local MLX, Apple Foundation, or any cloud provider you've connected. Tools span native plugins (v1–v6 ABI), remote MCP servers, and the Linux sandbox. Underneath everything: identity (signed requests, access keys), local storage (with opt-in SQLCipher encryption), and relay (public tunnels).

How the pieces fit together​

Layers​

LayerWhat it doesReference
Entry pointsChat window (⌘;), Settings (⌘ ,), HTTP API on :1337Chat, HTTP API, CLI
HarnessOrchestrator, delegated agents, Tasks, Memory, Skills/Methods, Schedules/Watchers — the continuity layerOrchestrator, Tasks, Memory, Skills, Methods, Schedules, Watchers
InferenceMLX local models, Apple Foundation Models, cloud providers — all behind the same pickerModels, Apple Intelligence, Inference Runtime
ToolsNative built-ins (web search, browser use, Apple Apps), registry plugins, remote MCP aggregation, and the SandboxTools & Plugins, Plugin Authoring, Sandbox Internals, Remote MCP Providers
FoundationsIdentity (signed requests, osk-v1 keys), local storage (opt-in SQLCipher), on-device Privacy Filter, Public Links (public tunnels)Identity Cryptography, Storage & Encryption, Privacy Filter, Public Links

Entry points​

Chat window​

A chat window summoned with ⌘; from anywhere on macOS. Each window holds tabs, a sidebar of agents and projects, and an inspector for history and file changes; you can open as many windows as you like. Each tab has its own agent, working-folder / Sandbox mode, model selection, and conversation history, so you can run several agents side by side. Chat →

The chat window is also where voice input lives: the microphone in the input bar, plus Wake Word activation and global Transcription Mode.

Settings​

⌘ ,. The sidebar groups General (General, Conversation, Voice, Themes, Credits, Workspaces, Mobile, Identity, Permissions, Privacy), Models (Local Models, Providers, Images), Agents (Orchestrator, Agents with its Templates and Runtimes pages, Channels), Capabilities (Web Search, Knowledge, Memory, Tools & MCP, Skills, Commands), Automation (Schedules, Watchers, Computer Use, Browser Use), and a collapsible Developer Tools section (Server, Sandbox, Insights). Native plugins and MCP connections live inside Tools & MCP; Claude plugin bundles live inside Skills. Workspaces connect you with teammates' shared agents, and Mobile pairs the Osaurus iPhone app.

HTTP API​

A local server on port 1337 (configurable). Speaks OpenAI Chat Completions, Anthropic Messages, Open Responses, and Ollama Chat APIs side by side, plus MCP endpoints, media generation, D3 decision requests, agent runs, Memory, pairing, loopback-only declarative configuration under /admin/config/*, and the osaurus import back end. An opt-in local access key can require keys from callers on this Mac too.

Harness​

The harness is what makes Osaurus more than a thin SDK shim:

  • Agent Loop — every chat is an agent loop. The model writes a markdown todo list, calls tools, iterates, and ends with a verified summary or pauses to ask one critical question.
  • Memory — persistent on-device memory with four layers (identity, pinned facts, episodes, and the raw transcript, which is read only for literal recall or as a search fallback). Distillation runs once per session, gated on a configured Core Model.
  • Skills & Methods — reusable capabilities. Skills are markdown packages of expertise; Methods are scored YAML workflows the agent saved from past runs. Both are discovered and loaded on demand via capabilities_discover / capabilities_load.
  • Schedules & Watchers — automation. Schedules run on a clock; watchers react to file system changes via FSEvents.
  • Runtimes — an agent can run on an external harness instead of the built-in loop. HarnessRuntimeStore holds the registered runtimes (Claude Code is seeded on first launch; Pi RPC programs are user-added), and each chat gets its own long-running harness process with a scoped agent token for calling back into Osaurus. Runtimes →
  • Templates and bundles — agent templates are portable, optionally signed agent definitions stored in ~/.osaurus/template-library/; Osaurus Bundles let an installer provision agents, skills, MCP servers, and runtimes through osaurus import, applied through the same declarative configuration engine.

Plugins, schedules, watchers, and the HTTP API all dispatch through the same agent loop — same engine, same loop tools, same intercepts. Sessions are tagged with their source (such as chat, plugin, http, channel, schedule, watcher, imported, or delegation) so you can audit what spawned each conversation in the chat sidebar.

Inference​

Three local options and a cloud surface, all behind the same model picker:

  • MLX — local transformer / SSM models, optimized for Apple Silicon via vmlx-swift-lm's BatchEngine (continuous batching, content-addressed prefix caching). Inference Runtime →
  • Apple Foundation Models — Apple's on-device system model (model: "foundation") on macOS 26+. Zero downloads, zero config.
  • Liquid Foundation Models — non-transformer architecture optimized for edge.
  • Cloud providers — OpenAI, Anthropic, Gemini, xAI, DeepSeek, MiniMax, Venice, AtlasCloud, Azure OpenAI, OpenRouter, Ollama, and more — plus the hosted Osaurus Router. API keys in macOS Keychain. Remote Providers →

Memory and agent context persist across all of them — switching from local Gemma to Claude 4 doesn't lose what your agent has learned about you.

Tools​

An append-only host ABI for native plugins, v1 through v6:

  • v1 — tools only
  • v2 — full host API: HTTP routes, SQLite-backed config, web app serving, agent dispatch, inference, events
  • v3–v6 — streaming cancellation, agent-context introspection, structured logging, and a host-side string free path, each added without breaking older plugins

Formerly-plugin capabilities like web search and browser use are native built-ins now — no plugin install needed.

Plus remote MCP providers to aggregate tools from external MCP servers (with elicitation, progress, and annotation-driven default approvals; ExternalMCPToolPolicy deny lists still apply on top), and the Sandbox for safe code execution (a Linux VM on macOS 26+, a Seatbelt-confined host runner on macOS 15). The sandbox itself accepts JSON-recipe plugins so users can extend an agent's capabilities without compiling anything.

Every tool — built-in, folder, sandbox, plugin, MCP-aggregated — returns the same canonical Tool Contract envelope.

Foundations​

The trust layer underneath everything:

  • Identity — secp256k1 master key in iCloud Keychain (biometric-gated), deterministic per-agent child keys, Apple App Attest device assertion, osk-v1 access keys for external callers (scoped, expirable, revocable).
  • Storage — SQLite across chat history, memory, methods, tool index, and plugin databases; FileVault covers at-rest protection by default, with opt-in SQLCipher whole-database encryption (key in macOS Keychain, device-bound). Large attachments spill to content-addressed blobs.
  • Public Links — secure WebSocket tunnels through agent.osaurus.ai per agent. The agent's cryptographic address is the routing key. No port forwarding.

These are the boundaries. See Security & Privacy for the user-facing summary, Identity Cryptography and Storage & Encryption for the specs.

Where to go next​

Build a thing:

Understand a piece: