Iris: AI Personal Assistant

A production-grade personal AI: a native SwiftUI iOS app backed by a TypeScript microservices platform that runs an LLM agent loop with real tools: email, calendar, voice calls, browser automation, scheduled tasks, and persistent memory.

SwiftUITypeScript / FastifyLLM Agent Loop SSE + WebSocketsPostgreSQL / PrismaTwilio Voice

The Idea

Most "AI assistant" demos answer questions. Iris is built to do things. The difference between a chatbot and an operator is tool access, memory, and a schedule. Iris reads and sends email across multiple accounts, manages calendars, places real phone calls, books flights, drives a cloud browser, and runs on timers so the highest-value work happens before you even ask.

Agentic Tool Loop

A multi-provider LLM core (Gemini, OpenAI, Anthropic) runs a build-context → call-model → execute-tools → repeat loop, with 25+ tools dispatched locally or over RPC.

Real-World Integrations

Gmail, Outlook, and IMAP email; calendar and contacts; web search; flight booking; Uber deep links; and cloud browser automation via Browserbase + Stagehand.

Voice Calls

Iris places and answers actual phone calls through Twilio, with low-latency speech powered by Gemini Live and Deepgram.

Memory & Scheduling

A persistent memory store of user facts personalizes every run, while scheduled tasks and reminders fire through BullMQ job queues. Briefings happen on a timer, not on demand.

Streaming-First iOS App

SwiftUI chat streams tokens over SSE while a parallel WebSocket carries async events. Running tasks surface in the thread in real time as they execute server-side.

Sandboxed Execution

Each user's agent runs in an isolated E2B sandbox, so tool execution is contained per user with a centralized tool registry controlling what the model can touch.

Architecture

The system is a Turborepo monorepo of TypeScript services in front of a native iOS client. The model is a commodity. The engineering is in the orchestration around it.

IRIS · SYSTEM TOPOLOGY iOS App · SwiftUI Chat · token streaming Live task events Tasks · Memories · Brain Push notifications Firebase Auth Google · Microsoft · Yahoo Apple OAuth + IMAP typed APIRoute contracts async/await · AsyncStream auto-reconnect HTTPS/SSE WebSocket FCM push Backend · TypeScript · Turborepo monorepo Fastify API REST · SSE · WS gateway Agent Loop Gemini · OpenAI · Anthropic BullMQ Queues scheduled tasks · reminders E2B Sandbox isolated per-user execution PostgreSQL · Prisma 13+ models · memory store Central Tool Registry 25+ scoped tools · 15-iteration cap real-world actions Tools · the hands of the agent Email (Gmail · Outlook · IMAP) · Calendar · Contacts · Web Search · Memory Twilio Voice calls (Gemini Live + Deepgram) · Browserbase + Stagehand browser · Flights · Uber read freely · write carefully · never act irreversibly without confirmation

Engineering Highlights

  • Hybrid real-time transport. SSE streams message tokens; a WebSocket carries async events (task updates, background results) with reconnection logic. If a disconnect exceeds the 60-second window, the client refetches state and deduplicates messages by server ID. No ghosts, no gaps.
  • Tasks live inside chat. Long-running agent tasks aren't a separate silo. They bubble up in the conversation with thread links and stream status as they execute, synced over WebSocket events.
  • Multi-account auth done properly. OAuth 2.0 flows for Google, Microsoft, Yahoo, and Apple, plus custom IMAP linking, so Iris operates across every inbox a person actually has.
  • Typed networking on iOS. A custom APIRoute protocol gives every endpoint a typed request/response contract, with separate HTTP and SSE clients built on async/await and AsyncStream, and NWPathMonitor driving auto-reconnect.
  • Trust is tiered. The agent loop caps at 15 tool iterations per run, every tool is scoped through a central registry, and execution is sandboxed per user: read freely, write carefully, never act irreversibly without confirmation.
  • Six environments, one codebase. The app ships with local-through-production configurations, Redis-backed queues, PlanetScale PostgreSQL, and Firebase for auth and push.

Design Philosophy

  • Schedule beats chat. The highest-value assistant work happens on a timer, not in response to a prompt. Briefings, reminders, and follow-ups run themselves.
  • Context is the product. The persistent memory of preferences, accounts, and past decisions is what separates an operator from a chatbot.
  • Real tools or it doesn't count. Sending an actual email, placing an actual call, and booking an actual flight are categorically different problems from generating text about them.

Status: in active personal use and continuous iteration across the iOS app and backend platform.