Personal AI assistant

Celeste

Private, local-first personal assistant: a FastAPI core on my PC with Android, web and desktop clients, Markdown memory and a voice orb, where the AI can only act through a risk-based tool router.

In daily personal use

Request a walkthrough

Project overview

Context and approach

Overview

Celeste is a distributed personal assistant that connects my PC, my phone, a durable Markdown memory and local or optional cloud AI models, without handing unrestricted control to any provider.

Problem

Personal tasks are scattered across PC, phone, notes, mail, calendar and reminders, and most assistants solve that by giving a cloud model broad access to everything. I wanted something useful every day that keeps the data local and never lets the model act on its own authority.

Solution

A FastAPI core is the security boundary: models receive declared tools, not shell access or credentials. A Tool Router classifies every action as READ, SAFE_WRITE, CONFIRM or RESTRICTED, sensitive changes require real confirmation, and memory lives in human-readable Markdown indexed with SQLite FTS5.

Screenshots

Celeste web client answering from its memory — Answering from Celeste Brain with the local Piper voice.

Celeste orb in thinking state with an orbital ring — Thinking state: the orb spins faster and shows an orbital ring.

Celeste web client at rest — Idle: a particle sphere that breathes and reacts to voice.

Key figures

151

Core tests passing

Architecture decisions (ADRs)

Clients: Android, web, desktop

Tool risk levels

Architecture

01
Clients
Android app, web client and desktop voice orb.
02
Celeste Core
FastAPI service with token auth and autostart on Windows.
03
Tool Router
READ, SAFE_WRITE, CONFIRM and RESTRICTED actions with audit.
04
AI providers
Ollama locally, Claude or OpenAI as interchangeable options.
05
Celeste Brain
Markdown notes with YAML frontmatter and an FTS5 index.

Main capabilities

Voice conversation with a local Piper voice and Whisper transcription.
Notes, search and memory in Markdown that I can open in Obsidian.
Durable reminders with proactive spoken announcements.
Gmail and Google Calendar through official OAuth, with confirmation before sending or changing.
Android client with Wake-on-LAN, offline notes (Room) and idempotent sync.
Swappable AI providers: Ollama, Claude or OpenAI.

Verification evidence

151 of 151 Core tests passed.
Used daily on Windows with autostart and a token-protected API.
Gmail and Calendar validated against a real account in draft mode.
Offline notes synced without duplicates after network loss.

Engineering decisions

— The Core, not the model, is the authority over data and actions.
— Markdown files are the source of truth; the SQLite index can always be rebuilt.
— Sensitive actions need explicit confirmation tied to the exact payload.
— Local models first; cloud providers are optional and replaceable.
— LinkedIn messaging was evaluated and rejected for lack of official free access.

Scope limitations

— Single-user project built for my own use.
— Source code is private; this page shows screenshots from a demo instance with empty memory.
— Home automation and an always-on server remain on the roadmap.

Want to see it running?

The repository is private. I can walk you through the code and a live demo.