AI chat

Valcraven ships a full AI chat assistant: a streaming chat UI at /app/chat, backed by a per-user AI agent that can call tools, read attachments, render inline charts and images, and remember the conversation across reloads. The chat page lives at apps/web/app/app/chat/[[...conversationId]]/page.tsx; the assistant itself runs in the separate realtime Worker (apps/realtime).

Chat is WebSocket-only. The assistant runs over a live WebSocket to a per-user agent (the AppAgent Durable Object hosted by apps/realtime). The old request/response SSE endpoint (POST /api/chat + lib/chat-engine.ts) was removed — the realtime Worker is now the only chat engine and is required for chat to work. There is no SSE fallback. See Realtime for how that Worker is structured, and the admin dev wiki at /admin/docs/agents-chat for the full protocol.

How it fits together

browser ──WS──▶ /agents/*  ──▶ realtime Worker (apps/realtime)
                                   │  AppAgent (one Durable Object per user)
                                   │  runs the streaming model + tool-use loop
                                   ▼
                        Anthropic / Workers AI  (tokens streamed back over WS)
                                   │
                        persistence ──▶ web API /api/conversations/*

• The web client holds the socket with the useAgentChat hook (apps/web/lib/use-agent-chat.ts), which wraps useAgent from agents/react. The chat page mounts it lazily through components/agent-chat-bridge.tsx.
• Per-user routing. The Worker ignores the client-supplied instance name and routes by the validated Better Auth session user id — one AppAgent per user. A request can never reach another user's agent. The connection is same-origin, so the session cookie rides along on the WebSocket upgrade automatically.
• The agent never touches the database directly. It persists both turns through the web app's own API (POST /api/conversations, POST /api/conversations/:id/messages), so history survives a reconnect or a Durable Object eviction.

The WebSocket protocol

One JSON frame per user turn goes up; several frames stream back down:

You rarely touch these frames directly — the useAgentChat hook exposes send(...) and a set of callbacks (onConversation, onToken, onDone, onError, onStatusChange) instead.

Model selection

The active model is resolved per turn, so an admin or the user can change it without a redeploy. The agent asks the web app which model to use (GET /api/chat/active-model) and falls back to a safe default on any failure so chat never hard-fails. Precedence:

per-user preference  >  admin default (DB)  >  CHAT_MODEL env  >  built-in default

Valcraven ships a two-tier model catalog:

• BYOK — proprietary models that need a provider key (Anthropic Claude runs today).
• Workers AI — open models hosted on Cloudflare Workers AI, no provider key required. If a Claude model is selected but no key is configured, chat automatically falls back to a default open model so it still works with zero config.

The admin default is set at /admin/ai-models; a user's own preference is set through PUT /api/settings/model-preference. The full catalog, the admin model switcher, and encrypted provider-key management are documented in the admin dev wiki at /admin/docs/ai-models. Usage and cost are surfaced at /admin/ai-usage (see /admin/docs/ai-usage).

Tools

The agent runs a full tool-use loop: on every turn the model can call tools, the handlers run, and their results feed back into the model until it stops asking for tools. The template ships four built-in tools (in apps/realtime/src/tools.ts):

Doc-aware answers

For any question about how the app works, the assistant calls search_docs, grounds its answer in the returned doc sections, cites the pages it used, and says it doesn't see the answer in the docs rather than making things up. Retrieval is keyword search (FTS5) over the docs indexed into your database; if it's unavailable the chat still works as a general assistant.

Two personas, by role (#653)

The assistant branches on the signed-in user's role so the same chat serves both your team and your customers:

The corpus scope for non-admins is a server-side security boundary, not just a prompt: a customer's search_docs has no corpus selector and is hard-forced to the customer guide, so a customer can never surface internal dev-wiki content — even if they try to talk the model into it. Role comes from the validated session, never from the client.

Make it your product's support bot. Replace this customer guide corpus (apps/web/content/docs/guide/) with your product's own help docs — that content is what non-admin users get answered from, so the support bot is only as good as those pages. Reshape the wording of either persona in getCustomInstructions(role, toolsEnabled) (see Customizing the assistant). The full role-branch design is in the dev wiki at /admin/docs/agents-chat.

Small open models run tool-free. Tools are only attached when the active model reliably supports them (the catalog's supportsTools flag). Claude keeps its tools; small Workers AI models run without them because they handle the tool-call protocol poorly.

Adding your own tool

A tool implements the AgentTool interface — a name, description, JSON-Schema input, and an async handler. Register it on the agent by subclassing AppAgent and calling registerTool (usually from the constructor), without editing the core loop:

this.registerTool({
  name: "log_meal",
  description: "Record a meal the user describes.",
  input_schema: {
    type: "object",
    properties: { description: { type: "string" } },
    required: ["description"],
  },
  handler: async (input, ctx) => {
    // ctx: { userId, cookie?, env, conversationId? }
    // Act AS the user through the web API:
    return callAsUser(ctx.env, ctx.cookie!, "/api/meals", {
      method: "POST",
      body: JSON.stringify({ description: input.description }),
    });
  },
});

The handler receives a ToolContext ({ userId, cookie?, env, conversationId? }). A handler that throws is surfaced to the model as an error result, so the model can recover and explain the failure rather than crashing the turn — throw on failure instead of swallowing it. The agent's extension surface (tools, scheduling, proactive delivery) is covered in the admin dev wiki at /admin/docs/agents.

Conversation persistence

Conversations and messages are ordinary rows in the app database, scoped to the signed-in user. The relevant API routes:

The agent creates the conversation on the first turn (if none was supplied), persists the user message before calling the model, and persists the assistant reply after streaming. On a cold start (Durable Object eviction) it re-hydrates recent turns from GET /api/conversations/:id/messages into an in-memory buffer. Every route scopes to auth.api.getSession(...), so a user only ever sees their own history.

Attachments and rich content

A user turn can carry file attachments (images, PDFs, and Office documents). The agent gates each attachment by what the active model can actually read — a model that can't take a PDF gets a visible "attachment omitted" note rather than a silent drop or a crash.

Assistant replies can also embed inline charts and images by emitting a marker on its own line:

[CHART:{"type":"bar","title":"Signups","labels":["Jan","Feb"],"series":[{"name":"Signups","data":[12,30]}]}]
[IMAGE:{"key":"ai-images/<userId>/<uuid>.png","alt":"a mountain at sunrise"}]

The app parses the marker out of the streamed text and renders a React component (a Recharts chart, or an image served from the auth-gated image route) in its place — the raw marker is never shown. Chart/image parsing is shared across every surface via @valcraven/core. See the admin dev wiki at /admin/docs/rich-chat-blocks for the full spec and how to add a new block type, and Storage & files for where generated images live.

Customizing the assistant

The chat personality and defaults live in apps/web/lib/chat-config.ts:

import { CHAT_CONFIG, SYSTEM_PROMPT_TEMPLATE, buildSystemPrompt, getCustomInstructions } from "@/lib/chat-config";

• CHAT_CONFIG — the fallback model (default claude-sonnet-4-6), maxTokens, contextWindowSize, and appName. model, maxTokens, and appName read from the CHAT_MODEL, CHAT_MAX_TOKENS, and APP_NAME env vars.
• SYSTEM_PROMPT_TEMPLATE — the prompt, with {{APP_NAME}}, {{CUSTOM_INSTRUCTIONS}}, and {{USER_CONTEXT}} placeholders.
• getCustomInstructions() — return a string here to inject domain-specific instructions (e.g. "You are an expert fitness coach…"). Empty by default.

Mirror the config into the Worker. The realtime Worker bundles independently (workerd) and cannot import from apps/web, so apps/realtime/src/chat-config.ts is a minimal replica of the web config. When you change the model or system prompt, update both files or the two will drift.

Tag extraction

Beyond the visible chart/image markers, the model can emit hidden [TAG:key=value] tags that trigger server-side side effects and are stripped from the displayed text. Register a handler with lib/chat-tags.ts:

import { registerTagHandler } from "@/lib/chat-tags";

registerTagHandler("PROFILE", async (key, value, userId) => {
  // e.g. persist a profile field the model inferred from the conversation
});

stripTags (re-exported from @valcraven/core) removes system tags while keeping the [CHART:]/[IMAGE:] block markers intact for rendering.

Where the shared logic lives

Chat logic that must stay identical across surfaces lives in the shared @valcraven/core package — capability-gated message building (buildModelMessages), model-aware context trimming (applyContextWindow), the rich-block parser, and the tag-strip helpers (stripSystemTags, stripAllTags). The realtime Worker imports these, so the WebSocket path can't drift from the shared spec.

Configuration

The agent calls Anthropic (and optionally Workers AI), so the realtime Worker needs its key:

In Docker dev the realtime-dev service reads apps/web/.env; outside Docker, copy apps/realtime/.dev.vars.example to apps/realtime/.dev.vars and set the key. Provider-key management and Workers AI setup are covered in /admin/docs/ai-models.

Related

• Voice I/O — speak to the assistant and have replies read back aloud.
• Realtime — the channel/publish layer the same Worker hosts (and the agent uses for proactive messages).
• Notifications — where undeliverable proactive messages land.