Voice

The Voice app is the most real-time part of TurboStarter AI. Instead of a request-response UI, it gives users a shared audio session with an agent that can listen, reason, speak back, and stream conversation state across web and mobile.

Capabilities

LiveKit gives this app more than a microphone button. It provides the realtime transport, room lifecycle, participant state, and media controls that make the experience feel like a proper call interface instead of a chat form with audio attached.

Web

On web, the app leans into the full browser surface area. The desktop experience is especially good for demos, internal copilots, sales assistants, and any workflow where screen sharing matters.

Mobile

On mobile, the same LiveKit room and agent stack is presented through a native-first session layout. The experience is optimized for touch controls, safe areas, and audio-session handling on real devices.

Shared infrastructure

Under the UI differences, the architecture stays consistent across platforms. Both apps request a room token from the shared API layer, join a LiveKit room, and then hand the live session off to a LiveKit agent worker.

Architecture

The voice app still uses the same monorepo boundaries as the rest of the AI product. The difference is that the backend work happens around room creation and agent sessions rather than around a single streamed HTTP response.

1. The user starts a session from the platform-specific voice UI in src/modules/voice/**.
2. The shared API route module in packages/api/src/modules/ai/voice.ts runs through the normal request lifecycle, including auth context and credit checks.
3. The token-creation logic in packages/ai/voice/src/api.ts creates a short-lived LiveKit participant token and room configuration.
4. The web and mobile SessionProvider implementations use LiveKit's token source helpers to join the room.
5. The LiveKit agent worker defined in packages/ai/voice/src/agent/main.ts joins that room and drives the conversation.

This keeps the app aligned with the rest of the starter while still giving voice its own realtime transport and deployment model.

Voice architecture

The most useful mental model for this app is the classic voice pipeline: speech comes in, gets transcribed, passed to an LLM, and then rendered back to audio. LiveKit Agents supports that pattern directly, and it is the right baseline to understand before looking at realtime speech-to-speech models.

import { AgentSession } from "@livekit/agents";

const session = new AgentSession({
  stt: "deepgram/nova-3:multi",
  llm: "openai/gpt-4.1-mini",
  tts: "cartesia/sonic-3:voice-id",
});

import { AgentSession } from "@livekit/agents";
import * as openai from "@livekit/agents-plugin-openai";

const session = new AgentSession({
  llm: new openai.realtime.RealtimeModel({ voice: "cedar" }),
});

LiveKit's own docs present both approaches side by side, which is a helpful way to reason about tradeoffs: pipeline mode gives you finer provider control, while realtime mode gives you a more tightly integrated speech experience. See the Voice AI quickstart, turn handling guide, and realtime models overview.

STT and TTS provider options

One of the strengths of LiveKit Agents is that you are not locked into a single speech stack. You can mix and match providers based on latency, language coverage, cost, and voice quality.

If you are exploring the space, start with STT models, TTS models, and realtime models. For product-specific guidance inside this docs set, see Speech, Transcription, OpenAI, and ElevenLabs.

Create a project and set environment variables

The repository already knows how to build and run the voice agent, but you still need a LiveKit project to point it at. You can use LiveKit Cloud for the smoothest path, or run the LiveKit server locally when you want full control during development.

brew install livekit-cli
lk cloud auth

LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your-livekit-api-key
LIVEKIT_API_SECRET=your-livekit-api-secret

# Optional: voice-model providers
OPENAI_API_KEY=your-openai-api-key
DEEPGRAM_API_KEY=your-deepgram-api-key
CARTESIA_API_KEY=your-cartesia-api-key
ELEVENLABS_API_KEY=your-elevenlabs-api-key

brew update && brew install livekit

livekit-server --dev

LIVEKIT_URL=ws://127.0.0.1:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret

Run the agent locally

Once your environment variables are in place, local development is straightforward. The easiest path is to let Turbo orchestrate the voice package task graph for you, because packages/ai/voice/turbo.json already declares dev -> download-files -> build.

pnpm with-env turbo dev --filter=@workspace/ai-voice

If you are already working in the full monorepo, pnpm dev from the repository root is also a valid path. The root script runs pnpm with-env turbo dev, so the voice package can be started as part of the wider development graph alongside the web and mobile apps.

pnpm install

pnpm --filter @workspace/ai-voice build

pnpm --filter @workspace/ai-voice download-files

pnpm --filter @workspace/ai-voice dev

pnpm dev

The package also includes pnpm --filter @workspace/ai-voice connect and pnpm --filter @workspace/ai-voice start for alternative LiveKit agent startup modes. If you prefer the Turbo task graph for those flows too, start is also wired to depend on download-files in packages/ai/voice/turbo.json. For the bigger picture on these modes, see LiveKit's voice quickstart.

Dashboard and playground

LiveKit Cloud gives you two especially useful surfaces while building. The dashboard is your operational control plane, and the playground is your fastest browser-based testing surface when you do not want to open the full app.

Dashboard

The LiveKit Cloud dashboard is where you create projects, manage API keys, inspect agent deployments, and review operational signals after deployment.

Playground

The Agents Playground is useful when you want to verify the agent itself before you involve the full product UI. It is especially handy while tuning prompts, testing interruptions, or validating a new STT/TTS provider combination.

You can use the playground against a locally running agent in dev mode or a deployed agent in LiveKit Cloud. LiveKit covers this flow in the Voice AI quickstart.

Deployment

The repository already contains the agent build and staging flow, so deployment is less about writing Docker logic and more about understanding what the existing scripts are doing for you.

deploy command

The main entrypoint is the package-level deploy script in packages/ai/voice/package.json. It stages a deployment workspace and then hands that staged directory off to the LiveKit CLI.

pnpm --filter @workspace/ai-voice run deploy

What the staging script does

The file packages/ai/voice/src/deployment/stage-agent-deploy.ts prepares a clean .lk-stage-ai-voice workspace at the repo root. That staging step keeps the agent deployment isolated from the rest of the monorepo while still letting the Docker build reuse workspace packages.

It copies:

• the root package.json
• pnpm-lock.yaml and pnpm-workspace.yaml
• the packages and tooling directories
• the agent-specific deployment Dockerfile

How the container build works

The Dockerfile in packages/ai/voice/src/deployment/Dockerfile is already set up for the agent. It installs dependencies for @workspace/ai-voice, builds the package, pre-downloads model files, and then starts the built worker with the production start script.

That means the important work has already been encoded into the repository:

• dependency installation is workspace-aware
• the build only targets @workspace/ai-voice
• download-files runs during image build
• the final container boots directly into the LiveKit agent worker

How LiveKit Cloud sees the deployment

After staging, the script runs lk agent deploy against .lk-stage-ai-voice. LiveKit Cloud then builds the image, stores deployment metadata, and exposes the resulting agent through the dashboard and playground surfaces.

If you are using a cloud deployment, keep provider credentials such as OPENAI_API_KEY, DEEPGRAM_API_KEY, CARTESIA_API_KEY, or ELEVENLABS_API_KEY in LiveKit Cloud secrets instead of baking them into the image. LiveKit documents this flow in its deployment overview.

LiveKit project metadata

The staged workspace also contains a livekit.toml file once the CLI has linked that deployment to a specific LiveKit project and agent. Think of it as deployment metadata owned by the LiveKit toolchain rather than application code you should hand-edit frequently.

Structure

The voice feature spans the shared agent package, the API layer, and the two frontends. Keeping these boundaries clear makes it much easier to change models or deployment strategy without rebuilding the UI from scratch.

Core

The shared voice backend logic lives in packages/ai/voice/src. This is where token generation, environment handling, agent instructions, the agent entrypoint, and deployment assets live.

API

The voice route wiring lives in packages/api/src/modules/ai/voice.ts. It sits inside the same Hono request pipeline as the other AI apps, so voice still benefits from shared auth, validation, and credit logic before the request reaches the LiveKit-specific code.

Web

The web route entrypoints live in apps/web/src/app/[locale]/(apps)/voice/**, while the actual feature implementation lives in apps/web/src/modules/voice/**. That module tree contains the welcome screen, controls, transcript, session provider, settings, and visualizer components.

Mobile

The mobile route entrypoints live in apps/mobile/src/app/(apps)/voice/**, while the feature logic lives in apps/mobile/src/modules/voice/**. That is where the mobile session provider, controls, transcript, video tile, chat composer, and animations are defined.

Related documentation

Voice sits at the intersection of speech, transcription, provider choice, and realtime product design. These pages are the best next stop if you want to go deeper into one part of the stack.

References

These are the best official LiveKit references to keep open while working on the voice app:

• Voice AI quickstart
• Turn handling
• STT models
• TTS models
• Realtime models
• Running LiveKit locally
• Agent deployment overview