Prompt + guide

Build your own custom Jarvie.

Jarvie is a custom Jarvis-style desktop agent you can build with a single detailed prompt. It talks with you in real time, uses tools in the background, and gives you a place to see real outputs instead of just chatting in a text box.

What Jarvie can do

  • Create a desktop voice agent that uses gpt-realtime-2 for live conversation and tool calling.
  • Let you talk naturally while it checks things, searches, writes code, or runs tools in the background.
  • Give it computer-control tools for opening apps, clicking, typing, scrolling, and inspecting the screen.
  • Show notes, web results, menus, tables, graphics, and task progress in a clean artifact panel.
  • Keep dangerous actions confirmation-based, including sending, deleting, buying, account changes, or sharing private information.

The problem

Most voice assistants can talk, but they do not actually do much work. They usually cannot operate your computer, keep working while you keep talking, or show structured outputs like tables, notes, search results, and task progress.

Jarvie is a starting point for a more useful personal agent: voice first, tool-using, visual, and built around real workflows instead of just conversation.

Full starter prompt

Paste this into Claude Code, Codex, or Cursor.

Build a Jarvis-like desktop AI companion named Jarvie using OpenAI gpt-realtime-2 for real-time voice conversation and tool calling.

The goal:
Create a desktop AI companion that I can talk to naturally while it does useful work in the background. It should feel like a personal Jarvis: conversational, fast, visual, and able to use tools.

Core experience:
- Use OpenAI gpt-realtime-2 for live voice conversation.
- Let me interrupt, ask follow-up questions, and talk about unrelated things while tools are still running.
- Give Jarvie a minimal animated face with natural blinking, expressive moods, and speech-synced mouth movement.
- Keep the UI simple: a small companion window plus an artifact panel for anything visual or structured.
- The artifact panel should show menus, web results, graphics, notes, database tables, code snippets, and task progress.
- The artifact panel should be able to expand fullscreen.

Computer control:
- Add tools for opening apps, clicking, typing, scrolling, reading the screen, and inspecting the current UI.
- Before doing anything risky, ask for confirmation.
- If a task involves sending messages, deleting data, buying something, changing account settings, or sharing private information, pause and ask me first.

Useful built-in tools:
- Check email and summarize what matters.
- Search the web and show sources in the artifact panel.
- Take notes and organize them.
- Create, search, edit, and confirm-delete database records.
- Draft messages, documents, and code.
- Open apps and perform repetitive workflows on my computer.

Database:
- Build a flexible local database system.
- It should support create, search, edit, filter, and confirmation-based delete.
- Give me simple tables for notes, people, tasks, ideas, saved links, and projects.
- Let Jarvie add new tables when a workflow needs structure.

Personality:
- Jarvie should be concise, useful, and calm.
- It should talk like a smart operator, not a chatbot.
- It should explain what it is doing while tools run, but not over-explain.
- It should ask good clarifying questions when the task is vague.

Setup:
- Use my OpenAI API key through an environment variable called OPENAI_API_KEY.
- If I paste a key during setup, move it into a local env file and do not print it again.
- Create clear run instructions.
- Run the app locally and help me test voice, tool calling, computer control, and the artifact panel.

OpenAI API key:
[PASTE_KEY_HERE]
  • Replace the API key placeholder with your own key locally.
  • Do not commit your API key to GitHub or paste it publicly.
  • Start with safe tools first, then add computer control after the basic voice loop works.