Featured Project · Multimodal Agent
Hermes Live Companion
A human-governed, multimodal AI companion built with voice, vision, memory, tools, and interruption control.
The starting question
Can today's devices already deliver part of the AI-hardware future?
The project began during a conversation about what a future AI device might become. A laptop already has a microphone, speakers, camera, screen, network, and compute. Hermes already had sessions, memory, tools, and vision.
The missing piece was not another model. It was a governed, real-time interaction layer that could connect those capabilities without taking over the existing system.
Can the devices and agents already on hand do it first?
What was built
One session. Six connected capabilities.
The MVP connects real inputs, agent reasoning, visible controls, and natural outputs in a local-first experience.
Voice loop
Microphone input, speech recognition, streamed responses, and text-to-speech.
Vision
User-approved camera captures and scene analysis inside the same conversation.
Memory & sessions
Multi-turn context with explicit reset, stop, and session controls.
Tools & approvals
Existing Hermes tools with visible progress and human approval boundaries.
Interruption control
Keyboard, button, and acoustic barge-in controls keep the human in charge.
Local-first safety
Loopback-only services, temporary tokens, bounded files, and cleanup rules.
System architecture
A visible local runtime, not a black box.
Voice and vision enter through controlled channels. Hermes coordinates memory and tools, then streams text and speech back to the user.
Browser and services remain on 127.0.0.1.
A dedicated worker preserves the regular Hermes environment.
Tool progress, approvals, stop, reset, and interruption stay observable.
Development method
Human-Governed Agent Development
The speed came from clear definitions, narrow phases, explicit checkpoints, and reversible changes—not from asking AI to build everything at once.
Define before building
Experience, boundaries, non-goals, safety rules, and acceptance criteria were written first.
Prove one chain at a time
The smallest end-to-end path was validated before voice, vision, and interruption were layered in.
Keep human decisions explicit
Direction, priority, risk approval, and release readiness remained human responsibilities.
Make progress reversible
Isolation, pinned dependencies, Git checkpoints, and preservation of the existing environment reduced risk.
Build timeline
From one question to a public submission.
The work was completed across two days of fragmented time. The 8h 46m figure is active development time, not elapsed calendar time.
- 01
Question and product concept
- 02
Plan and safety boundaries
- 03
Text and session foundation
- 04
Voice and interruption
- 05
Vision and approvals
- 06
Packaging and security review
- 07
Demo and Build Week submission
Product evidence
The interface makes agent activity inspectable.
Conversation, camera analysis, microphone status, runtime settings, and interruption controls remain visible in one operating surface.
- Voice, vision, and memory in one session
- Visible runtime and model status
- Human-accessible controls throughout
From MVP to delivery
Shipping is part of the product.
After the MVP worked, the project still needed to become safe, legible, and verifiable outside its original machine.
Clean and isolate the public package
Test, scan, document, and rebuild
Prepare the demo and project evidence
Submit to OpenAI Build Week through Devpost
The larger lesson
AI amplifies execution. Human value moves upstream: finding the problem, defining the system, controlling risk, and deciding when the result is ready for the real world.