The Yap App

Most translation apps break the rhythm of a real conversation: you talk, wait, then read. The Yap App needed to handle live two-way speech, printed signs, and even voice identity without making people fight the interface — and do it without leaking AI keys into a mobile binary.

I built two translation paths: a turn-based flow powered by Gemini for transcription and translation, and a hands-free Live Mode that streams Deepgram Nova-3 speech-to-text into Groq's Llama 3.3 70B for translation and ElevenLabs for spoken output, with reconnect and backoff when a stream drops. A Gemini-vision Sign Scanner translates printed text from the camera, ElevenLabs powers optional voice cloning, and every provider call runs server-side in Firebase Cloud Functions behind short-lived minted tokens. Contacts connect by QR code or deep link, with a short-lived nearby broadcast for meeting in person.

The Yap App handles real conversations: turn-by-turn on one device, hands-free Live Mode through earbuds, a camera Sign Scanner, and personal voice cloning — across 37 languages spanning everyday tongues, classical and Indigenous languages, and a few constructed ones. The Coastal Brand design system — deep teal anchored by warm earth tones — keeps the dense, real-time UI calm and legible.

• Built turn-based conversation translation plus a hands-free Live Mode that streams speech-to-text, translation, and voice in real time and routes output to earbuds.
• Shipped a camera Sign Scanner (vision OCR), personal voice cloning, and remote conversations you start by QR code or deep link — with a short-lived nearby broadcast for connecting in person.
• Routed every AI provider behind Firebase Cloud Functions so no model API keys ship in the app, and supported 37 languages including classical, Indigenous, and constructed tongues.