Desktop AI Assistant

J.A.R.V.I.S. on a Raspberry Pi 4

A voice-driven desk assistant on a Raspberry Pi 4 with a touchscreen: offline speech, a locally-run language model, three games and presence sensing, all self-written.

[ ← home ]

[ directory ]

The assembled unit from behind: a Raspberry Pi 4 on the back of a 7-inch touchscreen on a printed honeycomb stand, with a USB mic and round speaker — fig.01 · photograph

kind: project
course: ECE 4180 final project, Georgia Tech
category: ai
period: Jan 2025 - May 2025
status: completed

head -n 20 README

A solo project: a touchscreen desk assistant with fully offline speech and a language model running on the Pi itself, kept usable on a device with no GPU and a low-cost USB microphone.

A Raspberry Pi 4 with the official seven-inch touchscreen runs a full-screen Python app. Speech is entirely local: a spectral noise-reduction front end (I wrote a microphone calibration tool and a baseline-noise capture to make a cheap mic usable) feeds an offline recognizer, and espeak-ng gives it a robotic voice. A quantized Llama 3.2 1B model runs on the Pi for conversation, guided by an in-character system prompt; only weather and web lookups ever leave the device. Around that sit chess against a local Stockfish engine with voice moves, tic-tac-toe against a minimax opponent, voice hangman, reminders with a spoken alarm, a to-do list, volume control, a double-clap detector and an ultrasonic sensor that greets you when you sit down. About 2,400 lines of Python across fourteen files, on a printed stand.

grep -n highlights

01Built the whole assistant solo: about 2,400 lines of Python across fourteen files.
02Ran speech fully offline, and wrote a microphone calibration tool and a noise-reduction front end so an offline recognizer stayed usable on a cheap USB mic.
03Ran a quantized Llama 3.2 1B locally for conversation, with an in-character system prompt; only weather and web lookups leave the device.
04Added chess against Stockfish with voice moves, a minimax tic-tac-toe, voice hangman, reminders, a to-do list and volume control.
05Handled presence and gestures with an ultrasonic greeter and a double-clap detector, serializing the shared microphone with a lock.

ls ./media

Wiring diagram of the Raspberry Pi 4, display controller, ultrasonic sensor, USB microphone, speaker and power supply — fig.02How it goes together: the Pi, the display, the ultrasonic sensor and the audio peripherals.