teaching my doorbell to talk
I spent half of yesterday trying to make my doorbell talk on its own.
I frequently miss notifications from my smart doorbell while I’m at work or my phone is on silent - and there’s nothing more frustrating than missing a delivery that could have been left at the door.
My vision was this:
- Someone rings the doorbell
- Home Assistant verifies nobody is home
- Frigate verifies an unknown visitor is at the door
- My AI Assistant generates a greeting, “Nobody is home at the moment, can I ask what you’re here with regard to?” And speaks through the doorbell speaker
- The microphone then begins listening for a response through the doorbell, whisper transcription handles speech to text
- LLM processes the response and generates an appropriate response (i.e. “You can leave the package at the door,” or “Please hold while I alert Dan to your presence”)
No cloud. No app. All local.
The mic side turned out to be extraordinarily easy, getting audio OUT of the doorbell on the other hand, turned out to be absurdly difficult.
I tried everything. The streaming proxy, the manufacturer’s API, intercepting the app’s traffic, bypassing certificate pinning, packet capture. Every reasonable approach hit a wall.
Finally, after taking a break, I took a moment to reassess and do some research. It turns out there was an already an open source project that had reverse-engineered the camera’s proprietary binary protocol. This is way beyond my capabilities, but fortunately my AI assistant happens to be an expert in all things coding and was able to translate from Rust to Python and build a client from scratch.
It would have been cool if it worked perfectly right off the bat - but there were a few bugs to work out. Though in the end we did it, a custom ElevenLabs generated AI voice played clearly through my doorbell speaker.
Now a lot of you are probably thinking, is this practical? Debatable.
Is it safe? Questionably so.
Did I have an absolute blast? 100%
Now that the hard part is done, the next step is closing the loop so the house can actually hold a conversation with whoever’s at the door.
