My first ever "e2e" ML project during early covid quarantine was a crude vision model to detect pigeons on my balcony (and pigeons only) and play a loud random sound. It almost worked sometimes.
I did exactly this with BirdNet-Go and my Unifi doorbell cam. Unifi exposes an RTSP feed for each camera so it was easy for the tool to just "listen" to the doorbell and start classifying.
I have a spare e-ink display and my next weekend project to follow onto this is to wire it up so it shows some faux "woodcut" images of birds detected, or something like that.
Love your eink example! You’ve just sent me on a parts shopping trip :)
I’ve been unhappy with my little lcd board I’ve been using for the same here https://github.com/simonjgreen/OpenObservatory as the contrast is bad. I was considering using one of my trmnl displays, but it isn’t quite the right operating model for the frequency of updates I want.
Do you mean an RTSP audio only stream separate from the video+audio feed you might ingest to an NVR, or combined audio/video RTSP stream? Most manufacturers of security cameras with ordinary RTSP stream functions support provide both video and audio together. 'Frigate' for instance will auto detect audio stream on a reolink camera and it "just works" in the latest versions of Frigate.
Have you had any difficulty with simultaneously retrieving >1 RTSP stream from the camera at a time, 24x7? Such as full resolution video/audio to a NVR for security purposes and then a second stream to birdnet?
The journey to a magical, free bird sound ID app involves some obvious steps (the NSF gave them money), but the amount of timeline offramps is really high. The NSF didn't think this app was educational ("it's just telling them the answer, they don't even have to work for it!"), the funding ran out so they were thinking of charging money, eBird happened to have five years of bird sound recordings they could use, a computer vision guy on the team happened to learn that spectrograms are basically the same as pictures...
And the result is tremendously useful! They get great signal on where birds are, and they're making people so much more interested in birds and nature and science. And I think charging $1 for the app or, shudder, some sort of subscription model might've ruined the whole thing.
Funnily enough I was in the bathroom (that's where I do my most random thinking) just yesterday evening thinking how template matching in computer vision, e.g. https://docs.opencv.org/5.0/py_tutorials/py_imgproc/py_templ... is basically audio signal processing but instead of being temporal it is spatial. Different signals but same principle.
For me, mostly a lot of false negatives. I have a Pixel 8, and I have no problems using the microphone normally, but somehow, Merlin requires the bird to be within a couple of meters of me and sing for quite a while before Merlin will even say “hearing a bird”. And of course, you have to be dead silent; if anyone talks or coughs, it will mess up the spectogram badly.
Given that it's colloquially “Shazam for birds” and Shazam is just amazingly resistant to noise, it's a bit disappointing :-)
I think what makes Shazam magically "just work" is that they are searching for the exact embedding of the recorded version of the song - it doesn't work for a cover version, acapella, etc even if it is very similar. (Everyone would want an app that you can hum into and it will ID the song, but I guess it's a very tough problem.)
Merlin is solving the more general problem so makes sense that it's much more finicky and less accurate.
> an app that you can hum into and it will ID the song
YouTube Music and Google Search do OK with these types. Sometimes if I know a song well enough, I will open YT Music, activate the mic search, and sing a few bars, instead of typing the song title.
According to Gemini, SoundHound is also a good app for song-hummers to try.
I'm not sure why but this seems very phone dependent. I can stand next to another person, both running Merlin in silence, and mine will pick up much more than some people's phones. My phone is not particularly new (Zenfone 8).
Merlin got me into birding, and I've found this to be true, at least in some regions, especially if there's lots of background noise.
Unless a species has a really, really unique call or song, I've made it my personal rule to get eyes on it and, if I'm still not convinced, to take a picture and cross reference with iNaturalist and what folks on there think of it.
I've learned to curse the species that differ from others by the sheen of the backs of the males' necks or the angle of a wingbar. Also, gulls that only differ by things like the color of their feet or beaks.
Indeed - background noise will tend to lead to a lot more false positives, but you should use it as a guide rather than inherently trusting results. Same goes for using BirdNET-go and other models, though these do allow you to be very strict with detections and reduce false positives more.
Occasionally I'll see a screenshot of Merlin with, I dunno, a dozen different birds, and in the middle of the list: a catbird. I never have the heart to tell people that they just heard one very creative bird.
You can usually tell, though. Rare birds in the wrong place or time that come up only once are problematic, but if the bird is detected repeatedly there's a better chance. You should also be on the lookout for birds that mimic other birds (like the northern mockingbird and the gray catbird.)
Cornell / Merlin / eBird publishes a global species frequency DB indexed by location and date (so, how likely is a species to appear in a place, on a day of the year)... In a project that I was working on, this helped a lot when used to influence the "raw" identification based on appearance alone.
Tawny owls make so many different noises. The classic sound is two, one calls and one answers but they are very chatty with different sounds. They live all around, but never seen one.
I've got 33. I don't know how that stacks up against other but a large percentage of them came just from my backyard. Just got added a Baltimore Oriole that showed up this week.
I'm up to 71 thanks to the joys of taking it on holiday to other countries (less joy for my family), and occasionally going places just because I know there'll be new birds there.
Personal best for a single session is 23 birds in my back garden over the course of a couple of coffees. I have though spent a fair bit of time making the garden as attractive to different types of bird as possible - mainly through having a variety of different shaped feeders with different feed in. I can recommend the NatureSpy camera feeder too [0] - though I have had to modify it a little to make the food hole smaller otherwise magpies empty the thing in a single day just chucking all the seed on the floor. Dickheads.
I tried that with my Aqara camera. The mic there has no windshield, though; wind noise was terrible. Also I couldn’t get Aqara support to enable higher sampling rates in the firmware than 16kHz; BirdNET expects 48kHz audio samples.
Ended up installing a better microphone attached to a RPi3A+ and using a RaspberyPi 4 which I already owned for hosting BirdNET-Go
https://maciejb.me/posts/birdnet-go-setup/
Yeah, the bird use case is charming, but it's also a pretty good demonstration of how much information is sitting there waiting to be extracted from sensors that were installed for some completely different purpose.
Heads up. I noticed the screenshot of your markdown card has an ASCII block that goes below the baseline. `U+2588` (Full block) tends to do that depending on a lot of things.
I made a change to the markdown card sometime ago that uses only `U+2581` through `U+ 2587` instead to avoid that issue.
Nice! I never thought about using my security cameras but ended up building a regular Birdnet-Pi with a microphone. I took it one step further to make it portable so that I can use it when we hike or travel to new places. Added a little e-ink display to the front and it allows me to see Top 3 for the day, last detected, and total number for the day.
The documentary 'Listers ' (about two brothers who go all in on birding for a year) on YouTube is so hilarious and got me into birding (and using Merlin) this year. Highly recommend.
Really enjoyed it as well. I watched so many documentaries in college (20-25 years ago) and it really harkens back to a 90s/early00’s style documentary, in a fun way.
Why wouldn’t Ring integrate this into their cameras or at least provide a “premium” device that includes it? Seems like a marketing no brainer!
Security is all about trade offs and if the public perceives they are getting additional benefits from their devices then the trade off over surveillance becomes less so.
I’m actually surprised Flock hasn’t done it to their cameras already. “Stop cutting down our bird monitoring devices! We are doing good science!”
If they integrated actual positive benefits to their actions then society pushes back less
I just wonder, wouldn't ornithologists be interested in a system in where people voluntarily upload bird photos from their camera, helping them with bird spitting? They do that by hand already, after all. Think something like FlightRadar and crowd-sourced ADB-S info?
> I got a notification from Home Assistant that a fart was detected in the driveway. My neighbor was walking by my house on his nightly walk and ripped one as he passed by the driveway and the fart was detected.
I'd have to disable notifications for this in my neighbourhood to avoid alert fatigue.
Nice! I've tried to build something similar with Unifi cameras (https://github.com/dewey/birdhomie) and it worked pretty well, the species detection wasn't that accurate but I'll continue on that project in the winter.
I run BirdNET-Go as an app on my Home Assistant i7 M710q. I use the GPU-enabled BirdNET model and Perch V2, and it works swimmingly.
Input is three RTSP streams from various Reolink PoE cameras around our house, and it is pretty cool to be able to find and listen to the birds/frogs that come.
My impression is that the hardest part is knowing whether or not you're actually doing a good job. For vocal learners (like the ovenbird example linked above), it seems to be pretty easy using embeddings from existing models (like perch or birdnet). For non-learning species (possibly including your swifts), you can use timbre clues to differentiate many individuals, but may not be able to really nail down individual identity.
I guess the first step is to install the cameras and see what I can figure out from there. I never thought about the perching or vocalizations as an identifier.
Have you found a way to identify frogs? I also have a BirdNET (the older -PI, not -Go) setup, but haven't seen a way to get anything other than birds identified.
The full Perch V2 model at https://huggingface.co/tphakala/Perch-v2 can do some frogs and cicadas and such, but it is not very good. It will certainly detect it, whether the species will be right is a whole other question.
The sound that a frog makes is 'kwak' in Dutch. And that is also the name of this bird which makes a similar sound. Onomatopoeia for the frog predator.
I don't know how this possibly worked for the northern mockingbird. One full moon I stood and listened to one call continuously for an hour and I couldn't remember hearing the same call twice. Every bird I'd ever heard in person and more mimicked...several car alarms..sirens... Do they even have their own distinctive call?
Security cameras becoming a 24/7 backyard biodiversity monitor is already great. The fact that the same system eventually detected a neighbor farting in the driveway somehow makes it perfect.
Maybe you could try go2rtc[0]? It can ingest multiple stream formats and output them in a different one.
Also has some other nice features, like I use it to ingest one RTSP stream and share that to multiple applications, and to push audio out to my Tapo camera
I just turned my PTZ security camera into a VAR booth for my kid. Lawn (soccer) / Driveway (Floorball) / End. Live view, step back 10s, slowmo, frame step, scrubbing.
GLM 5.3 helped to root my camera so I could get proper endpoints for PTZ and stream. Claude built the api and frontend, hosted on Home Assistant.
Inspired by AvianVisitors [1] I made a similar project [2] with an android app for bird detection using Perch, and then displays the birds detected during the last 24 hours with nice art on a samsung frame tv in "art mode".
If you dont want to repurpose an old android device, my app also supports bird detections from birdnet-go and birdweather. It should be plug and play with this project.
I'm also working on a little project that is a walnut frame with a waveshare eink display and esp32 that polls for latest detection and display the picture.
I’m at a very steady state now years into running a BirdNET-GO instance. Disclaimer: I’m a contributor to the project.
I’m a pretty unhinged sorta engineer, so I actually recently created a 24/7 weather TV channel that combines real-time bird alerts, weather radar, timelapses and a sky cam feed (just a UniFi camera mounted on my home’s roof PV).
I noticed the linked blog mentions birds and bats. I thought bat chirps required higher sample rates than the microphones normally use. I have an Audiomoth and to set it up for bats you need to use the highest sample rate for proper ID.
I like this project. I have been using security cams around my place to keep track of animals meandering across the property. It's the best use for an outdoor solar-powered webcam if you ask me. You can get a feel for which animal to expect based on time of day or other characteristics. You also recognize those who regularly visit. I have a one-eyed raccoon that has been showing up for almost two years now. It's a female and the last time she visited the monitored fountain she had 5 rambunctious raccoonlets.
Thanks - this post reminded me to enter some sightings from yesterday into eBird (yes, another Cornell Lab birding app). A pair of rainbow bee eaters that I hadn't seen in the area before.
Came to say birdnet is fantastic but, at least in Australia, it regularly identifies calls with low probability (which are often wrong / skewed for location, but sometimes correct). Meaning you still need some wits about you. That said, it can correctly identify multiple birds in a single chorus which is cool. Just need to keep an eye out for those false positives.
Been working on a similar project for doing animal behavioral analysis on cameras setup in a barn. I got YOLO running in real-time on several streams on a RTX4070. Managing the training data and doing DINO->YOLO distillation has me writing my own data management suite that I plan on OSing soon.
Core use-case : push a message every time my dog is eating my shoes.
I have this too, I was looking to get more automated call recognition after we spotted a pheasant in my (suburban) back garden. I was going to buy a dedicated mike, but then realised that birdnet go works with any rtsp audio stream.
Wild that I could read this article, give DeepSeek about 4 sentences of prompts and have it running on my own homelab and unifi cameras in about 10 minutes.
The future mostly sucks but some parts are pretty cool.
how confident is it in the identification? i imagine sometimes the angle is or resolution is bad and it may detect a bird but not have a lot to go off of to say what kind. does it narrow the list down or just make a guess?
It looks like they are using audio. Even with poor audio on my phone I find that it works very well. Once in a blue moon I'll get an ID I am doubtful of from BirdNet but it's rare in my experience.
but the thing with AI is that it excels in creating things we would not doubt, regardless of the truth. in particular, i find it to be susceptible to avoiding doubt at the expense of truth.
though i would be hopeful for bird audio, as differentiating their calls is something the birds themselves excel at, so it could very well be a great usecase for ai
machine learning has been used to identify bird calls for a while now, it's more of a classifier than a content generator, so it identifies calls and provides a confidence level
while it's not perfect it's not really producing hallucinations of pterodactyls or anything like that
(i have a few https://www.birdweather.com/ devices strewn across North America and Europe because I am time poor and preferred a turnkey solution for those gifted to, but love the DIY efforts! no affiliation!)
I'm fairly satisfied with it and would recommend it to others who aren't interested in building their own solution. I like supporting the folks who are building it, although I'm unsure what the future looks like for the product. Could enshittify, could turn into Home Assistant Foundation [1] (which I also financially support). Time will tell. That's why I think its important to have open alternatives, similar to how one of the https://www.adsbexchange.com/ operators went rouge [2], sold to private equity, and others have filled the gap with new ADS-B network solutions.
Disclaimer: I’m also using this to see how far I can push supervised pure AI coding before it turns to mega slop, so know that before you dive in with the disparaging comments :D
I got a little bit into birds when I met the Merlin app. And I have thought about making a bird identification station, but I'm under the impression mostly the same species of birds visit where I live. I may be wrong though.
This would be more interesting with video rather than audio, no? I guess it depends where your security cameras are. Are there options for that as well?
I've done essentially the same thing before (using my existing Go2RTC and Frigate setup) but the challenge I ran into is that the microphones on my cameras just aren't very good.
I love AI. Thank you for such great software, anybody can do it now! Just like making videos, songs, pictures, stories, etc. What a great time to be alive! Oh, and F China for trying to sabotage our data center buildouts by leveraging useful idiots.
Ot to rain on your parade. But I made videos, songs and pictures, wrote stories back in ca. 2006 when I was 16 and read my first AI book. Without AI.
I see AI as a useful tool, but doing art with it is extremely bland compared to the real thing. I can't eben begin to describe what people who let the machine do the art miss out on.
In the non-fucked up timeline where Hilary won in 2016, Flock cameras is a nationwide network of automatic bird identification cameras to track bird population and migration.
I don't love it either, but it's better than the alternative of Peace President(tm) -- famous for strokes of genius such as "can't we just shoot [the protestors] in the legs"; asking the military to "shoot missiles into Mexico to destroy the drug labs; and the masterful decision to deny Iran nuclear weapons by withdrawing from the JCPOA and joining Israel in waging an illegitimate war.
Edit: Wow that's insane that your comment was flagged.
Yea tbh the only thing "wrong" about Hilary to me is that she rubs me the wrong way -- like I have a feeling she's just married to Bill so they can be the ultimate political power couple. But realistically that's such a terrible reason to dislike her, and is such a non-factor compared to the miles long laundry list of god-emperor Cheetolini's substantive flaws.
I just used AI(antigravity with gemini 3.6 flash, but i'm sure many others would work) to implement this and integrate it with my security cameras after reading the blog post 26 minutes ago. I suspect the author did the same thing. I also had it steal the idea from the AvianVisitors link in this thread and it displays a generated collage of the recently detected birds.
code: https://github.com/mattsahn/bird-away
Writeup: https://mattsahn.github.io/bird-away-blog/
I have a spare e-ink display and my next weekend project to follow onto this is to wire it up so it shows some faux "woodcut" images of birds detected, or something like that.
https://i.imgur.com/5XbM6bb.png
Also if anyone is interested, here is the server for the e-ink screen https://github.com/koryk/inky-impression-server
and here is a MCP for birdnet that I've been using for a bit https://github.com/koryk/birdnetpi-mcp-local-server
I’ve been unhappy with my little lcd board I’ve been using for the same here https://github.com/simonjgreen/OpenObservatory as the contrast is bad. I was considering using one of my trmnl displays, but it isn’t quite the right operating model for the frequency of updates I want.
Do you mean an RTSP audio only stream separate from the video+audio feed you might ingest to an NVR, or combined audio/video RTSP stream? Most manufacturers of security cameras with ordinary RTSP stream functions support provide both video and audio together. 'Frigate' for instance will auto detect audio stream on a reolink camera and it "just works" in the latest versions of Frigate.
Its very popular with friends and family, especially with older people that dont enjoy traditional tech.
The journey to a magical, free bird sound ID app involves some obvious steps (the NSF gave them money), but the amount of timeline offramps is really high. The NSF didn't think this app was educational ("it's just telling them the answer, they don't even have to work for it!"), the funding ran out so they were thinking of charging money, eBird happened to have five years of bird sound recordings they could use, a computer vision guy on the team happened to learn that spectrograms are basically the same as pictures...
And the result is tremendously useful! They get great signal on where birds are, and they're making people so much more interested in birds and nature and science. And I think charging $1 for the app or, shudder, some sort of subscription model might've ruined the whole thing.
https://www.youtube.com/watch?v=zl-wAqplQAo
Given that it's colloquially “Shazam for birds” and Shazam is just amazingly resistant to noise, it's a bit disappointing :-)
Merlin is solving the more general problem so makes sense that it's much more finicky and less accurate.
YouTube Music and Google Search do OK with these types. Sometimes if I know a song well enough, I will open YT Music, activate the mic search, and sing a few bars, instead of typing the song title.
According to Gemini, SoundHound is also a good app for song-hummers to try.
Unless a species has a really, really unique call or song, I've made it my personal rule to get eyes on it and, if I'm still not convinced, to take a picture and cross reference with iNaturalist and what folks on there think of it.
I've learned to curse the species that differ from others by the sheen of the backs of the males' necks or the angle of a wingbar. Also, gulls that only differ by things like the color of their feet or beaks.
Personal best for a single session is 23 birds in my back garden over the course of a couple of coffees. I have though spent a fair bit of time making the garden as attractive to different types of bird as possible - mainly through having a variety of different shaped feeders with different feed in. I can recommend the NatureSpy camera feeder too [0] - though I have had to modify it a little to make the food hole smaller otherwise magpies empty the thing in a single day just chucking all the seed on the floor. Dickheads.
[0] https://naturespy.org/collections/wifi-solar-bird-feeder-nes...
Ended up installing a better microphone attached to a RPi3A+ and using a RaspberyPi 4 which I already owned for hosting BirdNET-Go https://maciejb.me/posts/birdnet-go-setup/
Much better sound quality now!
I made a change to the markdown card sometime ago that uses only `U+2581` through `U+ 2587` instead to avoid that issue.
It's under a collapsed block here:
https://www.kyleniewiada.org/blog/2025/05/backyard-bird-trac...
Case I used to make it portable and fit the display https://makerworld.com/en/models/1336154-portable-pi-sdr-cas...
Security is all about trade offs and if the public perceives they are getting additional benefits from their devices then the trade off over surveillance becomes less so.
I’m actually surprised Flock hasn’t done it to their cameras already. “Stop cutting down our bird monitoring devices! We are doing good science!” If they integrated actual positive benefits to their actions then society pushes back less
I'd have to disable notifications for this in my neighbourhood to avoid alert fatigue.
Input is three RTSP streams from various Reolink PoE cameras around our house, and it is pretty cool to be able to find and listen to the birds/frogs that come.
Any advice on how to go about this.
My current plan is to put multiple cameras at the mouth of the chimney opening which is about 42"x42"(1mx1m).
Does this sound realistic to anyone with experience?
Here's one example: https://github.com/sammlapp/ovenbird-individual-recognition
My impression is that the hardest part is knowing whether or not you're actually doing a good job. For vocal learners (like the ovenbird example linked above), it seems to be pretty easy using embeddings from existing models (like perch or birdnet). For non-learning species (possibly including your swifts), you can use timbre clues to differentiate many individuals, but may not be able to really nail down individual identity.
Thank you for bring this up.
Also has some other nice features, like I use it to ingest one RTSP stream and share that to multiple applications, and to push audio out to my Tapo camera
[0] https://github.com/AlexxIT/go2rtc/
example: https://www.youtube.com/watch?v=TRUQsZQU60k
GLM 5.3 helped to root my camera so I could get proper endpoints for PTZ and stream. Claude built the api and frontend, hosted on Home Assistant.
I like the idea of doing something with the timestamp of the recording, or highlighting new (or recently unheard) visitors.
Inspired by AvianVisitors [1] I made a similar project [2] with an android app for bird detection using Perch, and then displays the birds detected during the last 24 hours with nice art on a samsung frame tv in "art mode".
If you dont want to repurpose an old android device, my app also supports bird detections from birdnet-go and birdweather. It should be plug and play with this project.
Have a look at [1] https://theodore.net/projects/AvianVisitors/ [2] https://github.com/simenf/birdframe
https://alec.is/posts/building-a-solar-powered-bird-station/
I'm also working on a little project that is a walnut frame with a waveshare eink display and esp32 that polls for latest detection and display the picture.
I’m a pretty unhinged sorta engineer, so I actually recently created a 24/7 weather TV channel that combines real-time bird alerts, weather radar, timelapses and a sky cam feed (just a UniFi camera mounted on my home’s roof PV).
https://www.youtube.com/live/sAC3o6KW2N0
Watchable from every TV in the house and shows the birds photo and name when it hears one. Whole thing is a single cursed FFMPEG command.
I like this project. I have been using security cams around my place to keep track of animals meandering across the property. It's the best use for an outdoor solar-powered webcam if you ask me. You can get a feel for which animal to expect based on time of day or other characteristics. You also recognize those who regularly visit. I have a one-eyed raccoon that has been showing up for almost two years now. It's a female and the last time she visited the monitored fountain she had 5 rambunctious raccoonlets.
Came to say birdnet is fantastic but, at least in Australia, it regularly identifies calls with low probability (which are often wrong / skewed for location, but sometimes correct). Meaning you still need some wits about you. That said, it can correctly identify multiple birds in a single chorus which is cool. Just need to keep an eye out for those false positives.
Core use-case : push a message every time my dog is eating my shoes.
I do need to tune the false positives though.
> No cloud services. No API calls.
> No subscription fees. No terms of service changes. No company shutting down the service in two years.
The future mostly sucks but some parts are pretty cool.
though i would be hopeful for bird audio, as differentiating their calls is something the birds themselves excel at, so it could very well be a great usecase for ai
while it's not perfect it's not really producing hallucinations of pterodactyls or anything like that
I really gotta set mine up!
(i have a few https://www.birdweather.com/ devices strewn across North America and Europe because I am time poor and preferred a turnkey solution for those gifted to, but love the DIY efforts! no affiliation!)
[1] https://www.openhomefoundation.org/
[2] https://news.ycombinator.com/item?id=45582907 (citations)
I’m using a AudioMoth USB mic so I can capture extremely high sample rate and also ultrasonic for bat detections.
https://github.com/simonjgreen/OpenObservatory
Disclaimer: I’m also using this to see how far I can push supervised pure AI coding before it turns to mega slop, so know that before you dive in with the disparaging comments :D
(you either go the path of a burner or a birdie, I don't make the rules)
I see AI as a useful tool, but doing art with it is extremely bland compared to the real thing. I can't eben begin to describe what people who let the machine do the art miss out on.
would have been a great Mr. Robot episode
Edit: Wow that's insane that your comment was flagged.
I'd still vote for her over the alternative the Republicans chose to offer to us (and did so). It was the only responsible thing to do.