How AudD identifies songs: the technology behind the API

When a developer sends an audio snippet to AudD’s API, the system analyzes the audio, searches a database of 160 million songs, and returns precise metadata — artist, title, album, and, on request, streaming links. This article breaks down how that works, from fingerprinting through matching to real-time stream recognition.

The foundation: audio fingerprinting

AudD’s recognition starts with audio fingerprinting — converting an audio signal into a compact digital signature.

Converting sound to a fingerprint

This is not simply re-encoding to MP3 or WAV. AudD uses a neural network to extract perceptually important features that stay consistent across audio quality, background noise, and compression — things like spectral structure, harmonic relationships, and how frequency content changes over time. The result is a mathematical fingerprint that captures what is distinctive about a recording while discarding irrelevant variation.

Robust feature extraction

Real-world audio is rarely clean. Songs arrive through radio static, heavy compression, or background chatter. The fingerprinting approach is built to survive that:

Noise resistance. The features that go into a fingerprint are chosen to survive background interference, so scratchy recordings and noisy live streams still recognize.
Format independence. Whether you send a high-quality WAV or a heavily compressed stream, the process targets the same core characteristics, so identification holds up across sources and quality levels.
Tempo tolerance. Songs sometimes play at slightly different speeds because of playback variation or pitch correction. The fingerprint accommodates minor tempo changes.

The matching engine

With a database of 160 million songs, finding the right match requires search that is both accurate and fast.

How matching works

A query fingerprint is not compared against all 160 million songs one by one. Indexing narrows the search space first, eliminating obvious non-matches, and a detailed comparison then runs against the remaining candidates. Each candidate receives a confidence score, and only matches that clear a quality threshold are returned. That threshold is what keeps results trustworthy: a weak coincidental overlap does not become a false positive.

On a Startup plan or higher, that confidence value is exposed to you as a score on the result, so your own application can apply a stricter threshold if your use case demands it.

Handling edge cases

Short fragments. Recognition can work from a short clip by matching against the reference fingerprints rather than needing a full track.
Live and alternate versions. Live performances and remixes that reuse the original master tend to match; performances that diverge significantly from the studio recording are harder, which is inherent to fingerprinting against masters.
Multiple songs in one file. When a file contains several tracks — a DJ set, a broadcast, a long video — the enterprise endpoint chunks it server-side and returns one match per recognized segment, with timestamps, so you get a full tracklist rather than a single result.

Speed and scale

The standard endpoint is built for a short clip and a single best match, typically returning in under two seconds. The enterprise endpoint trades that immediacy for completeness: it processes long audio and video in 12-second chunks and returns every match it finds. Because it meters per 12 seconds of audio processed, you control how much gets scanned with parameters like limit, every, and skip — full coverage when you need a complete tracklist, sampling when a verdict is enough.

Real-time processing for live streams

Recognizing live audio continuously is a different problem from analyzing a finite file. Instead of handing over a file and waiting for a response, you register a source once and AudD recognizes music from it over time, delivering results as songs play.

The flow uses stream methods on api.audd.io: register where results should be delivered with setCallbackUrl, add a source with addStream (a stream URL plus a radio_id you choose), and receive results via callbacks or longpoll. This is the surface behind 24/7 radio monitoring, Twitch and YouTube live tracking, and any source that runs continuously rather than ending. Because streams overlap their analysis windows, a song is not missed during a transition or a brief drop in signal.

Integration and the API surface

AudD’s recognition is packaged into a developer-friendly API, with official SDKs in eleven languages: Python, Node/TypeScript, Go, Rust, PHP, Swift, Kotlin, .NET, Java, C, and C++.

Flexible input

File uploads and raw bytes for stored audio.
A url for remote audio or video files.
Social and video URLs, parsed server-side by the enterprise endpoint.
Live stream URLs (HLS, Icecast, DASH, m3u/m3u8) plus the shortcuts twitch:<channel>, youtube:<video_id>, and youtube-ch:<channel_id>.

The response

Every match carries the core tags — artist, title, album, and label. On a Startup plan or higher you also get the ISRC and the match score. When you ask for them with return_metadata, results include provider blocks for Apple Music, Spotify, Deezer, and MusicBrainz, plus a song_link on lis.tn. For custom-catalog matches, the result carries the integer audio_id you assigned when you uploaded the track.

from audd import AudD

audd = AudD("your-api-token")  # get a token at dashboard.audd.io

result = audd.recognize(
    "https://audd.tech/example.mp3",
    return_metadata=["apple_music", "spotify"],
)

if result:
    print(result.artist, "—", result.title)
    print(result.apple_music.url)
else:
    print("no match")

Any field the API returns that the SDK does not model as a typed property — including newer or beta fields — is available on the result’s model_extra map in Python (and extras in Node), so you can read new metadata without waiting for an SDK update.

Custom recognition with your own catalog

AudD is not limited to the public catalog. You can upload your own recordings to a custom catalog — unreleased tracks, proprietary audio, or anything you control — and recognize against them. Matches against your catalog come back carrying the integer audio_id you assigned, so your application can distinguish a custom-catalog hit from a public-catalog hit.

The result: reliable recognition at scale

Neural-network fingerprinting, indexed matching against 160 million songs, chunked processing for long files, and continuous recognition for live streams work together to handle real-world audio while keeping each API call simple. That combination is what a content-analysis tool, a copyright scanner, or a radio monitor is actually relying on when it calls the API.

Ready to try it? Get a token at dashboard.audd.io and see the full reference at docs.audd.io.

Related