Run a one-week pilot: evaluate music recognition on your own audio

A vendor’s accuracy figure describes the vendor’s test audio. Your users send phone recordings, music under talk, 6-second clips, and sped-up edits. The only numbers worth deciding on are the ones you measure on that audio.

This page is a five-working-day plan, plus a buffer, that ends with a written recommendation: which endpoint to use, which score thresholds to apply, what an hour of audio will cost, and where recognition falls short for your content. It uses AudD in the examples; the test set and the method work for any vendor. The companion buyer’s checklist covers the questions to ask alongside the numbers.

The week at a glance

Day	Work	Output
1	Get a token, make a first call, write down the decision you need to make	Working call, one-paragraph goal
2	Build the labelled test set	`labels.csv` and an audio folder
3	Run the set	`results.csv`
4	Measure match rate, false matches, latency, and cost	A table per category
5	Choose standard, enterprise, or streams; set thresholds	A decision and threshold values
6–7	Re-run failures, write up results	A one-page report

Day 1: token, first call, and the question

Start with the public test token. It works on the standard endpoint and allows 10 requests a day, which is enough to confirm the integration and see the response shape:

curl -F api_token=test -F url=https://audd.tech/example.mp3 https://api.audd.io/

The test token won’t run a test set, and it doesn’t work on the enterprise endpoint. Get your own token at dashboard.audd.io for the rest of the week, and check which plan you’re on: ISRC, UPC, and the match score require a Startup plan or higher. If you plan to set thresholds on Day 5, you need score in your results.

pip install audd
export AUDD_API_TOKEN=your-api-token

Then write down, in one paragraph, the decision this pilot supports. “Can we auto-label the songs in user uploads, and what will it cost per month?” is a good one. “Is it accurate?” is not, because it has no threshold and no consequence. The paragraph decides which categories go into the test set and which metric matters most: a copyright filter cares about false matches, a “what’s playing” feature cares about match rate.

Day 2: build a labelled test set

A test set is audio plus the answer you expect. Aim for a few dozen clips in every category you care about; 200 to 500 clips in total is workable in a week. Take the audio from your own product (with whatever consent your privacy policy requires), not from a demo playlist.

Category	What to include	Why
Known matches	Commercially released songs, recorded the way your users capture them	Your baseline match rate
Known non-matches	Speech only, silence, ambient noise, your own unreleased recordings, royalty-free tracks you made	Measures false matches
Phone recordings	Songs played from a speaker and recorded on a phone across a room	The usual mobile input
Music under speech	Podcast segments, vlogs, streams with talk over music	Common in UGC and broadcast
Short clips	The shortest clips your product actually sends	Finds the length where matching stops working for you
Transformed audio	Sped-up, slowed, pitch-shifted, re-edited versions your users post	Shows how far transformations go before matches drop
Your genres and regions	Local-language releases, niche genres, recent releases	Coverage where your users actually are
Long files (if relevant)	DJ sets, episodes, broadcasts with a known tracklist	Tests the enterprise endpoint

Record the labels in a CSV:

file,category,expected_artist,expected_title,mode,duration_s
clips/0001.mp3,known_match,Imagine Dragons,Warriors,standard,
clips/0002.m4a,phone_recording,Imagine Dragons,Warriors,standard,
clips/0101.wav,known_non_match,,,standard,
clips/0201.mp3,music_under_speech,Daft Punk,Get Lucky,standard,
long/set-01.mp3,long_file,Daft Punk,Get Lucky,enterprise,3600

Leave expected_artist and expected_title empty for known non-matches.
mode is standard for clips and enterprise for long files.
duration_s is only needed for long files; it feeds the cost estimate.
For long files, label one track you know is in the file. To score a whole tracklist, add one row per expected track with the same file.

Keep the standard-endpoint clips under 10 MB. The standard endpoint analyzes the first 12 seconds of what you send, so trim clips to the moment you want tested. Audio formats and quality lists the accepted formats.

Day 3: run the set

This script runs every row through the standard endpoint, or through the enterprise endpoint with a chunk limit for long files, and writes one row per input to results.csv. It records the outcome, the returned song, the score, the time each call took from your network, and the requests it spent.

"""Run a labelled test set through AudD and write results.csv."""
import csv
import math
import time

from audd import AudD, AudDAPIError, AudDConnectionError

ENTERPRISE_LIMIT = 30  # max 12-second chunks per long file (30 chunks = 6 minutes)
EVERY, SKIP = 1, 0     # enterprise sampling; 1, 4 scans one chunk per minute

audd = AudD()  # reads AUDD_API_TOKEN


def norm(s):
    return (s or "").strip().lower()


def same_song(artist, title, row):
    return norm(row["expected_title"]) in norm(title) and norm(row["expected_artist"]) in norm(artist)


def enterprise_requests(duration_s, limit, every, skip):
    """Estimate chunks billed: one request per 12-second chunk scanned."""
    chunks = math.ceil(float(duration_s) / 12) if duration_s else limit
    scanned = sum(1 for i in range(chunks) if i % (every + skip) < every)
    return min(scanned, limit)


def outcome(matched, correct, row):
    expected = bool(row["expected_title"])
    if not matched:
        return "miss" if expected else "correct_no_match"
    if not expected:
        return "false_match"
    return "correct" if correct else "wrong_song"


def run_row(row):
    started = time.perf_counter()
    if row["mode"] == "enterprise":
        matches = audd.recognize_enterprise(
            row["file"], limit=ENTERPRISE_LIMIT, every=EVERY, skip=SKIP
        )
        elapsed = time.perf_counter() - started
        hit = next((m for m in matches if same_song(m.artist, m.title, row)), None)
        best = hit or (matches[0] if matches else None)
        requests = enterprise_requests(row["duration_s"], ENTERPRISE_LIMIT, EVERY, SKIP)
        return best, hit is not None, elapsed, requests, len(matches)

    result = audd.recognize(row["file"])
    elapsed = time.perf_counter() - started
    correct = result is not None and same_song(result.artist, result.title, row)
    return result, correct, elapsed, 1, int(result is not None)


with open("labels.csv", newline="") as f_in, open("results.csv", "w", newline="") as f_out:
    reader = csv.DictReader(f_in)
    writer = csv.writer(f_out)
    writer.writerow([
        "file", "category", "mode", "outcome", "artist", "title", "score",
        "isrc", "seconds", "requests", "matches", "error",
    ])
    for row in reader:
        try:
            song, correct, elapsed, requests, n = run_row(row)
        except (AudDAPIError, AudDConnectionError) as e:
            writer.writerow([row["file"], row["category"], row["mode"], "error",
                             "", "", "", "", "", "", "", str(e)])
            continue
        writer.writerow([
            row["file"], row["category"], row["mode"],
            outcome(song is not None, correct, row),
            song.artist if song else "",
            song.title if song else "",
            getattr(song, "score", "") if song else "",  # Startup plan or higher
            song.isrc if song else "",
            f"{elapsed:.2f}", requests, n, "",
        ])
        print(row["file"], "->", song.title if song else None)

A few things the script does on purpose:

Matching by contained title and artist. Release titles vary (“Warriors” vs “Warriors - Remastered”), so an exact string match would count correct answers as wrong. Review every wrong_song row by hand; some are a different release of the same recording.
A limit on every enterprise call. The enterprise endpoint bills one request per 12 seconds of audio it processes. Without limit, one long file can spend hundreds of requests. Raise ENTERPRISE_LIMIT once you have seen a few results. Enterprise cost control explains limit, every, and skip.
Requests are an estimate on enterprise. The script works out chunks from duration_s and the sampling settings. Compare the total with the usage in your dashboard after the run.
Errors are recorded, not retried. An error row is a finding (a format that’s rejected, a file over 10 MB). The SDK already retries the failures that are safe to retry.
Sequential calls. Timing one call at a time gives a clean latency number. The standard endpoint has no rate limits, so for a large set you can run rows in parallel with a thread pool once latency is measured; if you plan a large volume, tell AudD in advance at [email protected].

The CLI alternative

If you’d rather not write code, the audd command-line tool runs a folder as a batch and prints JSON lines, one per file, followed by a summary with requests_spent:

audd recognize ./clips --max-files 500 --dry-run          # plan and estimated requests
audd recognize ./clips --max-files 500 --format jsonl > results.jsonl
audd recognize long/set-01.mp3 --enterprise --limit 30 --format json > set-01.json

The CLI caches results by file contents, so a second run of the same clips spends nothing and is marked "cached": true. Pass --no-cache when you want fresh timings. --max-requests N stops a run before it spends more than N requests. Full flags: docs.audd.io/cli.

Day 4: measure

Group results.csv by category:

import csv
from collections import Counter, defaultdict
from statistics import median

by_cat = defaultdict(Counter)
seconds = defaultdict(list)
requests = 0
for r in csv.DictReader(open("results.csv")):
    by_cat[r["category"]][r["outcome"]] += 1
    if r["seconds"]:
        seconds[r["mode"]].append(float(r["seconds"]))
        requests += int(r["requests"])

for cat, c in sorted(by_cat.items()):
    print(f"{cat:22} {dict(c)}")
for mode, s in seconds.items():
    s.sort()
    print(f"{mode}: median {median(s):.2f}s, p95 {s[int(len(s) * 0.95) - 1]:.2f}s")
print("requests spent:", requests)

Four numbers matter.

Match rate per category: correct / (correct + miss + wrong_song) over rows that have an expected song. Report each category separately. A good overall rate can hide a poor rate on phone recordings, and phone recordings may be most of your traffic.

False matches: false_match rows among known non-matches, plus wrong_song rows. For a copyright filter this is the number that decides whether you can act automatically. Look at the score on each false match; Day 5 uses it.

Latency on your network: the median and 95th percentile from the seconds column. These include upload time from where the script ran, so run the pilot from the region and network your service will run in. Sending a URL instead of bytes changes the number; test the way you’ll call it in production.

Cost per hour of audio. Requests are the unit; multiply by the per-request price of your plan from audd.io.

How you process an hour of audio	Arithmetic	Requests per hour
Enterprise, every chunk (`every=1`, `skip=0`)	3,600 s / 12 s	300
Enterprise, one chunk per minute (`every=1`, `skip=4`)	300 / 5	60
Enterprise, one chunk per two minutes (`every=1`, `skip=9`)	300 / 10	30
Standard, one clip you cut per minute	60 calls	60
Standard, one clip per file, for 3-minute files	20 files × 1 call	20

Then scale to a month. For example, 2,000 hours of uploaded video a month, scanned with every=1, skip=4, is 2,000 × 60 = 120,000 requests. The same volume scanned end to end is 600,000. Sampling trades completeness for cost: a short cue that falls in a skipped gap won’t be found. Your Day 4 results show whether the sampled run still catches what you need.

Streams are priced per stream per month instead of per request; see docs.audd.io/streams for the current price.

Day 5: choose the endpoint and set thresholds

Pick the surface by the shape of your input, then confirm it with your numbers.

Your input	Surface	What to check in the results
One short clip, one song	Standard	Match rate on your clip lengths
Long files with many songs	Enterprise with `limit`, `skip`, `every`	Matches caught at your chosen sampling vs full coverage, cost per hour
Continuous live audio	Streams	Run one test stream for a day and compare against a known playlist

The reasoning is in standard, enterprise, or streams.

Next, set thresholds. Sort the matched rows by score and mark each correct or not. The point above which almost everything is correct is your auto-accept threshold. The point below which almost everything is wrong is your reject threshold. The band between them goes to a human, or is shown with less confidence. A blocking flow wants a high accept bar; a display flow can accept nearly anything that comes back. AudD does not publish a recommended cutoff, so use the values your data gives you. Interpreting the recognition score has the code shape.

If the score column is empty, your plan doesn’t include it; upgrade to a Startup plan or higher for the pilot, or the thresholds you set won’t carry over to production.

Days 6–7: re-run failures and write up results

Listen to every miss, wrong_song, and false_match. Most fall into a few causes: the clip is too short or too quiet, the music is under loud speech, the recording is a cover or live version (recognition matches recordings, not compositions), or the track isn’t in the database. Fix what’s yours to fix (trim to a better moment, send more of the file) and re-run those rows. Whatever is left is a limit you write down.

A one-page write-up is enough:

Decision and recommendation. The question from Day 1 and the answer.
Test set. Size, categories, where the audio came from.
Results by category. Match rate, false matches, median and p95 latency.
Endpoint and settings. Standard, enterprise (with limit, skip, every), or streams, and why.
Thresholds. Accept and reject scores, and the expected review volume at those values.
Cost. Requests per hour of audio, requests per month at expected and peak volume, and the plan that fits.
Known gaps. Categories that failed and what you’ll do about them (human review, a custom catalog for your own unreleased tracks, or accepting the miss).

Keep labels.csv and the script. When you change capture devices, add a market, or a new content type shows up, re-run the set and compare.

FAQ

How many clips do I need to evaluate a music recognition API? Enough that each category you care about has a few dozen examples. A total of 200 to 500 clips, split across known matches, known non-matches, and edge cases, is a practical size for one week. A single overall match rate hides the categories that fail.

Can I run the pilot on the AudD test token? Only the first call. The test token works on the standard endpoint for 10 requests a day, which is enough to confirm the integration but not to run a test set. Get your own token at dashboard.audd.io for the pilot itself.

How do I estimate the cost of recognizing an hour of audio? On the standard endpoint, count calls: each call is one request. On the enterprise endpoint, one hour of audio scanned end to end is 3,600 / 12 = 300 requests; with every=1 and skip=4 it is 60, and with every=1 and skip=9 it is 30. Multiply by the per-request price of your plan.

What accuracy should I expect? There is no single number that transfers to your audio, because results depend on what you send. Measure match rate and false matches on your own labelled clips, broken down by category, and use those numbers for the decision.

Related