---
title: "Run a one-week pilot: evaluate music recognition on your own audio"
description: "A day-by-day plan to test a music recognition API on your own audio: build a labelled test set, run it with a Python script, measure match rate, false matches, latency, and cost per hour, pick an endpoint, set thresholds, and write up the result."
slug: "/resources/articles/evaluate-music-recognition-on-your-audio"
section: "articles"
keywords: [audd, music recognition pilot, evaluate music recognition api, test set, match rate, false positives, cost per hour, python]
---

# Run a one-week pilot: evaluate music recognition on your own audio

A vendor's accuracy figure describes the vendor's test audio. Your users send
phone recordings, music under talk, 6-second clips, and sped-up edits. The only
numbers worth deciding on are the ones you measure on that audio.

This page is a five-working-day plan, plus a buffer, that ends with a written
recommendation: which endpoint to use, which score thresholds to apply, what
an hour of audio will cost, and where recognition falls short for your
content. It uses AudD in the examples; the test set and the method work for
any vendor. The companion
[buyer's checklist](/resources/articles/questions-to-ask-a-music-recognition-vendor)
covers the questions to ask alongside the numbers.

## The week at a glance

| Day | Work | Output |
|---|---|---|
| 1 | Get a token, make a first call, write down the decision you need to make | Working call, one-paragraph goal |
| 2 | Build the labelled test set | `labels.csv` and an audio folder |
| 3 | Run the set | `results.csv` |
| 4 | Measure match rate, false matches, latency, and cost | A table per category |
| 5 | Choose standard, enterprise, or streams; set thresholds | A decision and threshold values |
| 6–7 | Re-run failures, write up results | A one-page report |

## Day 1: token, first call, and the question

Start with the public `test` token. It works on the standard endpoint and
allows 10 requests a day, which is enough to confirm the integration and see
the response shape:

```bash
curl -F api_token=test -F url=https://audd.tech/example.mp3 https://api.audd.io/
```

The test token won't run a test set, and it doesn't work on the enterprise
endpoint. Get your own token at [dashboard.audd.io](https://dashboard.audd.io)
for the rest of the week, and check which plan you're on: ISRC, UPC, and the
match `score` require a Startup plan or higher. If you plan to set thresholds
on Day 5, you need `score` in your results.

```bash
pip install audd
export AUDD_API_TOKEN=your-api-token
```

Then write down, in one paragraph, the decision this pilot supports. "Can we
auto-label the songs in user uploads, and what will it cost per month?" is a
good one. "Is it accurate?" is not, because it has no threshold and no
consequence. The paragraph decides which categories go into the test set and
which metric matters most: a copyright filter cares about false matches, a
"what's playing" feature cares about match rate.

## Day 2: build a labelled test set

A test set is audio plus the answer you expect. Aim for a few dozen clips in
every category you care about; 200 to 500 clips in total is workable in a
week. Take the audio from your own product (with whatever consent your
privacy policy requires), not from a demo playlist.

| Category | What to include | Why |
|---|---|---|
| Known matches | Commercially released songs, recorded the way your users capture them | Your baseline match rate |
| Known non-matches | Speech only, silence, ambient noise, your own unreleased recordings, royalty-free tracks you made | Measures false matches |
| Phone recordings | Songs played from a speaker and recorded on a phone across a room | The usual mobile input |
| Music under speech | Podcast segments, vlogs, streams with talk over music | Common in UGC and broadcast |
| Short clips | The shortest clips your product actually sends | Finds the length where matching stops working for you |
| Transformed audio | Sped-up, slowed, pitch-shifted, re-edited versions your users post | Shows how far transformations go before matches drop |
| Your genres and regions | Local-language releases, niche genres, recent releases | Coverage where your users actually are |
| Long files (if relevant) | DJ sets, episodes, broadcasts with a known tracklist | Tests the enterprise endpoint |

Record the labels in a CSV:

```csv
file,category,expected_artist,expected_title,mode,duration_s
clips/0001.mp3,known_match,Imagine Dragons,Warriors,standard,
clips/0002.m4a,phone_recording,Imagine Dragons,Warriors,standard,
clips/0101.wav,known_non_match,,,standard,
clips/0201.mp3,music_under_speech,Daft Punk,Get Lucky,standard,
long/set-01.mp3,long_file,Daft Punk,Get Lucky,enterprise,3600
```

- Leave `expected_artist` and `expected_title` empty for known non-matches.
- `mode` is `standard` for clips and `enterprise` for long files.
- `duration_s` is only needed for long files; it feeds the cost estimate.
- For long files, label one track you know is in the file. To score a whole
  tracklist, add one row per expected track with the same file.

Keep the standard-endpoint clips under 10 MB. The standard endpoint analyzes
the first 12 seconds of what you send, so trim clips to the moment you want
tested. [Audio formats and quality](/resources/concepts/audio-format-and-quality)
lists the accepted formats.

## Day 3: run the set

This script runs every row through the standard endpoint, or through the
enterprise endpoint with a chunk `limit` for long files, and writes one row per
input to `results.csv`. It records the outcome, the returned song, the score,
the time each call took from your network, and the requests it spent.

```python
"""Run a labelled test set through AudD and write results.csv."""
import csv
import math
import time

from audd import AudD, AudDAPIError, AudDConnectionError

ENTERPRISE_LIMIT = 30  # max 12-second chunks per long file (30 chunks = 6 minutes)
EVERY, SKIP = 1, 0     # enterprise sampling; 1, 4 scans one chunk per minute

audd = AudD()  # reads AUDD_API_TOKEN


def norm(s):
    return (s or "").strip().lower()


def same_song(artist, title, row):
    return norm(row["expected_title"]) in norm(title) and norm(row["expected_artist"]) in norm(artist)


def enterprise_requests(duration_s, limit, every, skip):
    """Estimate chunks billed: one request per 12-second chunk scanned."""
    chunks = math.ceil(float(duration_s) / 12) if duration_s else limit
    scanned = sum(1 for i in range(chunks) if i % (every + skip) < every)
    return min(scanned, limit)


def outcome(matched, correct, row):
    expected = bool(row["expected_title"])
    if not matched:
        return "miss" if expected else "correct_no_match"
    if not expected:
        return "false_match"
    return "correct" if correct else "wrong_song"


def run_row(row):
    started = time.perf_counter()
    if row["mode"] == "enterprise":
        matches = audd.recognize_enterprise(
            row["file"], limit=ENTERPRISE_LIMIT, every=EVERY, skip=SKIP
        )
        elapsed = time.perf_counter() - started
        hit = next((m for m in matches if same_song(m.artist, m.title, row)), None)
        best = hit or (matches[0] if matches else None)
        requests = enterprise_requests(row["duration_s"], ENTERPRISE_LIMIT, EVERY, SKIP)
        return best, hit is not None, elapsed, requests, len(matches)

    result = audd.recognize(row["file"])
    elapsed = time.perf_counter() - started
    correct = result is not None and same_song(result.artist, result.title, row)
    return result, correct, elapsed, 1, int(result is not None)


with open("labels.csv", newline="") as f_in, open("results.csv", "w", newline="") as f_out:
    reader = csv.DictReader(f_in)
    writer = csv.writer(f_out)
    writer.writerow([
        "file", "category", "mode", "outcome", "artist", "title", "score",
        "isrc", "seconds", "requests", "matches", "error",
    ])
    for row in reader:
        try:
            song, correct, elapsed, requests, n = run_row(row)
        except (AudDAPIError, AudDConnectionError) as e:
            writer.writerow([row["file"], row["category"], row["mode"], "error",
                             "", "", "", "", "", "", "", str(e)])
            continue
        writer.writerow([
            row["file"], row["category"], row["mode"],
            outcome(song is not None, correct, row),
            song.artist if song else "",
            song.title if song else "",
            getattr(song, "score", "") if song else "",  # Startup plan or higher
            song.isrc if song else "",
            f"{elapsed:.2f}", requests, n, "",
        ])
        print(row["file"], "->", song.title if song else None)
```

A few things the script does on purpose:

- **Matching by contained title and artist.** Release titles vary
  ("Warriors" vs "Warriors - Remastered"), so an exact string match would
  count correct answers as wrong. Review every `wrong_song` row by hand; some
  are a different release of the same recording.
- **A `limit` on every enterprise call.** The enterprise endpoint bills one
  request per 12 seconds of audio it processes. Without `limit`, one long file
  can spend hundreds of requests. Raise `ENTERPRISE_LIMIT` once you have seen
  a few results. [Enterprise cost control](/resources/concepts/enterprise-cost-control)
  explains `limit`, `every`, and `skip`.
- **Requests are an estimate on enterprise.** The script works out chunks from
  `duration_s` and the sampling settings. Compare the total with the usage in
  your [dashboard](https://dashboard.audd.io) after the run.
- **Errors are recorded, not retried.** An error row is a finding (a format
  that's rejected, a file over 10 MB). The SDK already retries the failures
  that are safe to retry.
- **Sequential calls.** Timing one call at a time gives a clean latency
  number. The standard endpoint has no rate limits, so for a large set you can
  run rows in parallel with a thread pool once latency is measured; if you
  plan a large volume, tell AudD in advance at
  [api@audd.io](mailto:api@audd.io).

### The CLI alternative

If you'd rather not write code, the `audd` command-line tool runs a folder as
a batch and prints JSON lines, one per file, followed by a summary with
`requests_spent`:

```bash
audd recognize ./clips --max-files 500 --dry-run          # plan and estimated requests
audd recognize ./clips --max-files 500 --format jsonl > results.jsonl
audd recognize long/set-01.mp3 --enterprise --limit 30 --format json > set-01.json
```

The CLI caches results by file contents, so a second run of the same clips
spends nothing and is marked `"cached": true`. Pass `--no-cache` when you want
fresh timings. `--max-requests N` stops a run before it spends more than N
requests. Full flags: [docs.audd.io/cli](https://docs.audd.io/cli).

## Day 4: measure

Group `results.csv` by category:

```python
import csv
from collections import Counter, defaultdict
from statistics import median

by_cat = defaultdict(Counter)
seconds = defaultdict(list)
requests = 0
for r in csv.DictReader(open("results.csv")):
    by_cat[r["category"]][r["outcome"]] += 1
    if r["seconds"]:
        seconds[r["mode"]].append(float(r["seconds"]))
        requests += int(r["requests"])

for cat, c in sorted(by_cat.items()):
    print(f"{cat:22} {dict(c)}")
for mode, s in seconds.items():
    s.sort()
    print(f"{mode}: median {median(s):.2f}s, p95 {s[int(len(s) * 0.95) - 1]:.2f}s")
print("requests spent:", requests)
```

Four numbers matter.

**Match rate** per category: `correct / (correct + miss + wrong_song)` over
rows that have an expected song. Report each category separately. A good
overall rate can hide a poor rate on phone recordings, and phone recordings
may be most of your traffic.

**False matches**: `false_match` rows among known non-matches, plus
`wrong_song` rows. For a copyright filter this is the number that decides
whether you can act automatically. Look at the score on each false match; Day 5
uses it.

**Latency on your network**: the median and 95th percentile from the
`seconds` column. These include upload time from where the script ran, so run
the pilot from the region and network your service will run in. Sending a URL
instead of bytes changes the number; test the way you'll call it in
production.

**Cost per hour of audio**. Requests are the unit; multiply by the
per-request price of your plan from [audd.io](https://audd.io).

| How you process an hour of audio | Arithmetic | Requests per hour |
|---|---|---|
| Enterprise, every chunk (`every=1`, `skip=0`) | 3,600 s / 12 s | 300 |
| Enterprise, one chunk per minute (`every=1`, `skip=4`) | 300 / 5 | 60 |
| Enterprise, one chunk per two minutes (`every=1`, `skip=9`) | 300 / 10 | 30 |
| Standard, one clip you cut per minute | 60 calls | 60 |
| Standard, one clip per file, for 3-minute files | 20 files × 1 call | 20 |

Then scale to a month. For example, 2,000 hours of uploaded video a month,
scanned with `every=1, skip=4`, is 2,000 × 60 = 120,000 requests. The same
volume scanned end to end is 600,000. Sampling trades completeness for cost: a
short cue that falls in a skipped gap won't be found. Your Day 4 results show
whether the sampled run still catches what you need.

Streams are priced per stream per month instead of per request; see
[docs.audd.io/streams](https://docs.audd.io/streams) for the current price.

## Day 5: choose the endpoint and set thresholds

Pick the surface by the shape of your input, then confirm it with your
numbers.

| Your input | Surface | What to check in the results |
|---|---|---|
| One short clip, one song | Standard | Match rate on your clip lengths |
| Long files with many songs | Enterprise with `limit`, `skip`, `every` | Matches caught at your chosen sampling vs full coverage, cost per hour |
| Continuous live audio | Streams | Run one test stream for a day and compare against a known playlist |

The reasoning is in
[standard, enterprise, or streams](/resources/concepts/standard-vs-enterprise-vs-streams).

Next, set thresholds. Sort the matched rows by `score` and mark each correct
or not. The point above which almost everything is correct is your
auto-accept threshold. The point below which almost everything is wrong is
your reject threshold. The band between them goes to a human, or is shown with
less confidence. A blocking flow wants a high accept bar; a display flow can
accept nearly anything that comes back. AudD does not publish a recommended
cutoff, so use the values your data gives you.
[Interpreting the recognition score](/resources/concepts/score-thresholds)
has the code shape.

If the `score` column is empty, your plan doesn't include it; upgrade to a
Startup plan or higher for the pilot, or the thresholds you set won't carry
over to production.

## Days 6–7: re-run failures and write up results

Listen to every `miss`, `wrong_song`, and `false_match`. Most fall into a few
causes: the clip is too short or too quiet, the music is under loud speech,
the recording is a cover or live version (recognition matches recordings, not
compositions), or the track isn't in the database. Fix what's yours to fix
(trim to a better moment, send more of the file) and re-run those rows.
Whatever is left is a limit you write down.

A one-page write-up is enough:

1. **Decision and recommendation.** The question from Day 1 and the answer.
2. **Test set.** Size, categories, where the audio came from.
3. **Results by category.** Match rate, false matches, median and p95
   latency.
4. **Endpoint and settings.** Standard, enterprise (with `limit`, `skip`,
   `every`), or streams, and why.
5. **Thresholds.** Accept and reject scores, and the expected review volume
   at those values.
6. **Cost.** Requests per hour of audio, requests per month at expected and
   peak volume, and the plan that fits.
7. **Known gaps.** Categories that failed and what you'll do about them (human
   review, a [custom catalog](/resources/concepts/custom-vs-public-db) for
   your own unreleased tracks, or accepting the miss).

Keep `labels.csv` and the script. When you change capture devices, add a
market, or a new content type shows up, re-run the set and compare.

## FAQ

**How many clips do I need to evaluate a music recognition API?**
Enough that each category you care about has a few dozen examples. A total of 200 to 500 clips, split across known matches, known non-matches, and edge cases, is a practical size for one week. A single overall match rate hides the categories that fail.

**Can I run the pilot on the AudD test token?**
Only the first call. The test token works on the standard endpoint for 10 requests a day, which is enough to confirm the integration but not to run a test set. Get your own token at dashboard.audd.io for the pilot itself.

**How do I estimate the cost of recognizing an hour of audio?**
On the standard endpoint, count calls: each call is one request. On the enterprise endpoint, one hour of audio scanned end to end is 3,600 / 12 = 300 requests; with every=1 and skip=4 it is 60, and with every=1 and skip=9 it is 30. Multiply by the per-request price of your plan.

**What accuracy should I expect?**
There is no single number that transfers to your audio, because results depend on what you send. Measure match rate and false matches on your own labelled clips, broken down by category, and use those numbers for the decision.

---

**Related**

- [Questions to ask before choosing a music recognition API](/resources/articles/questions-to-ask-a-music-recognition-vendor)
- [Standard, enterprise, or streams: how to choose](/resources/concepts/standard-vs-enterprise-vs-streams)
- [Interpreting the recognition score](/resources/concepts/score-thresholds)
- [Enterprise cost control](/resources/concepts/enterprise-cost-control)
- [Command-line music recognition](/resources/recipes/command-line-music-recognition)
- [Music recognition API pricing](/resources/articles/music-recognition-api-pricing)

## Sitemap

See the [resources sitemap](https://audd.io/resources/sitemap.md) for every guide and the [glossary](https://audd.io/resources/reference/glossary.md) for AudD terms.