Building music recognition for mobile apps: iOS and Android guide

Building music recognition from scratch takes months; using a dedicated API with native SDKs reduces it to hours, and gives you a large reference catalog out of the box.

This guide covers implementing music recognition on both platforms: capturing audio with native recorders, calling AudD’s Swift and Kotlin SDKs, handling permissions, and optimizing for performance and battery.

Planning your implementation

Before writing code, define your requirements:

Recognition triggers. Will users tap a button, or should recognition happen automatically? Button-triggered recognition conserves battery and gives users control; automatic recognition is smoother but needs careful resource management.

Audio sources. Microphone audio, audio from video files, or both? Each source needs a slightly different capture path.

Metadata. Decide what you need beyond title and artist — album artwork, release date, streaming links. Each requested provider adds a little latency.

Performance. Define acceptable response times. AudD’s standard endpoint returns in under 2 seconds once the clip reaches the server; total round-trip includes recording and upload time.

AudD’s standard endpoint is the right surface for mobile “what’s this song?” features: it’s built for a short audio clip, returns the single best match, and matches against a database of 160 million songs. A clip of a handful of seconds is plenty.

iOS implementation

On iOS, capture audio with AVFoundation and recognize it with the official Swift SDK (github.com/AudDMusic/audd-swift).

Setting up audio recording

Add the microphone permission to Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone to identify music playing around you.</string>

Record a short clip with AVAudioRecorder:

import AVFoundation

class AudioRecorder {
    private var audioRecorder: AVAudioRecorder?
    private let session = AVAudioSession.sharedInstance()

    func setupSession() throws {
        try session.setCategory(.record, mode: .measurement, options: [])
        try session.setActive(true)
    }

    func startRecording(to url: URL) throws {
        let settings: [String: Any] = [
            AVFormatIDKey: kAudioFormatLinearPCM,
            AVSampleRateKey: 44100,
            AVNumberOfChannelsKey: 1,
            AVLinearPCMBitDepthKey: 16,
        ]
        audioRecorder = try AVAudioRecorder(url: url, settings: settings)
        audioRecorder?.record()
    }

    func stopRecording() {
        audioRecorder?.stop()
        audioRecorder = nil
    }
}

This captures CD-quality mono audio (44.1 kHz, 16-bit) — good recognition accuracy with a modest file size. Record about seven seconds; that’s enough for the standard endpoint and keeps you well under the 10 MB upload cap.

Recognizing the clip

Pass the recorded file to the Swift SDK. It handles authentication, the request, and response parsing, so you call one method:

import AudD

let audd = AudD(apiToken: "test") // get your own token at dashboard.audd.io

func identify(fileURL: URL) async {
    do {
        let song = try await audd.recognize(
            file: fileURL,
            returnMetadata: [.appleMusic, .spotify]
        )
        if let song {
            print("\(song.artist ?? "?") — \(song.title ?? "?")")
            print("Apple Music:", song.streamingURL(for: .appleMusic) ?? song.songLink ?? "")
        } else {
            // result: null — a successful no-match, NOT an error
            print("no match — try again with the music a bit louder")
        }
    } catch {
        print("recognition failed:", error)
    }
}

The key distinction: a nil song is a successful no-match (the API returns result: null), not a failure. Treat it as “try again,” never as an error. Every field on the result is optional — guard accordingly.

Handling permissions

Request microphone access before recording:

import AVFoundation

func requestMicrophonePermission(completion: @escaping (Bool) -> Void) {
    switch AVAudioSession.sharedInstance().recordPermission {
    case .granted:
        completion(true)
    case .denied:
        completion(false)
    case .undetermined:
        AVAudioSession.sharedInstance().requestRecordPermission { granted in
            DispatchQueue.main.async { completion(granted) }
        }
    @unknown default:
        completion(false)
    }
}

Handle denial gracefully by pointing users to Settings to enable microphone access.

Android implementation

On Android, capture audio with MediaRecorder and recognize it with the official Kotlin SDK (io.audd:audd-kotlin on Maven Central).

Setting up audio recording

Add permissions to AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />

Record a short clip:

import android.content.Context
import android.media.MediaRecorder
import android.os.Build
import java.io.File

class AudioRecorder(private val context: Context) {
    private var recorder: MediaRecorder? = null
    private var recording = false

    fun startRecording(outputFile: File) {
        recorder = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
            MediaRecorder(context)
        } else {
            @Suppress("DEPRECATION")
            MediaRecorder()
        }
        recorder?.apply {
            setAudioSource(MediaRecorder.AudioSource.MIC)
            setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
            setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
            setAudioSamplingRate(44100)
            setAudioEncodingBitRate(128000)
            setOutputFile(outputFile.absolutePath)
            prepare()
            start()
            recording = true
        }
    }

    fun stopRecording() {
        if (recording) {
            recorder?.apply { stop(); release() }
            recorder = null
            recording = false
        }
    }
}

Recording AAC in an MP4 container at 128 kbps keeps the upload small while staying well within the formats AudD decodes server-side.

Recognizing the clip

Pass the recorded file to the Kotlin SDK:

import io.audd.AudD

val audd = AudD("test") // get your own token at dashboard.audd.io

suspend fun identify(audioFile: File) {
    try {
        val song = audd.recognize(audioFile, returnMetadata = listOf("apple_music", "spotify"))
        if (song != null) {
            println("${song.artist} — ${song.title}")
            println("Apple Music: ${song.streamingUrl("apple_music") ?: song.songLink}")
        } else {
            // result: null — a successful no-match, NOT an error
            println("no match — try again with the music a bit louder")
        }
    } catch (e: Exception) {
        println("recognition failed: ${e.message}")
    }
}

As on iOS, a null result is a successful no-match, not an error. All fields are nullable.

Managing permissions

Request RECORD_AUDIO at runtime:

import android.Manifest
import android.content.pm.PackageManager
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
import androidx.fragment.app.FragmentActivity

class PermissionManager(private val activity: FragmentActivity) {
    companion object { const val RECORD_AUDIO_REQUEST = 1001 }

    fun ensurePermission(): Boolean {
        val granted = ContextCompat.checkSelfPermission(
            activity, Manifest.permission.RECORD_AUDIO
        ) == PackageManager.PERMISSION_GRANTED

        if (!granted) {
            ActivityCompat.requestPermissions(
                activity,
                arrayOf(Manifest.permission.RECORD_AUDIO),
                RECORD_AUDIO_REQUEST
            )
        }
        return granted
    }
}

Cross-platform considerations

UI conventions. Design recognition controls that feel native on each platform — iOS bottom sheets and navigation bars, Android material patterns — while keeping the recognition flow consistent.

Audio quality. Both platforms record high-quality audio, but defaults differ. A handful of seconds of reasonably clean audio is enough; the fingerprinting tolerates background noise and compression.

Background processing. iOS restricts background audio more than Android. Design recognition to start and finish in the foreground rather than monitoring continuously.

Battery. Record only when the user requests recognition. Avoid continuous microphone capture; cache results to skip repeat recognition of the same audio.

Performance optimization

Keep clips short. The standard endpoint analyzes up to about 12 seconds of audio; a clip in that range matches reliably while keeping upload size and latency down. Recording well beyond that mostly adds bytes the endpoint won’t analyze.

Cache results. Store recognized metadata locally so you don’t re-recognize the same audio.

Handle the network gracefully. Use request timeouts and let the SDK’s built-in retry handle transient connection failures. The SDK retries before your bytes reach the server but never repeats a recognition call after the upload completes — re-sending would risk double-billing.

Error handling and edge cases

Separate the outcomes so your UI responds correctly:

No match. The SDK returns nil/null. Not an error — render “try again,” optionally hinting the user to hold the device closer to the speaker.
Server errors. Bad token, exhausted quota, undecodable audio. The SDKs surface these as typed errors so you can branch on the cause.
Quota during testing. The test token is capped at 10 requests/day on the standard endpoint. If recognition suddenly fails with a quota error in development, switch to a real token from dashboard.audd.io.

Testing

Unit-test the pieces — recording, the recognition call, result parsing — separately, mocking the SDK where you can.

Integration-test the full flow from capture to display using known audio samples to verify accuracy and timing.

Test on real devices with varying microphone quality, and under different network conditions including slow and intermittent connectivity.

Production deployment

Token management. Never hardcode the api_token in client code. Route recognition through your backend, or store the token in platform secure storage and treat it as a bearer secret.

Monitoring. Track recognition success rates, response times, and error frequencies, and watch API usage to avoid surprise costs.

Gradual rollout. Ship recognition to a subset of users first to catch issues before full release.

Conclusion

Capture a short clip with the platform’s native recorder, hand it to AudD’s Swift or Kotlin SDK, and render the result. Focus on the fundamentals — clear permission prompts, a clean no-match path, and battery-aware recording — and the feature stays reliable.

AudD gives you 300 free requests on signup with no card, so you can wire up the flow and test it on a real device before committing to a plan.

Related