Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

cochlea

A headless, deterministic audio engine for AI agents. Write a score as data, render it offline to byte-identical PCM, then listen through numbers — loudness, onsets, pitch, key, spectrograms — and assert what you heard. Compose → render → probe → verify, with no human ear (and no audio device) anywhere in the loop.

There is no realtime path in this project, and there never will be one. Offline render is ground truth.

What the agent sees instead of hearing a WAV file.

Why

An agent can’t listen to audio, and it shouldn’t have to read raw PCM either — a few minutes of 48kHz float audio is tens of megabytes, which is a bad way to spend a context window. Cochlea’s answer is a small set of primitives an agent can compose:

  • Score IR — ticks, tempo, tracks, notes, automation, and an optional master bus (gain + brick-wall limiter), expressed as data (Rust builder or RON). Standard MIDI Files import and export with timing intact.
  • Deterministic render — the same score renders to the same PCM bytes every time on the pinned CI target, enforced at the toolchain level, not by convention. See Determinism Contract.
  • Feature reports — loudness, true peak, onsets, pitch plus a quantized melody (note events an agent can diff against what it wrote), an MFCC timbre digest, key, tempo (with octave-alternative candidates and stability), rhythm (grid alignment with a straight-vs-triplet hypothesis test, syncopation, a trustable clear_rhythm), a chord timeline and per-section key (harmony), stereo width, structure — as a few kilobytes of JSON, or a sub-kilobyte text digest sized for an LLM’s context window. Every read tool takes a --from/--to window, so a long file can be probed a few bars at a time.
  • Spectrograms — a small PNG when a report alone doesn’t answer the question; optionally annotated with the detected beats, onsets, and pitch, and diffable as a signed A→B heat map.
  • Verify — an assertion DSL over a render (true_peak_below, pitch_matches_score, tempo_is, …), so an agent can retry on a failed assertion instead of asking a human “does that sound right?”

Every crate here works standalone. cochlea probe runs on any WAV, FLAC, mp3, or ogg file with no score in sight — that’s the adoption wedge, and it’s enforced structurally (features/spectro depend on neither score nor synth, checked via cargo tree in CI).

Where to start

  • Writing a score — the RON grammar, every preset and parameter, the verify assertions, a worked example: Score Format Reference. The same text is served in-band by cochlea reference and the MCP score_reference tool, and a test pins all three to the same generator.
  • Building instruments and scores, or wiring cochlea into a bigger system: Design & API Surface.
  • The determinism contract, and the fundsp/rustfft audits behind it: Determinism Contract.
  • Running cochlea as an MCP server so an agent calls render/probe/diff as tools: MCP Server.

Install

cargo install cochlea       # the CLI: render / probe / lint / spectro
cargo install cochlea-mcp   # the MCP stdio server
cargo add cochlea-features  # or any crate, as a library dependency

All 9 crates are on crates.io. Source, issues, and the workspace layout live at github.com/richer-richard/cochlea.

cochlea score reference (RON data form, version 1)

A score is one Score(...) value in RON syntax. Render it with cochlea render score.ron --out mix.wav (add --verify to run the embedded assertions), or via the render_score MCP tool.

Top level

Score(
    version: 1,                      // required, must be 1
    sample_rate: 48000,              // Hz
    ppq: 960,                        // ticks per quarter note (960 is standard)
    time_signature: (4, 4),          // optional, default (4, 4)
    tempo: [(tick: 0, bpm: 120.0)],  // tempo map: step changes at ticks
    tracks: [ Track(...), ... ],
    master: Master(...),             // optional master bus (below)
    verify: [ ... ],                 // optional embedded assertions (below)
)

Time is integer ticks at ppq per quarter note; positions in the data form are (bar, beat) pairs, both 1-based. Anything off the tick grid is an error, never a rounding.

Tracks and notes

Track(
    name: "lead",                    // unique per score
    instrument: Preset("saw_lead"),  // one of the preset names below
    inserts: [Preset("reverb")],     // optional per-track effect chain
    notes: [
        Note(at: (1, 1), dur: "1/4", pitch: "A4", vel: 96),
        // `off:` shifts the position by a duration past the beat —
        // e.g. the second eighth of beat 1:
        Note(at: (1, 1), off: "1/8", dur: "1/8", pitch: "C5", vel: 80),
    ],
    automation: [ Auto(...), ... ],  // optional (below)
)
  • pitch: note name + octave ("C4" = middle C, "F#3", "Bb2").
  • vel: 1–127 (MIDI-style; velocity maps to amplitude squared).
  • dur: a fraction of a whole note as a string — "1/4" quarter, "3/16", dotted "1/4.", triplet "1/8t".

A track name is free-form text and need only be unique within the score — unless you export stems (cochlea render --stems, or stems_dir on the MCP render_score tool), which writes one <track>.wav per track and so turns the name into a file name. Then it must be a portable one, checked the same way on every platform so a score exports the same stems everywhere instead of working on one host and failing on another: no path separator (/, \) and no : (a drive or an alternate data stream on Windows); none of <>"|?* and no control characters; not a Windows device name (CON, PRN, AUX, NUL, COM0COM9, LPT0LPT9, matched before the first dot whatever the extension); short enough to be a file name; and no two tracks differing only by case, since those are one file on macOS and Windows. A name that breaks a rule fails the render with the reason and the track quoted, before anything is written — it is never silently rewritten. cochlea import lifts track names straight out of a MIDI file, so an imported score is the usual place to meet this.

Automation

Auto(param: "cutoff_hz", keys: [
    Key(at: (1, 1), value: 400.0, ease: EaseInOut),
    Key(at: (3, 1), value: 4000.0),
])
  • param must be automatable on the track’s instrument (see the catalog below). Engine-level "gain" (linear, default 1.0) and "pan" (-1.0..1.0 constant-power, default 0.0) work on every track; a single Key sets a constant value.
  • ease shapes the segment leaving that key: Linear (default), Hold, EaseIn, EaseOut, EaseInOut, Bezier(x1, y1, x2, y2).
  • Automation is control-rate: sampled every 64 samples (~1.3 ms at 48 kHz). Note timing is sample-accurate.

Master bus

master: Master(
    gain_db: 3.0,                    // optional, default 0.0 (-40..=24)
    limiter: Limiter(
        ceiling_db: -1.0,            // required (-40..=0)
        lookahead_ms: 5.0,           // optional, default 5.0 (0..=50)
        release_ms: 50.0,            // optional, default 50.0 (1..=1000)
    ),
)

Applied to the f64 stem sum after mixing: gain first, then a brick-wall lookahead limiter whose sample-peak ceiling holds exactly (inter-sample true peaks can read fractionally higher — leave ~1 dB of headroom under a TruePeakBelow target). This is the tool for hitting loudness targets: push with gain_db, let the limiter hold the ceiling, and assert both with IntegratedLufs + TruePeakBelow. Omit master: entirely for a untouched bus; per-track stems are always exported pre-master.

Instrument presets

  • chord_pad (poly 16)
    • param cutoff_hz (Hz, 40..12000, default 1200)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • fm_bell (poly 12)
    • param brightness (index, 0..12, default 3)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • kick (mono)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • marimba (poly 12)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • noise_hat (poly 8)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • organ (poly 16)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • pluck (poly 16)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • saw_lead (poly 8)
    • param cutoff_hz (Hz, 40..18000, default 2400)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • sine (poly 16)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • snare (poly 4)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)
  • square_bass (mono)
    • param gain (linear, 0..4, default 1)
    • param pan (pan, -1..1, default 0)

Inserts (per-track effects): reverb.

Embedded assertions (verify:)

cochlea render score.ron --verify (or render_score with verify: true) runs these against the finished render and fails on any miss. Positions are (bar, beat).

verify: [
    IntegratedLufs(target: -14.0, tol: 0.5),      // mix loudness, LUFS
    TruePeakBelow(dbtp: -1.0),                     // headroom, dBTP
    OnsetAt(track: "drums", at: (17, 1), tol_ms: 5.0),
    PitchMatchesScore(track: "lead", tol_cents: 10.0),  // monophonic tracks
    Monotone(track: "pad", param: "cutoff_hz",
             from: (1, 1), to: (3, 1), direction: Rising),  // authored curve
    BrightnessRises(track: "pad", from: (1, 1), to: (3, 1),
                    min_ratio: 1.3),               // rendered audio actually brightens
    BrightnessFalls(track: "pad", from: (3, 1), to: (5, 1), min_ratio: 1.3),
    NoDiscontinuity(track: "lead", db: 40.0),      // click detector
    SilentAfter(at: (64, 1)),
    TempoIs(bpm: 110.0, tol_bpm: 2.0),             // optional min_bpm/max_bpm range
    HasClearRhythm(expected: true),                // grid-based rhythm trust flag
    GridAlignmentAtLeast(min: 0.9),                // fraction of onsets on the grid
    StereoWidthWithin(min: 0.03, max: 0.2),
    LraBelow(lu: 8.0),                             // loudness range, LU
    SectionCount(min: 1, max: 3),                  // detected structure sections
]

Monotone checks the authored automation curve (a score-side lint); BrightnessRises/BrightnessFalls listen to the rendered stem’s spectral centroid — assert both to prove the sweep was written and audibly happened.

Worked example

Score(
    version: 1,
    sample_rate: 48000,
    ppq: 960,
    tempo: [(tick: 0, bpm: 120.0)],
    tracks: [
        Track(
            name: "kick",
            instrument: Preset("kick"),
            notes: [
                Note(at: (1, 1), dur: "1/8", pitch: "A1", vel: 112),
                Note(at: (1, 2), dur: "1/8", pitch: "A1", vel: 100),
                Note(at: (1, 3), dur: "1/8", pitch: "A1", vel: 112),
                Note(at: (1, 4), dur: "1/8", pitch: "A1", vel: 100),
            ],
        ),
        Track(
            name: "pad",
            instrument: Preset("chord_pad"),
            inserts: [Preset("reverb")],
            notes: [
                Note(at: (1, 1), dur: "1/1", pitch: "A2", vel: 74),
                Note(at: (1, 1), dur: "1/1", pitch: "C3", vel: 70),
                Note(at: (1, 1), dur: "1/1", pitch: "E3", vel: 70),
            ],
            automation: [
                Auto(param: "cutoff_hz", keys: [
                    Key(at: (1, 1), value: 400.0, ease: EaseInOut),
                    Key(at: (2, 1), value: 3000.0),
                ]),
            ],
        ),
    ],
    verify: [
        TruePeakBelow(dbtp: -1.0),
        TempoIs(bpm: 120.0, tol_bpm: 2.0),
        BrightnessRises(track: "pad", from: (1, 1), to: (2, 1), min_ratio: 1.2),
    ],
)

Loop: lint_score to catch authoring mistakes cheaply, render_score (with verify: true), then probe_audio/probe_digest to read the result and spectrogram to look at it.

cochlea v1 plan: crate-by-crate public API surface

Written in Phase 0, before implementation. Where implementation reality diverges, this file is updated in the same PR (deviations get a Deviation marker). Determinism decisions and the dependency audits live in docs/determinism.md; the contract summary lives in the README.

Phase 0 decisions (deltas and pins)

  • fenestra-anim 0.1.0 verified on crates.io. It provides everything required: mul_div(u64, u64, u64, Rounding{Floor,Ceil,Round}) -> u64 (u128 intermediate, panics on div-by-zero and u64 overflow), typed Track<T: Interpolate> / Key<T> / key(at, value).ease(e), easing set (linear, hold, ease_in, ease_out, ease_in_out, spring, CubicBezier, Spring), Interpolate (f32 and (f32, f32)), and serde behind the serde feature. MSRV 1.88, unsafe_code = "forbid".
  • keys! lives in cochlea-score, not upstream. fenestra-anim exports the key() constructor; the keys![...] macro from the kickoff sketch is authoring sugar over cochlea tick positions (bar(1).beat(1) etc.), which fenestra-anim knows nothing about. Defining it here is not a fork of easing math — it expands to fenestra_anim::key(ticks, value).ease(e). No upstream PR needed.
  • Spring easing is rejected in v1 automation. Track::sample(frame, fps) grounds springs in seconds via an integer fps; automation runs in tick space where “ticks per second” is tempo-dependent and generally non-integer. Rather than evaluate springs subtly wrong, score validation rejects Ease::Spring on automation tracks (linear/hold/bezier are fine — they never read fps). Phase 2 can PR fractional-rate sampling upstream.
  • Tempo is stored as integer nanoseconds per quarter note (u64), converted from Bpm(f64) once at authoring time (round), like MIDI’s µs-per-quarter but 1000× finer. All tick→sample math after that point is exact rational (see score crate below). Valid BPM range [1.0, 4000.0], validated.
  • Denormal policy: honor denormals everywhere — no FTZ/DAZ, no algorithmic offsets. Full rationale in docs/determinism.md; short form: IEEE 754 fully specifies denormal arithmetic, so not flushing is the most deterministic and the most cross-platform-uniform choice, and offline rendering doesn’t care about the occasional slow filter tail. We never call fundsp’s prevent_denormals().
  • Pins (resolved on crates.io 2026-07-03): fundsp 0.23.0 (default-features off, std only — keeps symphonia and fft-convolver out), ebur128 0.1.10, rustfft 6.4.1, hound 3.5.1, ron 0.12.2, libm 0.2.16, image 0.25.10 (png only), serde 1.0.228, rayon 1.12.0, clap 4.6.1, proptest 1.11. Toolchain pinned 1.95.0; MSRV 1.88 (floor set by fenestra-anim and image).
  • Naming deviation, recorded: the user-facing name Instrument belongs to the score IR (score::Instrument::preset("saw_lead") — what a score references, serializable data). The synth-side trait (how a name becomes sound: param registry + fundsp voice graphs) is synth::Patch. The kickoff sketch used Instrument for both sides; one name for two types in two crates would be permanently confusing at call sites.
  • Stereo throughout v1. Every track renders 2 channels; mono voice graphs are panned center. Analysis that wants mono (YIN) downmixes (l + r) / 2.
  • Audit-driven rules (details in docs/determinism.md, enforced via clippy.toml bans): voices tick sample-by-sample, never fundsp process(); fundsp feedback()/fdn() combinators banned (internal thread-global x86 FTZ); ADSR/voice graphs freshly constructed per note, never pooled-and-reset (fundsp ADSR closure state survives reset()); pluck is a hand-rolled Karplus-Strong AudioNode (fundsp’s Pluck seeds its excitation from stateful funutd RNG) with counter-RNG excitation; fundsp noise()/hold() unused — all noise is the in-repo counter RNG; analysis FFTs construct FftPlannerScalar explicitly; fundsp presets use the prelude64 constructors (f64 internal state, f32 node interface); ebur128 true/sample peaks return linear amplitude — we convert to dBTP/dBFS via libm, and -inf loudness readings mean “silence/no data”, not errors.
  • The reverb insert is in-repo (P2 finding): every fundsp stereo reverb constructor (reverb_stereo, reverb4_stereo_delays) builds on fdn() internally, which flips thread-global x86 FTZ — and clippy bans can’t see call sites inside fundsp. The reverb insert is a Freeverb-style Schroeder (cochlea-synth), pure arithmetic per tick, tail ~2.5 s. Envelopes are likewise our own piecewise-linear ADSR evaluated through fundsp::envelope closures (pure arithmetic; the offline schedule knows note length, so no live gate state exists at all).

Dependency graph (acyclic, law-checked in CI)

score    → fenestra-anim, ron, serde
synth    → score, fundsp, libm
render   → score, synth, hound, rayon
features → ebur128, rustfft, hound, libm, serde        (NEVER score/synth)
spectro  → rustfft, hound, image, libm                  (NEVER score/synth)
decode   → features, symphonia                          (NEVER score/synth)
verify   → score, features, render
cli      → everything
mcp      → score, synth, render, features, spectro, verify

crates/score

Newtypes: Ticks(pub u64), Ppq(pub u32), SampleRate(pub u32), Bpm(pub f64), Vel(pub u8) (1..=127), Pitch(pub u8) (MIDI number, consts like Pitch::A4 = 69, FromStr for “A4”/“C#3”/“Bb2”, fn hz(self) -> f64 via libm), Dur { num: u32, den: u32 } (fraction of a whole note — Dur::quarter() = 1/4, eighth, half, whole, sixteenth, .dotted() (×3/2), .triplet() (×2/3), Dur::ticks(u64) escape hatch; resolved against PPQ exactly: ticks = ppq * 4 * num / den via mul_div).

Positions: Pos — built by the terse constructors from the sketch:

#![allow(unused)]
fn main() {
pub fn bar(n: u32) -> Pos;              // 1-based
impl Pos {
    pub fn beat(self, b: u32) -> Pos;   // 1-based within the bar
    pub fn plus(self, d: Dur) -> Pos;   // sub-beat offset, exact
}
}

Pos resolves to Ticks against (Ppq, TimeSignature) — 4/4 in v1 but the math takes (num, den) so time-signature support isn’t load-bearing anywhere. Resolution is exact integer math; a Pos that lands on a non-integer tick (impossible at 960 PPQ for any Dur with den | 3840) is a validation error, not a rounding.

Tempo map:

#![allow(unused)]
fn main() {
pub struct TempoMap { /* sorted (Ticks, NanosPerQuarter) steps, ppq, sample_rate */ }
impl TempoMap {
    pub fn sample_at(&self, t: Ticks) -> u64;   // THE conversion — see below
    pub fn tick_at(&self, sample: u64) -> Ticks; // inverse, Floor
    pub fn ns_at(&self, t: Ticks) -> u64;
}
}

sample_at is the one documented rounding rule: per tempo segment i starting at tick T_i with anchor sample S_i,

sample_at(t) = S_i + mul_div(t - T_i, npq_i * sr, ppq * 1_000_000_000, Rounding::Round)

Anchors are computed left-to-right with the same formula, so each tempo change rounds at most once (±0.5 samples, no accumulation) and within a segment there is zero drift. Monotonicity and exactness vs a direct u128 reference are property-tested. u64 range check: worst case npq * sr = 6e10 × 192_000 ≈ 1.2e16 < u64::MAX; enforced by the BPM and sample-rate validation ranges (SR in [8_000, 192_000]).

Score builder (mirrors the sketch):

#![allow(unused)]
fn main() {
let score = Score::new(SampleRate(48_000), Ppq(960))
    .time_signature(4, 4)
    .tempo(Ticks(0), Bpm(120.0))
    .track("lead", Instrument::preset("saw_lead"))
    .insert("lead", Insert::preset("reverb"))      // per-track insert chain
    .note("lead", bar(1).beat(1), Dur::quarter(), Pitch::A4, Vel(96))
    .automate("lead", Param::CUTOFF_HZ,
        keys![(bar(1), 400.0, ease_in_out()), (bar(3), 4_000.0)]);
}
  • score::Instrument — serializable reference: Instrument::preset(&str) | Instrument::custom(&str) (a name resolved from a caller-supplied bank at render time; the closure itself lives in synth — keeps score pure data, and RON containing a custom name fails rendering with a clear error unless the bank supplies it. Code-only, documented as non-serializable in spirit: lint warns on custom refs in .ron files).
  • Param(&'static str | owned) newtype with consts Param::CUTOFF_HZ, Param::GAIN, Param::PAN, Param::custom("..."). Snake_case strings in RON.
  • Automation: keys![...] expands to fenestra-anim keys; positions are Pos/Ticks, values f32, ease defaults linear. Stored per (track, param). Sampled at block starts by the renderer (control rate ≈1.3 ms at 64 samples / 48 kHz — documented); blocks split at event boundaries keep note-ons/offs sample-accurate.
  • Notes: Note { at: Ticks, dur: Ticks, pitch: Pitch, vel: Vel } post- resolution; builder keeps Pos/Dur authoring forms in the RON mirror.

RON data form (version = 1, round-trip tested both directions, one committed example under examples/scores/):

Score(
    version: 1,
    sample_rate: 48000,
    ppq: 960,
    time_signature: (4, 4),
    tempo: [(tick: 0, bpm: 120.0)],
    tracks: [
        Track(
            name: "lead",
            instrument: Preset("saw_lead"),
            inserts: [Preset("reverb")],
            notes: [Note(at: (1, 1), dur: "1/4", pitch: "A4", vel: 96)],
            automation: [
                Auto(param: "cutoff_hz", keys: [
                    Key(at: (1, 1), value: 400.0, ease: EaseInOut),
                    Key(at: (3, 1), value: 4000.0),
                ]),
            ],
        ),
    ],
    verify: [],   // assertion data forms, see crates/verify
)

Durations/pitches are strings in RON (“1/4”, “1/8.”, “A4”, “C#3”); positions are (bar, beat) tuples with an optional third sub-beat element (bar, beat, "1/16"). Ease names: Linear | Hold | EaseIn | EaseOut | EaseInOut | Bezier(x1, y1, x2, y2).

Static validation (score.validate(&impl Catalog) -> Vec<LintFinding>): overlapping notes on a mono instrument, params outside declared ranges, automation targeting unknown params, tracks with no events, BPM/SR out of range, spring ease on automation, custom instrument refs in data form. Catalog is a small trait (param specs + polyphony per instrument name) implemented by synth’s registry — keeps score free of synth while letting lints know param ranges.

Tests: unit (bar/beat/dur resolution exactness, tempo anchors), property (monotone sample_at, zero drift over 1e9 ticks vs u128 reference, tick_at(sample_at(t)) == t where exact), RON round-trip.


crates/synth

#![allow(unused)]
fn main() {
pub trait Patch: Send + Sync {
    fn name(&self) -> &str;
    fn params(&self) -> &[ParamSpec];          // typed registry
    fn polyphony(&self) -> Polyphony;          // Mono | Poly(NonZeroU8)
    fn release_ticks_hint(&self) -> f64;       // release tail in seconds — voice lifetime is static
    fn voice(&self, note: VoiceCtx) -> Box<dyn AudioUnit>;  // fresh fundsp graph per note
}

pub struct ParamSpec { pub param: Param, pub unit: Unit, pub range: RangeInclusive<f32>, pub default: f32 }
pub struct VoiceCtx { pub pitch: Pitch, pub vel: Vel, pub sample_rate: SampleRate,
                      pub note_seed: u64,     // hash(track, note index) — feeds counter RNG
                      pub start_sample: u64 } // absolute, for (seed, sample_index) noise keying
}
  • Param application: each automatable param is a fundsp shared variable (shared()/var()) wired into the voice graph; the renderer sets them at block starts. Settings are plain f32 stores — deterministic.
  • presets() registry: sine, saw_lead (filtered saw + ADSR), square_bass (mono), chord_pad (detuned saws + LPF), noise_hat (counter-RNG noise + fast envelope), pluck (Karplus-Strong, stateful but deterministic; excitation from the counter RNG, not fundsp’s noise). Registry implements score::Catalog for lints.
  • Insert presets: reverb (fundsp reverb, fixed parameters, applied per-track post voice-sum). Registry entry like patches.
  • Counter RNG (in-repo, the only randomness in the workspace):
#![allow(unused)]
fn main() {
pub fn crng(seed: u64, index: u64) -> u64;    // SplitMix64-style finalizer of seed ^ mix(index)
pub fn crng_f32(seed: u64, index: u64) -> f32; // uniform [-1, 1)
}

Noise is a pure function of (seed, sample_index) — random access, no state. fundsp’s noise()/hold()/Pluck are not used (audited: funutd- backed, stateful); the hat’s noise source is a custom AudioNode over crng, and pluck is a hand-rolled Karplus-Strong node with crng excitation.

  • All transcendentals in our DSP code go through libm. fundsp internals are covered by the audit in determinism.md.

Tests: two identical voices produce identical buffers; crng reference vectors; param registry ranges match preset defaults; each preset renders non-silence and decays after release.


crates/render

#![allow(unused)]
fn main() {
pub fn render(score: &Score) -> Result<Rendered, RenderError>;                    // presets only
pub fn render_with(score: &Score, bank: &PatchBank) -> Result<Rendered, RenderError>; // + customs

pub struct Rendered {
    // interleaved stereo f32; mix defined as f64-sum of the f32 stems in
    // fixed track order, then f64→f32 — so mix == sum(stems) byte-exactly
    pub fn mix(&self) -> &[f32];
    pub fn stems(&self) -> impl Iterator<Item = (&str, &[f32])>;
    pub fn sample_rate(&self) -> SampleRate;
    pub fn write_wav(&self, path: &Path) -> Result<...>;      // 32-bit float WAV (hound)
    pub fn write_stems(&self, dir: &Path) -> Result<...>;
}
}
  • Event schedule (pure): score → per-track sorted Vec<Event> with sample indices resolved once via TempoMap::sample_at (the single rounding). Events: NoteOn/NoteOff. Ordering: by sample, then track order, then note index — total and documented.
  • Blocks: fixed 64-sample max, split at event boundaries. Automation sampled at block-start ticks (TempoMap::tick_at of block start) and written to shared params before the block ticks.
  • Voices: per-track pool sized by Patch::polyphony. Allocation and stealing (oldest note) are functions of the schedule alone; voice lifetime is note span + release tail computed statically. Voices tick sample-by-sample (AudioUnit::tick), never fundsp block process() — one code path, one rounding story (see determinism.md on SIMD).
  • Track render: voices sum into an f64 stereo accumulator in voice-index order → insert chain (f32 in fundsp) → f64 track gain/pan → f32 stem.
  • Master: stems summed at f64 in fixed track order → f32 mix.
  • Parallelism: rayon over tracks; each track fully independent; deterministic fold in track order afterward. parallel == serial tested byte-for-byte.

Tests: same score twice → byte-equal; parallel == serial; stems f64-sum == mix byte-equal; voice stealing scenario (poly 2, 3 overlapping notes → oldest stolen, schedule-derived); golden PCM SHA-256 on the Tier 1 target (cfg-gated).


crates/features

Input type (no score, no synth — ever):

#![allow(unused)]
fn main() {
pub struct Audio { pub samples: Vec<f32> /* interleaved */, pub channels: u16, pub sample_rate: u32 }
impl Audio { pub fn from_wav(path: &Path) -> Result<Self, ...>;
             pub fn mono(&self) -> Vec<f32>; }

pub fn probe(audio: &Audio, opts: &ProbeOpts) -> Report;
}

Report (serde, schema_version: 2 — v1 lacked loudness.lra, tempo, stereo, and structure; v3 split tempo/rhythm — 0.2.0 Deviation ledger below applies):

{
  "schema_version": 3,
  "source": { "sample_rate": 48000, "channels": 2, "samples": 480000, "duration_ms": 10000.0 },
  "loudness": { "integrated_lufs": -14.2, "momentary_max_lufs": -10.1,
                "true_peak_dbtp": -1.3, "sample_peak_dbfs": -1.5, "lra": 4.2 },
  "tempo":    { "bpm": 110.3, "confidence": 0.79, "stability": 1.0,
                "candidates": [ { "bpm": 110.3, "salience": 0.79 }, { "bpm": 55.1, "salience": 0.89 } ],
                "beat_count": 32, "mean_beat_interval_ms": 545.4 },
  "rhythm":   { "grid_alignment": 0.98, "offbeat_ratio": 0.56,
                "onset_rate_per_s": 2.79, "clear_rhythm": true },
  "stereo":   { "width": 0.01, "correlation": 1.0, "balance": -0.0 },
  "structure": { "boundaries_ms": [8000.0], "section_count": 2, "confidence": 0.74 },
  "onsets":   { "count": 17, "times_ms": [...] },
  "pitch":    { "voiced_ratio": 0.82, "median_f0_hz": 440.1,
                "segments": [ { "start_ms": 0.0, "end_ms": 500.0, "f0_hz": 440.1, "midi_nearest": 69, "cents_off": 0.4 } ] },
  "key":      { "tonic": "C", "mode": "major", "confidence": 0.87, "chroma": [/* 12 */] },
  "silence":  { "leading_ms": 0.0, "trailing_ms": 812.5, "last_audible_sample": 441000, "floor_dbfs": -60.0 },
  "clipping": { "clipped_samples": 0, "true_peak_over_0dbtp": false }
}
  • Loudness: ebur128 (I | M | TRUE_PEAK | SAMPLE_PEAK), stereo default channel map. Integrated LUFS, momentary max, true peak dBTP.
  • Onsets: STFT (rustfft, 1024/256 hop, Hann via libm), half-wave- rectified spectral flux, median-based adaptive threshold, peak picking with minimum inter-onset gap. Times in ms (sample-derived).
  • Pitch: YIN (difference function + CMNDF, threshold 0.1, parabolic interpolation), 2048 window / 512 hop on mono downmix; per-segment median f0. Run per stem for multi-voice material — the caller (cli) does that; features stays per-buffer.
  • Key: 12-bin chroma from STFT magnitude (log-frequency weighting), Krumhansl-Schmuckler correlation against 24 major/minor templates → (tonic, mode, confidence = top correlation vs runner-up margin folded in).
  • Silence/tail: windowed RMS (50 ms hop) under floor (default −60 dBFS); last-audible sample. Clipping: |sample| ≥ 1.0 count; true peak > 0 dBTP flag.
  • Score-context enrichment (nearest tick for onsets, cents-vs-score) is NOT here — verify/cli join features output against the score.

Tests (fixtures synthesized in-test with libm, no synth dep): 440 Hz sine → A4 within 1 cent, LUFS of a −18 dBFS sine ≈ −18 LUFS (K-weighting at 1 kHz ≈ 0), click track → onsets at known samples ±2 ms, C major triad → C major, tail detection on a gated burst, clipping counter on a driven square.


crates/spectro

#![allow(unused)]
fn main() {
pub struct SpectroOpts { fft: usize /*2048*/, hop: usize /*512*/, mels: usize /*128*/,
                         floor_db: f32 /*-80*/, fmin: f32, fmax: f32 }
pub fn mel_spectrogram(audio: &Audio, opts: &SpectroOpts) -> MelImage;    // matrix + axis metadata
pub fn render_png(img: &MelImage, ruler: Ruler, markers: &[Marker]) -> RgbImage;
pub fn contact_sheet(img: &MelImage, markers: &[Marker], per_tile: usize) -> RgbImage;

pub struct Marker { pub sample: u64, pub label: String }   // bar markers WITHOUT score types
}

Hand-rolled mel filterbank (Slaney-style, documented), log magnitude with dB floor, 256-entry viridis LUT (const table in-repo), time ruler in seconds, caller-supplied bar markers. Contact sheets tile N-bar sections vertically so an agent reviews a whole piece in one vision call. Audio type is re-used from features? — no: spectro must not depend on features either? (law only covers score/synth; a features dep would be harmless but keep spectro standalone: it defines its own tiny input or takes &[f32], sr directly. Decision: functions take (samples: &[f32], channels, sample_rate) plain args; cli/verify adapt.)

Tier 3 sentinels: committed PNGs for fixture signals; image diff with per-pixel tolerance + max-fraction-differing threshold (helper in this crate, used by demo tests).


crates/decode

#![allow(unused)]
fn main() {
pub fn load(path: &Path) -> Result<Audio, DecodeError>;   // Audio = cochlea_features::Audio
}

Wave 2: lossless-only real-world file input. Dispatches on file extension: .wav/.wave delegates straight to cochlea_features::Audio::from_wav (hound); .flac goes through symphonia’s bundled FLAC reader+decoder (the symphonia crate, default-features = false, features = ["flac"] — no other format/codec, so no lossy decode sneaks in via a shared feature). Depends on cochlea-features only for the Audio type; never score/synth, same law as features/spectro.

FLAC is lossless by spec, so a correct decode reproduces the source PCM exactly — but symphonia-bundle-flac left-justifies every sample into the full 32-bit range regardless of the stream’s true bit depth (its own decode_inner comment: “the decoder uses a 32bit sample format as a common denominator”). Normalizing by always dividing by 2^31 (not by a bit-depth- derived scale) is what makes FLAC decode land on the same f32 bits as the WAV twin — verified, not assumed, by tests/sample_exact.rs against tiny committed FLAC fixtures with WAV twins (tests/fixtures/).

mp3/ogg (lossy) are explicitly next, not this round (docs/superpowers/ specs/2026-07-09-agent-audio-v2-design.md §2/§6).


crates/verify

#![allow(unused)]
fn main() {
use cochlea_verify::VerifyExt;   // extension trait on Rendered

let report = rendered.verify(&score)
    .integrated_lufs(-14.0, Tol(0.5))
    .true_peak_below(-1.0)
    .onset_at("drums", bar(17).beat(1), Ms(5.0))
    .pitch_matches_score("lead", Cents(10.0))
    .monotone("lead", Param::CUTOFF_HZ, bar(1)..bar(3), Direction::Rising)
    .no_discontinuity("lead", Db(40.0))
    .silent_after(bar(64))
    .run();                       // -> VerifyReport

pub struct VerifyReport { pub passed: bool, pub failures: Vec<Failure>, ... } // serde JSON
}
  • Newtypes Tol(f32), Ms(f32), Cents(f32), Db(f32).
  • onset_at: nearest detected onset on that track’s stem within tolerance.
  • pitch_matches_score: YIN per note window on the stem vs score pitch.
  • monotone: the authored automation curve sampled at block rate over the range must be monotone in the given direction (validates the authored sweep; spectral verification would conflate instrument response).
  • no_discontinuity: max sample-to-sample jump (in dB of |Δ|) away from note on/off boundaries ± a guard window — click detector.
  • silent_after: windowed RMS below floor for everything after the tick.
  • Wave-2 assertions over the v2 analyzers (same builder + RON dual form): tempo_is(bpm, BpmTol) / TempoIs(bpm, tol_bpm, [min_bpm, max_bpm]) — optional search-range override, the escape hatch for >~170 BPM material where the octave prior favors half-time; has_clear_rhythm(bool); stereo_width_within(min, max); lra_below(lu); section_count(min, max). Undefined-metric policy (stated in verify::checks): bounded-above checks pass on an undefined metric, value assertions fail on one.
  • Every assertion is also a serde data form embeddable in score RON under verify:; Verifier::from_specs(&[VerifySpec]) builds the same run. CLI cochlea render score.ron --verify runs them, writes the JSON failure report to stdout (or --report path), exits nonzero on failure.

crates/cli (cochlea binary)

cochlea render score.ron --out mix.wav [--stems dir/] [--verify] [--report report.json]
cochlea probe input.{wav,flac} [--json report.json] [--spectro spec.png]
                               [--digest] [--segments timeline.json] [--window-ms 1000]
cochlea diff a.{wav,flac} b.{wav,flac} [--json compare.json] [--tier2] [--window-ms 1000]
cochlea lint score.ron
cochlea spectro input.{wav,flac} --out spec.png [--sheet --bars-per-tile 8]

clap derive; probe ships in P3, the rest complete in P4; probe --digest/--segments and diff land in v2 wave 1 (the token-cheap read path: digest text instead of JSON, feature-space diff with a byte-identical / tier2-equivalent / different verdict). probe with no flags prints the JSON report to stdout. Exit codes: 0 ok, 1 verify/lint failures (and diff --tier2 when the verdict is not equivalent), 2 usage/IO errors. --window-ms rejects non-finite or sub-1 ms values at the flag boundary (NaN defeats downstream range checks; sub-millisecond windows would round to one-sample segments and explode the timeline), and distinct output flags pointing at one path are a usage error, not a silent last-write-wins.

The cochlea-mcp sibling binary (crates/mcp, v2 wave 1) serves the same pipeline as MCP tools over newline-delimited JSON-RPC 2.0 on stdio: render_score, probe_audio, spectrogram, lint_score, probe_digest, audio_diff. Hand-rolled protocol, no async runtime; see docs/mcp.md.


Demos (P5, demos/ as workspace tests or examples)

  1. metronome — click track; unit test asserts scheduled events land sample-exact; probe onsets within 2 ms.
  2. chord_pad — chord progression on chord_pad; chroma/key assertions.
  3. title_cue — 10 s cinematic sting: filter sweep (automation), volume envelope; asserts LUFS target, monotone cutoff, silent_after.

Each demo = a .ron score + a test running render → verify → probe, plus a committed spectrogram sentinel.


0.2.0 deviations from this plan (2026-07-21)

  • Tempo/rhythm split (schema v3). tempo.confidence became pulse clarity (normalized autocovariance), tempo gained candidates and stability, and clear_rhythm moved to a new grid-based rhythm section (grid_alignment, offbeat_ratio). The v2 confidence sketch above is superseded; the drum-groove demo now asserts HasClearRhythm(true).
  • Verify additions: GridAlignmentAtLeast, BrightnessRises/ BrightnessFalls (render-side sweep verification over the stem’s spectral centroid — the output-side companion to Monotone, which remains authored-curve-only by design).
  • Synth: eight presets (kick, snare added); chord_pad is stereo (±0.35 constant-power saw spread).
  • Self-describing reference: cochlea_score::authoring_reference feeds cochlea reference, the MCP score_reference tool, and the book’s Score Format page, pinned together by tests.
  • MCP: seven tools; inline image content for spectrograms (out_path optional); --root confinement; canonical-path clobber guards.
  • Structure detection computes a banded similarity matrix with a deterministic frame-count cap (the O(n²) full matrix is gone).

0.3.0 deviations from this plan (2026-07-22)

  • Melody + timbre (schema v4). pitch.melody (quantized note events off the YIN track — the compose loop’s read-back half) and a top-level timbre MFCC digest; CompareReport v3 adds a spectral-shape distance and rhythm.grid_changed.
  • Triplet grids. Rhythm alignment tests straight-16th and eighth-note-triplet hypotheses and reports the winner (rhythm.grid) — swing is recognized, not scored sloppy.
  • The zoom lens. Audio::window + --from/--to on probe/spectro/ diff (and from_s/to_s on the MCP tools); source.start_ms anchors windowed reports.
  • Spectro: analysis overlays (beats/onsets/pitch as plain data) and signed A→B difference heat maps.
  • Master bus. The score IR gains Master/Limiter (RON master:); the render bus is now Σ stems → master → f32, byte-inert when absent. “Bus routing” stays out of scope — this is one fixed output stage, not routing.
  • Lossy probe input. mp3/ogg decode via symphonia (“compressed- format probe (phase 2)” partially delivered) — analysis input only, contract documented in cochlea-decode.
  • MIDI import (import_midi / cochlea import / MCP tool): “MIDI import/export, out of scope for v1” is half-lifted — import only, hand-rolled SMF 0/1, timing exact, instrumentation a labeled guess. Export remains out of scope.
  • MCP: eight tools (import_midi), window/annotate params, inline diff heat maps.

Still deliberately out (design questions, not omissions): sampled instruments (external sample bytes vs the score-is-the-audio contract — needs content-hash pinning and a resampling determinism decision before any code), tempo ramps, time-signature changes, sample-accurate automation, bus/send routing, realtime anything.

0.4.0 deviations from this plan (2026-07-24)

  • Harmony (schema v5). A top-level harmony section: a chord timeline (template-matched from the shared chroma STFT) plus per-section key — the two questions the single global key can’t answer (“what’s the progression”, “what key is the bridge in”). Also loudness.short_term_max_lufs and a standalone loudness_timeline (the dynamics curve), and downbeat fields on the full TempoReport (beats_per_bar, downbeats_ms, bar_beat_at).
  • MIDI export. The other half of import: export_midi / cochlea export / the MCP export_midi tool. “MIDI import/export, out of scope for v1” is now fully lifted — timing exports exactly, instruments become rough GM labels (the inverse of the importer’s family mapping).
  • Integer PCM output. 16/24-bit WAV via --bits (WavBitDepth), for a small ordinary file; float32 stays the lossless ground truth.
  • fm_bell preset. The palette’s ninth voice and its first non-subtractive one — harmonic FM with an automatable brightness (modulation index), answering the “rich ears, thin voice” critique that every prior voice was a filtered saw/square.
  • Python bindings. bindings/python (pyo3 + maturin): an assert_audio fluent API and a pytest plugin. A detached crate, deliberately outside the determinism build.
  • Golden-audio eval. cochlea eval scores a directory of candidate renders against a directory of goldens by filename, exit 1 on any regression — the generative-model regression harness, plus a GitHub composite action.
  • Hardening (adversarial-review fixes). Authored ticks are bounded at Ticks::MAX, the read path caps decoded samples and rejects degenerate audio shape, and the MCP server contains any tool panic with catch_unwind instead of dying.
  • MCP: nine tools (export_midi added).

0.5.0 (2026-08-06)

  • Two non-subtractive voices. marimba (modal — a struck bar as a fundamental plus tuned octave partials, faded out before retirement so the ring never clicks) and organ (additive — a drawbar harmonic stack). The palette is now eleven presets, three of them non-subtractive (with fm_bell).
  • Timeline surfaces. cochlea probe --loudness/--beats and the MCP loudness_timeline / beat_grid tools expose the loudness-over-time curve and the full beat grid (every beat, downbeats, candidates, stability) — the per-time detail the compact report summaries drop. MCP is now eleven tools.
  • Overwrite guard generalized. One shared same_file (raw-or-canonical) now backs every CLI write guard (probe/diff/import/export/render), closing an aliased-path data-loss hole the old raw-string compare left open.

0.6.0 (2026-08-07)

  • transcribe: the audio→score arrow. cochlea transcribe and the MCP transcribe_audio tool turn a rendered or recorded file back into an editable RON score — melody note events read against a detected (or given) tempo, quantized to a note grid, with velocity estimated from peak level. Monophonic by construction, and every guess surfaces as a warning. MCP is now twelve tools.
    • The conversion lives in score (transcribe.rs) and takes plain NoteObservation data, never audio — so the dependency law is untouched: features and score stay independent leaves, and the front ends (CLI, MCP) do the five-line join. This is the same “receives plain data, never the other crate’s types” pattern the spectrogram annotation path already uses, pointed the other way.
  • Malformed-MIDI hardening. A time-signature denominator exponent past 31 overflowed an unchecked shift in the importer (panic in debug, garbage in release), reachable by a single byte flip. Now checked, warned, and covered by a dedicated malformed-input suite.

0.7.1 (2026-08-13)

An adversarial pass over 0.7.0’s own fixes.

  • Presence, not existence. 0.7.0’s stem-path containment check was guarded by if path.exists(), and exists() follows a symlink — so a link whose target did not exist yet reported “nothing here”, skipped the check, and was followed by File::create regardless, creating the stem outside the stems directory and outside --root at exit 0. The test is now symlink_metadata (lstat), and a link that cannot be resolved is refused rather than guessed at. General lesson for this workspace: when the question is “is something sitting at this path”, exists() is the wrong call — it answers “does the far end exist”.
  • One same-file rule, and it folds case. 0.7.0 taught the stem set that Lead and lead are one file on macOS and Windows, but left the guard between a stem and the mix/report/score comparing paths exactly — so --out d/Lead.wav --stems d with a track named lead still destroyed the mix at exit 0, on the platform most of this project’s development happens on. The rule is cochlea_render::same_target_file, enforced on every platform for the same portability reason the stem-name rule is (see docs/determinism.md).
  • The overwrite sweep finally reaches every subcommand. spectro and eval were never wired into same_filespectro audio.png --out audio.png decoded the audio and wrote the PNG over it (the .wav case is refused by the PNG encoder, which looks like a guard and isn’t), and eval --json d/a.wav replaced a golden candidate with its own report after reporting that the candidate passed.
  • Sharing a rule is not sharing a check. The fix above put the comparison in one place and left the resolving in front of it to each caller — so the Python spectrogram binding, a third front door onto the same call, had no guard at all, and the MCP server’s same_file and the CLI’s same_file were two functions with one name and different semantics. cochlea_render::same_file is now the whole predicate, resolving included, and all three front ends call exactly it. This is the same lesson as the one below it, found one level up: a rule applied per-call-site gets missed at the next call site, and “shared helper” is not evidence that it wasn’t.
  • A fallback that gives up is a guard that answers wrong. The path resolver canonicalized the parent when the file did not exist yet and returned the path untouched when the parent did not exist either — exactly the case where a run creates its own output directory, so --stems ./new --report new/lead.wav compared two spellings of one file as different and let the report land on a stem. The fallback folds components lexically now. The direction of a wrong answer is the whole design here: lexical folding can only merge two spellings (refusing a pair that would have been fine), never split one file into two.
  • The position path is bounded where positions are made. Pos::resolve multiplied (bar - 1) by ticks_per_bar unchecked, and neither factor was bounded — bar is a u32 from the file and the time signature’s beats-per-bar had no ceiling — so a crafted score overflowed u64 (panic in debug, silent wrap in release). Separately, Score::resolve never applied Ticks::MAX, so a far-future verify: position reached mul_div at render time and panicked after the mix was written. Every position now resolves through one bounded path — both arms, grid and raw: bounding the grid alone closed the RON route and left the same panic a line above it in the Rust API, since Ticks is a public newtype over a public u64 and verify(...).silent_after(Ticks(1 << 40)) goes straight into Score::resolve. Bounding the funnel retired the per-builder check_tick calls entirely — one bound, one place — with each builder now passing only the noun for the error message (“tempo change”, “automation key”), since a shared check should not cost a specific message. What stays at the builder is the note end check: at + dur is not a position, and a raw-tick Dur is unbounded.
  • Bound the input, don’t clamp the output. TimeSignature::validate accepted any nonzero numerator, and export_midi then squeezed it into the SMF meta event’s single byte with unwrap_or(u8::MAX) — a score that said 300/4 exported a file that said 255/4, silently. The bound belongs at the door (beats is 1..=255 now, named after the byte that has to hold it): one check replaces a truncation, an unreachable bar 2, and an error message that advertised u32::MAX as a legal value.
  • A pub fn that hardens its arithmetic has to check what it divides by. Pos::resolve gained checked multiplication and a tick ceiling while still calling ticks_per_beatwhole_note_ticks / unit — on a caller-built TimeSignature with public fields, before any validation. unit: 0 was a division-by-zero panic in the function that had just been hardened against untrusted numbers.

0.7.0 (2026-08-11)

  • Stem names are validated as file names. write_stems_as derived <dir>/<track>.wav from a track name without checking it, and a track name is free-form score data (hand-authored, or lifted verbatim from a MIDI track-name meta event by the importer). Path::join discards the base for an absolute argument, so a path-shaped track name wrote outside the stems directory — and, over MCP, outside --root while every path argument stayed inside it. The rule is now one public function, cochlea_render::stem_file_name (one ordinary, portable file name — separators, :, <>"|?*, control characters and the Win32 device names all refused on every platform, so a score means the same thing on every host), applied at the write sink and pre-flighted by both front ends so nothing is written when a name is refused. The same “one public rule, several callers” shape as Bpm::validate and validate_window_ms.
  • Containment is checked at the path, not just the name. A code review of the above found the name rule was necessary but not sufficient: a symlink already sitting at <stems>/<track>.wav is followed by File::create, so a legitimate name still wrote outside the stems directory and outside --root (reproduced). write_stems_as now canonicalizes the stems directory and resolves any existing stem target before writing, refusing one that lands outside — the same treatment ToolCtx::resolve_write gives the MCP server’s own path arguments, which stem paths never passed through. A link that stays inside is left alone.
  • The output-collision guards were completed. same_file canonicalized both sides, which fails for a path that does not exist yet, so for two outputs it degraded to a raw string compare and --out d/stems/lead.wav --stems d/stems/../stems destroyed the mix at exit 0; it now resolves through the canonical parent plus file name. The CLI gained the missing stem-vs-score pair (a score need not be named .ron), and the MCP render_score, which had no stem-collision guard at all, gained stem-vs-out_path and stem-vs-score_path.

Determinism: decisions, audits, and rationale

This file records the Phase 0 determinism decisions and the dependency audits behind them. The contract itself (three tiers) is summarized in the README; this is the evidence and the fine print.

The three tiers, restated precisely

  • Tier 1 — byte-identical PCM for identical inputs on the pinned target: x86_64-unknown-linux-gnu, toolchain 1.95.0 (rust-toolchain.toml). Two renders of the same score are byte-equal on any machine (tested on every CI platform); committed golden hashes are asserted on the pinned target (and on aarch64-macos, the bless machine). Empirical note: the first Tier 1 CI run (2026-07-03) matched the aarch64-macos-blessed hash exactly — with every DSP path on libm + pure arithmetic, the render is in practice byte-identical across these architectures, stronger than the tier promises. The contract remains per-pinned-target so a future divergence is a re-bless, not a breach.
  • Tier 2 — feature tolerances across platforms: integrated LUFS within 0.1 LU, onsets within 2 ms, pitch within 5 cents. These absorb the cross-target float differences Tier 1 does not promise away.
  • Tier 3 — spectrogram sentinels: image diff with per-pixel tolerance and a max-fraction-differing threshold.

Why “audio is a fold” shapes everything

Filters, envelopes, delays, and reverbs carry state; sample N depends on all samples before it. So the reproducibility unit is the whole render, not the sample. Consequences: fixed summation order everywhere (voices in index order, stems in track order), one rounding rule at the tick→sample boundary, and stochastic sources that are random-access (counter-based) so they don’t inherit the fold’s ordering sensitivity.

Denormal policy: honor denormals everywhere

Decision: no FTZ/DAZ, no algorithmic offsets. We never call fundsp’s prevent_denormals() (fundsp/src/denormal.rs — it sets x86 MXCSR to 0x9fc0, i.e. FTZ|DAZ, and is a no-op on aarch64). The audit found fundsp calls it itself inside the Feedback/Feedback2/FDN node family — with no restore, so one tick of a feedback() node flips FTZ on thread-globally on x86. Those combinators are therefore banned via clippy disallowed-methods; the stock Reverb doesn’t use them and is fine. (ebur128 also flips FTZ inside its K-weighting loop, but scoped with a restore-on-drop, analysis-side only — accepted, see its audit section.)

Rationale, in order:

  1. Determinism. IEEE 754 fully specifies denormal (subnormal) results for + − × ÷ and sqrt. Honoring them is bit-deterministic on one machine and bit-uniform across architectures. Flushing is neither: the kickoff’s FTZ option can’t even be implemented uniformly (x86 MXCSR vs aarch64 FPCR; fundsp only covers x86, and our workspace forbids the unsafe needed to touch FPCR ourselves).
  2. The performance argument doesn’t apply offline. FTZ exists so realtime reverb/filter tails don’t blow the audio callback budget. We render offline; a rare 10–100× slowdown on a handful of samples in a decaying tail costs milliseconds of wall clock, not glitches.
  3. The f64 master bus makes denormals rarer where it matters. f32 subnormals start at ~1.2e−38; summing at f64 keeps the bus far from its own subnormal range (~2.2e−308).

Trade-off accepted: fundsp’s f32 filter internals may briefly process subnormal state values at tail ends, slower than flushed. If profiling ever shows a pathological case, the phase 2 revisit is an algorithmic fix applied uniformly (documented tiny offset), never an FPU-flag fix.

Toolchain and codegen pins

  • rust-toolchain.toml pins 1.95.0; CI installs exactly that via rustup’s toolchain-file support. Bumping the toolchain is a deliberate commit that re-blesses golden hashes if they move.
  • No fast-math: we never enable -ffast-math-style flags and don’t set RUSTFLAGS codegen options that relax float semantics.
  • No implicit FMA: Rust never contracts a * b + c into fma by default; mul_add is in the clippy disallowed-methods ban list so fusion is always an explicit, #[expect]-documented choice.
  • Std float transcendentals are banned in this workspace by clippy.toml disallowed-methods (delegated to platform libm — different results per OS/libc). DSP code uses the libm crate, which is pure-Rust and bit-stable across platforms. Std sqrt is exempt: IEEE-exact, hardware instruction everywhere.

fenestra-anim 0.1.0 (audited by reading the published source)

What the timebase and automation path actually execute:

  • mul_div(a, b, c, Rounding) -> u64 (src/rational.rs): u128 intermediate, explicit Floor | Ceil | Round (ties up), panics on c == 0 and on u64 overflow of the result. Pure integer math — exact and platform-independent. This is the only tick→sample primitive we use.
  • Track<T>::sample(frame, fps)locate() (src/track.rs): segment lookup by partition_point (integer), progress u = (run as f64 / span as f64) as f32 (exact-division semantics, deterministic), then easing:
    • Ease::Linear, Ease::Hold: pure arithmetic. Deterministic everywhere.
    • Ease::Bezier (src/bezier.rs): fixed 16-iteration Newton solve using only * + - / and clamp — no transcendentals. Deterministic everywhere.
    • Ease::Spring (src/spring.rs): std f32 exp/cos/sin/sqrt — platform libm, NOT cross-platform bit-stable. v1 rejects springs on automation tracks at validation (also for the seconds-grounding reason in docs/plan.md), so no fenestra-anim transcendental runs in any v1 signal path. Springs stay available to non-audio consumers upstream.
  • Interpolate for f32: a + (b - a) * t, pure arithmetic.

Conclusion: cochlea’s automation evaluation (linear/hold/bezier over ticks) is bit-deterministic across platforms, stronger than Tier 1 requires.

fundsp 0.23.0 audit

Manifest facts (verified in the vendored crate): depends on libm directly; wide (SIMD f32x8) is unconditional; funutd is the RNG dependency; default features files (symphonia) and fft (fft-convolver) are disabled in our workspace pin (default-features = false, features = ["std"]), which keeps symphonia out of Cargo.lock entirely (also enforced by a deny.toml ban).

Audit findings (full-source read of the vendored crate; per-node-family detail with file:line citations retained in the audit transcript, digest here):

  • The node interface is f32, full stop. AudioNode::tick/process are hardcoded to Frame<f32, _> (audionode.rs:75,79); prelude64 only upgrades internal state precision (Sine<f64> phase, filter state) and still emits f32 per sample. Consequence: fundsp renders per-voice f32; cochlea’s f64 buses (voice sum, master sum) live outside fundsp’s node boundary, in render. We use the prelude64 node constructors where offered (f64 internal accumulators cost nothing offline).
  • tick() and process() provably diverge. Sine::tick computes sin via libm::sinf (oscillator.rs:71lib.rs:480); Sine::process computes it via wide’s f32x8 SIMD polynomial (oscillator.rs:82lib.rs:632) — different algorithms, no bit-equality contract. SIMD floor/ceil are even approximations ((x ± 0.4999999).round(), lib.rs:326-332). Rule: voices tick sample-by-sample; AudioNode::process/AudioUnit::process are in the clippy disallowed-methods ban list. One code path, one rounding story.
  • Scalar transcendentals all route through libm — exhaustively confirmed: Num/Float/Real impls for f32/f64 call libm::{sinf,cosf,tanf,expf,exp2f,logf,log2f,log10f,tanhf,atanf,powf,...} (lib.rs:168-280,444-594,773-825). No std float intrinsics in the scalar path. Per-sample transcendentals in nodes we use: Sine::tick (sin), Moog::tick/Rez::tick (tanh) — all libm. Biquad/Lowpole/Highpole coefficient math (tan/sin/cos/exp) runs at construction/set-parameter time only. None of Biquad, Moog, Rez, Reverb, WaveSynth, Pluck, ADSR override process() anyway — the divergence risk is concentrated in oscillators like Sine; the tick-only ban covers everything uniformly.
  • No entropy anywhere. Zero hits for SystemTime|thread_rng|rand::|Instant|getrandom|OsRng across the crate; no rand dependency. All node seeding derives from the structural graph hash chained through ping()/AttoHash (deterministic function of node IDs and combinator positions), overridable per node via Setting::Seed(u64). Two structurally identical graphs get identical output. Cochlea presets set explicit seeds/phases anyway so sound identity survives graph refactors.
  • fundsp’s own denormal handling is x86-only, scattered, and side-effectful: prevent_denormals() is called from exactly one node family — Feedback/Feedback2/FDN (feedback.rs:129,136,241,248,357,370) — and sets MXCSR FTZ|DAZ thread-globally with no restore, a no-op on aarch64. Under the honor-denormals policy those combinators are banned (feedback, feedback2, fdn, fdn2, prevent_denormals in the clippy ban list). Verified consequence: every fundsp stereo reverb constructor is off-limitsreverb_stereo builds fdn::<U32> directly (prelude.rs:1755) and reverb4_stereo_delays builds two fdns (prelude.rs:1938-1939); the clippy ban cannot see those internal call sites, so the rule is: no fundsp reverb constructors at all. The reverb insert is instead an in-repo Freeverb-style Schroeder (8 damped combs + 4 allpasses per channel, cochlea-synth/src/nodes.rs), pure arithmetic per tick.
  • ADSR reset is a trap: adsr_live’s closure captures attacked + two Shared cells that reset() cannot see (adsr.rs:29-33). Voices are therefore always freshly constructed per note, never pooled-and-reset — which the render engine does anyway by design.
  • Net/Sequencer use hashmaps only for keyed lookup (execution order is a Vec-based topological sort, net.rs:834-917), but their live-edit frontends exist for realtime use; cochlea builds static An<_> graphs and never uses Net commit APIs.
  • funutd (fundsp’s RNG dep) stays out of our signal path: fundsp’s noise()/pluck()/hold() nodes (funutd-seeded, and Pluck’s excitation is a stateful funutd::Rnd replay) are not used by cochlea presets. Noise and Karplus-Strong excitation come from the in-repo counter RNG keyed (seed, sample_index); the KS voice is hand-rolled (delay line + damping FIR as a custom AudioNode) rather than fundsp’s Pluck. funutd remains in Cargo.lock as an unused-at-runtime transitive dep.
  • Versions that can move last-bit float behavior — fundsp, libm, wide, funutd — are pinned by Cargo.lock (committed); toolchain pinned; no -C target-cpu=native anywhere.

Per-preset Tier 1 verdicts (all under the tick-only rule): sine safe (libm sin); saw_lead/square_bass/chord_pad safe (WaveSynth’s interpolation is pure arithmetic, filters libm-at-construction, ADSR fresh per voice — implemented as our own piecewise-linear closure over note time rather than fundsp’s adsr_live, sidestepping its closure-state reset trap entirely); noise_hat safe (counter-RNG noise + filter); pluck safe (hand-rolled KS, counter-RNG excitation); reverb insert safe (in-repo Schroeder — fundsp’s reverb constructors are all FDN-based and banned, see above).

ebur128 0.1.10 audit

API facts the features crate builds on (file:line citations in the audit transcript):

  • Construct EbuR128::new(channels, rate, mode); we use Mode::I | Mode::TRUE_PEAK (TRUE_PEAK implies SAMPLE_PEAK and M). Feed interleaved add_frames_f32(&[f32]). Default 2-channel map is [Left, Right] — correct for us, no set_channel needed.
  • Readouts: loudness_global() → LUFS (needs Mode::I), loudness_momentary() → LUFS over the last 400 ms (momentary max is ours to track: feed in 100 ms chunks and take the running max of readings), true_peak(ch)/sample_peak(ch)linear amplitude — we convert to dBTP/dBFS via 20·log10 (libm) ourselves.
  • Gating confirmed BS.1770-4: absolute −70 LUFS gate (energies below the first histogram boundary are discarded before recording, history.rs:264-267, boundary derived from the −70 LUFS energy) and relative −10 LU gate (history.rs:320-334), 400 ms blocks at 75% overlap.
  • Silence/not-enough-audio is not an error: loudness_* return Ok(-inf), peaks Ok(0.0). The features crate maps -inf to a JSON null-with-reason, never an error.
  • True peak: rate-dependent oversampling (4× below 96 kHz), 48-tap Hanning-windowed-sinc polyphase FIR with coefficients computed once at construction; FIR math is f32; no SIMD; the precision-true-peak feature (FMA in the FIR) stays off.
  • Internal math is otherwise f64. One arch-conditional path, flagged: the K-weighting filter loop enables scoped hardware FTZ on x86_64 (filter.rs:376-428, restored on drop) and approximates it on other arches by flushing filter state below f64::EPSILON at block boundaries. So loudness readings can differ at ULP level between x86_64 and aarch64. This is analysis-side only (never touches PCM), scoped (does not leak MXCSR state to our thread beyond the call), and absorbed by the Tier 2 0.1 LU tolerance with ~5 orders of magnitude of headroom. Accepted.

rustfft 6.4.1 audit

  • FftPlanner::new() does runtime CPU-feature dispatch (AVX+FMA → SSE4.1 → NEON → WASM → scalar). Same binary, different machine ⇒ different FFT bits; and a cloud CI runner migration could silently flip the path on the “pinned” target.
  • Decision: analysis code constructs FftPlannerScalar explicitly, everywhere (rustfft::FftPlanner::new is in the clippy ban list). One code path on every machine, immune to runner-hardware drift. Our FFTs are small (1024–2048 points at audio hop rates); scalar is more than fast enough offline, and this choice makes features/spectro deterministic per-binary, not just per-platform.
  • Planning is a pure function of (len, direction) for the scalar planner; process() panics on length mismatch (caller invariant); output is unnormalized (we only use magnitudes, and normalize where needed, consistently).
  • rustfft is used only in features and spectro — never in the PCM render path — so even its residual cross-toolchain variance is bounded by Tier 2 tolerances and Tier 3 image diffs by construction. MSRV 1.61, deps purely numeric.

Rounding rules (the complete list)

  1. Bpm(f64) → integer nanoseconds-per-quarter (round), once, at authoring/parse time. The stored score is exact from then on.
  2. Tick → sample: mul_div(Δticks, npq · sr, ppq · 1e9, Rounding::Round) per tempo segment with left-to-right integer anchors; applied once at event-schedule time. Property-tested monotone and drift-free.
  3. Sample → tick (block starts → automation domain): same segment math with Rounding::Floor (a block start belongs to the tick it is inside).
  4. f64 master bus → f32 output samples: default Rust as f32 (round-to-nearest-even), documented here, applied at the very end.

Nothing else in the pipeline rounds between integer domains.

v2 analyzers and decode (wave 2 additions)

The wave-2 surface introduces no new determinism mechanism — it reuses the existing rules — but each addition deserves its line in this ledger:

  • FLAC decode (cochlea-decode, symphonia 0.6, FLAC feature only). FLAC reconstruction is pure integer arithmetic by spec; the only place a decode could diverge from a WAV twin is float normalization. symphonia-bundle-flac left-justifies every sample into 32 bits (its own documented “common denominator”), so the crate divides by 2^31 unconditionally — exactly equal to the WAV path’s per-depth 2^(bits-1) divide (both are the same power-of-two rescale, exact in IEEE 754). Enforced bit-for-bit by committed WAV/FLAC twin fixtures. Zero-packet streams take their shape from STREAMINFO and their sample vector stays empty — no fabricated rates.
  • Tempo (tempo.rs), stereo (stereo.rs), structure (structure.rs), segment timeline (segments.rs). All transcendentals via libm; the onsets-grade STFT they share comes from the same FftPlannerScalar path as everything else; autocorrelation and the novelty kernel use fixed ascending summation order; every float sort uses total_cmp. The analyzers are pure functions of the buffer — probe() computes shared intermediates (STFT, onset report, YIN track) once and fans them out, which is a pure refactor precisely because the passes were deterministic duplicates.
  • Non-finite input policy. Float WAVs can legally encode NaN/±inf; IEEE 754 comparison semantics would let a single NaN sample masquerade as silence in one analyzer and a real peak in another. Audio::from_wav rejects non-finite samples at ingestion (NonFiniteSample), so no analyzer ever sees one. Degenerate options (NaN window/frame lengths, non-finite silence floors) are guarded with is_finite() checks — a plain <= 0.0 range check is NaN-blind.
  • Text outputs (digest, compare). Fixed-precision {:.N} formatting only, stable field order, no wall clock, no hash-map iteration — byte- deterministic per platform for identical input; the numbers inside stay Tier-2 across platforms like every other analysis float.

0.2.0 additions (2026-07-21)

  • Tempo/rhythm split. Pulse clarity (mean-removed, length-unbiased autocovariance), tempo candidates, windowed stability, and the grid-alignment rhythm analyzer are all pure arithmetic + libm over the same shared STFT, fixed summation order throughout. The Ellis beat-DP envelope is normalized by its own standard deviation — a pure function of the buffer, so normalization changes nothing about determinism, only the penalty calibration.
  • kick/snare presets, stereo chord_pad. Drum envelopes are rational 1/(1+t/tau) decays — pure arithmetic per sample, no new transcendental call sites; the pad’s two saws pan through the same constant-power fd::pan used everywhere. Golden re-bless: the drum-groove golden moved to 0xE613_FEF7_9664_B529 (deliberate DSP + score change: real kit, per-track pans, stereo pad); first-light was untouched (it uses none of the changed patches) and its hash did not move — an incidental cross-check that the preset work leaked into nothing it shouldn’t have.
  • Banded structure similarity. Computes exactly the entries the checkerboard kernel reads, in the same order, same arithmetic — byte-identical novelty curves to the full matrix, minus the O(n²) waste. The frame-count cap changes the effective frame length only as a deterministic function of input length.
  • Spectral centroid (centroid.rs). Weighted mean over the shared scalar-FFT magnitudes; a silent frame’s centroid is None, never a ratio of float-noise sums.
  • MCP base64/PNG. encode_png runs the identical image encoder as the file path; base64 is a pure byte mapping — the inline spectrogram and the PNG on disk are the same bytes.

0.3.0 additions (2026-07-22)

  • Master bus (gain + limiter). The bus becomes Σ stems (f64) → master → single f32 rounding — rounding rule 4 is unchanged and still applied exactly once. A default master returns before touching a sample, so master-less scores render byte-identically to 0.2.0 (both golden hashes confirmed unchanged). The limiter is pure arithmetic plus two libm calls at setup (pow for the ceiling, exp for the release coefficient): per-frame peaks, a forward sliding-window maximum via a monotonic deque (integer logic, no float accumulation), and a one-pole release. Gain is clamped by ceiling / windowed_peak at every frame, so the sample-peak ceiling is exact by construction on every platform — no fast-math, no FMA, nothing target-dependent.
  • Melody + timbre analyzers. Melody is integer quantization and run grouping over the existing YIN track (libm::log2/exp2 only, the same calls the pitch report already made). MFCC is a hand-rolled mel filterbank + orthonormal DCT-II over the shared scalar-FFT magnitudes — libm transcendentals, fixed summation order, and a per-frame dynamic-range floor that is a pure function of the frame.
  • Triplet-grid rhythm hypothesis. Grid geometry only — the same classifier run at two subdivision counts, winner by aligned count with a deterministic tie-break (straight). No new signal pass.
  • Lossy decode (mp3/ogg) is Tier-2-adjacent, not Tier-1. The lossy path is analysis input only: symphonia is pure Rust and our feature set keeps its opt-simd runtime dispatch off, so decoding the same file with the same build is reproducible — but no bit-exactness claim is made (the codec discarded the original samples), and nothing lossy can reach the render path. The FLAC module and its WAV-twin bit-exactness test are untouched.
  • MIDI import. Integer tick arithmetic end to end: SMF division becomes PPQ verbatim, deltas sum in u64 with overflow checks, tempo metas convert through the same Bpm → ns/quarter rounding rule 1 as authored tempos. No float time anywhere in the parser.
  • Spectrogram overlays and diffs. Drawing is integer pixel writes at positions derived from sample offsets; the diff image is per-cell f32 subtraction of two already-computed spectrograms. MelSpec::hz_band uses the same HTK mel formula as the filterbank (libm), so overlay placement is as deterministic as the spectrogram itself.

0.4.0 additions (2026-07-24)

  • Harmony (chords + per-section key). Reads the same shared chroma STFT the key estimate already builds. Chord detection is a cosine match against L2-normalized binary templates in fixed order; the per-section key runs the exact Krumhansl-Schmuckler correlation the global key uses. Only libm::log2 and the exempt std sqrt — no new transcendental call sites, fixed summation order throughout.
  • Loudness timeline + short-term max. ebur128 is fed in fixed-size chunks and polled at a fixed hop; the running maxima are tracked in order. Same audit as integrated LUFS — -inf/undefined maps to null, never a non-finite float.
  • Integer PCM output (WavBitDepth). 16/24-bit WAV is a deterministic clamp to [-1, 1], scale by 2^frac, round to nearest, clamp to the signed range — no dither (dither would trade determinism for a lower noise floor, the wrong trade here). Float32 stays the render’s ground truth; the integer paths are a lossy convenience, not a second source of truth.
  • fm_bell preset. Harmonic FM (a sine frequency-modulated by a harmonic-ratio sine), amplitude and modulation-index both driven by the same control-rate ADSR envelopes every other voice uses — pure arithmetic per sample, libm only at construction. It uses none of the patches first_light/drum_groove play, so both golden hashes are unchanged (the same incidental cross-check the drum presets got).
  • MIDI export. Score ticks become SMF ticks verbatim (PPQ is the file division); only the tempo value is microsecond-quantized, which is all SMF can carry. Delta-times are integer VLQ. No float time anywhere in the writer — the exact inverse of the import path’s integer arithmetic.
  • Hardening. The authored-tick bound (Ticks::MAX) is a pure integer comparison; the decode sample caps are integer counts checked as they accumulate; the MCP catch_unwind backstop wraps dispatch only and never touches the render fold. None of the three affects rendered bytes.

0.5.0 additions (2026-08-06)

  • marimba and organ presets. Both are pure arithmetic over exact harmonics: marimba sums modal sine partials under 1/(1+t/tau)² decays (plus a short linear fade before retirement, the same guard pluck uses); organ is a weighted sine sum under the piecewise-linear ADSR. No new transcendental call sites, no fundsp feedback/fdn/process. Golden hashes are unchanged — no flagship score plays them.
  • Loudness/beat-grid surfaces. No new DSP — they reuse the existing loudness_timeline and estimate_tempo analyzers (ebur128 polled at a fixed hop; the shared scalar-FFT tempo path), served over new CLI flags and MCP tools.

0.6.0 additions (2026-08-07)

  • transcribe (audio → score). Analysis times are inherently f64 milliseconds — a tracker measures frames, not ticks — so this path has one float→integer conversion, and it is confined to exactly one place: ms_to_ticks in crates/score/src/transcribe.rs, applied once per note boundary and immediately rounded to u64 with libm::round. Nothing accumulates in float: quantization, the minimum-duration floor, and the Ticks::MAX bound are all integer arithmetic on the rounded value. This is the same shape as the existing authoring-time rounding (Bpm → integer ns per quarter): float in, integer immediately, integer thereafter.
    • Non-finite and negative inputs are rejected at that boundary rather than propagated, so a NaN from an analyzer can never reach the tick domain.
    • Ordering is deterministic: observations sort by (start_ms, midi) with total_cmp, so equal starts break ties by pitch rather than by input order.
    • No new DSP and no new transcendental call sites — velocity estimation is libm::log10 over an existing peak fold, and the render path is untouched. Golden hashes are unchanged.
  • MIDI time-signature denominator. The importer’s 1u32 << payload[1] became checked_shl. This is a malformed-input fix, not a numeric one: the shifted value never reaches the render fold (an out-of-range denominator is warned about and the default 4/4 kept), so rendered bytes are unaffected.

0.7.1 additions (2026-08-13)

  • Position arithmetic is checked, and the answer is the same everywhere. Pos::resolve’s (bar - 1) * ticks_per_bar was unchecked with both factors caller-supplied. In debug that panicked; in release — the profile every published binary is built with — it wrapped, which is the worse outcome for this workspace: the same score loaded to different ticks under different build profiles, i.e. a determinism failure dressed as an overflow bug. Now checked_mul/checked_add plus the existing Ticks::MAX bound, so a position either resolves to one tick on every build and every host, or is refused. No rendered sample changes; the golden PCM hashes are unchanged.
  • Case folding joins the portability rule. The “one score means one thing on every host” argument recorded below for stem names now also covers the paths those names collide with: two outputs differing only by case are refused on Linux too, because they are one file on macOS and Windows. Same trade, same reason — the answer should not depend on which machine asked. The NFC/NFD limit below is unchanged.
  • A time signature’s numerator is bounded by what can carry it. beats was any nonzero u32 at the door and a clamped u8 at the MIDI exit, so a score could say 300/4 and its exported .mid could say 255/4 with nothing to mark the loss. Export is a second representation of the same score, and this workspace’s rule for a second representation is that it either round-trips or errors — never quietly disagrees. Bounding beats to 1..=255 (TimeSignature::MAX_BEATS, named after the byte the SMF meta event gives it) makes the export exact by construction. No sample changes; no real signature reaches the ceiling.

0.7.0 additions (2026-08-11)

  • Stem-name validation: no bit-level change, one cross-platform rule. The stem-name fix (cochlea_render::stem_file_name, plus containment of the resolved stem path) touches only which files a render is willing to write, never a sample. No DSP, no new transcendental call sites, no change to the schedule or the bus. Both golden PCM hashes are unchanged and were re-confirmed on the Tier 1 target by CI.
  • Why the name rule is enforced on every platform, not per-host. A score is portable data, so the checks that only bite on Windows — \ as a separator, : as a drive or alternate-data-stream marker, the <>"|?* set, the CON/NUL/COM1 device names — are refused on Unix too. The alternative is worse than a false rejection: C: was previously accepted on Unix and rejected on Windows, so one score exported stems on the Tier 1 target and errored on a Tier 2 one. That is the same commitment as the bit-exact render, applied to the filesystem edge — the answer should not depend on which machine asked.
    • Known limit, recorded rather than half-solved: names colliding only under Unicode normalization (NFC vs NFD) are not caught, because normalization would mean a new dependency in a workspace that keeps its graph deliberately small. Case-insensitive collisions are caught.

Golden-audio testing

The core idea behind cochlea: audio is too big and too opaque to diff by eye or by byte, but its features are small, meaningful, and — on the pinned target — byte-reproducible. That makes audio testable the way any other output is testable: render (or synthesize) it, compare against a checked-in reference, and fail the build when it moves outside tolerance.

This is the golden-audio pattern, and it’s what cochlea is really for. No human ear in the loop.

The three tiers of “did it change”

cochlea answers “did this audio change” at three levels of strictness (see determinism.md for the full contract):

  • Tier 1 — byte-identical PCM. The exact same samples. Use this for a deterministic renderer on a fixed target: any difference is a real change.
  • Tier 2 — feature-equivalent. The audio measures the same within cross-platform tolerances (integrated LUFS within 0.1 LU, onsets within 2 ms, pitch within 5 cents). Use this across platforms, or for a model whose output isn’t bit-exact but should stay perceptually stable.
  • Tier 3 — spectrogram sentinel. An image diff of the mel spectrogram with a pixel tolerance, for a visual regression check.

cochlea diff — compare two files

cochlea diff candidate.wav golden.wav --tier2

Exit code 0 means byte-identical or Tier-2 equivalent; exit code 1 means they differ. That single exit-code gate is all a CI step needs. Add --json report.json for the full per-dimension comparison, or --spectro diff.png for a signed A→B difference heat map (red = louder in B, blue = quieter).

cochlea eval — score a directory of outputs

For evaluating a generative model (TTS, voice, music-gen), you usually have a directory of outputs to check against a directory of references. cochlea eval does the whole set in one deterministic pass:

cochlea eval --candidates out/ --references golden/ --json eval.json

It matches files by name, compares each pair, prints a per-file verdict table and an aggregate pass rate, and exits 1 if any pair regressed (or a reference is missing). Add --exact to demand byte-identity instead of Tier-2 equivalence. This is a reference-render regression oracle for an audio model or DSP library: check the golden set in once, and every change is scored against it with no listening and no flakiness.

In Python (pytest)

The Python bindings turn the same check into an ordinary assertion:

from cochlea import assert_audio

def test_tts_output_has_not_regressed():
    synth("hello world", "out.wav")
    assert_audio("out.wav").matches("golden/hello_world.wav")   # tier-2

or, checking properties instead of a golden:

assert_audio("out.wav").true_peak_below(-1.0).pitch_matches("A4").not_clipping()

In CI (GitHub Actions)

The bundled composite action wraps cochlea eval:

# .github/workflows/audio-regression.yml
name: audio regression
on: [pull_request]
jobs:
  golden:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: cargo run -p cochlea -- render examples/scores/first_light.ron --out /tmp/out.wav
      - uses: ./.github/actions/golden-audio
        with:
          candidates: /tmp
          references: examples/golden
          tier: tier2

See .github/actions/golden-audio/action.yml.

Blessing a golden

When a change to the audio is intended, re-bless the reference deliberately — regenerate the golden file and commit it with a note on why the sound changed. For cochlea’s own render goldens that’s cochlea render … --out golden.wav; for the internal PCM-hash and spectrogram sentinels see the “Blessing goldens” notes in the project README. The discipline is the same everywhere: a golden only changes when a human decides the new sound is correct.

cochlea-mcp

An MCP (Model Context Protocol) stdio server over the cochlea libraries. Any MCP client — Claude Code, another agent, a script — gets render / probe / spectro / lint / digest / loudness / beats / diff / import / export / transcribe / reference as tool calls, so it can compose, render, and “listen” to audio through numbers and images without shelling out to the cochlea binary or reading raw PCM.

The protocol is hand-rolled JSON-RPC 2.0 over newline-delimited stdio: one JSON object per line in, at most one JSON object per line out. No async runtime — offline batch tools (render a score, extract a report, write a PNG) need none. Every response is a pure function of its request: no wall clock, no session state, so identical requests produce identical responses.

Tools

Each tool mirrors the matching cochlea CLI subcommand’s semantics exactly (see crates/cli/src/main.rs) over the same library calls — this is a second front end onto the same offline pipeline, not a reimplementation.

ToolArgumentsReturns
render_scorescore_path (string, required), out_path (string, required), stems_dir (string, optional), verify (bool, default false)Text summary: frame count, duration, sample rate, peak dBFS, stems written; if verify is set, the full verify-report JSON is appended and the call reports isError: true on a failed verification.
probe_audioaudio_path (string, required), from_s/to_s (numbers, optional)The full feature report, schema v5 (loudness/LUFS/true peak/LRA + short-term max, onsets, YIN pitch + quantized melody notes, MFCC timbre digest, chroma/key, a chord timeline and per-section key (harmony), tempo with candidates + stability, rhythm with grid alignment + straight-vs-triplet grid + clear_rhythm, stereo image, structural sections, silence, clipping) as pretty JSON. Works on any WAV, FLAC, mp3, or ogg — no score required. from_s/to_s zoom into a time window: report times become relative to the cut, source.start_ms anchors them.
spectrogramaudio_path (string, required), out_path (string, optional), sheet (bool, default false), bars_per_tile (integer, default 8), annotate (bool, default false), from_s/to_s (numbers, optional)The image itself, inline, as an MCP image content block (base64 PNG) whenever it fits the ~700 KB cap — a client with no filesystem access still gets to look at the audio — plus a text summary with pixel dimensions. out_path additionally (or, over the cap, instead) writes the PNG to disk. annotate: true draws the detected beat grid (orange, top), onsets (cyan, bottom), and pitch segments (magenta) on the image; sheet: true tiles a contact sheet instead (the two are mutually exclusive).
lint_scorescore_path (string, required)Text: "ok: no lint findings", or the JSON list of findings. isError: true iff any finding is Severity::Error, matching cochlea lint’s exit-1 threshold.
probe_digestaudio_path (string, required), window_ms (number, default 1000)A ~40-line deterministic text digest (cochlea_features::digest_text) instead of a full JSON report — the token-cheap way to “listen” to an audio file. Prefer this over probe_audio unless the caller needs exact numbers to assert against.
loudness_timelineaudio_path (string, required), hop_ms (number, default 100)The loudness-over-time curve of the whole file as JSON: momentary (400 ms) and short-term (3 s) LUFS sampled every hop_ms, each point’s time measured from the start of the file. The dynamics view the single integrated-LUFS / LRA summary in probe_audio can’t give — where a mix gets loud, where a gate opens, how the level moves. (For a windowed, anchored analysis use probe_audio with from_s/to_s.)
beat_gridaudio_path (string, required)The full beat grid of the whole file as JSON (a TempoReport): every detected beat time (ms, from the start of the file), the estimated downbeats, the tempo with its octave-alternative candidates, and a windowed stability score — the per-beat detail the compact tempo field inside probe_audio drops.
score_reference(none)The complete score-authoring reference as Markdown: the RON grammar (including the master: gain/limiter section), the live instrument-preset catalog (names, polyphony, every automatable param with unit/range/default — generated from the same registry that validates scores, so it cannot go stale), all embeddable verify: assertions, and a worked example that the test suite itself parses and renders. An agent should call this before its first render_score.
audio_diffaudio_path_a (string, required), audio_path_b (string, required), window_ms (number, default 1000), json (bool, default false), spectrogram (bool, default false)Feature-space comparison text (cochlea_features::compare_text): a verdict (byte-identical / tier2-equivalent / different (dimensions...)) plus per-dimension deltas, now including a timbre (MFCC) distance. json: true appends the full CompareReport; spectrogram: true also returns the signed A→B difference heat map inline (red = louder in B, blue = quieter, black = unchanged). A different verdict is a normal, successful answer — not isError.
import_midimidi_path (string, required), out_path (string, required), sample_rate (integer, default 48000)Converts a Standard MIDI File (format 0/1, metrical division) to a RON score at out_path. Timing imports exactly; GM programs map to rough preset families and channel-10 percussion to kick/snare/hat tracks, with every mapping guess listed in the response for re-voicing.
export_midiscore_path (string, required), out_path (string, required)The inverse of import_midi: converts a RON score to a Standard MIDI File (format 1) at out_path. Timing exports exactly (score ticks → SMF ticks, tempo map and time signature carry over); presets become rough General MIDI program labels, since a synth voice isn’t a GM instrument. Use it to hand a composed score to a DAW or notation tool.
transcribe_audioaudio_path (string, required), out_path (string, required), bpm (number, optional), grid (string, default "1/16"), preset (string, default "sine"), track_name (string, default "lead"), ppq (integer, default 960)The inverse of render_score, and the arrow that closes the compose loop: audio in, an editable RON score out. Pitch-tracks the melody, reads its timing against bpm (detected from the audio when omitted), quantizes to grid ("none" keeps raw analyzer timing), and estimates each note’s velocity from its peak level. Deliberately monophonic — chords, drums, and dense mixes come back as whichever single line the tracker locked onto. Every assumption (tempo and where it came from, the grid, the preset, clamped/repaired/dropped notes) is in the response text; treat the result as a draft to re-voice.

Tool-level failures (a bad path, a render error, a failed verify or lint) come back as a normal tools/call success response with isError: true and the reason in the text content — never a JSON-RPC error. JSON-RPC errors (-32700/-32601/-32602) are reserved for protocol problems: malformed JSON, an unknown method, or missing/malformed arguments on a known tool. audio_diff’s different verdict is not one of these failures — see its row above.

Confinement (--root)

By default the server reads and writes wherever the caller points it — appropriate for a personal, local loop. For anything less trusted, launch with --root DIR: every path argument on every tool, reads and writes alike, must then resolve (canonically — symlinks and .. are resolved first) inside DIR, and anything else is refused as an Invalid Params error before the filesystem is touched. This is defense against a confused or prompt-injected client, not a sandbox against hostile local processes.

Confinement covers paths the score implies, not just the ones the caller types. render_score with a stems_dir derives one file per track from the track name, and a track name is free-form score data — it can arrive from a hand-authored RON file or, through import_midi, from a MIDI track-name meta event. A name spelled as a path (/etc/x, ../../x) would otherwise escape both the stems directory and DIR while every path argument stayed legitimately inside it. Two checks close that:

  • The name, against cochlea_render::stem_file_name before the render starts — one ordinary, portable file name, with separators, :, <>"|?*, control characters and the Win32 device names refused on every platform so a score means the same thing on every host.
  • The path it lands on, when the stem is written. A well-formed name is not enough: a symlink already sitting at <stems>/<track>.wav would be followed straight out of the directory, so anything already at that path is resolved and checked for containment. Presence is tested with symlink_metadata, not exists()exists() follows the link, so a link pointing at a file that does not exist yet used to read as “nothing here” and skip the check while File::create followed it anyway (fixed in 0.7.1). A link that cannot be resolved is refused — including a dangling one whose target would have been inside the directory, since a lexical check of an unresolvable target is exactly the check that cannot see a redirect. One that resolves and stays inside the stems directory is ordinary and still works.

A stem that would land on the mix (out_path) or the score (score_path) is refused too, so a stems_dir overlapping either cannot destroy them. “Same file” here folds case (Lead.wav and lead.wav are one file on macOS and Windows), on every platform — the same rule the CLI uses, and the same rule that governs every out_path-versus-input check on this server.

claude mcp add cochlea -- cochlea-mcp --root ~/music-workspace

Client setup

Claude Code:

claude mcp add cochlea -- cargo run -p cochlea-mcp --release

or, against an already-built binary:

claude mcp add cochlea -- /path/to/target/release/cochlea-mcp

Any other stdio MCP client: launch cochlea-mcp (or cargo run -p cochlea-mcp) as a subprocess and speak JSON-RPC 2.0 over its stdin/stdout, one object per line. stdout carries only protocol responses; all logging goes to stderr, so it’s safe to leave stderr connected to a terminal or log file without corrupting the transport.

Example

Request (tools/call for probe_audio, sent as one line):

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"probe_audio","arguments":{"audio_path":"mix.wav"}}}

Response (one line back; the pretty-printed report is escaped into the text field, shown here unescaped for readability):

{"jsonrpc":"2.0","id":1,"result":{"content":[{"type":"text","text":"{\n  \"schema_version\": 5,\n  \"source\": {\n    \"sample_rate\": 48000,\n    \"channels\": 2,\n    ...\n  },\n  ...\n}"}],"isError":false}}

Testing this crate

crates/mcp/src/lib.rs exists so the dispatch logic (server::Server) can be driven in-process from integration tests — Server::handle_line(&str) -> Option<String> takes one request line and returns at most one response line, with no stdin/stdout/subprocess involved. crates/mcp/src/main.rs is just the framing loop around it. See crates/mcp/tests/protocol.rs for JSON-RPC conformance and crates/mcp/tests/tools_e2e.rs for a real render → probe → spectrogram round trip against examples/scores/first_light.ron.