cochlea
A headless, deterministic audio engine for AI agents. Write a score as data, render it offline to byte-identical PCM, then listen through numbers — loudness, onsets, pitch, key, spectrograms — and assert what you heard. Compose → render → probe → verify, with no human ear (and no audio device) anywhere in the loop.
There is no realtime path in this project, and there never will be one. Offline render is ground truth.
What the agent sees instead of hearing a WAV file.
Why
An agent can’t listen to audio, and it shouldn’t have to read raw PCM either — a few minutes of 48kHz float audio is tens of megabytes, which is a bad way to spend a context window. Cochlea’s answer is a small set of primitives an agent can compose:
- Score IR — ticks, tempo, tracks, notes, automation, and an optional master bus (gain + brick-wall limiter), expressed as data (Rust builder or RON). Standard MIDI Files import and export with timing intact.
- Deterministic render — the same score renders to the same PCM bytes every time on the pinned CI target, enforced at the toolchain level, not by convention. See Determinism Contract.
- Feature reports — loudness, true peak, onsets, pitch plus a
quantized melody (note events an agent can diff against what it
wrote), an MFCC timbre digest, key, tempo (with octave-alternative
candidates and stability), rhythm (grid alignment with a
straight-vs-triplet hypothesis test, syncopation, a trustable
clear_rhythm), a chord timeline and per-section key (harmony), stereo width, structure — as a few kilobytes of JSON, or a sub-kilobyte text digest sized for an LLM’s context window. Every read tool takes a--from/--towindow, so a long file can be probed a few bars at a time. - Spectrograms — a small PNG when a report alone doesn’t answer the question; optionally annotated with the detected beats, onsets, and pitch, and diffable as a signed A→B heat map.
- Verify — an assertion DSL over a render (
true_peak_below,pitch_matches_score,tempo_is, …), so an agent can retry on a failed assertion instead of asking a human “does that sound right?”
Every crate here works standalone. cochlea probe runs on any WAV,
FLAC, mp3, or ogg file with no score in sight — that’s the adoption
wedge, and it’s enforced structurally (features/spectro depend on
neither score nor synth, checked via cargo tree in CI).
Where to start
- Writing a score — the RON grammar, every preset and parameter, the
verify assertions, a worked example: Score Format
Reference. The same text is served in-band by
cochlea referenceand the MCPscore_referencetool, and a test pins all three to the same generator. - Building instruments and scores, or wiring cochlea into a bigger system: Design & API Surface.
- The determinism contract, and the fundsp/rustfft audits behind it: Determinism Contract.
- Running cochlea as an MCP server so an agent calls render/probe/diff as tools: MCP Server.
Install
cargo install cochlea # the CLI: render / probe / lint / spectro
cargo install cochlea-mcp # the MCP stdio server
cargo add cochlea-features # or any crate, as a library dependency
All 9 crates are on crates.io. Source, issues, and the workspace layout live at github.com/richer-richard/cochlea.
cochlea score reference (RON data form, version 1)
A score is one Score(...) value in RON
syntax. Render it with cochlea render score.ron --out mix.wav (add
--verify to run the embedded assertions), or via the render_score MCP
tool.
Top level
Score(
version: 1, // required, must be 1
sample_rate: 48000, // Hz
ppq: 960, // ticks per quarter note (960 is standard)
time_signature: (4, 4), // optional, default (4, 4)
tempo: [(tick: 0, bpm: 120.0)], // tempo map: step changes at ticks
tracks: [ Track(...), ... ],
master: Master(...), // optional master bus (below)
verify: [ ... ], // optional embedded assertions (below)
)
Time is integer ticks at ppq per quarter note; positions in the data
form are (bar, beat) pairs, both 1-based. Anything off the tick grid is
an error, never a rounding.
Tracks and notes
Track(
name: "lead", // unique per score
instrument: Preset("saw_lead"), // one of the preset names below
inserts: [Preset("reverb")], // optional per-track effect chain
notes: [
Note(at: (1, 1), dur: "1/4", pitch: "A4", vel: 96),
// `off:` shifts the position by a duration past the beat —
// e.g. the second eighth of beat 1:
Note(at: (1, 1), off: "1/8", dur: "1/8", pitch: "C5", vel: 80),
],
automation: [ Auto(...), ... ], // optional (below)
)
pitch: note name + octave ("C4"= middle C,"F#3","Bb2").vel: 1–127 (MIDI-style; velocity maps to amplitude squared).dur: a fraction of a whole note as a string —"1/4"quarter,"3/16", dotted"1/4.", triplet"1/8t".
A track name is free-form text and need only be unique within the score
— unless you export stems (cochlea render --stems, or stems_dir on the
MCP render_score tool), which writes one <track>.wav per track and so
turns the name into a file name. Then it must be a portable one, checked
the same way on every platform so a score exports the same stems
everywhere instead of working on one host and failing on another: no path
separator (/, \) and no : (a drive or an alternate data stream on
Windows); none of <>"|?* and no control characters; not a Windows device
name (CON, PRN, AUX, NUL, COM0–COM9, LPT0–LPT9, matched
before the first dot whatever the extension); short enough to be a file
name; and no two tracks differing only by case, since those are one file
on macOS and Windows. A name that breaks a rule fails the render with the
reason and the track quoted, before anything is written — it is never
silently rewritten. cochlea import lifts track names straight out of a
MIDI file, so an imported score is the usual place to meet this.
Automation
Auto(param: "cutoff_hz", keys: [
Key(at: (1, 1), value: 400.0, ease: EaseInOut),
Key(at: (3, 1), value: 4000.0),
])
parammust be automatable on the track’s instrument (see the catalog below). Engine-level"gain"(linear, default 1.0) and"pan"(-1.0..1.0 constant-power, default 0.0) work on every track; a singleKeysets a constant value.easeshapes the segment leaving that key:Linear(default),Hold,EaseIn,EaseOut,EaseInOut,Bezier(x1, y1, x2, y2).- Automation is control-rate: sampled every 64 samples (~1.3 ms at 48 kHz). Note timing is sample-accurate.
Master bus
master: Master(
gain_db: 3.0, // optional, default 0.0 (-40..=24)
limiter: Limiter(
ceiling_db: -1.0, // required (-40..=0)
lookahead_ms: 5.0, // optional, default 5.0 (0..=50)
release_ms: 50.0, // optional, default 50.0 (1..=1000)
),
)
Applied to the f64 stem sum after mixing: gain first, then a brick-wall
lookahead limiter whose sample-peak ceiling holds exactly (inter-sample
true peaks can read fractionally higher — leave ~1 dB of headroom under a
TruePeakBelow target). This is the tool for hitting loudness targets:
push with gain_db, let the limiter hold the ceiling, and assert both
with IntegratedLufs + TruePeakBelow. Omit master: entirely for a
untouched bus; per-track stems are always exported pre-master.
Instrument presets
chord_pad(poly 16)- param
cutoff_hz(Hz, 40..12000, default 1200) - param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
fm_bell(poly 12)- param
brightness(index, 0..12, default 3) - param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
kick(mono)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
marimba(poly 12)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
noise_hat(poly 8)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
organ(poly 16)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
pluck(poly 16)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
saw_lead(poly 8)- param
cutoff_hz(Hz, 40..18000, default 2400) - param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
sine(poly 16)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
snare(poly 4)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
square_bass(mono)- param
gain(linear, 0..4, default 1) - param
pan(pan, -1..1, default 0)
- param
Inserts (per-track effects): reverb.
Embedded assertions (verify:)
cochlea render score.ron --verify (or render_score with
verify: true) runs these against the finished render and fails on any
miss. Positions are (bar, beat).
verify: [
IntegratedLufs(target: -14.0, tol: 0.5), // mix loudness, LUFS
TruePeakBelow(dbtp: -1.0), // headroom, dBTP
OnsetAt(track: "drums", at: (17, 1), tol_ms: 5.0),
PitchMatchesScore(track: "lead", tol_cents: 10.0), // monophonic tracks
Monotone(track: "pad", param: "cutoff_hz",
from: (1, 1), to: (3, 1), direction: Rising), // authored curve
BrightnessRises(track: "pad", from: (1, 1), to: (3, 1),
min_ratio: 1.3), // rendered audio actually brightens
BrightnessFalls(track: "pad", from: (3, 1), to: (5, 1), min_ratio: 1.3),
NoDiscontinuity(track: "lead", db: 40.0), // click detector
SilentAfter(at: (64, 1)),
TempoIs(bpm: 110.0, tol_bpm: 2.0), // optional min_bpm/max_bpm range
HasClearRhythm(expected: true), // grid-based rhythm trust flag
GridAlignmentAtLeast(min: 0.9), // fraction of onsets on the grid
StereoWidthWithin(min: 0.03, max: 0.2),
LraBelow(lu: 8.0), // loudness range, LU
SectionCount(min: 1, max: 3), // detected structure sections
]
Monotone checks the authored automation curve (a score-side lint);
BrightnessRises/BrightnessFalls listen to the rendered stem’s
spectral centroid — assert both to prove the sweep was written and
audibly happened.
Worked example
Score(
version: 1,
sample_rate: 48000,
ppq: 960,
tempo: [(tick: 0, bpm: 120.0)],
tracks: [
Track(
name: "kick",
instrument: Preset("kick"),
notes: [
Note(at: (1, 1), dur: "1/8", pitch: "A1", vel: 112),
Note(at: (1, 2), dur: "1/8", pitch: "A1", vel: 100),
Note(at: (1, 3), dur: "1/8", pitch: "A1", vel: 112),
Note(at: (1, 4), dur: "1/8", pitch: "A1", vel: 100),
],
),
Track(
name: "pad",
instrument: Preset("chord_pad"),
inserts: [Preset("reverb")],
notes: [
Note(at: (1, 1), dur: "1/1", pitch: "A2", vel: 74),
Note(at: (1, 1), dur: "1/1", pitch: "C3", vel: 70),
Note(at: (1, 1), dur: "1/1", pitch: "E3", vel: 70),
],
automation: [
Auto(param: "cutoff_hz", keys: [
Key(at: (1, 1), value: 400.0, ease: EaseInOut),
Key(at: (2, 1), value: 3000.0),
]),
],
),
],
verify: [
TruePeakBelow(dbtp: -1.0),
TempoIs(bpm: 120.0, tol_bpm: 2.0),
BrightnessRises(track: "pad", from: (1, 1), to: (2, 1), min_ratio: 1.2),
],
)
Loop: lint_score to catch authoring mistakes cheaply, render_score
(with verify: true), then probe_audio/probe_digest to read the
result and spectrogram to look at it.
cochlea v1 plan: crate-by-crate public API surface
Written in Phase 0, before implementation. Where implementation reality
diverges, this file is updated in the same PR (deviations get a Deviation
marker). Determinism decisions and the dependency audits live in
docs/determinism.md; the contract summary lives in the README.
Phase 0 decisions (deltas and pins)
- fenestra-anim 0.1.0 verified on crates.io. It provides everything
required:
mul_div(u64, u64, u64, Rounding{Floor,Ceil,Round}) -> u64(u128 intermediate, panics on div-by-zero and u64 overflow), typedTrack<T: Interpolate>/Key<T>/key(at, value).ease(e), easing set (linear,hold,ease_in,ease_out,ease_in_out,spring,CubicBezier,Spring),Interpolate(f32 and (f32, f32)), and serde behind theserdefeature. MSRV 1.88,unsafe_code = "forbid". keys!lives in cochlea-score, not upstream. fenestra-anim exports thekey()constructor; thekeys![...]macro from the kickoff sketch is authoring sugar over cochlea tick positions (bar(1).beat(1)etc.), which fenestra-anim knows nothing about. Defining it here is not a fork of easing math — it expands tofenestra_anim::key(ticks, value).ease(e). No upstream PR needed.- Spring easing is rejected in v1 automation.
Track::sample(frame, fps)grounds springs in seconds via an integerfps; automation runs in tick space where “ticks per second” is tempo-dependent and generally non-integer. Rather than evaluate springs subtly wrong, score validation rejectsEase::Springon automation tracks (linear/hold/bezier are fine — they never readfps). Phase 2 can PR fractional-rate sampling upstream. - Tempo is stored as integer nanoseconds per quarter note (
u64), converted fromBpm(f64)once at authoring time (round), like MIDI’s µs-per-quarter but 1000× finer. All tick→sample math after that point is exact rational (see score crate below). Valid BPM range [1.0, 4000.0], validated. - Denormal policy: honor denormals everywhere — no FTZ/DAZ, no
algorithmic offsets. Full rationale in
docs/determinism.md; short form: IEEE 754 fully specifies denormal arithmetic, so not flushing is the most deterministic and the most cross-platform-uniform choice, and offline rendering doesn’t care about the occasional slow filter tail. We never call fundsp’sprevent_denormals(). - Pins (resolved on crates.io 2026-07-03): fundsp 0.23.0
(default-features off,
stdonly — keeps symphonia and fft-convolver out), ebur128 0.1.10, rustfft 6.4.1, hound 3.5.1, ron 0.12.2, libm 0.2.16, image 0.25.10 (png only), serde 1.0.228, rayon 1.12.0, clap 4.6.1, proptest 1.11. Toolchain pinned 1.95.0; MSRV 1.88 (floor set by fenestra-anim and image). - Naming deviation, recorded: the user-facing name
Instrumentbelongs to the score IR (score::Instrument::preset("saw_lead")— what a score references, serializable data). The synth-side trait (how a name becomes sound: param registry + fundsp voice graphs) issynth::Patch. The kickoff sketch usedInstrumentfor both sides; one name for two types in two crates would be permanently confusing at call sites. - Stereo throughout v1. Every track renders 2 channels; mono voice
graphs are panned center. Analysis that wants mono (YIN) downmixes
(l + r) / 2. - Audit-driven rules (details in
docs/determinism.md, enforced viaclippy.tomlbans): voices tick sample-by-sample, never fundspprocess(); fundspfeedback()/fdn()combinators banned (internal thread-global x86 FTZ); ADSR/voice graphs freshly constructed per note, never pooled-and-reset (fundsp ADSR closure state survivesreset());pluckis a hand-rolled Karplus-StrongAudioNode(fundsp’sPluckseeds its excitation from stateful funutd RNG) with counter-RNG excitation; fundspnoise()/hold()unused — all noise is the in-repo counter RNG; analysis FFTs constructFftPlannerScalarexplicitly; fundsp presets use theprelude64constructors (f64 internal state, f32 node interface); ebur128 true/sample peaks return linear amplitude — we convert to dBTP/dBFS via libm, and-infloudness readings mean “silence/no data”, not errors. - The reverb insert is in-repo (P2 finding): every fundsp stereo reverb
constructor (
reverb_stereo,reverb4_stereo_delays) builds onfdn()internally, which flips thread-global x86 FTZ — and clippy bans can’t see call sites inside fundsp. Thereverbinsert is a Freeverb-style Schroeder (cochlea-synth), pure arithmetic per tick, tail ~2.5 s. Envelopes are likewise our own piecewise-linear ADSR evaluated throughfundsp::envelopeclosures (pure arithmetic; the offline schedule knows note length, so no live gate state exists at all).
Dependency graph (acyclic, law-checked in CI)
score → fenestra-anim, ron, serde
synth → score, fundsp, libm
render → score, synth, hound, rayon
features → ebur128, rustfft, hound, libm, serde (NEVER score/synth)
spectro → rustfft, hound, image, libm (NEVER score/synth)
decode → features, symphonia (NEVER score/synth)
verify → score, features, render
cli → everything
mcp → score, synth, render, features, spectro, verify
crates/score
Newtypes: Ticks(pub u64), Ppq(pub u32), SampleRate(pub u32),
Bpm(pub f64), Vel(pub u8) (1..=127), Pitch(pub u8) (MIDI number, consts
like Pitch::A4 = 69, FromStr for “A4”/“C#3”/“Bb2”, fn hz(self) -> f64
via libm), Dur { num: u32, den: u32 } (fraction of a whole note —
Dur::quarter() = 1/4, eighth, half, whole, sixteenth,
.dotted() (×3/2), .triplet() (×2/3), Dur::ticks(u64) escape hatch;
resolved against PPQ exactly: ticks = ppq * 4 * num / den via mul_div).
Positions: Pos — built by the terse constructors from the sketch:
#![allow(unused)]
fn main() {
pub fn bar(n: u32) -> Pos; // 1-based
impl Pos {
pub fn beat(self, b: u32) -> Pos; // 1-based within the bar
pub fn plus(self, d: Dur) -> Pos; // sub-beat offset, exact
}
}
Pos resolves to Ticks against (Ppq, TimeSignature) — 4/4 in v1 but the
math takes (num, den) so time-signature support isn’t load-bearing
anywhere. Resolution is exact integer math; a Pos that lands on a
non-integer tick (impossible at 960 PPQ for any Dur with den | 3840) is a
validation error, not a rounding.
Tempo map:
#![allow(unused)]
fn main() {
pub struct TempoMap { /* sorted (Ticks, NanosPerQuarter) steps, ppq, sample_rate */ }
impl TempoMap {
pub fn sample_at(&self, t: Ticks) -> u64; // THE conversion — see below
pub fn tick_at(&self, sample: u64) -> Ticks; // inverse, Floor
pub fn ns_at(&self, t: Ticks) -> u64;
}
}
sample_at is the one documented rounding rule: per tempo segment i
starting at tick T_i with anchor sample S_i,
sample_at(t) = S_i + mul_div(t - T_i, npq_i * sr, ppq * 1_000_000_000, Rounding::Round)
Anchors are computed left-to-right with the same formula, so each tempo
change rounds at most once (±0.5 samples, no accumulation) and within a
segment there is zero drift. Monotonicity and exactness vs a direct u128
reference are property-tested. u64 range check: worst case
npq * sr = 6e10 × 192_000 ≈ 1.2e16 < u64::MAX; enforced by the BPM and
sample-rate validation ranges (SR in [8_000, 192_000]).
Score builder (mirrors the sketch):
#![allow(unused)]
fn main() {
let score = Score::new(SampleRate(48_000), Ppq(960))
.time_signature(4, 4)
.tempo(Ticks(0), Bpm(120.0))
.track("lead", Instrument::preset("saw_lead"))
.insert("lead", Insert::preset("reverb")) // per-track insert chain
.note("lead", bar(1).beat(1), Dur::quarter(), Pitch::A4, Vel(96))
.automate("lead", Param::CUTOFF_HZ,
keys![(bar(1), 400.0, ease_in_out()), (bar(3), 4_000.0)]);
}
score::Instrument— serializable reference:Instrument::preset(&str)|Instrument::custom(&str)(a name resolved from a caller-supplied bank at render time; the closure itself lives in synth — keeps score pure data, and RON containing a custom name fails rendering with a clear error unless the bank supplies it. Code-only, documented as non-serializable in spirit: lint warns on custom refs in.ronfiles).Param(&'static str | owned)newtype with constsParam::CUTOFF_HZ,Param::GAIN,Param::PAN,Param::custom("..."). Snake_case strings in RON.- Automation:
keys![...]expands to fenestra-anim keys; positions arePos/Ticks, values f32, ease defaults linear. Stored per (track, param). Sampled at block starts by the renderer (control rate ≈1.3 ms at 64 samples / 48 kHz — documented); blocks split at event boundaries keep note-ons/offs sample-accurate. - Notes:
Note { at: Ticks, dur: Ticks, pitch: Pitch, vel: Vel }post- resolution; builder keepsPos/Durauthoring forms in the RON mirror.
RON data form (version = 1, round-trip tested both directions, one
committed example under examples/scores/):
Score(
version: 1,
sample_rate: 48000,
ppq: 960,
time_signature: (4, 4),
tempo: [(tick: 0, bpm: 120.0)],
tracks: [
Track(
name: "lead",
instrument: Preset("saw_lead"),
inserts: [Preset("reverb")],
notes: [Note(at: (1, 1), dur: "1/4", pitch: "A4", vel: 96)],
automation: [
Auto(param: "cutoff_hz", keys: [
Key(at: (1, 1), value: 400.0, ease: EaseInOut),
Key(at: (3, 1), value: 4000.0),
]),
],
),
],
verify: [], // assertion data forms, see crates/verify
)
Durations/pitches are strings in RON (“1/4”, “1/8.”, “A4”, “C#3”); positions
are (bar, beat) tuples with an optional third sub-beat element
(bar, beat, "1/16"). Ease names: Linear | Hold | EaseIn | EaseOut | EaseInOut | Bezier(x1, y1, x2, y2).
Static validation (score.validate(&impl Catalog) -> Vec<LintFinding>):
overlapping notes on a mono instrument, params outside declared ranges,
automation targeting unknown params, tracks with no events, BPM/SR out of
range, spring ease on automation, custom instrument refs in data form.
Catalog is a small trait (param specs + polyphony per instrument name)
implemented by synth’s registry — keeps score free of synth while letting
lints know param ranges.
Tests: unit (bar/beat/dur resolution exactness, tempo anchors), property
(monotone sample_at, zero drift over 1e9 ticks vs u128 reference,
tick_at(sample_at(t)) == t where exact), RON round-trip.
crates/synth
#![allow(unused)]
fn main() {
pub trait Patch: Send + Sync {
fn name(&self) -> &str;
fn params(&self) -> &[ParamSpec]; // typed registry
fn polyphony(&self) -> Polyphony; // Mono | Poly(NonZeroU8)
fn release_ticks_hint(&self) -> f64; // release tail in seconds — voice lifetime is static
fn voice(&self, note: VoiceCtx) -> Box<dyn AudioUnit>; // fresh fundsp graph per note
}
pub struct ParamSpec { pub param: Param, pub unit: Unit, pub range: RangeInclusive<f32>, pub default: f32 }
pub struct VoiceCtx { pub pitch: Pitch, pub vel: Vel, pub sample_rate: SampleRate,
pub note_seed: u64, // hash(track, note index) — feeds counter RNG
pub start_sample: u64 } // absolute, for (seed, sample_index) noise keying
}
- Param application: each automatable param is a fundsp shared variable
(
shared()/var()) wired into the voice graph; the renderer sets them at block starts. Settings are plain f32 stores — deterministic. presets()registry:sine,saw_lead(filtered saw + ADSR),square_bass(mono),chord_pad(detuned saws + LPF),noise_hat(counter-RNG noise + fast envelope),pluck(Karplus-Strong, stateful but deterministic; excitation from the counter RNG, not fundsp’s noise). Registry implementsscore::Catalogfor lints.Insertpresets:reverb(fundsp reverb, fixed parameters, applied per-track post voice-sum). Registry entry like patches.- Counter RNG (in-repo, the only randomness in the workspace):
#![allow(unused)]
fn main() {
pub fn crng(seed: u64, index: u64) -> u64; // SplitMix64-style finalizer of seed ^ mix(index)
pub fn crng_f32(seed: u64, index: u64) -> f32; // uniform [-1, 1)
}
Noise is a pure function of (seed, sample_index) — random access, no
state. fundsp’s noise()/hold()/Pluck are not used (audited: funutd-
backed, stateful); the hat’s noise source is a custom AudioNode over
crng, and pluck is a hand-rolled Karplus-Strong node with crng
excitation.
- All transcendentals in our DSP code go through
libm. fundsp internals are covered by the audit in determinism.md.
Tests: two identical voices produce identical buffers; crng reference
vectors; param registry ranges match preset defaults; each preset renders
non-silence and decays after release.
crates/render
#![allow(unused)]
fn main() {
pub fn render(score: &Score) -> Result<Rendered, RenderError>; // presets only
pub fn render_with(score: &Score, bank: &PatchBank) -> Result<Rendered, RenderError>; // + customs
pub struct Rendered {
// interleaved stereo f32; mix defined as f64-sum of the f32 stems in
// fixed track order, then f64→f32 — so mix == sum(stems) byte-exactly
pub fn mix(&self) -> &[f32];
pub fn stems(&self) -> impl Iterator<Item = (&str, &[f32])>;
pub fn sample_rate(&self) -> SampleRate;
pub fn write_wav(&self, path: &Path) -> Result<...>; // 32-bit float WAV (hound)
pub fn write_stems(&self, dir: &Path) -> Result<...>;
}
}
- Event schedule (pure): score → per-track sorted
Vec<Event>with sample indices resolved once viaTempoMap::sample_at(the single rounding). Events: NoteOn/NoteOff. Ordering: by sample, then track order, then note index — total and documented. - Blocks: fixed 64-sample max, split at event boundaries. Automation
sampled at block-start ticks (
TempoMap::tick_atof block start) and written to shared params before the block ticks. - Voices: per-track pool sized by
Patch::polyphony. Allocation and stealing (oldest note) are functions of the schedule alone; voice lifetime isnote span + release tailcomputed statically. Voices tick sample-by-sample (AudioUnit::tick), never fundsp blockprocess()— one code path, one rounding story (see determinism.md on SIMD). - Track render: voices sum into an f64 stereo accumulator in voice-index order → insert chain (f32 in fundsp) → f64 track gain/pan → f32 stem.
- Master: stems summed at f64 in fixed track order → f32 mix.
- Parallelism: rayon over tracks; each track fully independent;
deterministic fold in track order afterward.
parallel == serialtested byte-for-byte.
Tests: same score twice → byte-equal; parallel == serial; stems f64-sum == mix byte-equal; voice stealing scenario (poly 2, 3 overlapping notes → oldest stolen, schedule-derived); golden PCM SHA-256 on the Tier 1 target (cfg-gated).
crates/features
Input type (no score, no synth — ever):
#![allow(unused)]
fn main() {
pub struct Audio { pub samples: Vec<f32> /* interleaved */, pub channels: u16, pub sample_rate: u32 }
impl Audio { pub fn from_wav(path: &Path) -> Result<Self, ...>;
pub fn mono(&self) -> Vec<f32>; }
pub fn probe(audio: &Audio, opts: &ProbeOpts) -> Report;
}
Report (serde, schema_version: 2 — v1 lacked loudness.lra, tempo,
stereo, and structure; v3 split tempo/rhythm — 0.2.0 Deviation ledger below applies):
{
"schema_version": 3,
"source": { "sample_rate": 48000, "channels": 2, "samples": 480000, "duration_ms": 10000.0 },
"loudness": { "integrated_lufs": -14.2, "momentary_max_lufs": -10.1,
"true_peak_dbtp": -1.3, "sample_peak_dbfs": -1.5, "lra": 4.2 },
"tempo": { "bpm": 110.3, "confidence": 0.79, "stability": 1.0,
"candidates": [ { "bpm": 110.3, "salience": 0.79 }, { "bpm": 55.1, "salience": 0.89 } ],
"beat_count": 32, "mean_beat_interval_ms": 545.4 },
"rhythm": { "grid_alignment": 0.98, "offbeat_ratio": 0.56,
"onset_rate_per_s": 2.79, "clear_rhythm": true },
"stereo": { "width": 0.01, "correlation": 1.0, "balance": -0.0 },
"structure": { "boundaries_ms": [8000.0], "section_count": 2, "confidence": 0.74 },
"onsets": { "count": 17, "times_ms": [...] },
"pitch": { "voiced_ratio": 0.82, "median_f0_hz": 440.1,
"segments": [ { "start_ms": 0.0, "end_ms": 500.0, "f0_hz": 440.1, "midi_nearest": 69, "cents_off": 0.4 } ] },
"key": { "tonic": "C", "mode": "major", "confidence": 0.87, "chroma": [/* 12 */] },
"silence": { "leading_ms": 0.0, "trailing_ms": 812.5, "last_audible_sample": 441000, "floor_dbfs": -60.0 },
"clipping": { "clipped_samples": 0, "true_peak_over_0dbtp": false }
}
- Loudness: ebur128 (
I | M | TRUE_PEAK | SAMPLE_PEAK), stereo default channel map. Integrated LUFS, momentary max, true peak dBTP. - Onsets: STFT (rustfft, 1024/256 hop, Hann via libm), half-wave- rectified spectral flux, median-based adaptive threshold, peak picking with minimum inter-onset gap. Times in ms (sample-derived).
- Pitch: YIN (difference function + CMNDF, threshold 0.1, parabolic interpolation), 2048 window / 512 hop on mono downmix; per-segment median f0. Run per stem for multi-voice material — the caller (cli) does that; features stays per-buffer.
- Key: 12-bin chroma from STFT magnitude (log-frequency weighting), Krumhansl-Schmuckler correlation against 24 major/minor templates → (tonic, mode, confidence = top correlation vs runner-up margin folded in).
- Silence/tail: windowed RMS (50 ms hop) under floor (default −60 dBFS); last-audible sample. Clipping: |sample| ≥ 1.0 count; true peak > 0 dBTP flag.
- Score-context enrichment (nearest tick for onsets, cents-vs-score) is NOT here — verify/cli join features output against the score.
Tests (fixtures synthesized in-test with libm, no synth dep): 440 Hz sine → A4 within 1 cent, LUFS of a −18 dBFS sine ≈ −18 LUFS (K-weighting at 1 kHz ≈ 0), click track → onsets at known samples ±2 ms, C major triad → C major, tail detection on a gated burst, clipping counter on a driven square.
crates/spectro
#![allow(unused)]
fn main() {
pub struct SpectroOpts { fft: usize /*2048*/, hop: usize /*512*/, mels: usize /*128*/,
floor_db: f32 /*-80*/, fmin: f32, fmax: f32 }
pub fn mel_spectrogram(audio: &Audio, opts: &SpectroOpts) -> MelImage; // matrix + axis metadata
pub fn render_png(img: &MelImage, ruler: Ruler, markers: &[Marker]) -> RgbImage;
pub fn contact_sheet(img: &MelImage, markers: &[Marker], per_tile: usize) -> RgbImage;
pub struct Marker { pub sample: u64, pub label: String } // bar markers WITHOUT score types
}
Hand-rolled mel filterbank (Slaney-style, documented), log magnitude with dB
floor, 256-entry viridis LUT (const table in-repo), time ruler in seconds,
caller-supplied bar markers. Contact sheets tile N-bar sections vertically so
an agent reviews a whole piece in one vision call. Audio type is re-used
from features? — no: spectro must not depend on features either? (law
only covers score/synth; a features dep would be harmless but keep spectro
standalone: it defines its own tiny input or takes &[f32], sr directly.
Decision: functions take (samples: &[f32], channels, sample_rate) plain
args; cli/verify adapt.)
Tier 3 sentinels: committed PNGs for fixture signals; image diff with per-pixel tolerance + max-fraction-differing threshold (helper in this crate, used by demo tests).
crates/decode
#![allow(unused)]
fn main() {
pub fn load(path: &Path) -> Result<Audio, DecodeError>; // Audio = cochlea_features::Audio
}
Wave 2: lossless-only real-world file input. Dispatches on file extension:
.wav/.wave delegates straight to cochlea_features::Audio::from_wav
(hound); .flac goes through symphonia’s bundled FLAC reader+decoder (the
symphonia crate, default-features = false, features = ["flac"] — no
other format/codec, so no lossy decode sneaks in via a shared feature).
Depends on cochlea-features only for the Audio type; never score/synth,
same law as features/spectro.
FLAC is lossless by spec, so a correct decode reproduces the source PCM
exactly — but symphonia-bundle-flac left-justifies every sample into the
full 32-bit range regardless of the stream’s true bit depth (its own
decode_inner comment: “the decoder uses a 32bit sample format as a common
denominator”). Normalizing by always dividing by 2^31 (not by a bit-depth-
derived scale) is what makes FLAC decode land on the same f32 bits as the
WAV twin — verified, not assumed, by tests/sample_exact.rs against tiny
committed FLAC fixtures with WAV twins (tests/fixtures/).
mp3/ogg (lossy) are explicitly next, not this round (docs/superpowers/ specs/2026-07-09-agent-audio-v2-design.md §2/§6).
crates/verify
#![allow(unused)]
fn main() {
use cochlea_verify::VerifyExt; // extension trait on Rendered
let report = rendered.verify(&score)
.integrated_lufs(-14.0, Tol(0.5))
.true_peak_below(-1.0)
.onset_at("drums", bar(17).beat(1), Ms(5.0))
.pitch_matches_score("lead", Cents(10.0))
.monotone("lead", Param::CUTOFF_HZ, bar(1)..bar(3), Direction::Rising)
.no_discontinuity("lead", Db(40.0))
.silent_after(bar(64))
.run(); // -> VerifyReport
pub struct VerifyReport { pub passed: bool, pub failures: Vec<Failure>, ... } // serde JSON
}
- Newtypes
Tol(f32),Ms(f32),Cents(f32),Db(f32). onset_at: nearest detected onset on that track’s stem within tolerance.pitch_matches_score: YIN per note window on the stem vs score pitch.monotone: the authored automation curve sampled at block rate over the range must be monotone in the given direction (validates the authored sweep; spectral verification would conflate instrument response).no_discontinuity: max sample-to-sample jump (in dB of |Δ|) away from note on/off boundaries ± a guard window — click detector.silent_after: windowed RMS below floor for everything after the tick.- Wave-2 assertions over the v2 analyzers (same builder + RON dual form):
tempo_is(bpm, BpmTol)/TempoIs(bpm, tol_bpm, [min_bpm, max_bpm])— optional search-range override, the escape hatch for >~170 BPM material where the octave prior favors half-time;has_clear_rhythm(bool);stereo_width_within(min, max);lra_below(lu);section_count(min, max). Undefined-metric policy (stated inverify::checks): bounded-above checks pass on an undefined metric, value assertions fail on one. - Every assertion is also a serde data form embeddable in score RON under
verify:;Verifier::from_specs(&[VerifySpec])builds the same run. CLIcochlea render score.ron --verifyruns them, writes the JSON failure report to stdout (or--report path), exits nonzero on failure.
crates/cli (cochlea binary)
cochlea render score.ron --out mix.wav [--stems dir/] [--verify] [--report report.json]
cochlea probe input.{wav,flac} [--json report.json] [--spectro spec.png]
[--digest] [--segments timeline.json] [--window-ms 1000]
cochlea diff a.{wav,flac} b.{wav,flac} [--json compare.json] [--tier2] [--window-ms 1000]
cochlea lint score.ron
cochlea spectro input.{wav,flac} --out spec.png [--sheet --bars-per-tile 8]
clap derive; probe ships in P3, the rest complete in P4; probe --digest/--segments and diff land in v2 wave 1 (the token-cheap read
path: digest text instead of JSON, feature-space diff with a
byte-identical / tier2-equivalent / different verdict). probe with no
flags prints the JSON report to stdout. Exit codes: 0 ok, 1 verify/lint
failures (and diff --tier2 when the verdict is not equivalent), 2
usage/IO errors. --window-ms rejects non-finite or sub-1 ms values at
the flag boundary (NaN defeats downstream range checks; sub-millisecond
windows would round to one-sample segments and explode the timeline), and
distinct output flags pointing at one path are a usage error, not a
silent last-write-wins.
The cochlea-mcp sibling binary (crates/mcp, v2 wave 1) serves the same
pipeline as MCP tools over newline-delimited JSON-RPC 2.0 on stdio:
render_score, probe_audio, spectrogram, lint_score, probe_digest,
audio_diff. Hand-rolled protocol, no async runtime; see docs/mcp.md.
Demos (P5, demos/ as workspace tests or examples)
metronome— click track; unit test asserts scheduled events land sample-exact; probe onsets within 2 ms.chord_pad— chord progression onchord_pad; chroma/key assertions.title_cue— 10 s cinematic sting: filter sweep (automation), volume envelope; asserts LUFS target, monotone cutoff,silent_after.
Each demo = a .ron score + a test running render → verify → probe, plus a
committed spectrogram sentinel.
0.2.0 deviations from this plan (2026-07-21)
- Tempo/rhythm split (schema v3).
tempo.confidencebecame pulse clarity (normalized autocovariance),tempogainedcandidatesandstability, andclear_rhythmmoved to a new grid-basedrhythmsection (grid_alignment,offbeat_ratio). The v2 confidence sketch above is superseded; the drum-groove demo now assertsHasClearRhythm(true). - Verify additions:
GridAlignmentAtLeast,BrightnessRises/BrightnessFalls(render-side sweep verification over the stem’s spectral centroid — the output-side companion toMonotone, which remains authored-curve-only by design). - Synth: eight presets (
kick,snareadded);chord_padis stereo (±0.35 constant-power saw spread). - Self-describing reference:
cochlea_score::authoring_referencefeedscochlea reference, the MCPscore_referencetool, and the book’s Score Format page, pinned together by tests. - MCP: seven tools; inline image content for spectrograms
(
out_pathoptional);--rootconfinement; canonical-path clobber guards. - Structure detection computes a banded similarity matrix with a deterministic frame-count cap (the O(n²) full matrix is gone).
0.3.0 deviations from this plan (2026-07-22)
- Melody + timbre (schema v4).
pitch.melody(quantized note events off the YIN track — the compose loop’s read-back half) and a top-leveltimbreMFCC digest;CompareReportv3 adds a spectral-shape distance andrhythm.grid_changed. - Triplet grids. Rhythm alignment tests straight-16th and
eighth-note-triplet hypotheses and reports the winner (
rhythm.grid) — swing is recognized, not scored sloppy. - The zoom lens.
Audio::window+--from/--toon probe/spectro/ diff (andfrom_s/to_son the MCP tools);source.start_msanchors windowed reports. - Spectro: analysis overlays (beats/onsets/pitch as plain data) and signed A→B difference heat maps.
- Master bus. The score IR gains
Master/Limiter(RONmaster:); the render bus is nowΣ stems → master → f32, byte-inert when absent. “Bus routing” stays out of scope — this is one fixed output stage, not routing. - Lossy probe input. mp3/ogg decode via symphonia (“compressed-
format probe (phase 2)” partially delivered) — analysis input only,
contract documented in
cochlea-decode. - MIDI import (
import_midi/cochlea import/ MCP tool): “MIDI import/export, out of scope for v1” is half-lifted — import only, hand-rolled SMF 0/1, timing exact, instrumentation a labeled guess. Export remains out of scope. - MCP: eight tools (
import_midi), window/annotate params, inline diff heat maps.
Still deliberately out (design questions, not omissions): sampled instruments (external sample bytes vs the score-is-the-audio contract — needs content-hash pinning and a resampling determinism decision before any code), tempo ramps, time-signature changes, sample-accurate automation, bus/send routing, realtime anything.
0.4.0 deviations from this plan (2026-07-24)
- Harmony (schema v5). A top-level
harmonysection: a chord timeline (template-matched from the shared chroma STFT) plus per-section key — the two questions the single globalkeycan’t answer (“what’s the progression”, “what key is the bridge in”). Alsoloudness.short_term_max_lufsand a standaloneloudness_timeline(the dynamics curve), and downbeat fields on the fullTempoReport(beats_per_bar,downbeats_ms,bar_beat_at). - MIDI export. The other half of import:
export_midi/cochlea export/ the MCPexport_miditool. “MIDI import/export, out of scope for v1” is now fully lifted — timing exports exactly, instruments become rough GM labels (the inverse of the importer’s family mapping). - Integer PCM output. 16/24-bit WAV via
--bits(WavBitDepth), for a small ordinary file; float32 stays the lossless ground truth. fm_bellpreset. The palette’s ninth voice and its first non-subtractive one — harmonic FM with an automatablebrightness(modulation index), answering the “rich ears, thin voice” critique that every prior voice was a filtered saw/square.- Python bindings.
bindings/python(pyo3 + maturin): anassert_audiofluent API and a pytest plugin. A detached crate, deliberately outside the determinism build. - Golden-audio eval.
cochlea evalscores a directory of candidate renders against a directory of goldens by filename, exit 1 on any regression — the generative-model regression harness, plus a GitHub composite action. - Hardening (adversarial-review fixes). Authored ticks are bounded at
Ticks::MAX, the read path caps decoded samples and rejects degenerate audio shape, and the MCP server contains any tool panic withcatch_unwindinstead of dying. - MCP: nine tools (
export_midiadded).
0.5.0 (2026-08-06)
- Two non-subtractive voices.
marimba(modal — a struck bar as a fundamental plus tuned octave partials, faded out before retirement so the ring never clicks) andorgan(additive — a drawbar harmonic stack). The palette is now eleven presets, three of them non-subtractive (withfm_bell). - Timeline surfaces.
cochlea probe --loudness/--beatsand the MCPloudness_timeline/beat_gridtools expose the loudness-over-time curve and the full beat grid (every beat, downbeats, candidates, stability) — the per-time detail the compact report summaries drop. MCP is now eleven tools. - Overwrite guard generalized. One shared
same_file(raw-or-canonical) now backs every CLI write guard (probe/diff/import/export/render), closing an aliased-path data-loss hole the old raw-string compare left open.
0.6.0 (2026-08-07)
transcribe: the audio→score arrow.cochlea transcribeand the MCPtranscribe_audiotool turn a rendered or recorded file back into an editable RON score — melody note events read against a detected (or given) tempo, quantized to a note grid, with velocity estimated from peak level. Monophonic by construction, and every guess surfaces as a warning. MCP is now twelve tools.- The conversion lives in
score(transcribe.rs) and takes plainNoteObservationdata, never audio — so the dependency law is untouched:featuresandscorestay independent leaves, and the front ends (CLI, MCP) do the five-line join. This is the same “receives plain data, never the other crate’s types” pattern the spectrogram annotation path already uses, pointed the other way.
- The conversion lives in
- Malformed-MIDI hardening. A time-signature denominator exponent past 31 overflowed an unchecked shift in the importer (panic in debug, garbage in release), reachable by a single byte flip. Now checked, warned, and covered by a dedicated malformed-input suite.
0.7.1 (2026-08-13)
An adversarial pass over 0.7.0’s own fixes.
- Presence, not existence. 0.7.0’s stem-path containment check was
guarded by
if path.exists(), andexists()follows a symlink — so a link whose target did not exist yet reported “nothing here”, skipped the check, and was followed byFile::createregardless, creating the stem outside the stems directory and outside--rootat exit 0. The test is nowsymlink_metadata(lstat), and a link that cannot be resolved is refused rather than guessed at. General lesson for this workspace: when the question is “is something sitting at this path”,exists()is the wrong call — it answers “does the far end exist”. - One same-file rule, and it folds case. 0.7.0 taught the stem set
that
Leadandleadare one file on macOS and Windows, but left the guard between a stem and the mix/report/score comparing paths exactly — so--out d/Lead.wav --stems dwith a track namedleadstill destroyed the mix at exit 0, on the platform most of this project’s development happens on. The rule iscochlea_render::same_target_file, enforced on every platform for the same portability reason the stem-name rule is (seedocs/determinism.md). - The overwrite sweep finally reaches every subcommand.
spectroandevalwere never wired intosame_file—spectro audio.png --out audio.pngdecoded the audio and wrote the PNG over it (the.wavcase is refused by the PNG encoder, which looks like a guard and isn’t), andeval --json d/a.wavreplaced a golden candidate with its own report after reporting that the candidate passed. - Sharing a rule is not sharing a check. The fix above put the
comparison in one place and left the resolving in front of it to each
caller — so the Python
spectrogrambinding, a third front door onto the same call, had no guard at all, and the MCP server’ssame_fileand the CLI’ssame_filewere two functions with one name and different semantics.cochlea_render::same_fileis now the whole predicate, resolving included, and all three front ends call exactly it. This is the same lesson as the one below it, found one level up: a rule applied per-call-site gets missed at the next call site, and “shared helper” is not evidence that it wasn’t. - A fallback that gives up is a guard that answers wrong. The path
resolver canonicalized the parent when the file did not exist yet and
returned the path untouched when the parent did not exist either —
exactly the case where a run creates its own output directory, so
--stems ./new --report new/lead.wavcompared two spellings of one file as different and let the report land on a stem. The fallback folds components lexically now. The direction of a wrong answer is the whole design here: lexical folding can only merge two spellings (refusing a pair that would have been fine), never split one file into two. - The position path is bounded where positions are made.
Pos::resolvemultiplied(bar - 1)byticks_per_barunchecked, and neither factor was bounded —baris au32from the file and the time signature’s beats-per-bar had no ceiling — so a crafted score overflowedu64(panic in debug, silent wrap in release). Separately,Score::resolvenever appliedTicks::MAX, so a far-futureverify:position reachedmul_divat render time and panicked after the mix was written. Every position now resolves through one bounded path — both arms, grid and raw: bounding the grid alone closed the RON route and left the same panic a line above it in the Rust API, sinceTicksis a public newtype over a publicu64andverify(...).silent_after(Ticks(1 << 40))goes straight intoScore::resolve. Bounding the funnel retired the per-buildercheck_tickcalls entirely — one bound, one place — with each builder now passing only the noun for the error message (“tempo change”, “automation key”), since a shared check should not cost a specific message. What stays at the builder is the note end check:at + duris not a position, and a raw-tickDuris unbounded. - Bound the input, don’t clamp the output.
TimeSignature::validateaccepted any nonzero numerator, andexport_midithen squeezed it into the SMF meta event’s single byte withunwrap_or(u8::MAX)— a score that said 300/4 exported a file that said 255/4, silently. The bound belongs at the door (beatsis 1..=255 now, named after the byte that has to hold it): one check replaces a truncation, an unreachable bar 2, and an error message that advertisedu32::MAXas a legal value. - A
pub fnthat hardens its arithmetic has to check what it divides by.Pos::resolvegained checked multiplication and a tick ceiling while still callingticks_per_beat—whole_note_ticks / unit— on a caller-builtTimeSignaturewith public fields, before any validation.unit: 0was a division-by-zero panic in the function that had just been hardened against untrusted numbers.
0.7.0 (2026-08-11)
- Stem names are validated as file names.
write_stems_asderived<dir>/<track>.wavfrom a track name without checking it, and a track name is free-form score data (hand-authored, or lifted verbatim from a MIDI track-name meta event by the importer).Path::joindiscards the base for an absolute argument, so a path-shaped track name wrote outside the stems directory — and, over MCP, outside--rootwhile every path argument stayed inside it. The rule is now one public function,cochlea_render::stem_file_name(one ordinary, portable file name — separators,:,<>"|?*, control characters and the Win32 device names all refused on every platform, so a score means the same thing on every host), applied at the write sink and pre-flighted by both front ends so nothing is written when a name is refused. The same “one public rule, several callers” shape asBpm::validateandvalidate_window_ms. - Containment is checked at the path, not just the name. A code review
of the above found the name rule was necessary but not sufficient: a
symlink already sitting at
<stems>/<track>.wavis followed byFile::create, so a legitimate name still wrote outside the stems directory and outside--root(reproduced).write_stems_asnow canonicalizes the stems directory and resolves any existing stem target before writing, refusing one that lands outside — the same treatmentToolCtx::resolve_writegives the MCP server’s own path arguments, which stem paths never passed through. A link that stays inside is left alone. - The output-collision guards were completed.
same_filecanonicalized both sides, which fails for a path that does not exist yet, so for two outputs it degraded to a raw string compare and--out d/stems/lead.wav --stems d/stems/../stemsdestroyed the mix at exit 0; it now resolves through the canonical parent plus file name. The CLI gained the missing stem-vs-score pair (a score need not be named.ron), and the MCPrender_score, which had no stem-collision guard at all, gained stem-vs-out_pathand stem-vs-score_path.
Determinism: decisions, audits, and rationale
This file records the Phase 0 determinism decisions and the dependency audits behind them. The contract itself (three tiers) is summarized in the README; this is the evidence and the fine print.
The three tiers, restated precisely
- Tier 1 — byte-identical PCM for identical inputs on the pinned target:
x86_64-unknown-linux-gnu, toolchain1.95.0(rust-toolchain.toml). Two renders of the same score are byte-equal on any machine (tested on every CI platform); committed golden hashes are asserted on the pinned target (and on aarch64-macos, the bless machine). Empirical note: the first Tier 1 CI run (2026-07-03) matched the aarch64-macos-blessed hash exactly — with every DSP path on libm + pure arithmetic, the render is in practice byte-identical across these architectures, stronger than the tier promises. The contract remains per-pinned-target so a future divergence is a re-bless, not a breach. - Tier 2 — feature tolerances across platforms: integrated LUFS within 0.1 LU, onsets within 2 ms, pitch within 5 cents. These absorb the cross-target float differences Tier 1 does not promise away.
- Tier 3 — spectrogram sentinels: image diff with per-pixel tolerance and a max-fraction-differing threshold.
Why “audio is a fold” shapes everything
Filters, envelopes, delays, and reverbs carry state; sample N depends on all samples before it. So the reproducibility unit is the whole render, not the sample. Consequences: fixed summation order everywhere (voices in index order, stems in track order), one rounding rule at the tick→sample boundary, and stochastic sources that are random-access (counter-based) so they don’t inherit the fold’s ordering sensitivity.
Denormal policy: honor denormals everywhere
Decision: no FTZ/DAZ, no algorithmic offsets. We never call fundsp’s
prevent_denormals() (fundsp/src/denormal.rs — it sets x86 MXCSR to
0x9fc0, i.e. FTZ|DAZ, and is a no-op on aarch64). The audit found fundsp
calls it itself inside the Feedback/Feedback2/FDN node family — with
no restore, so one tick of a feedback() node flips FTZ on thread-globally
on x86. Those combinators are therefore banned via clippy
disallowed-methods; the stock Reverb doesn’t use them and is fine.
(ebur128 also flips FTZ inside its K-weighting loop, but scoped with a
restore-on-drop, analysis-side only — accepted, see its audit section.)
Rationale, in order:
- Determinism. IEEE 754 fully specifies denormal (subnormal) results
for
+ − × ÷andsqrt. Honoring them is bit-deterministic on one machine and bit-uniform across architectures. Flushing is neither: the kickoff’s FTZ option can’t even be implemented uniformly (x86 MXCSR vs aarch64 FPCR; fundsp only covers x86, and our workspace forbids theunsafeneeded to touch FPCR ourselves). - The performance argument doesn’t apply offline. FTZ exists so realtime reverb/filter tails don’t blow the audio callback budget. We render offline; a rare 10–100× slowdown on a handful of samples in a decaying tail costs milliseconds of wall clock, not glitches.
- The f64 master bus makes denormals rarer where it matters. f32 subnormals start at ~1.2e−38; summing at f64 keeps the bus far from its own subnormal range (~2.2e−308).
Trade-off accepted: fundsp’s f32 filter internals may briefly process subnormal state values at tail ends, slower than flushed. If profiling ever shows a pathological case, the phase 2 revisit is an algorithmic fix applied uniformly (documented tiny offset), never an FPU-flag fix.
Toolchain and codegen pins
rust-toolchain.tomlpins1.95.0; CI installs exactly that via rustup’s toolchain-file support. Bumping the toolchain is a deliberate commit that re-blesses golden hashes if they move.- No fast-math: we never enable
-ffast-math-style flags and don’t setRUSTFLAGScodegen options that relax float semantics. - No implicit FMA: Rust never contracts
a * b + cinto fma by default;mul_addis in the clippydisallowed-methodsban list so fusion is always an explicit,#[expect]-documented choice. - Std float transcendentals are banned in this workspace by
clippy.tomldisallowed-methods(delegated to platform libm — different results per OS/libc). DSP code uses thelibmcrate, which is pure-Rust and bit-stable across platforms. Stdsqrtis exempt: IEEE-exact, hardware instruction everywhere.
fenestra-anim 0.1.0 (audited by reading the published source)
What the timebase and automation path actually execute:
mul_div(a, b, c, Rounding) -> u64(src/rational.rs): u128 intermediate, explicitFloor | Ceil | Round(ties up), panics onc == 0and on u64 overflow of the result. Pure integer math — exact and platform-independent. This is the only tick→sample primitive we use.Track<T>::sample(frame, fps)→locate()(src/track.rs): segment lookup bypartition_point(integer), progressu = (run as f64 / span as f64) as f32(exact-division semantics, deterministic), then easing:Ease::Linear,Ease::Hold: pure arithmetic. Deterministic everywhere.Ease::Bezier(src/bezier.rs): fixed 16-iteration Newton solve using only* + - /andclamp— no transcendentals. Deterministic everywhere.Ease::Spring(src/spring.rs): std f32exp/cos/sin/sqrt— platform libm, NOT cross-platform bit-stable. v1 rejects springs on automation tracks at validation (also for the seconds-grounding reason indocs/plan.md), so no fenestra-anim transcendental runs in any v1 signal path. Springs stay available to non-audio consumers upstream.
Interpolate for f32:a + (b - a) * t, pure arithmetic.
Conclusion: cochlea’s automation evaluation (linear/hold/bezier over ticks) is bit-deterministic across platforms, stronger than Tier 1 requires.
fundsp 0.23.0 audit
Manifest facts (verified in the vendored crate): depends on libm directly;
wide (SIMD f32x8) is unconditional; funutd is the RNG dependency;
default features files (symphonia) and fft (fft-convolver) are disabled
in our workspace pin (default-features = false, features = ["std"]), which
keeps symphonia out of Cargo.lock entirely (also enforced by a deny.toml
ban).
Audit findings (full-source read of the vendored crate; per-node-family detail with file:line citations retained in the audit transcript, digest here):
- The node interface is f32, full stop.
AudioNode::tick/processare hardcoded toFrame<f32, _>(audionode.rs:75,79);prelude64only upgrades internal state precision (Sine<f64>phase, filter state) and still emits f32 per sample. Consequence: fundsp renders per-voice f32; cochlea’s f64 buses (voice sum, master sum) live outside fundsp’s node boundary, inrender. We use theprelude64node constructors where offered (f64 internal accumulators cost nothing offline). tick()andprocess()provably diverge.Sine::tickcomputessinvialibm::sinf(oscillator.rs:71→lib.rs:480);Sine::processcomputes it viawide’s f32x8 SIMD polynomial (oscillator.rs:82→lib.rs:632) — different algorithms, no bit-equality contract. SIMDfloor/ceilare even approximations ((x ± 0.4999999).round(),lib.rs:326-332). Rule: voices tick sample-by-sample;AudioNode::process/AudioUnit::processare in the clippydisallowed-methodsban list. One code path, one rounding story.- Scalar transcendentals all route through libm — exhaustively
confirmed:
Num/Float/Realimpls for f32/f64 calllibm::{sinf,cosf,tanf,expf,exp2f,logf,log2f,log10f,tanhf,atanf,powf,...}(lib.rs:168-280,444-594,773-825). No std float intrinsics in the scalar path. Per-sample transcendentals in nodes we use:Sine::tick(sin),Moog::tick/Rez::tick(tanh) — all libm. Biquad/Lowpole/Highpole coefficient math (tan/sin/cos/exp) runs at construction/set-parameter time only. None of Biquad, Moog, Rez, Reverb, WaveSynth, Pluck, ADSR overrideprocess()anyway — the divergence risk is concentrated in oscillators likeSine; the tick-only ban covers everything uniformly. - No entropy anywhere. Zero hits for
SystemTime|thread_rng|rand::|Instant|getrandom|OsRngacross the crate; noranddependency. All node seeding derives from the structural graph hash chained throughping()/AttoHash(deterministic function of node IDs and combinator positions), overridable per node viaSetting::Seed(u64). Two structurally identical graphs get identical output. Cochlea presets set explicit seeds/phases anyway so sound identity survives graph refactors. - fundsp’s own denormal handling is x86-only, scattered, and
side-effectful:
prevent_denormals()is called from exactly one node family —Feedback/Feedback2/FDN (feedback.rs:129,136,241,248,357,370) — and sets MXCSR FTZ|DAZ thread-globally with no restore, a no-op on aarch64. Under the honor-denormals policy those combinators are banned (feedback,feedback2,fdn,fdn2,prevent_denormalsin the clippy ban list). Verified consequence: every fundsp stereo reverb constructor is off-limits —reverb_stereobuildsfdn::<U32>directly (prelude.rs:1755) andreverb4_stereo_delaysbuilds twofdns (prelude.rs:1938-1939); the clippy ban cannot see those internal call sites, so the rule is: no fundsp reverb constructors at all. The reverb insert is instead an in-repo Freeverb-style Schroeder (8 damped combs + 4 allpasses per channel,cochlea-synth/src/nodes.rs), pure arithmetic per tick. - ADSR reset is a trap:
adsr_live’s closure capturesattacked+ twoSharedcells thatreset()cannot see (adsr.rs:29-33). Voices are therefore always freshly constructed per note, never pooled-and-reset — which the render engine does anyway by design. Net/Sequenceruse hashmaps only for keyed lookup (execution order is a Vec-based topological sort,net.rs:834-917), but their live-edit frontends exist for realtime use; cochlea builds staticAn<_>graphs and never usesNetcommit APIs.- funutd (fundsp’s RNG dep) stays out of our signal path: fundsp’s
noise()/pluck()/hold()nodes (funutd-seeded, andPluck’s excitation is a statefulfunutd::Rndreplay) are not used by cochlea presets. Noise and Karplus-Strong excitation come from the in-repo counter RNG keyed(seed, sample_index); the KS voice is hand-rolled (delay line + damping FIR as a customAudioNode) rather than fundsp’sPluck. funutd remains in Cargo.lock as an unused-at-runtime transitive dep. - Versions that can move last-bit float behavior —
fundsp,libm,wide,funutd— are pinned by Cargo.lock (committed); toolchain pinned; no-C target-cpu=nativeanywhere.
Per-preset Tier 1 verdicts (all under the tick-only rule): sine safe
(libm sin); saw_lead/square_bass/chord_pad safe (WaveSynth’s
interpolation is pure arithmetic, filters libm-at-construction, ADSR fresh
per voice — implemented as our own piecewise-linear closure over note time
rather than fundsp’s adsr_live, sidestepping its closure-state reset trap
entirely); noise_hat safe (counter-RNG noise + filter); pluck safe
(hand-rolled KS, counter-RNG excitation); reverb insert safe (in-repo
Schroeder — fundsp’s reverb constructors are all FDN-based and banned, see
above).
ebur128 0.1.10 audit
API facts the features crate builds on (file:line citations in the audit transcript):
- Construct
EbuR128::new(channels, rate, mode); we useMode::I | Mode::TRUE_PEAK(TRUE_PEAK implies SAMPLE_PEAK and M). Feed interleavedadd_frames_f32(&[f32]). Default 2-channel map is[Left, Right]— correct for us, noset_channelneeded. - Readouts:
loudness_global()→ LUFS (needsMode::I),loudness_momentary()→ LUFS over the last 400 ms (momentary max is ours to track: feed in 100 ms chunks and take the running max of readings),true_peak(ch)/sample_peak(ch)→ linear amplitude — we convert to dBTP/dBFS via20·log10(libm) ourselves. - Gating confirmed BS.1770-4: absolute −70 LUFS gate (energies below the
first histogram boundary are discarded before recording,
history.rs:264-267, boundary derived from the −70 LUFS energy) and relative −10 LU gate (history.rs:320-334), 400 ms blocks at 75% overlap. - Silence/not-enough-audio is not an error:
loudness_*returnOk(-inf), peaksOk(0.0). The features crate maps-infto a JSONnull-with-reason, never an error. - True peak: rate-dependent oversampling (4× below 96 kHz), 48-tap
Hanning-windowed-sinc polyphase FIR with coefficients computed once at
construction; FIR math is f32; no SIMD; the
precision-true-peakfeature (FMA in the FIR) stays off. - Internal math is otherwise f64. One arch-conditional path, flagged: the
K-weighting filter loop enables scoped hardware FTZ on x86_64
(
filter.rs:376-428, restored on drop) and approximates it on other arches by flushing filter state belowf64::EPSILONat block boundaries. So loudness readings can differ at ULP level between x86_64 and aarch64. This is analysis-side only (never touches PCM), scoped (does not leak MXCSR state to our thread beyond the call), and absorbed by the Tier 2 0.1 LU tolerance with ~5 orders of magnitude of headroom. Accepted.
rustfft 6.4.1 audit
FftPlanner::new()does runtime CPU-feature dispatch (AVX+FMA → SSE4.1 → NEON → WASM → scalar). Same binary, different machine ⇒ different FFT bits; and a cloud CI runner migration could silently flip the path on the “pinned” target.- Decision: analysis code constructs
FftPlannerScalarexplicitly, everywhere (rustfft::FftPlanner::newis in the clippy ban list). One code path on every machine, immune to runner-hardware drift. Our FFTs are small (1024–2048 points at audio hop rates); scalar is more than fast enough offline, and this choice makes features/spectro deterministic per-binary, not just per-platform. - Planning is a pure function of
(len, direction)for the scalar planner;process()panics on length mismatch (caller invariant); output is unnormalized (we only use magnitudes, and normalize where needed, consistently). - rustfft is used only in
featuresandspectro— never in the PCM render path — so even its residual cross-toolchain variance is bounded by Tier 2 tolerances and Tier 3 image diffs by construction. MSRV 1.61, deps purely numeric.
Rounding rules (the complete list)
Bpm(f64)→ integer nanoseconds-per-quarter (round), once, at authoring/parse time. The stored score is exact from then on.- Tick → sample:
mul_div(Δticks, npq · sr, ppq · 1e9, Rounding::Round)per tempo segment with left-to-right integer anchors; applied once at event-schedule time. Property-tested monotone and drift-free. - Sample → tick (block starts → automation domain): same segment math with
Rounding::Floor(a block start belongs to the tick it is inside). - f64 master bus → f32 output samples: default Rust
as f32(round-to-nearest-even), documented here, applied at the very end.
Nothing else in the pipeline rounds between integer domains.
v2 analyzers and decode (wave 2 additions)
The wave-2 surface introduces no new determinism mechanism — it reuses the existing rules — but each addition deserves its line in this ledger:
- FLAC decode (
cochlea-decode, symphonia 0.6, FLAC feature only). FLAC reconstruction is pure integer arithmetic by spec; the only place a decode could diverge from a WAV twin is float normalization.symphonia-bundle-flacleft-justifies every sample into 32 bits (its own documented “common denominator”), so the crate divides by2^31unconditionally — exactly equal to the WAV path’s per-depth2^(bits-1)divide (both are the same power-of-two rescale, exact in IEEE 754). Enforced bit-for-bit by committed WAV/FLAC twin fixtures. Zero-packet streams take their shape from STREAMINFO and their sample vector stays empty — no fabricated rates. - Tempo (
tempo.rs), stereo (stereo.rs), structure (structure.rs), segment timeline (segments.rs). All transcendentals vialibm; the onsets-grade STFT they share comes from the sameFftPlannerScalarpath as everything else; autocorrelation and the novelty kernel use fixed ascending summation order; every float sort usestotal_cmp. The analyzers are pure functions of the buffer —probe()computes shared intermediates (STFT, onset report, YIN track) once and fans them out, which is a pure refactor precisely because the passes were deterministic duplicates. - Non-finite input policy. Float WAVs can legally encode NaN/±inf;
IEEE 754 comparison semantics would let a single NaN sample masquerade
as silence in one analyzer and a real peak in another.
Audio::from_wavrejects non-finite samples at ingestion (NonFiniteSample), so no analyzer ever sees one. Degenerate options (NaN window/frame lengths, non-finite silence floors) are guarded withis_finite()checks — a plain<= 0.0range check is NaN-blind. - Text outputs (digest, compare). Fixed-precision
{:.N}formatting only, stable field order, no wall clock, no hash-map iteration — byte- deterministic per platform for identical input; the numbers inside stay Tier-2 across platforms like every other analysis float.
0.2.0 additions (2026-07-21)
- Tempo/rhythm split. Pulse clarity (mean-removed, length-unbiased
autocovariance), tempo candidates, windowed stability, and the
grid-alignment rhythm analyzer are all pure arithmetic +
libmover the same shared STFT, fixed summation order throughout. The Ellis beat-DP envelope is normalized by its own standard deviation — a pure function of the buffer, so normalization changes nothing about determinism, only the penalty calibration. kick/snarepresets, stereochord_pad. Drum envelopes are rational1/(1+t/tau)decays — pure arithmetic per sample, no new transcendental call sites; the pad’s two saws pan through the same constant-powerfd::panused everywhere. Golden re-bless: the drum-groove golden moved to0xE613_FEF7_9664_B529(deliberate DSP + score change: real kit, per-track pans, stereo pad); first-light was untouched (it uses none of the changed patches) and its hash did not move — an incidental cross-check that the preset work leaked into nothing it shouldn’t have.- Banded structure similarity. Computes exactly the entries the checkerboard kernel reads, in the same order, same arithmetic — byte-identical novelty curves to the full matrix, minus the O(n²) waste. The frame-count cap changes the effective frame length only as a deterministic function of input length.
- Spectral centroid (
centroid.rs). Weighted mean over the shared scalar-FFT magnitudes; a silent frame’s centroid isNone, never a ratio of float-noise sums. - MCP base64/PNG.
encode_pngruns the identicalimageencoder as the file path; base64 is a pure byte mapping — the inline spectrogram and the PNG on disk are the same bytes.
0.3.0 additions (2026-07-22)
- Master bus (gain + limiter). The bus becomes
Σ stems (f64) → master → single f32 rounding— rounding rule 4 is unchanged and still applied exactly once. A default master returns before touching a sample, so master-less scores render byte-identically to 0.2.0 (both golden hashes confirmed unchanged). The limiter is pure arithmetic plus twolibmcalls at setup (powfor the ceiling,expfor the release coefficient): per-frame peaks, a forward sliding-window maximum via a monotonic deque (integer logic, no float accumulation), and a one-pole release. Gain is clamped byceiling / windowed_peakat every frame, so the sample-peak ceiling is exact by construction on every platform — no fast-math, no FMA, nothing target-dependent. - Melody + timbre analyzers. Melody is integer quantization and run
grouping over the existing YIN track (
libm::log2/exp2only, the same calls the pitch report already made). MFCC is a hand-rolled mel filterbank + orthonormal DCT-II over the shared scalar-FFT magnitudes —libmtranscendentals, fixed summation order, and a per-frame dynamic-range floor that is a pure function of the frame. - Triplet-grid rhythm hypothesis. Grid geometry only — the same classifier run at two subdivision counts, winner by aligned count with a deterministic tie-break (straight). No new signal pass.
- Lossy decode (mp3/ogg) is Tier-2-adjacent, not Tier-1. The lossy
path is analysis input only: symphonia is pure Rust and our feature
set keeps its
opt-simdruntime dispatch off, so decoding the same file with the same build is reproducible — but no bit-exactness claim is made (the codec discarded the original samples), and nothing lossy can reach the render path. The FLAC module and its WAV-twin bit-exactness test are untouched. - MIDI import. Integer tick arithmetic end to end: SMF division
becomes PPQ verbatim, deltas sum in u64 with overflow checks, tempo
metas convert through the same
Bpm → ns/quarterrounding rule 1 as authored tempos. No float time anywhere in the parser. - Spectrogram overlays and diffs. Drawing is integer pixel writes at
positions derived from sample offsets; the diff image is per-cell f32
subtraction of two already-computed spectrograms.
MelSpec::hz_banduses the same HTK mel formula as the filterbank (libm), so overlay placement is as deterministic as the spectrogram itself.
0.4.0 additions (2026-07-24)
- Harmony (chords + per-section key). Reads the same shared chroma
STFT the key estimate already builds. Chord detection is a cosine match
against L2-normalized binary templates in fixed order; the per-section
key runs the exact Krumhansl-Schmuckler correlation the global key uses.
Only
libm::log2and the exempt stdsqrt— no new transcendental call sites, fixed summation order throughout. - Loudness timeline + short-term max. ebur128 is fed in fixed-size
chunks and polled at a fixed hop; the running maxima are tracked in
order. Same audit as integrated LUFS —
-inf/undefined maps tonull, never a non-finite float. - Integer PCM output (
WavBitDepth). 16/24-bit WAV is a deterministic clamp to[-1, 1], scale by2^frac, round to nearest, clamp to the signed range — no dither (dither would trade determinism for a lower noise floor, the wrong trade here). Float32 stays the render’s ground truth; the integer paths are a lossy convenience, not a second source of truth. fm_bellpreset. Harmonic FM (asinefrequency-modulated by a harmonic-ratiosine), amplitude and modulation-index both driven by the same control-rate ADSR envelopes every other voice uses — pure arithmetic per sample,libmonly at construction. It uses none of the patchesfirst_light/drum_grooveplay, so both golden hashes are unchanged (the same incidental cross-check the drum presets got).- MIDI export. Score ticks become SMF ticks verbatim (PPQ is the file division); only the tempo value is microsecond-quantized, which is all SMF can carry. Delta-times are integer VLQ. No float time anywhere in the writer — the exact inverse of the import path’s integer arithmetic.
- Hardening. The authored-tick bound (
Ticks::MAX) is a pure integer comparison; the decode sample caps are integer counts checked as they accumulate; the MCPcatch_unwindbackstop wraps dispatch only and never touches the render fold. None of the three affects rendered bytes.
0.5.0 additions (2026-08-06)
marimbaandorganpresets. Both are pure arithmetic over exact harmonics:marimbasums modal sine partials under1/(1+t/tau)²decays (plus a short linear fade before retirement, the same guardpluckuses);organis a weighted sine sum under the piecewise-linear ADSR. No new transcendental call sites, no fundspfeedback/fdn/process. Golden hashes are unchanged — no flagship score plays them.- Loudness/beat-grid surfaces. No new DSP — they reuse the existing
loudness_timelineandestimate_tempoanalyzers (ebur128 polled at a fixed hop; the shared scalar-FFT tempo path), served over new CLI flags and MCP tools.
0.6.0 additions (2026-08-07)
transcribe(audio → score). Analysis times are inherentlyf64milliseconds — a tracker measures frames, not ticks — so this path has one float→integer conversion, and it is confined to exactly one place:ms_to_ticksincrates/score/src/transcribe.rs, applied once per note boundary and immediately rounded tou64withlibm::round. Nothing accumulates in float: quantization, the minimum-duration floor, and theTicks::MAXbound are all integer arithmetic on the rounded value. This is the same shape as the existing authoring-time rounding (Bpm→ integer ns per quarter): float in, integer immediately, integer thereafter.- Non-finite and negative inputs are rejected at that boundary rather than propagated, so a NaN from an analyzer can never reach the tick domain.
- Ordering is deterministic: observations sort by
(start_ms, midi)withtotal_cmp, so equal starts break ties by pitch rather than by input order. - No new DSP and no new transcendental call sites — velocity estimation
is
libm::log10over an existing peak fold, and the render path is untouched. Golden hashes are unchanged.
- MIDI time-signature denominator. The importer’s
1u32 << payload[1]becamechecked_shl. This is a malformed-input fix, not a numeric one: the shifted value never reaches the render fold (an out-of-range denominator is warned about and the default 4/4 kept), so rendered bytes are unaffected.
0.7.1 additions (2026-08-13)
- Position arithmetic is checked, and the answer is the same everywhere.
Pos::resolve’s(bar - 1) * ticks_per_barwas unchecked with both factors caller-supplied. In debug that panicked; in release — the profile every published binary is built with — it wrapped, which is the worse outcome for this workspace: the same score loaded to different ticks under different build profiles, i.e. a determinism failure dressed as an overflow bug. Nowchecked_mul/checked_addplus the existingTicks::MAXbound, so a position either resolves to one tick on every build and every host, or is refused. No rendered sample changes; the golden PCM hashes are unchanged. - Case folding joins the portability rule. The “one score means one thing on every host” argument recorded below for stem names now also covers the paths those names collide with: two outputs differing only by case are refused on Linux too, because they are one file on macOS and Windows. Same trade, same reason — the answer should not depend on which machine asked. The NFC/NFD limit below is unchanged.
- A time signature’s numerator is bounded by what can carry it.
beatswas any nonzerou32at the door and a clampedu8at the MIDI exit, so a score could say 300/4 and its exported.midcould say 255/4 with nothing to mark the loss. Export is a second representation of the same score, and this workspace’s rule for a second representation is that it either round-trips or errors — never quietly disagrees. Boundingbeatsto 1..=255 (TimeSignature::MAX_BEATS, named after the byte the SMF meta event gives it) makes the export exact by construction. No sample changes; no real signature reaches the ceiling.
0.7.0 additions (2026-08-11)
- Stem-name validation: no bit-level change, one cross-platform rule.
The stem-name fix (
cochlea_render::stem_file_name, plus containment of the resolved stem path) touches only which files a render is willing to write, never a sample. No DSP, no new transcendental call sites, no change to the schedule or the bus. Both golden PCM hashes are unchanged and were re-confirmed on the Tier 1 target by CI. - Why the name rule is enforced on every platform, not per-host. A
score is portable data, so the checks that only bite on Windows —
\as a separator,:as a drive or alternate-data-stream marker, the<>"|?*set, theCON/NUL/COM1device names — are refused on Unix too. The alternative is worse than a false rejection:C:was previously accepted on Unix and rejected on Windows, so one score exported stems on the Tier 1 target and errored on a Tier 2 one. That is the same commitment as the bit-exact render, applied to the filesystem edge — the answer should not depend on which machine asked.- Known limit, recorded rather than half-solved: names colliding only under Unicode normalization (NFC vs NFD) are not caught, because normalization would mean a new dependency in a workspace that keeps its graph deliberately small. Case-insensitive collisions are caught.
Golden-audio testing
The core idea behind cochlea: audio is too big and too opaque to diff by eye or by byte, but its features are small, meaningful, and — on the pinned target — byte-reproducible. That makes audio testable the way any other output is testable: render (or synthesize) it, compare against a checked-in reference, and fail the build when it moves outside tolerance.
This is the golden-audio pattern, and it’s what cochlea is really for. No human ear in the loop.
The three tiers of “did it change”
cochlea answers “did this audio change” at three levels of strictness (see determinism.md for the full contract):
- Tier 1 — byte-identical PCM. The exact same samples. Use this for a deterministic renderer on a fixed target: any difference is a real change.
- Tier 2 — feature-equivalent. The audio measures the same within cross-platform tolerances (integrated LUFS within 0.1 LU, onsets within 2 ms, pitch within 5 cents). Use this across platforms, or for a model whose output isn’t bit-exact but should stay perceptually stable.
- Tier 3 — spectrogram sentinel. An image diff of the mel spectrogram with a pixel tolerance, for a visual regression check.
cochlea diff — compare two files
cochlea diff candidate.wav golden.wav --tier2
Exit code 0 means byte-identical or Tier-2 equivalent; exit code 1 means
they differ. That single exit-code gate is all a CI step needs. Add --json report.json for the full per-dimension comparison, or --spectro diff.png for
a signed A→B difference heat map (red = louder in B, blue = quieter).
cochlea eval — score a directory of outputs
For evaluating a generative model (TTS, voice, music-gen), you usually have a
directory of outputs to check against a directory of references. cochlea eval does the whole set in one deterministic pass:
cochlea eval --candidates out/ --references golden/ --json eval.json
It matches files by name, compares each pair, prints a per-file verdict table
and an aggregate pass rate, and exits 1 if any pair regressed (or a reference
is missing). Add --exact to demand byte-identity instead of Tier-2
equivalence. This is a reference-render regression oracle for an audio model or
DSP library: check the golden set in once, and every change is scored against
it with no listening and no flakiness.
In Python (pytest)
The Python bindings turn the same check into an ordinary assertion:
from cochlea import assert_audio
def test_tts_output_has_not_regressed():
synth("hello world", "out.wav")
assert_audio("out.wav").matches("golden/hello_world.wav") # tier-2
or, checking properties instead of a golden:
assert_audio("out.wav").true_peak_below(-1.0).pitch_matches("A4").not_clipping()
In CI (GitHub Actions)
The bundled composite action wraps cochlea eval:
# .github/workflows/audio-regression.yml
name: audio regression
on: [pull_request]
jobs:
golden:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: cargo run -p cochlea -- render examples/scores/first_light.ron --out /tmp/out.wav
- uses: ./.github/actions/golden-audio
with:
candidates: /tmp
references: examples/golden
tier: tier2
See .github/actions/golden-audio/action.yml.
Blessing a golden
When a change to the audio is intended, re-bless the reference deliberately —
regenerate the golden file and commit it with a note on why the sound changed.
For cochlea’s own render goldens that’s cochlea render … --out golden.wav; for
the internal PCM-hash and spectrogram sentinels see the “Blessing goldens” notes
in the project README. The discipline is the same everywhere: a golden only
changes when a human decides the new sound is correct.
cochlea-mcp
An MCP (Model Context Protocol) stdio server over the cochlea libraries.
Any MCP client — Claude Code, another agent, a script — gets render /
probe / spectro / lint / digest / loudness / beats / diff /
import / export / transcribe / reference as tool calls, so it can
compose, render, and “listen” to audio through
numbers and images without shelling out to the cochlea binary or
reading raw PCM.
The protocol is hand-rolled JSON-RPC 2.0 over newline-delimited stdio: one JSON object per line in, at most one JSON object per line out. No async runtime — offline batch tools (render a score, extract a report, write a PNG) need none. Every response is a pure function of its request: no wall clock, no session state, so identical requests produce identical responses.
Tools
Each tool mirrors the matching cochlea CLI subcommand’s semantics exactly
(see crates/cli/src/main.rs) over the same library calls — this is a
second front end onto the same offline pipeline, not a reimplementation.
| Tool | Arguments | Returns |
|---|---|---|
render_score | score_path (string, required), out_path (string, required), stems_dir (string, optional), verify (bool, default false) | Text summary: frame count, duration, sample rate, peak dBFS, stems written; if verify is set, the full verify-report JSON is appended and the call reports isError: true on a failed verification. |
probe_audio | audio_path (string, required), from_s/to_s (numbers, optional) | The full feature report, schema v5 (loudness/LUFS/true peak/LRA + short-term max, onsets, YIN pitch + quantized melody notes, MFCC timbre digest, chroma/key, a chord timeline and per-section key (harmony), tempo with candidates + stability, rhythm with grid alignment + straight-vs-triplet grid + clear_rhythm, stereo image, structural sections, silence, clipping) as pretty JSON. Works on any WAV, FLAC, mp3, or ogg — no score required. from_s/to_s zoom into a time window: report times become relative to the cut, source.start_ms anchors them. |
spectrogram | audio_path (string, required), out_path (string, optional), sheet (bool, default false), bars_per_tile (integer, default 8), annotate (bool, default false), from_s/to_s (numbers, optional) | The image itself, inline, as an MCP image content block (base64 PNG) whenever it fits the ~700 KB cap — a client with no filesystem access still gets to look at the audio — plus a text summary with pixel dimensions. out_path additionally (or, over the cap, instead) writes the PNG to disk. annotate: true draws the detected beat grid (orange, top), onsets (cyan, bottom), and pitch segments (magenta) on the image; sheet: true tiles a contact sheet instead (the two are mutually exclusive). |
lint_score | score_path (string, required) | Text: "ok: no lint findings", or the JSON list of findings. isError: true iff any finding is Severity::Error, matching cochlea lint’s exit-1 threshold. |
probe_digest | audio_path (string, required), window_ms (number, default 1000) | A ~40-line deterministic text digest (cochlea_features::digest_text) instead of a full JSON report — the token-cheap way to “listen” to an audio file. Prefer this over probe_audio unless the caller needs exact numbers to assert against. |
loudness_timeline | audio_path (string, required), hop_ms (number, default 100) | The loudness-over-time curve of the whole file as JSON: momentary (400 ms) and short-term (3 s) LUFS sampled every hop_ms, each point’s time measured from the start of the file. The dynamics view the single integrated-LUFS / LRA summary in probe_audio can’t give — where a mix gets loud, where a gate opens, how the level moves. (For a windowed, anchored analysis use probe_audio with from_s/to_s.) |
beat_grid | audio_path (string, required) | The full beat grid of the whole file as JSON (a TempoReport): every detected beat time (ms, from the start of the file), the estimated downbeats, the tempo with its octave-alternative candidates, and a windowed stability score — the per-beat detail the compact tempo field inside probe_audio drops. |
score_reference | (none) | The complete score-authoring reference as Markdown: the RON grammar (including the master: gain/limiter section), the live instrument-preset catalog (names, polyphony, every automatable param with unit/range/default — generated from the same registry that validates scores, so it cannot go stale), all embeddable verify: assertions, and a worked example that the test suite itself parses and renders. An agent should call this before its first render_score. |
audio_diff | audio_path_a (string, required), audio_path_b (string, required), window_ms (number, default 1000), json (bool, default false), spectrogram (bool, default false) | Feature-space comparison text (cochlea_features::compare_text): a verdict (byte-identical / tier2-equivalent / different (dimensions...)) plus per-dimension deltas, now including a timbre (MFCC) distance. json: true appends the full CompareReport; spectrogram: true also returns the signed A→B difference heat map inline (red = louder in B, blue = quieter, black = unchanged). A different verdict is a normal, successful answer — not isError. |
import_midi | midi_path (string, required), out_path (string, required), sample_rate (integer, default 48000) | Converts a Standard MIDI File (format 0/1, metrical division) to a RON score at out_path. Timing imports exactly; GM programs map to rough preset families and channel-10 percussion to kick/snare/hat tracks, with every mapping guess listed in the response for re-voicing. |
export_midi | score_path (string, required), out_path (string, required) | The inverse of import_midi: converts a RON score to a Standard MIDI File (format 1) at out_path. Timing exports exactly (score ticks → SMF ticks, tempo map and time signature carry over); presets become rough General MIDI program labels, since a synth voice isn’t a GM instrument. Use it to hand a composed score to a DAW or notation tool. |
transcribe_audio | audio_path (string, required), out_path (string, required), bpm (number, optional), grid (string, default "1/16"), preset (string, default "sine"), track_name (string, default "lead"), ppq (integer, default 960) | The inverse of render_score, and the arrow that closes the compose loop: audio in, an editable RON score out. Pitch-tracks the melody, reads its timing against bpm (detected from the audio when omitted), quantizes to grid ("none" keeps raw analyzer timing), and estimates each note’s velocity from its peak level. Deliberately monophonic — chords, drums, and dense mixes come back as whichever single line the tracker locked onto. Every assumption (tempo and where it came from, the grid, the preset, clamped/repaired/dropped notes) is in the response text; treat the result as a draft to re-voice. |
Tool-level failures (a bad path, a render error, a failed verify or lint)
come back as a normal tools/call success response with isError: true
and the reason in the text content — never a JSON-RPC error. JSON-RPC
errors (-32700/-32601/-32602) are reserved for protocol problems:
malformed JSON, an unknown method, or missing/malformed arguments on a
known tool. audio_diff’s different verdict is not one of these
failures — see its row above.
Confinement (--root)
By default the server reads and writes wherever the caller points it —
appropriate for a personal, local loop. For anything less trusted, launch
with --root DIR: every path argument on every tool, reads and writes
alike, must then resolve (canonically — symlinks and .. are resolved
first) inside DIR, and anything else is refused as an Invalid Params
error before the filesystem is touched. This is defense against a
confused or prompt-injected client, not a sandbox against hostile local
processes.
Confinement covers paths the score implies, not just the ones the caller
types. render_score with a stems_dir derives one file per track from
the track name, and a track name is free-form score data — it can arrive from a
hand-authored RON file or, through import_midi, from a MIDI track-name
meta event. A name spelled as a path (/etc/x, ../../x) would otherwise
escape both the stems directory and DIR while every path argument stayed
legitimately inside it. Two checks close that:
- The name, against
cochlea_render::stem_file_namebefore the render starts — one ordinary, portable file name, with separators,:,<>"|?*, control characters and the Win32 device names refused on every platform so a score means the same thing on every host. - The path it lands on, when the stem is written. A well-formed name is
not enough: a symlink already sitting at
<stems>/<track>.wavwould be followed straight out of the directory, so anything already at that path is resolved and checked for containment. Presence is tested withsymlink_metadata, notexists()—exists()follows the link, so a link pointing at a file that does not exist yet used to read as “nothing here” and skip the check whileFile::createfollowed it anyway (fixed in 0.7.1). A link that cannot be resolved is refused — including a dangling one whose target would have been inside the directory, since a lexical check of an unresolvable target is exactly the check that cannot see a redirect. One that resolves and stays inside the stems directory is ordinary and still works.
A stem that would land on the mix (out_path) or the score (score_path)
is refused too, so a stems_dir overlapping either cannot destroy them.
“Same file” here folds case (Lead.wav and lead.wav are one file on
macOS and Windows), on every platform — the same rule the CLI uses, and the
same rule that governs every out_path-versus-input check on this server.
claude mcp add cochlea -- cochlea-mcp --root ~/music-workspace
Client setup
Claude Code:
claude mcp add cochlea -- cargo run -p cochlea-mcp --release
or, against an already-built binary:
claude mcp add cochlea -- /path/to/target/release/cochlea-mcp
Any other stdio MCP client: launch cochlea-mcp (or cargo run -p cochlea-mcp) as a subprocess and speak JSON-RPC 2.0 over its stdin/stdout,
one object per line. stdout carries only protocol responses; all logging
goes to stderr, so it’s safe to leave stderr connected to a terminal or log
file without corrupting the transport.
Example
Request (tools/call for probe_audio, sent as one line):
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"probe_audio","arguments":{"audio_path":"mix.wav"}}}
Response (one line back; the pretty-printed report is escaped into the
text field, shown here unescaped for readability):
{"jsonrpc":"2.0","id":1,"result":{"content":[{"type":"text","text":"{\n \"schema_version\": 5,\n \"source\": {\n \"sample_rate\": 48000,\n \"channels\": 2,\n ...\n },\n ...\n}"}],"isError":false}}
Testing this crate
crates/mcp/src/lib.rs exists so the dispatch logic (server::Server) can
be driven in-process from integration tests — Server::handle_line(&str) -> Option<String> takes one request line and returns at most one response
line, with no stdin/stdout/subprocess involved. crates/mcp/src/main.rs is
just the framing loop around it. See crates/mcp/tests/protocol.rs for
JSON-RPC conformance and crates/mcp/tests/tools_e2e.rs for a real
render → probe → spectrogram round trip against
examples/scores/first_light.ron.