architecture.md

Architecture

See also: Plugins

Files

File	Role
core.js	Engine — decode, paginate, plugin registry, instance factory, page I/O. The only required file.
stats.js	Block-level stat engine — min/max/energy/clip/dc per BLOCK_SIZE-sample block during decode. Powers waveform, loudness, stat queries without touching PCM.
cache.js	Page cache — LRU eviction to OPFS, on-demand restore. Large files stay playable without exhausting RAM.
plan.js	Edit pipeline — non-destructive edit list, plan builder, stream renderer. a.gain(-3).trim() becomes a declarative plan materialized on read.
audio.js	Full bundle — core + stats + cache + plan + all plugins. The default import.
deck.js	Playback output — the deck (an AudioWorklet on the page's one AudioContext; Node: @audio/speaker) and its main-thread transport. Engine-free: the worker facade uses it as is.
fn/*.js	Plugins — each file is one op, stat, or method. Self-contained, independently importable.
bin/cli.js	CLI — arg parsing, plugin auto-discovery, batch/glob/macro/playback.

Three import paths

import audio from 'audio'            // full bundle — all ops, stats, cache
import audio from 'audio/core.js'    // bare engine — no ops, no cache
import gain from 'audio/fn/gain.js'  // individual plugin

Worker engine

audio/worker runs the whole library inside a Worker; the main thread holds a thin facade (imports none of the engine). The edit list is the wire protocol — ops post [type, opts] tuples, the worker replays them through the real engine, so undo, serialization and stream≡read carry across the boundary unchanged. Op methods are proxied against the worker's live registry (plugins registered worker-side appear automatically); read/encode results transfer zero-copy; facades sharing a worker reference each other by instance id (a.mix(b)). Custom worker entry = your plugin imports + import 'audio/worker' (the file self-hosts in worker scope); the engine talks on a MessageChannel handed over once ({ '@audio': port }), so the app's own messages on the Worker never meet its protocol, and expose(a) hands an instance the app made to the page (audioWorker.adopt(id, { worker })). Breakpoint curves {t, v} replace function params, which can't serialize.

Playback crosses the boundary without the main thread: the voice runs in the worker and posts straight into the page's deck worklet through a port the page handed it, so the page can stall for seconds without a dropout; the facade keeps the clock and the events.

Motivation: DSP on the calling thread competes with the ~23ms/block playback budget, so heavy chains (compressor, denoise, declick) jank the UI. The worker moves render off-thread while the facade keeps the media-element API. Main-thread stays the zero-setup default — the worker is the recommended path for editor UIs. Validated against wavearea, which runs its editor on audio/worker.

Playback

voice (engine: main thread or worker) ──port──▶ deck (AudioWorklet) ──▶ speakers
                                  ◀──reports──        ──reports──▶ transport (main thread)

• Voice (fn/play.js) renders the instance's timeline with stream() into runs: planar blocks placed on an axis in seconds. A loop is unrolled on it: each pass's last 10 ms crossfade (equal power) into the 10 ms before the span's start. It renders up to 2 s ahead of the deck's head, yielding to the event loop, so a busy thread doesn't starve the output. On an edit it renders again from just ahead of the head and splices the run in there; a seek splices at once, a hand-off (b.play({ from: a })) continues the axis from another instance.
• Deck (deck.js, one AudioWorkletNode per playback, channels as the source has) reads with a varispeed head: playbackRate glides (~50 ms), the source rate converts to the device's through a windowed sinc (bit-exact copy at unit rate). It crossfades a splice (linear for an edit or hand-off: the same sound changed; equal power for a seek), ramps start, pause, stop and underrun over 5 ms, and reports its head.
• Transport (main thread) maps the reports to what the speakers play now, compensating the output latency (getOutputTimestamp): currentTime, timeupdate, ended, and meter values released when heard.

One AudioContext per page (audio.context, on a global symbol shared by every copy of the library), made on first use and resumed by the first gesture; the deck resamples, so any source rate plays on it. Node runs the same deck in a pump into @audio/speaker.

The stream itself (plan.js) crossfades too: an edit that changes what renders at its position (a cut, an insert, an op added or removed) plays the old render into the new over ~20 ms, the new one warmed up at the same place; a parameter change ramps in place.

Stream-first

No operation touches full PCM at once. Audio is stored in pages. Decode streams pages progressively. Every output — stream(), play(), save(), stat() — walks pages one chunk at a time through the edit pipeline. read() materializes into memory but still processes through the same chunk pipeline internally.

A 2-hour file never needs 2 hours of float arrays in memory unless you read() the whole thing.

Non-destructive editing

Ops don't mutate source pages. a.gain(-3).crop({at: 1, duration: 5}) pushes two entries to a.edits. Source data stays immutable. The edit list replays on demand — undo, serialize, or reapply.

Plan

compilePlan(a, len, final) compiles a.edits into a plan object:

• Segment map — copy instructions: which source ranges map to which output positions. Structural ops (crop, remove, insert, repeat, pad, reverse, speed) rewrite segments. Each segment is [from, count, to, rate?, ref?, interp?] — "read count samples from from, write at to."

• Sample pipeline — per-block transforms applied in order (gain, fade, filter, pan). Each op receives separate input and output buffers (Float32Array[] per channel, BLOCK_SIZE samples). Each pipeline stage owns its output buffer set, sized to that stage's channel width (channel-changing ops like remix are a per-stage property, not a special case) — previous stage's output is the next stage's input, zero allocation in the hot path. Ops read from input, write to output (never alias). Stateful ops carry state across blocks via ctx.

  The engine also resolves cross-cutting concerns once, for every op:

  • Range scoping — {at, duration} on an op without native range handling is applied by the engine: input copied through outside the range, the op invoked only on the in-range sub-block.
  • Automation — function-valued numeric params are sampled by the engine in 128-sample sub-blocks (gain/pan opt into per-sample via auto: 'sample'; ops with genuine function args like filter(fn)/transform(fn) exclude them via fnArgs). Breakpoint curves {t, v} are the serializable equivalent — same sampling, but they survive toJSON() and the worker boundary.
  • Click-free patching — when a streamed plan's values refine (progressive normalize, live edits), numeric changes ramp linearly across one block instead of stepping.
  • Mid-stream edits — stream() watches a.version; an edit pushed while streaming recompiles the plan: values patch in place (ramped), and a change of what renders here (segments, pipeline, latency, width) crossfades (~20 ms) from the old render into the new. play() splices edits in ahead of the head (Playback).

• Stages — edits apply in order: a structural op works on the audio as processed so far. When one follows pipeline ops, the pending pipeline bakes into a stage (a nested plan the new segment map reads as its source) and a fresh pipeline starts. So a.gain(-6).insert(b) leaves b untouched, and a fade, range or automation issued before a splice keeps its time coordinates. A stage streams through a persistent cursor: contiguous reads continue its processor state exactly, a jump re-seeks with the same warm-up as seeking. Edited instances read as sources (insert, paste, mix) stream the same way.

• Limit — the safe output boundary. During incremental streaming (final=false), adjustLimit tracks how far output is deterministic given partial source data.

buildPlan(a) is the cached wrapper for fully-decoded audio — calls compilePlan(a, len, true) once per version.

Clipboard edits (copy, cut, paste) stay in the edit list. During compilation, Copy and Cut capture a shared-page reference to the compiled prefix and crop that reference; Cut removes the range in the same edit, while Paste inserts the clipboard using the normal insertion plan. Removal and insertion bake the processed prefix like any structural op, so earlier effects keep their time coordinates and do not process pasted samples. The clipboard is local to each plan replay: Undo, clone, worker calls and serialization all use the same history. Clipboard chains defer incremental output until decode completes.

During streaming, compilePlan is called repeatedly as more data arrives (final=false). Each call recomputes the segment map and limit from scratch — stateless, pure, cheap. The limit tells the stream loop how far it can safely render without waiting for more source data.

a.edits ──→ compilePlan(a, len, final) ──→ { segs, pipeline, totalLen, sr, limit }
                                                    ↓
source pages ──→ segment map ──→ sample pipeline ──→ output blocks
(Float32)        (structural)    (per-block ops)    (stream or flat)

Incremental streaming

Structural ops (crop, repeat, pad, reverse, etc.) stream incrementally — output begins before decode completes. adjustLimit(limit, type, ctx) is a pure function that transforms the safe output boundary per op type during compilation:

• crop: limit clamps to cropped range
• insert: limit extends by inserted length (when insertion point is reachable)
• remove: limit shrinks by removed portion
• pad: limit extends by pad amount
• reverse: limit collapses to 0 for open-end reverse (needs total), passes through for ranged
• speed: limit scales by rate
• negative at: limit drops to 0 (position depends on total)

The stream loop waits when outPos >= plan.limit, resumes when new source data arrives and recompilation extends the limit.

Progressive resolve

Ops with resolve (trim, normalize) can work with partial stats during incremental streaming. stats.snapshot() returns whatever block stats are available so far (marked partial: true). The resolved result refines as more data decodes — e.g. trim can determine head silence early, normalize can apply gain from available peaks.

Pages and blocks

Audio is fragmented at two levels — pages for storage, blocks for processing:

	PAGE_SIZE (1024 × BLOCK_SIZE)	BLOCK_SIZE (1024)
What	Samples per storage chunk	Samples per processing unit
Stores	Float32Array[] per channel	min, max, energy, clip, dc per block
Purpose	Memory management, cache eviction, decode streaming	Op processing, stat computation, waveform resolution
Memory	PCM data — evictable to OPFS	Stats — always resident (~7 MB for 2h stereo)

The edit pipeline materializes one block at a time: read source → apply structural ops (segment map) → run sample transforms (rotating pre-allocated input/output buffers) → yield output block. Peak memory is two blocks per channel regardless of file length or op chain depth.

Cache eviction works at page granularity: cold pages offload to OPFS, hot pages near the playhead stay resident.

Ops and stats both process in BLOCK_SIZE chunks — same unit, same granularity. Stateful ops like filters carry state sample-to-sample within and across blocks.

Both are configurable — set before creating any instances:

audio.BLOCK_SIZE = 256
audio.PAGE_SIZE = 2048 * audio.BLOCK_SIZE

Each instance's stats record stats.blockSize — the block size used at decode time.

Waveform zoom and block resolution

BLOCK_SIZE sets the finest pre-computed stat resolution. Three zoom regimes:

• Zoomed out (many samples per pixel): stat('max', { bins }) aggregates multiple blocks into fewer bins. Pre-computed stats suffice — no PCM access.
• 1:1 (one block per pixel): stats render directly.
• Zoomed in (fewer samples than one block per pixel): read PCM from pages for sample-accurate waveform. Page cache makes this fast for the visible region.

Changing BLOCK_SIZE cannot retroactively refine existing stats. For finer resolution in a zoomed region, read the PCM.

contract atoms

audio.use(atom) hosts contract audio.js factories natively: params map to op params (engine automation/curves/ramps apply), declared tail composes a trailing pad, declared latency gets plan-level delay compensation, streaming: false atoms run as whole-render ops (materialize → one call → continue from the result), and multi-bus atoms read their sidechain from the key option. audio.plugins maps names to published packages, installed with audio and autowired: registry ops are methods from the start and their package loads by dynamic import at LOAD, the async gate every render passes; ops with a load hook resolve their module the same way (audio.atoms remains as a deprecated alias).

Meta, markers, regions

Container tags (a.meta), point markers (a.markers), and time-span regions (a.regions) live alongside PCM but follow a different path.

On decode, the streaming loop buffers the first 256 KB of the encoded source (enough for ID3v2 tags, WAV chunks before data, and FLAC metadata blocks) and hands them to fn/meta.js for parsing. The parser returns normalized keys (title, artist, ...) plus unknown blocks preserved under .raw. Pictures arrive as raw Uint8Array with a lazy .url getter — a CD of 20 tracks with 500 KB covers doesn't eagerly create 10 MB of Blob URLs.

Markers and regions are stored in content-sample coordinates on a._.markers / a._.regions: a sample of the source, or of a sound an edit put in (ref, the inserted instance). A mark that falls in silence (a pad, a gap opened, a take written past the end) keeps to the sound nearest it and its distance from it (offset, in that sound's samples; a region's run past its sound, below / above). The instance getters project through the edit plan on read:

content marks ──→ buildPlan(a).segs ──→ plays(sample, ref, segs) ──→ output markers (+ offset / rate)

plays walks plan segments [from, count, to, rate, ref], through stages and the timeline a whole render keeps, and emits one output position, with its rate, per segment that plays the sample. This makes crop shift, remove drop, reverse flip, and speed/stretch scale — uniformly, without per-op plumbing, the distance into silence carried the same way.

On save, WAV/MP3/FLAC buffer the whole encoded output and splice meta back in; other formats stream. Pass meta: false to strip.