These sources ask how sound can be analyzed, compressed or generated. Some are Hacker News threads, described without their linked articles. This page lists nine sources, oldest first where dated.
Synthesizing Music from JSON
The post explains how pl_synth works and how the author made the JavaScript version faster. It shows code for the synthesis loop, the editor and song storage. Evidence is shown code and timing figures for the author's own project.
WildFX: A DAW-Powered Pipeline for In-the-Wild Audio FX Graph Modeling
WildFX is a Docker pipeline that drives the REAPER DAW with real plugins. The paper tests it by training a model to estimate effect graphs. The authors say the results fall short of the earlier work.
Eleven Music
A long Hacker News thread, mostly debate about AI music and art. A few commenters compare outputs with Suno. The evidence is opinion and anecdote, with some linked tracks but no measurements.
Neural audio codecs: how to get audio into LLMs
The thread discusses a Kyutai explainer on turning audio into tokens for language models. The explainer's author, posting as vvolhejn, replies often. Commenters debate MP3 as a tokenizer and why voice models seem to miss pitch.
How the cochlea computes (2024)
The thread discusses a 2024 blog article on whether the ear does a Fourier transform. Most replies argue over definitions and whether the title is clickbait. Others add facts about phase, neurons and speech.
How To Train a Generative Kick Drum Model on Your Old Linux Desktop With 6GB of VRAM
The post walks through three models trained to generate kick drums. It covers data cleaning, training choices, deployment and problems that came up. Evidence is the author's own project and a few audio samples.
Resonate
The page explains how banks of simple resonators compute spectra sample by sample. It lists two conference papers, a talk and code packages. Evidence is formulas, spectrogram comparisons and audio demos, not a benchmark table.
AudioX: A Unified Framework for Anything-to-Audio Generation
A project page from a university team. It carries the abstract, a list of tasks and many sample comparisons against other models. The samples are chosen by the authors, and they were not played here.
Toward a Sparse Interpretable Audio Codec
The page proposes encoding audio as a set of events with times, not fixed frames. It gives model details, a streaming method and five audio examples. The page itself says reconstruction quality is not yet good enough.
Connections to other subjects
These links are this wiki's own reading, not claims the sources make.
- WildFX: A DAW-Powered Pipeline for In-the-Wild Audio FX Graph Modeling and Logic Pro MCP Server for Claude, Cursor, and AI Agents, from Music as code and DAW control. This Logic Pro server returns confirmed, uncertain or failed, never a bare success. WildFX reports that headless REAPER gives no error reporting, so steps were checked manually. Both meet one problem: DAWs expect someone watching the screen. An agent needs a third answer besides yes and no.
Glossary
- Adversarial loss
- A training signal from a second model that tries to spot fake outputs.
- Basilar membrane
- The strip inside the cochlea whose different parts vibrate at different frequencies.
- Blind estimation
- Working out how audio was processed from the result alone.
- Classifier-free guidance
- A trick that exaggerates the pull of the text keywords during generation.
- Cochlea
- The fluid-filled spiral organ in the inner ear that senses sound.
- Codec
- Software that compresses audio for storage and rebuilds it later.
- Constant-Q transform
- A spectrum method whose frequency bins are spaced like musical notes.
- DAW
- Digital audio workstation, the software used to record and edit music.
- Demo scene
- A hobby community that makes tiny programs with striking graphics and music.
- Diffusion model
- A model that learns to remove noise step by step.
- Diffusion transformer
- A generative model that builds output by removing noise step by step.
- Directed acyclic graph
- A set of nodes and one-way links with no loops.
- Docker
- A tool that packages software and its dependencies into an isolated container.
- Eleven Music
- An AI service from ElevenLabs that generates songs from text prompts.
- Envelope
- A shape describing how a sound's volume rises and falls over time.
- EWMA
- An exponentially weighted moving average, which weights recent values more than old ones.
- Fair use
- A legal defense that lets some uses of copyrighted work go without permission.
- FFT
- The fast Fourier transform, the standard way to compute a spectrum.
- Foley
- Sound effects made to match action on screen.
- Formant
- A band of frequencies that a vocal tract strengthens when making vowels.
- Fourier transform
- A math tool that splits a signal into pure sine waves of different frequencies.
- Gabor transform
- A Fourier-style analysis that uses a smooth window to keep timing information.
- ICLR
- A major yearly machine learning research conference.
- Impulse response
- The recorded way a room or object rings after a short sound.
- Inception Score
- A number that rates the quality and variety of generated samples.
- Inpainting
- Filling in a missing part of a recording.
- Latent space
- A compressed representation in which each point stands for a possible sound.
- LFO
- A low frequency oscillator, used to wobble another sound setting.
- Lookup table
- A precomputed list of values read instead of calculating each time.
- Mel spectrogram
- A picture of sound energy over time, with frequencies spaced like human hearing.
- Moshi
- A speech-based language model built by Kyutai.
- Neural audio codec
- A learned model that compresses audio into a small stream of numbers or tokens.
- Opus
- A common audio codec for voice calls and streaming.
- Oscillator
- A generator of a repeating wave such as a sine or square.
- Phase
- Where a wave is in its cycle at a given moment.
- Phasor
- A rotating complex number that tracks the phase of a wave.
- Plugin
- An add-on program that applies an effect or makes sound inside a DAW.
- REAPER
- A commercial DAW that can be scripted.
- Residual vector quantization
- Coding each slice of audio with several small dictionaries, one correcting the previous.
- Resonator
- A simple filter that responds strongly to input near one chosen frequency.
- Short-time Fourier transform
- Fourier analysis applied to short slices of a signal in turn.
- Sidechain
- A routing where one track's signal controls an effect on another track.
- Sparse
- Using only a few active elements, with the rest left empty.
- Spectrogram
- A picture of how sound energy is spread over frequency and time.
- Stems
- Separate recordings of single parts of a song, such as drums or vocals.
- Suno
- A rival AI service that generates full songs from prompts.
- T-SNE
- A method that flattens many-dimensional data into a plot while keeping neighbors close.
- Token
- A discrete unit that a language model reads and writes.
- Tracker
- A music editor where notes are entered in rows of patterns.
- U-Net
- A neural network shaped like a U, used here to remove noise.
- VAE
- A variational autoencoder, a model that compresses data into a small code and rebuilds it.
- Vocoder
- A model that turns a spectrogram back into an audio waveform.
- VRAM
- The memory on a graphics card.
- WASM
- WebAssembly, a compact code format that browsers run at near native speed.
- Wavelet transform
- A way to analyze a signal using short wave shapes of varying width.