cochlear

Joined 1 year ago

Karma
71
Submissions
21
Member since
Dec 7
I'm interested in sparse, interpretable and easy-to-manipulate representations of audio.

https://blog.cochlea.xyz/sparse-interpretable-audio-codec-paper.html

View on Hacker News

Recent submissions

Comment · 2 months ago
Funny, I guess this idea is in the air (har har).

I've made a couple things along these lines over the past year. One is a fun, Halloween-themed toy:

https://blog.cochlea.xyz/theremin.html

The other is a (slightly) more serious exploration of multivariate time signals as a control signal for an instrument. In the latter case, the instrument is learned/inferred from a segment of input audio:

https://blog.cochlea.xyz/resonancemodel.html

This is a really neat idea! Like others here, I'm not a huge fan of the hyper-dense and dripping-with-ai UI, though.

Comment · 1 year ago
Comment · 1 year ago
How funny, I actually corresponded with one of the authors of the "Spiking Music..." paper when it first showed up on arxiv. I'll definitely give the amp-modeling paper a read, looks to be right up my alley!

Now that I understand the basics of how this works, I'd like to use a (much) more efficient version of the simulation as an infinite-dataset generator and try to learn a neural operator, or NERF like model that, given a spring mesh configuration, a sparse control signal, and a time, can produce an approximation of the simulation in a parallel and sample-rate-independent manner. This also (maybe) opens the door to spatial audio, such that you could approximate sound-pressure levels at a particular point in time _and_ space. At this point, I'm just dreaming out-loud a bit.

Comment · 1 year ago
Thanks for the bug report, I'll look into this and see if I can make it better! There's definitely more optimization juice to be squeezed, I think, and it'd probably be smart to allow the number of nodes in the simulation to be adjusted.

Working with AudioWorklets (https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkle...) has been really cool, and I've been surprised at what's possible, but I _haven't_ yet figured out how to get good feedback about when the custom processor node is "falling behind" in terms of not delivering the next buffer quickly enough.

Comment · 1 year ago
Same here! Not a physics engine per se, but I've been eyeing Taichi Lang (https://github.com/taichi-dev/taichi) as a potential next stop for running this on a much larger scale.

My assumption has been that any physics engine that does soft-body physics would work in this regard, just at a much higher sampling rate than one would normally use in a gaming scenario. This simulation is actually only running at 22050hz, rather than today's standard 44100hz sampling rate.

Comment · 1 year ago
Thanks for the bug report, I definitely do see now that things fall apart at certain extreme values. I'll look into it and see if I can fix. My assumption is that it's some sort of numeric overflow or underflow resulting in NaN values, and that I'll probably just need to set the slider boundaries more carefully. Thanks again!
Comment · 1 year ago
These are both great points and I'll use them to refine my writing on the subject, I appreciate the feedback!

Apologies if it isn't clear, but the animated gifs are meant as an illustration of the iterative encoding process, where the encoder decomposes the signal step-by-step, as in matching pursuit. I'll be sure to clarify that point.

I'll add a paragraph on compression rates/ratios, although that isn't necessarily the main focus here; codecs may compress a signal, but they might also transform it into a more useful, easy-to-understand and easy-to-manipulate representation.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton