Cochleagram

Tools for psychoacoustics

Installation

1. git clone [email protected]:CBMM/cochleagram && cd cochleagram
2. Install Haskell
3. Install ghcjs
4. git submodule update --init --recursive
5. ./init-sandbox.sh
6. cabal configure --ghcjs
7. cabal install --only-dep
8. cabal install

Running

cochleagram uses your computer's microphone, so the browser protects you by making sure cochleagram only runs over a secured connection. This makes it a little bit tough to test your changes cochleagram on your local computer - you need an SSL-enabled server.

http-server is a quick way to do this.

You'll need to make a self-signed certificate.

Then point your server to the cert and key files, from the cochleagram compiler output directory:

cd dist/build/cochleagram/cochleagram.jsexe/
http-server -S -K ~/key.pem -C ~/cert.pem

At this point you can reach the files via the browser at https://localhost:8080

If this gives you trouble please feel free to get in touch.

Brainstorming - Hack with us!

Teaching & demos

• Show basic principles (filters, power spectrum, spectrograms, cochleagrams)

• Use data from computer mic (real time visualization)

• Use data from file on the web (zoomable visualization)

• Tune parameters

• Demos usable after the talk is over

Researchy demos

• Template matching in spectrogram

• Auditory scene / texture / cepstra analysis

• Sound generation & playback from scene statistics

• Stereo source localization demos (too ambitious)?

Goals, Themes

• Separate the sound exploration from grungy coding details

• Abstract the most common tasks into nice libraries

• Push science findings toward demo-level teaching tools

Related work & Inspiration

• Web Audio Spectrogram RT spectrogram & distortion demo

• Web Audio Demos Many demos of the web audio api

• Web Audio API Mozilla Dev Network. Browser-based audio analysis & production.

• VisPy Luke Campagnola (Paul Manis Lab, Allen Inst.) RT spectrograms & general data vis

• MathBox Web math visualization

• Pandoc Generate slides from markdown

• Explorable Explanations

• Neurons Nicky Case

• Parable of the Polygons Nicky Case

Brainstorming session with McDermott lab

• Demo walking through mechanism of generating cochleagram and connection to auditory system. Show traveling wave in basalar membrane. (Sam)

• How hard to specify (how many dimensions) (how many sliders) for parametizing cochleagram.

• Export figures

• Javascript NN?

• Other ways of visualizing sound than spectrograms?

• Code box to change the signal?

• Online training in some way or another (reverse correlation?)

• Provide filters to the thing.

• Sliding correlation matrix.