Categories: Advanced, Featured, PocketBeagle, PocketBeagle 2
The details of my project are on github: https://github.com/reesul/pb-pru-matrix
See an actual video of this running here:
Project Summary
The PocketBeagle (PB) has an AM335x processor for the processing. I capture audio with ADC input in chunks with a sliding window — sampling rate was set based on processing speed (unoptimized w/ numpy calls) and responsiveness. Audio is processed with FFT -> power log spectrum -> log-frequency power spectrum to make the basis for the image. The image is then an opencv color mapping for each (pseudo-log) frequency bin’s power, and send to the PRU via /dev/mem so it can be displayed to the 64×32 LED matrix. The PRU is running a driver to display 8-bit color images, FPS somewhere in 20-30 range.
I chose a pocket beagle, partly because I’ve worked as an Apps Engineer at TI, partly b I have met the awesome Beagle folks several times, and partly bc I wanted to try running on an SoC with fast IO core + reasonable A-core running Linux. It’s worked out well, and the examples were definitely present enough out there to get me started on all aspects of the project. That said, it helps a lot to have worked professionally with such devices to heterogeneous SoCs running a mixture of HLOS/RTOS/No-OS (esp. including Linux) and have a signal processing / embedded systems background.
Version 1 details
First version of this project is complete as of 2024 EoY. I started sometime in 2023, but didn’t put much time into it / slowed down by silly roadblocks.
This v1 includes:
- Custom cape (PCB that beagle board attaches to) that provides audio input and image output signalling to the matrix panel
- Revisions needed to reduce analog noise, reduce pinout/external connection, etc.
- PRU code that reads an image from shared memory, and manages output pins for the matrix such that we visualize an 8-bit, 32×64 pixel image.
- Framerate is 20-30 FPS and looks about reasonably smooth and responsive, albeit a bit jittery.
- Updated device tree overlay for setting ADC configuartion (20kS/s) (see dtb files in my repo)
- Data arrives through IIO driver and read with sysfs
- I noted that bb-overlay repo (origin of these DTS) has an ./install.sh that doesn’t work. I had to look in /boot/uEnv.txt to apply the DTBO file (name_overlays)
- Sampling rate was chosen based on processing speed vs. responsiveness. I wanted 40 kS/s for 20 kHz bandwidth, but this just wasn’t feasible. As a result, I have some aliasing in the upper bands that I’ll need to resolve later.
- A bunch of python code to do the audio capture, processing (FFTs and such), and image generation
- I signal to the PRU by opening /dev/mem to access the whole memory space and write to PRU’s shared SRAM. Images too big for rpmsg anyway. In theory though (thx Jason K. for the tip), I can just alloc a chunk of DDR and access that between both cores — should be fast enough, but maybe still need double-buffer.
- Buffer sizes, sampling rates, and processing code are tuned to be snappy — audio sync tests look visually acceptable (<200ms). Image processing at >30fps, unsure about display
- I have some filtering to make the frequency bands look smoother in both space (pixels) and time (frames). Could spend lots of time tuning.
- Add some power switches
- Check signal levels on home audio system and see if I need to reduce / amplify voltage. Almost certainty the former per Klipsch RP-600m, but must measure at realistic listening level
The fact that this works with Python code and limited optimizations shows that AM335x is still a plenty beefy processor. I’m at ~70% utilization of the 1x A8. That said, I really want a new revision of pocket beagle with a more modern (probably TI) processor, so long as there’s a PRU. I’ll be keeping an eye out for a new PB iteration 😉 But This would probably also mean a few extra components on the PCB for ADC
There’s a main runner script that will turn off the PRU, rebuild firmware, start PRU, apply sysfs settings for ADC capture, and kick off the audio processing + image generation
I have plans for a v2, so expect an update in the future 🙂 I’d like to add extra panels, clean up some audio ground noise, make the PCB more ‘user friendly’ (including enable line-out signal levels from AV receiver). If I can move to next generation of PB with higher powered SoC, I’d like to add more complex audio processing like beat detection
I have a bit of work to do as well for others who might like to implement / copy this project. Without replicating the PCB, hardware wiring is the ugliest part to get input ADC from audio and output 7x PRU pins and 6x GPIOs to the LED panel
Audio Spectrogram with PocketBeagle
Advanced, Featured, PocketBeagle, PocketBeagle 2
Comments are not currently available for this post.