Visuals

By default, no visualizer is attached to the tracker. It simply prints dots to stdout when a beat occurs. You may override this behavior by specifying the name of a handler after beat tracking arguments:

python -m beatviewer <Beat Tracking Args> <Handler Name> <Handler Args>

Possible handler names are beep, fireworks, fluid, galaxy, rectangles, socket, tunnel, warp and waves.

Contents

• Beep On Beat
• Warp
• Pygame Visuals
• Socket
• Web Visuals
• OBS script

Beep On Beat

Handler name: beep

Make sounds when beats and onsets are detected. If you are on Windows, it uses the winsound library. If you are on Linux, it uses and external package called beep.

beatviewer beep [-h] [--beats-only] [--onsets-only]

Argument	Default	Description
beats-only	False	Only beep when beats occur
onsets-only	False	Only beep when onsets occur

Warp

Handler name: warp

This visualisation synchronizes a video by warping time so that checkpoints in the video (visual beats) correspond to audio beats. The checkpoints can be set automatically using a preprocessing step (see Directogram Computation), or manually (see Video Annotator). Once set, you simply must keep the video file and the accompanying JSON file in the same folder, and call beatviewer:

python -m beatviewer warp [-h] [--rewind REWIND] [--timing TIMING] [--before BEFORE] [--after AFTER] [--margin MARGIN] [--size SIZE] [--virtual-cam] [--prev-threshold PREV_THRESHOLD] [--next-threshold NEXT_THRESHOLD] [--jumpcut] video_path

Argument	Default	Description
video_path		Path to the video file
rewind	0	Probability of going back in time at each checkpoint
timing	linear	Timing function and arguments, see below
before	120	Number of frames stored in the buffer before its cursor
after	120	Number of frames stored in the buffer after its cursor
margin	120	Number of frames above the default buffer size to trigger the cleanup
size	900	Length of the largest video edge
virtual-cam	False	Send video output to a virtual camera instead of a window
prev-threshold	0.1	Duration (in seconds) above which the timing is updated (see below). Increase to reduce flickering.
next-threshold	0.1	Duration (in seconds) above which the timing is updated (see below). Increase to reduce flickering.
jumpcut	False	Jump to a random checkpoint when a beat occurs instead of warped playback

Timing functions. Time warping between checkpoints is, by default, linear. However, it often looks better to emphazise the visual beats, so several warping easing functions are available. Possible choices are linear, power or bezier. The last two require arguments, e.g. --easing "power(1.5)" or --easing "bezier(0.3,0,0.7,1)".

Function	Argument	Behavior
linear	-	Linearly interpolates between previous and next checkpoints
power	float	If t is the progression between the previous and next checkpoints, it gets raised to the specified power. A power of 1 means linear interpolation. A power greater than 1 starts slowly and accelerates. A power greater than 1 starts fast and decelerates.
bezier	x1, y1, x2, y2	Uses a cubic Bezier interpolation, given two control points. Behaves as CSS bezier interpolation.
ease-in	-	Same as bezier(0.42,0,1,1)
ease-out	-	Same as bezier(0,0,0.58,1)
ease-weak	-	Same as bezier(0.3,0,0.7,1)
ease-medium	-	Same as bezier(0.42,0,0.58,1)
ease-strong	-	Same as bezier(0.75,0,0.25,1)

Beat timing distance thresholds. Warping is achieved by interpolating a playback cursor between the previous and the next beat instant given a previous and a next video checkpoints. The next beat instant is predicted as the previous beat instant plus the estimated beat period, i.e. the BPM. When a new beat occurs, one of the two instants is updated: the (temporally) closest one is chosen. The temporal distance threshold is used to prevent any update when the detected beat occurs relatively close to the predicted one. Larger threshold reduces flickering, as the program will wait for the lag to get large before updating.

It also works with GIF files. Here are some that you might want to try:

Filename	Checkpoints
alien.gif	0 9 18 27 36 45 54 63 72 81 90
catjam.gif	5 19 31 44 56 68 79 90 103 115 128 140 152
charlie-brown.gif	4 12
kirua.gif	2 8
mickey.gif	5 13
steps.gif	2 16
walk.gif	13 27
zoom.gif	15

JSON File Format. You can manually write the JSON file that goes with the video, to manually specify the checkpoints. Here is a sample file:

{
    "path": "catjam.gif",
    "frame_count": 157,
    "frame_rate": 24.0,
    "width": 640,
    "height": 640,
    "checkpoints": [5, 17, 29, 42, 54, 66, 77, 89, 101, 113, 126, 138, 150]
}

Pygame Visuals

The following visuals rely on Pygame 2D drawing features.

Tip: Hide Pygame support prompt

You may disable Pygame's support prompt by setting the environment variable PYGAME_HIDE_SUPPORT_PROMPT to 1.

Galaxy

Handler name: galaxy

This draws a number of dots, slowly coming towards the screen, that are pushed around when a beat occurs.

beatviewer galaxy [-h] [--fps FPS] [--width WIDTH] [--height HEIGHT] [--fullscreen] [--n N] [--bw] [--z0 Z0] [--dz DZ] [--dxy DXY] [--vmax VMAX] [--blur BLUR]

Argument	Default	Description
fps	60	Framerate
width	1280	Window width
height	720	Window height
fullscreen	False	Make the window fullscreen
n	200	Number of dots
z0	0.1	Z-index (between 0 and 1) of the camera plane. Lower values increase the FOV.
dz	0.015	Dot speed along the Z axis.
dxy	0.02	Acceleration amplitude for the XY axes. Greater values will make beat impacts stronger.
vmax	0.05	Dot speed limit on the X and Y axis. Both range from 0 to 1.
bw	False	Disable colors
blur	2	Dot blurring factor. Dots get sharper as the value increases.
random-dz	0	Z-axis speed deviation probability
base-radius	1	Dot radius

Rectangles

Handler name: rectangles

This draws rectangles when beats occur, along with color changes.

beatviewer rectangles [--fps FPS] [--width WIDTH] [--height HEIGHT] [--fullscreen] [--palette {bw,rgb,dark,chromakey,hippy,contrast}] [--scale-curve SCALE_CURVE] [--duration DURATION] [--beat-breadth BEAT_BREADTH] [--onset-breadth ONSET_BREADTH] [--onsets]

Argument	Default	Description
fps	60	Framerate
width	1280	Window width
height	720	Window height
fullscreen	False	Make the window fullscreen
palette	dark	Color palette, possible values are bw, rgb, dark, hippy, chromakey and contrast
scale-curve	0.3	Power of non-linear scale progression. Values smaller than 1 produce an ease-out effect.
duration	5	How long a rectangle takes to go off screen, in seconds
beat-breadth	0.97	Relative size of inner beat rectangles. Smaller values create thicker rectangles.
onset-breadth	0.99	Relative size of inner onset rectangles. Smaller values create thicker rectangles.
onsets	False	Also handle onsets

Tunnel

Handler name: tunnel

This draws a tunnel that comes closer to the screen in sync with the beat.

beatviewer tunnel [-h] [--width WIDTH] [--height HEIGHT] [--n N] [--z0 Z0]

Argument	Default	Description
fps	60	Framerate
width	1280	Window width
height	720	Window height
fullscreen	False	Make the window fullscreen
n	20	Number of tunnel parts
z0	0.05	Z-index (between 0 and 1) of the camera plane. Lower values increase FOV.

Waves

Handler name: waves

This draws waves propagating through ropes along with the occuring beats.

beatviewer waves [-h] [--fps FPS] [--width WIDTH] [--height HEIGHT] [--fullscreen] [--line-count LINE_COUNT] [--line-offset LINE_OFFSET] [--dot-count DOT_COUNT] [--dot-offset DOT_OFFSET] [--amplitude AMPLITUDE] [--sigma SIGMA] [--offset OFFSET] [--abs-distance] [--buffer-size BUFFER_SIZE]

Argument	Default	Description
fps	60	Framerate
width	1280	Window width
height	720	Window height
fullscreen	False	Make the window fullscreen
line-count	20	Number of lines (or ropes)
line-offset	0.001	Time differnce between two lines, in seconds
dot-count	50	Number of dots per line
dot-offset	0.0007	Time difference between two dots, in seconds
amplitude	100	Gaussian perturbation amplitude, in pixels
sigma	0.1	Gaussian perturbation width
offset	0	Static perturbation offset, in seconds
abs-distance	False	Use absolute distance instead of Euclidean norm for spacing lines and dots
buffer-size	20	Gaussian buffer size

Socket

Handler name: socket

This handler acts as a websocket server. Any connected socket will receive a message whenever a beat, onset or BPM change event occurs. Messages are simple strings: BEAT, ONSET and BPM <float>.

beatviewer socket [-h] [--web] [--host HOST] [--port PORT] [--mute-beats] [--mute-onsets] [--mute-bpm]

Argument	Default	Description
web	False	Use the WebSockets protocol (for the web simulations)
host	localhost	Server host
port	8765	Server port
mute-beats	False	Do not send BEAT messages
mute-onsets	False	Do not send ONSET messages
mute-bpm	False	Do not send BPM messages

Regular socket server is used to communicate with the OBS script.

Web Visuals

Web visuals are HTML pages with a JavaScript code that acts as a socket client. Under the hood, the program uses the socket handler, and then starts a basic HTTP server to serve an HTML file (those files can be seen in the web folder).

You could simply start beatviewer with the socket handler and open the HTML files within your browser, it would do the same. Also, it should be possible (untested at this day) to make this work remotely.

Fireworks

Handler name: fireworks

Each beat triggers a new firework, with a random color and a random number of branches.

Parameter	Default	Description
uri	ws://localhost:8765	Socket server URI
p	0.5	Geometric law power for generating the number of branches for a firework.
palette	green	Color palette. Possible values are green, colors, dark.
line_width	40	Line width.

Fluid Simulation

Handler name: fluid

Each beat adds smoke particules with a random velocity inside a Eulerian fluid simulator, adapted from the work of Matthias Müller for Ten Minute Physics.

Some GET parameters can be changed:

Parameter	Default	Description
uri	ws://localhost:8765	Socket server URI
velocity	20.0	Maximum air flow velocity when opening the valve.
palette	red	Smoke particules color palette. Possible values are red, green, blue, dark and hippy.
iters	40	Number of simulation iteration for each frame
dt	0.01667	Simulation time step
over_relaxation	1.9	Simulation over relaxation (see Müller video)

OBS script

This repository contains an OBS script called BeatViewer, that communicates with the regular socket visualizer.

Installation

If this is the first time you are using a Python OBS script, you will have to set Python path for OBS, and add the script to OBS:

1. In OBS, go to Tools > Scripts
2. Go to the Python params tab
3. Set Python path to the folder containing Python's executable
4. In OBS, go to Tools > Scripts
5. Click on the plus sign, and open obs_beatviewer.py

Usage

1. Start beatviewer with the socket handler (--web flag should be off, here we use regular sockets)
2. Open OBS
3. Go to Tools > Scripts, find the obs_beatviewer.py script, and under the Socket Server option panel, click the Connect button. Now OBS should be connected to the socket server, which should have logged this
4. Check one or more of the available visuals

If you provide a source for the Debugging Text Source option, the provided Text Source will be used to display current tracker status, such as connection state and last BPM value.

Blinking Slideshow

Change the source path of an Image Source when a beat occurs and briefly toggle source visibility.

Option	Default	Description
Source name	BlinkingSlideshow	The name of the Image Source which the source will be changed
Images Folder	—	Path to a folder containing the images to display (ordered by name)
Blinking Duration	100	Duration of a blink in milliseconds (absolute duration) or in percent of beat duration (relative duration)
Duration Relative to BPM	False	Switch between absolute and relative duration
Random Index Jumps	False	Go through images in a random order
Automatic Scaling	True	Automatically scale the source when a image is loaded

fftools

Dedicated assets must be prepared for this effect to work. You might want to look at the Average or Shots tools from my set of FFmpeg Tools for this: you can either split a video into multiple shots, or create long-exposure type images from videos to decompose movements.

Seek On Beat

Automatically seeks within a media source when a beat occurs, causing a jump cut effect.

Option	Default	Description
Source name	SeekOnBeat	The name of the Media Source which the source will be changed