Colin Calnan  /  Builds  /  Music and AI

Gig Splitter

I record every gig on a phone clipped to the speaker stand. One command turns that two-hour video into a file per song, named by what I actually played, and a 30-second chorus reel for each one.

Status
Running. Used on eight gigs so far
Built
May 2026. The first working version in one night
Stack
Python, librosa, ffmpeg, faster-whisper, forced alignment, Claude
How
Built in Claude Code, every commit
In
One portrait video of a whole gig, 1080 by 1920
Out
A file per song, a reel per song, captions from the real lyrics, a title card
A reel it cut: Dancing in the Dark, Predator Ridge, May 2026. It picked the window and burned the captions in. I didn't edit it.
Three frames from cut reels, each from a different gig: a covered winery patio, a barn door open onto a vineyard, and a sunlit courtyard. Same setup every time, one phone on a stand.
Three gigs, one setup: a phone on the speaker stand, recording the whole set. The input costs nothing, because it happens every gig anyway.

How it works

Two hours in, a folder of reels out.

Run as python split_gig.py ~/Movies/Gigs/<gig>/IMG_xxxx.MOV. Every step skips work already on disk, so a rerun picks up where it stopped.

  1. Find the songs by how loud they are

    ffmpeg pulls the audio out and librosa measures its loudness, smoothed over 15 seconds so a quiet verse doesn't split a song in half. Anything above a threshold for more than 90 seconds is a song. Anything below is me talking.

    The energy analysis for one gig: a jagged line of loudness across 120 minutes, a red dashed threshold across the middle, and 23 shaded blue bands, one per detected song, with the quiet gaps between them left white.
    One gig, 120 minutes. Each shaded band is a song it found. The white gaps are me talking between them.
  2. Cut each song out without re-encoding

    ffmpeg stream-copies each band to its own file, so the cut is fast and nothing loses quality.

  3. Name the song from garbled lyrics

    Whisper transcribes a minute of each song, and Claude names it from the transcript, knowing the words will be wrong but the phrases close. If the minute turns out to be banter, it tries later in the song.

    Predator Ridge, reels folderreal output, one bug included
    song_02_Van_Morrison_-_Bright_Side_of_the_Road_reel.mp4
    song_03_The_Beatles_-_All_You_Need_Is_Love_reel.mp4
    song_04_Creedence_Clearwater_Revival_-_Have_You_Ever_Seen_the_Rain_reel.mp4
    song_07_Sting_-_Englishman_in_New_York_reel.mp4
    song_09_Van_Morrison_-_Brown_Eyed_Girl_reel.mp4
    song_10_Bruce_Springsteen_-_Dancing_in_the_Dark_reel.mp4
    song_11_Hall_Oates_-_Maneater_reel.mp4
    song_13_Me_by_Ben_E_King_-_Stand_reel.mp4
    song_21_ACDC_-_Highway_to_Hell_reel.mp4
    song_26_ABBA_-_Dancing_Queen_reel.mp4

    Ten of the 23 reels from 8 May 2026. Song 13 is "Stand By Me", and it's explained under "Fixed since".

  4. Find the chorus

    A second Whisper pass gets word timings for 90 seconds of the song. The reel starts two seconds before the phrase that repeats most, which is almost always the chorus, the bit people know. No repeat, and it starts a quarter of the way in.

    split_gig.pyfind_chorus()
    def find_chorus(transcript: str) -> str:
        words = transcript.lower().split()
        best_phrase, best_score = "", 0
        for n in range(8, 3, -1):
            seen = {}
            for i in range(len(words) - n):
                phrase = " ".join(words[i:i+n])
                seen[phrase] = seen.get(phrase, 0) + 1
            for phrase, count in seen.items():
                score = count * len(phrase)
                if count > 1 and score > best_score:
                    best_phrase, best_score = phrase, score
        return best_phrase
  5. Cut the reel

    30 seconds at 1080 by 1920, karaoke captions burned in from a subtitle file, and the captions saved as editable JSON next to it so a wrong word can be fixed and re-burned without rerunning anything.

    A folder listing of the output: song_02_Bryan_Adams_-_Summer_of_69_reel.mp4, song_04_Creedence_Clearwater_Revival_-_Have_You_Ever_Seen_the_Rain_reel.mp4 and so on, each mp4 paired with its own captions json file.
    The output folder from another gig. Every reel has its captions beside it, and the folder reads as the set list.

How AI was used

Built with Claude, and runs on it too.

  • Built in Claude Code

    47 of its 48 commits carry Claude as co-author, from the first one on 9 May, the night it was built. It came before I packaged the method as my spec-driven development skill, so there's no specs folder behind this one.

  • A model that listens

    Whisper transcribes a minute of each song to identify it, and a 90-second window with word timings to find the chorus. A forced aligner then places the real lyrics against the audio.

  • A model that names the song

    Claude Sonnet gets the garbled lyrics, is told the words will be wrong but the phrases close, and answers in one fixed format. It turned out it had been right all along about "Stand By Me". My parser wasn't.

  • Plain code where plain code is enough

    Finding the songs is loudness and a threshold. Finding the chorus is counting repeated phrases. No model in either, so they're free, fast and predictable.

  • What stayed mine

    Playing the gig, picking which reels are worth posting, and the look of the captions, which is matched to the ones I do by hand in InShot.


The hard parts

What fought back.

  • Shazam can't recognise a live cover

    The first version used it, and it failed on every song, because it matches studio recordings and a live acoustic cover isn't one. It was gone the same night. Reading the lyrics and letting Claude name the song is the decision the whole tool rests on.

  • I had the loudness backwards

    My first guess was that the crowd chatting between songs would be louder than me. It was the other way round, so songs are the loud parts.

  • A truncated cache hid half the gig

    A cut-short audio cache meant only 15 of 28 songs were found on the first real run. It now checks the cached audio's length against the video before trusting it.

  • Knowing which job to hand back

    From late May I ran it with captions off and captioned in InShot, because the transcribed words were wrong too often to post. The captions only came back once they were built on the real lyrics, below.


Fixed since

The words on screen are the real words now.

The first version captioned each reel with whatever Whisper heard, and a live cover in a noisy room is hard to hear. In September I went back to it, measured what was actually wrong, and fixed the parts that numbers said were worth fixing.

  • Transcription was already at its ceiling

    Against lyrics I typed by hand for three songs, every model I tried landed between 21% and 27% of words wrong, close to the published state of the art for sung lyrics. Asking Claude to correct the transcript made it worse, 25% to 41%, because on "Maneater" it wrote the real chorus instead of the verse I'd actually sung. So I stopped trying to hear the words better.

  • The words come from the lyrics instead

    Claude has already named the song, so the pipeline looks the lyrics up in LRCLIB, a free open lyrics database that had 66 of the 68 songs in my set. A forced aligner then places each word against the audio, and lands within about a tenth of a second of where the transcriber independently heard it. On "Maneater" the transcript said "by the power of a JQR". The reel now says "by the purr of a Jaguar".

  • Two guards, because an aligner can't say no

    Hand an aligner any words and any audio and it will place them. So a caption has to pass two checks before it's drawn. Wrong song is caught by how many words it shares with the transcript: right songs scored 0.59 to 0.92, wrong songs 0.04 to 0.09. Right song but the wrong verse is caught by timing: the right passage drifts 0.05 to 0.15 seconds from the transcript, the wrong one 6 to 18. If either check fails, the reel falls back to the honest transcript. A caption that's a quarter wrong beats a verse that reads perfectly and was never sung.

  • Lines break where the song breaks

    One caption read "door So many have", the end of one sung line running into the start of the next. Every word was right and it still looked like a mistake. Captions now follow the lyric's own line breaks, and long lines split into even halves.

  • A title card

    The song and the venue sit on screen for the first five seconds, taken from the gig folder's name, the one place I typed them myself. It shows the venue but not the date, because a date makes a reel look stale the day it's posted late. If the folder name has a typo it refuses to guess, since a wrong title burned into a posted video can't be edited.

  • "Stand By Me", finally

    Claude replies as "Song Title" by Artist, and the parser split on the first " by ". So "Stand By Me" by Ben E. King came out as Me_by_Ben_E_King_-_Stand, at four gigs. Claude had answered correctly every time. The parser now trusts the quotes, and the four files are renamed.

  • Nine times slower than it needed to be

    The script ran itself at background priority, with a comment saying that cost "a few percent". Measured on the same sample it was 18.5 seconds at normal priority and 172.7 at background, nine times slower. Separately, the blurred background behind each reel was being computed at full size. Blurring a thumbnail and scaling it up looks the same, and took a thirty-second reel from 78.5 seconds to 21.3. Transcription moved to faster-whisper and now runs four songs at once.

Word error rates, overlap and drift figures, and timings from the project's measurement tools and commit history, September 2026.


Still gets wrong

Where it breaks.

  • One threshold doesn't fit every room

    At Cadence in August the default threshold welded songs together into one 9 minute 47 second segment. I fixed it by trying other values and splitting two songs by hand, and got one of those splits wrong.

  • No model looks at the picture

    The models only hear and read. The reel window comes from the lyrics and the framing is a fixed ffmpeg filter. Letting a model choose the best frame for each song is the next layer.

  • A filter that does nothing

    Each reel gets a blurred copy of the video behind it, to fill the frame. My footage is already portrait, so the blur never shows.


Runs

Eight gigs so far.

GigWhat happened
8 May 2026Predator Ridge. 28 songs found, 23 reels cut. The truncated-cache bug turned up here.
May 2026Britannia Brewing.
29 MayEx Nihilo. The first run with captions switched off.
7 JunFolktale Fine Cider. "Stand By Me" again.
22 JunEx Nihilo. 26 of 26 songs found.
19 AugCadence. The threshold failed and two songs were split by hand.
SeptemberTwo more gigs, including Intrigue. 20 of 20 songs there.
23Reels cut from one two-hour recording, Predator Ridge in May.
8Gigs run through it so far.
26 of 26Songs found at Ex Nihilo in June.

Runs from the output folders and the project history. The August runs took about 75 minutes for 26 reels.

  • Python
  • librosa
  • ffmpeg
  • Whisper
  • Claude Sonnet
  • Gradio
  • Docker