Pitch Tracking with Probabalistic Yin

In this workshop video, we implement pitch tracking in digital audio from scratch! We first go through a classical algorithm called YIN, and then we show how to improve it using a Hidden Markov Model, upgrading it to a version called probabalistic YIN. Finally, we show some applications, including making singing bells, creating an autotuner like Cher, and changing a song so that it's the same note the whole time.
This is a long video, but it's worth it!

Jupyter notebook code here:
https://ctraliedotcom.github.io/pYIN/...
(longer notes coming soon!)

Table of contents:
00:00 Intro Sequence
00:16 Motivation
05:10 Repetitions in pitched waveforms
09:19 Autocorrelation concept
11:22 Frequencies, notes, octave errors
16:31 Mathematical definition of autocorrelation
20:04 Speeding up autocorrelation with the FFT
25:50 Framing
30:24 Basic YIN And Computation
40:02 Normalized YIN
42:45 Basic fundamental frequency system and sonification
50:12 Refinement with parabolic interpolation
57:44 Preparing all YIN estimates for probabalistic YIN
1:05:22 Introducing probabalistic YIN and HMMs
1:10:36 State space
1:15:17 Transition model
1:19:48 Unrolling state transitions over time
1:21:06 Unvoiced states
1:27:42 Observation model
1:32:13 Coding up probabalistic YIN!
2:00:02 Debugging probabalistic YIN code
2:03:31 Backtracing the optimal frequency trajectory
2:09:27 Probabalistic YIN results
2:14:58 Real time (causal) probabalistic YIN
2:19:20 Research notes
2:22:33 FM synthesis sonification
2:28:13 Making an autotuner!
2:39:19 Everything the same note (lol)
2:43:31 Outro



Special thank you to Brendan Sellers (mrbrendansellers on Instagram) for letting me use his vocals!

------------------
Errata
I had a slight typo in my parabolic interpolation. The a coefficient can be negative, so to prevent divide by 0, I should simply say
a[a == 0] = 1

------------------
References
[1] De Cheveigné, Alain, and Hideki Kawahara. "YIN, a fundamental frequency estimator for speech and music." The Journal of the Acoustical Society of America 111.4 (2002): 1917-1930.

[2] Mauch, Matthias, and Simon Dixon. "pYIN: A fundamental frequency estimator using probabilistic threshold distributions." 2014 ieee international conference on acoustics, speech and signal processing (icassp). IEEE, 2014.

[3] Georgieva, Elena, Pablo Ripollés, and Brian McFee. "The Changing Sound of Music: An Exploratory Corpus Study of Vocal Trends Over Time." Proceedings of the International Society for Music Information Retrieval Conference. International Society for Music Information Retrieval, 2024.

[4] Ellis, Daniel PW. "Beat tracking by dynamic programming." Journal of New Music Research 36.1 (2007): 51-60.