Harmonic Polarity:
A New View of Relationships Between Sounds
— A "Mathematical" Idea

The phenomenon in which multiple frequencies, especially those aligned in a harmonic series, are integrated and perceived as a single pitch at the fundamental frequency is apparently called harmonic fusion.

Even when a sound can be regarded as several sine waves at different pitches sounding together, we hear it at the pitch of the fundamental, while also perceiving differences that we describe as differences in "timbre."

I suspect that this relates not only to the mechanism of timbre, but also to musical phenomena more generally. Musical keys involve a tonic or a central note, and chord theory includes the idea of function. Since function changes with the key, it is a tonal concept. Could these ideas be connected with perceptual processes that integrate multiple frequencies, such as harmonic fusion?

So I began with this question: given a frequency spectrum, could we calculate something like "how much harmonic fusion is taking place" at each frequency, and obtain a spectrum representing the degree of harmonic fusion?

Let $P\left(f\right)$ be a real-valued intensity function of the frequency spectrum. I mainly use the squared magnitude of the Fourier transform of the audio waveform.

When modeling harmonic fusion, it seems natural to consider the following:

This seems a natural starting point.

But I also want to evaluate the inverse relationship:

Whereas a harmonic has a frequency $n$ times $f_0$, what is called a subharmonic has a frequency $\frac{1}{n}$ times $f_0$. This is the inverse of the former relationship: frequency $f_0$ and its subharmonic $\frac{f_0}{n}$ satisfy

Thus, $f_0$ is the $n$th harmonic of $\frac{f_0}{n}$. When this relationship exists, my design gives $\frac{f_0}{n}$ a higher evaluation as a fundamental, and subtracts from $f_0$ the contribution associated with $f_0$ being a harmonic of $\frac{f_0}{n}$.

More specifically:

This gives the following expression:

I call this "harmonic polarity." In theory, $N=\infty$ is also possible when the series converges; for computation, I use $N$ because the sum is truncated at a finite upper limit.

I chose the word "polarity" because this function assigns a positive or negative value to each frequency:

That is the interpretation. I picture an "attraction" between frequencies. It feels a little like a Coulomb's law of sound.

Higher harmonics usually attenuate in harmonic fusion. I therefore use weights $w_n$ such as $\frac{1}{n}$ or $\frac{1}{n^2}$, mainly $\frac{1}{n^2}$. A sawtooth wave is a representative waveform containing all harmonics, and its amplitudes decay as $\frac{1}{n}$. So if $P$ uses squared magnitude, I think $\frac{1}{n^2}$ is an appropriate choice.

Also, because this sum starts at $n=2$, $Q$ does not include the spectrum's own value at frequency $f$. It is a function that evaluates relationships with other frequencies alone.

I will also assign symbols to the sums on the harmonic and subharmonic sides, respectively:

The symbols H and S come from "harmonics" and "subharmonics."

With this notation, we can write

as the difference between the two sums.

I obtained a MIDI file of Bach's C major Prelude from Mutopia and created an audio recording rendered with sine waves.

First, here is the spectrum of this recording:

This is $P\left(f\right)$, chosen here as the squared magnitude of the audio's Fourier transform: the distribution of frequencies actually present in the sound. To calculate a spectrum for the whole recording, I average the STFT (short-time Fourier transform, a method for finding a spectrum at each moment) over the entire duration. Note that the frequency axis is logarithmic.

Calculating its harmonic polarity $Q\left(f\right)$ gives:

The values now extend over a wider pitch range than the original spectrum. Can you see that there are more positive values on the left, in the low-frequency region, and more negative values on the right, in the middle and higher regions? This comes from the nature of the calculation of $Q$ itself, and the same general tendency occurs with other recordings.

It is easier to see why by considering a single pure tone. If there is a value $v$ at frequency $f_0$, then

hold. This happens at each harmonic order, producing the harmonic-polarity graph below.

Positive values appear on the left and negative values on the right. In ordinary audio, these contributions overlap to produce graphs like the one above.

As mentioned earlier, $Q\left(f\right)$ does not include the value at frequency $f$ itself; it evaluates only harmonic relationships, namely the presence of harmonics and subharmonics. Consequently, as we have seen, its values extend into regions where no sound frequencies are actually present. It is a virtual evaluation of relationships alone. To evaluate harmonic fusion in a fuller sense, we should also include the strength of the fundamental itself. For example, define

and then use

with normalization to bring the scale into line with $P$. Here I call this the "harmonic-fusion evaluation" function. For the recording above, its distribution is as follows.

One point deserves attention: what I am doing here is somewhat unusual and differs from ordinary harmonic-fusion evaluation. Such evaluation is normally used, for example, to estimate the fundamental frequency of an instrument's sound.

Here, however, I apply it to a time-averaged frequency spectrum that represents an aggregation of the sounds used throughout a piece. I am examining which frequencies can be evaluated as fundamentals, and to what extent, when we consider harmonic-fusion relationships within that collection of sounds.

With that interpretation in mind, under the current analysis settings the largest value of $Q\left(f\right)$ in this piece appears at $f\approx 48.83\text{Hz}$. In musical terms, this corresponds to G (near G1).

A musical key does not depend on pitch register. This way of treating sounds arises when we adopt octave equivalence, assigning the same note name to pitches an octave apart. To evaluate relationships in this direction, let us aggregate the spectrum across octaves.

For frequencies within one octave, we calculate the average, or simply the sum, of the values at all octave-related frequencies and arrange the results within that octave. Since an octave is a factor of two, the entire chain associated with a frequency $f$ is

and we refer to the values at these frequencies for all integer k. I call the resulting distribution, with note names on the horizontal axis, a pitch-class distribution or pitch-class spectrum. Pitch classes need not be restricted to 12-tone equal temperament; the calculation can be performed continuously over an octave.

For the recording above, aggregating the frequency-spectrum values $P\left(f\right)$ gives:

I find it interesting that the characteristics of C major are already visible (C D E F G A B). The values are roughly proportional to how often each note is used in the piece. This suggests that C really does occur frequently here. However, simply using a piece's tonic frequently is not necessarily enough to determine its key. Another feature is that G is the most prominent after C, while A and B seem to be used less than the other notes of C major.

Now let us perform the octave aggregation for harmonic polarity $Q\left(f\right)$. The result is:

I think this is even more interesting to observe. First, the largest value has shifted to F. This is probably because F's third harmonic corresponds to C, so the many occurrences of C contribute to it. F major is also the subdominant of C major, which may have contributed as well.

Some pitch classes have positive values and others negative values. Averaging within ±50 cents of each note position in 12-tone equal temperament gives:

Positive: C, F, A♯/B♭

Negative: C♯/D♭, D, D♯/E♭, E, F♯/G♭, G, G♯/A♭, A, B

As explained at the beginning, positive values indicate the degree to which a sound is on the fundamental side, drawing other sounds in as harmonics; negative values indicate the degree to which it is on the harmonic side, being drawn into other sounds. Because we are aggregating across octaves, however, this differs from the ordinary interpretation: it shows the average tendency across an octave family, or pitch class.

It is particularly easy to understand why E and G are on the negative side. C's third harmonic corresponds to G, and its fifth harmonic corresponds to E, although the latter is slightly displaced from its equal-tempered position.

And $P\left(f\right)+\frac{Q\left(f\right)}{W}$, the harmonic-fusion evaluation, looks like this:

It may not look very different from $P\left(f\right)$ alone, but F, which had a large positive harmonic-polarity value, has increased, while D and E have decreased. It is also interesting that G♯/A♭, A, A♯/B♭, and B have become nearly equal in value, making that region flatter.

Because this uses an average spectrum for the whole piece, relationships from multiple chords are mixed together. Of course, we can also calculate harmonic polarity from a spectrum at a particular moment, or averaged over a chosen interval, to examine the combination of sounds used during that time.

Instead of a musical recording, let us now examine the harmonic polarity of a simpler chord: C, E, and G at their equal-tempered positions, with equal amplitudes.

Equal amplitudes may play a role here, but the main peaks are formed largely by the third-harmonic contributions. On the positive side, we find

C F A, an F major chord;

on the negative side, D G B, a G major chord.

This simply reflects the positions related to each of C, E, and G by a factor of three (×3 and ×1/3).

Let us add the spectral intensity $P\left(f\right)$ and look at the harmonic-fusion evaluation.

Although C, E, and G had equal amplitudes in the recording, the harmonic-fusion evaluation slightly reduces E and G, which lie on the harmonic side, making C the pitch class with the largest value. In this way, the harmonic-fusion evaluation function can, to some extent, evaluate which sounds lie on the fundamental side of harmonic relationships. The C major chord provides an easy-to-understand example.

The chord in this example had equal amplitudes, which for this case we can loosely think of as equal volumes. In actual music, though, chords are played with a variety of relative sound levels, and with amplitudes spread over multiple frequencies. I think it is these more general chords that are especially interesting to examine through harmonic polarity. We can calculate it for other chord types in 12-tone equal temperament, but there is no need to restrict ourselves to that tuning: any combination of frequencies can be used.

(From here, I introduce an idea that goes a little beyond harmonic polarity.)

So far, we have looked at harmonic polarity $Q\left(f\right)$, which evaluates harmonic and subharmonic components from $n=2$ onward with positive and negative contributions, and at the harmonic-fusion evaluation $P\left(f\right)+\frac{Q\left(f\right)}{W}$, which combines it with spectral intensity $P\left(f\right)$. Now I would like to discuss another interesting perspective on harmonic relationships.

Looking at the first few terms of the harmonic series,

these actually correspond to

or, in C major, C C G C E G.

Since an octave is a factor of two, the essential terms are

the odd-numbered terms. In this way, the harmonic series seems connected with major tonality.

Now look at the subharmonic series. Interestingly,

correspond to

these notes. If we now reinterpret "$\frac{1}{3}$" as do, we get

these notes: a minor chord. The familiar ratio 10:12:15 agrees with the portion $\frac{1}{4}$, $\frac{1}{5}$, $\frac{1}{6}$, with $\frac{1}{6}$ as the root.

This suggests a connection between the subharmonic series and minor tonality.

Could harmonic polarity evaluate this? I considered several possibilities. The issue is the "1" in the series, the actual sound represented by $P\left(f\right)$. To include it in the harmonic series, using a positive sign as in harmonic-fusion evaluation makes sense. But to include it in the subharmonic series when considering a minor chord, we could give $P\left(f\right)$ a negative sign and calculate $-P\left(f\right)+\frac{Q\left(f\right)}{W}$. The negative component of harmonic polarity $Q\left(f\right)$ comes from the subharmonic series starting at n=2, so this places $P\left(f\right)$ on that same side.

Let us actually calculate $-P\left(f\right)+\frac{Q\left(f\right)}{W}$ for a C minor chord:

As expected, G has the largest negative magnitude, revealing the characteristics of the subharmonic series.

However, I thought it was somewhat cumbersome to change the sign of $P$ depending on whether I was examining major-type or minor-type relationships. So I tried designing

this complex number. I call it the "harmonic polarity vector." As a complex number, it is a kind of two-dimensional vector.

The advantage of this representation is that it contains both evaluations proposed above, obtained by changing the sign of $P$. Its projections onto +45° and −45° are, respectively,

so they give quantities of the same kind. Looking at the complex number itself rather than its projections lets us examine whether a frequency leans toward harmonic-series fusion or toward belonging to a subharmonic series, including positions in between.

For C major:

For C minor:

For C major Prelude (Bach):

There is, however, information that the harmonic polarity vector loses: cases where a frequency has both harmonic-series and subharmonic-series relationships. For example, in a chord containing F and G, F lies on C's ×$\frac{1}{3}$ side and G on its ×3 side, so C has both relationships. In the harmonic polarity vector, this collapses onto the real axis and becomes mixed with the magnitude of $P\left(f\right)$, the amount of sound actually present.

That led me to the following expression:

Like three primary colors, this complex number places the actual spectral intensity $P\left(f\right)$, the harmonic-series relationship term $H\left(f\right)$, and the subharmonic-series relationship term $S\left(f\right)$ along three equally spaced directions in the complex plane. Expanding it into real and imaginary parts gives

so it contains the same imaginary component as the harmonic polarity vector, multiplied by a constant! (Recall that $Q\left(f\right)=H\left(f\right)-S\left(f\right)$.)

With $H\left(f\right)$ placed at +120° and $S\left(f\right)$ at −120°, strong contributions from both combine into a vector pointing near ±180°. If the real-axis contribution is also strong, cancellation moves the result closer to the origin.

I call this the "harmonic characteristic vector." I am still experimenting with the idea that this complex value could express a sound's character in terms of harmonic relationships.

For C major:

For C minor:

For C major Prelude (Bach):

I will end this article here. How was it? There is still much I would like to discuss about the harmonic characteristic vector, including

and other properties. But research and discoveries are still ongoing, and I have not yet assembled enough material to explain its properties systematically. The explanation of harmonic polarity alone has also made this a fairly long page, so I decided to leave those topics out this time.

I think harmonic polarity itself was a new concept, so I hope it offered a fresh perspective. Calculations of harmonic fusion are not in themselves new: they appear in several papers and software implementations, although methods incorporating subharmonics seem less common.

In particular, might "the idea that subharmonics are, in a sense, the inverse of the harmonic series" be useful beyond the harmonic-polarity calculation presented here?
This also means that we can regard the subharmonic series as "a sequence of frequencies whose nth harmonic is a fixed frequency."

"The subharmonic series does not physically exist, so the theory cannot hold."

This concept is often met with that response. Yet the relationship between harmonics and chord consonance also holds for sine waves, or pure tones, which contain no overtones.
I find it a little unfortunate that the view of harmonics and subharmonics as numerical "relationships" between frequencies does not seem to be widely recognized. I hope this article encourages some readers to adopt that perspective.
Even if the calculations were difficult to follow, I hope you can take at least this idea away with you.

Even just looking at the graphs, I hope you can see what is interesting about the harmonic characteristic vector, although I have only briefly introduced its formula here. In the C major Prelude graph above, the phase angle varies considerably from note to note. I think this may visualize differences in the harmonic roles of those notes.

Still, this article has only shown simple examples using a few recordings. I have therefore made web apps so that you can carry out experiments and observations yourself.

For a simpler app, try:
Fundamental Perception Simulator

It can plot the functions introduced in this article from an audio file or microphone input. In its "Behind Chord Theory" section, you can also create chords by adjusting volume sliders for notes in 12-tone equal temperament, then view graphs of their harmonic relationships.

For more detailed parameter controls and graph export, try:
TONAL / WAVE WEB

This app loads audio files and calculates graphs from the harmonic-polarity family of functions. It supports PNG and CSV export. It also offers several display modes: for example, you can select a pitch or pitch class and display its harmonic-relationship values over time, or draw curves of pitch or pitch-class values in the corresponding complex plane. Other features include continuous analysis of harmonic relationships using spectra averaged over musical time intervals, such as quarter notes matched to a piece's tempo.

Please try calculating harmonic polarity for your favorite chords, pieces, or your own music, and see what you discover.

My aim is to build a theory of sound that can represent the space of sounds as people experience it, much as color science represents color space. This is still far from a practical framework, but I intend to keep researching it. If you are interested, please visit my X account or this website again.

Thank you for reading. This was Wooly.