project07:P3

Research

In my search for as many performances I could possibly find, I stumbled across a particular special one. Under the direction of Benjamin Zander, the Boston Philharmonic Youth Orcestra does not only give a stunning performance of a complex work (see: La Valse), but I noticed a big screen behind the orchestra with where moving geometric coloured shapes are projected. This projection is a creation by Stephen Malinowski. You can find more information on Stephen here.

Stephen Malinowski and the Music Animation Machine

Stephen's Project is called the Music Animation Machine, a framework he has been working on since 1974 (!!). Since the release of YouTube in 2005, he has been able to make quite a name for himself and he has been working fulltime on the research since 2010. The Music Animation Machine can be seen as research into the field of visualizing music. This makes the project very interesting for musicians and non-musicians. Music can be confusing for the untrained ear. While trained musicians will always 'feel' the directions of harmony - tension and relief, the fundamentals of western music - and are happily surprised by any unexpected twists and turns, the untrained ear might find all this information confusing. This may explain why in the last 10 or so decades, the complexity of music has seen a drastic step backwards, to appeal to the bigger market. Stephen's work may help this audience understand not only more complex works of classical music by giving them a visual aid of what is to come, but also find the intricacies and finer details one may find in classical music. As his research is vast and comprehensive (and somewhat behind the scope of my original research), below you may find a list of papers which I used to expand upon the research of Stephen Malinowski.

Visualizing Harmony

While reading a written score of music can reveal any crucial piece of information of the music for trained musicians, untrained musicians may become non the wiser. However with recent advances in computing power, we can process MIDI information in totally new ways. For example, the piano roll notation system displayed in Synthesia, where every note falls towards a keyboard so timing and pitch can be observed instantly, revealing many patterns otherwise unnoticed. The software has become popular among young piano students who now don't have the need to learn to read any form of classical music notation.

However, new iterations have emerged with the works of Stephen Malinowski. Most notably: his use of color. Color may prove to be the most intuitive way to provide visual aid in an otherwise completely auditory form of art. Stephen Malinowski has used colors in many different ways: separated by instrument, voice and pitch. The latter is most important to my research. We can generate color codes according to HSL (Hue, Saturatin and Lumination) and shift hue to match with pitch. However, this may not provide any information corresponding changes in harmony, so Stephen used a shift in hue to correspond with the circle of fifths, the deepest fundamental layer of western music. You can read how he achieved this here. Now, the hue value of each note displays it's position in the circle of fifths. If we process the entire piece of music with the same logic, we can find harmonic changes throughout the piece - this information is absolutely essential and the algorithm provides a simple method of providing fundamental information about the piece of music.

=== B. Klemenc et al.: Visual and Aural: Visualization of Harmony in Music with Colour ===

However, a paper by B. Klemenc and colleagues suggests we can improve upon this by placing each notes in a broader context. Simply put: by averaging pitch between clusters of notes, the given saturation to each note is slightly averaged. By calculating distance between the pitch of the note and it's surrounding notes we get the interval. Any interval can be very stable, unstable or anywhere in between. This is again proved by the circle of fifths. A C and a G have a very stable relation so they are very closeby one another, however a C and an F-sharp have the most unstable relationship, so they are opposite one another in the circle. The distance to the circle is mapped by saturation. By averaging a C and a G we get a value close by the circle, meaning a saturation of (almost) 1.0, by averaging a C and an F#, we end up in the middle of the circle, resulting in a saturatin of (exactly) 0 - so grey. In short, expected intervals have a bright and shiny color, more unstable relationships like chromatic scales or jazz chords are greyish.