The article, “Early Computer Art In The 50’s & 60’s” touches on the Musicolor system, which was retired in 1957. Although Musicolor was an experimental program created for entertainment, it was also an artificial means of experiencing chromesthesia (although not referenced in the article, I believe this was the concept being presented). Firstly, this video https://www.youtube.com/watch?v=2xDnxkzQtdI elaborates on our course reading of Kandinsky’s “Concerning the Spiritual in Art” and discusses chromesthesia and synesthesia.

Synesthesia is when one of our senses stimulates another sense or associates what we experience with another sensory output.  For example, I may hear a violin playing a G, and I see the color blue. A person can smell a sunflower and hear a C major chord or see a triangle. Chromesthesia, which is a type of synesthesia, is a rare condition where individuals experience the perception of color while listening to music. There are many artists who are said to have (had) chromesthesia. A few are Vincent Van Gogh, Stevie Wonder, Pharrell Williams, Billy Joel, Kanye West, and yes… Wassily Kandinsky. What an amazing gift (in my opinion) for the mind to be able to translate sound into colors, although I understand there are some disadvantages.

Since Musicolor has long been retired, I searched for a similar program that emulates Chromesthesia. I was looking for something that could help those who do not have chromesthesia understand and experience it from the point of view of someone who does have the condition. I was looking for software or an app where one could define the color parameters they see for each note. The program could then paint, draw, or otherwise convey the colors according to the notes played, programmatically or manually. Of course, like Musicolor, it would serve as a tool and entertainment for musicians and artists, as well as anyone could use it to create art. I found several tools; three notable projects are “AudioPaint”, “SOVIS”, and Pitch to Color Calculator.

However, as an avid user of Oculus/Meta, I remembered an app I have used and began searching for others. There are two apps that bring us down the same road as Musicolor began. I hope that with developing technology, these both become more than entertainment. Each has the potential to be a more educational and immersive visual experience. I have demonstrated both of these apps in my attached video. 

1.  “MUSIC VISION”, which is still in experimental stages and has not yet been released on the Meta App Store, translates music notes or chords into color in real time. I used two songs which I thought had strong chord progressions to demonstrate.
2. “VIRTUOSO” gives us everything but the option to change the color, as you can see from my demonstration in the video. 

Side note: I added “Tilt Brush” by Google, because it also has the potential to simulate synesthesia. I believe Tilt Brush is open source (as ‘Open Brush’) if there are any other coders in here! “Animating Virtual Worlds” (AME 494) is a great course to get started if you have some coding experience and want to start developing in Unity 😜.  

So we have the technology to make this happen (in our virtual world) so we can experience this through immersion, but the developers may not know the benefits of including this feature. Imagine if we could assign colors to notes in these settings panels in these immersive VR apps. A little bit of C# and C++ will do the trick! It’s something that’s on my list to play/experiment with in the future as I continue on my color theory journey.