When I was in high school, I often pondered some of life's bigger mysteries and conundrums, typically of the "unstoppable object vs. immovable post" or "why does 7-11 need locks on the doors?" variety. Steven Wright and Emo Phillips were big influences in those formative years. Yeah, I was a hit at parties.
One of those questions that stuck, however, was what would happen if the audio and video of the TV somehow got mixed up... so that you would "see" the sound and "hear" the images. What would a rainbow "sound" like? How would a Beethoven symphony "look"? I later discovered that there are actually people who are kind of wired this way. It's a neurological condition, probably described in an Oliver Sacks book I haven't yet read, called synesthesia, where experiences in one sense (e.g., sound) lead automatically to experiences in a second sense (e.g., sight). As you might imagine, many famous artists (Hockney, Kandinsky), composers (Ellington, Liszt) and performers (John Mayer) are so gifted. And if you're a fan of the a cappella group, The Bobs, you may know this song about the condition.
One synesthete is Eric Haeker, a Philadelphia musician/composer and founder of the non-profit Pieris Music. As part of his educational mission, Eric wanted to share his visualizations of music with the rest of the world, and developed a system to do so. It goes way beyond the typical oscilloscope manipulations you see on iTunes or WMP, instead having a distinct graphical object for, e.g., each instrument of a musical score. That's perfect for Bach, as seen in the mesmerising application of Haeker's creation below. A hi-res version, with explanation, is found at the former Pieris website. Another visualization, of a Chopin-like piece, can be found here.
In September, Eric was granted a patent for his work.
|
U.S. Patent No. 7,589,727 Method and apparatus for generating visual images based on musical compositions Inventor: Eric P. Haeker (Philadelphia, PA) Issued: September 15, 2009 1. A method of producing a graphical representation of a musical work comprising a plurality of individual musical lines comprising notes, said method comprising the steps of: (1) obtaining an electronic version of said musical work; (2) translating using a processor said notes of each individual musical line of said electronic version into a separate x, y graph in which a y value of said notes is representative of a pitch of said note and an x value is representative of a relative time of said note in said musical work and a duration of said note; (3) importing using said processor each said graph into three-dimensional animation software; (4) generating using said processor a visual object corresponding to each individual musical line of said musical work; and
|
Commentary: There are at least three interesting things about this patent that factored into my decision to feature it here. The first two are discussed above: a) it's for a non-profit arts group -- not your typical patentee; and b) it's damn cool to watch.
The third thing, of course, is the Bilski angle. After initially being given a patenability pass, this claim was rejected under 101 in a second office action that came just two weeks after the Federal Circuit's Bilski decision. Strangely, the examiner cited not to Bilski, but instead to the Love Memorandum of May 2008, which enunciated the same machine-or-transformation test the CAFC later adopted. After amending to include the "using a processsor" language, the claim was allowed.
Whether the simple inclusion of a generic "processor" suffices to pass the machine prong of Bilski is at best debatable -- and it's been debated on this very blog in the past. But a more intriguing question is whether this claim passes the transformation prong. As noted previously at 12:01, the Bilski opinion on this point invokes In re Abele in a way that leaves many unanswered questions. The BPAI has addressed some of those questions in Ex parte Hardwick (rejecting claim that did not specify input's origins, or that they were transformed into something not "cognizable by a human operator"), Ex parte Gardiner (rejecting claim when input did not represent a physical item), and most recently in Ex parte Aoyama (claims failing "because the data does not represent physical and tangible objects," and not reaching a more general question about "mere calculation of a number based on inputs of other numbers"). Whether the transformed article must be physical or tangible is also a subject of debate on this blog, with some commenters taking some extreme positions. But that question is relevant to the Haeker patent.
Are Haeker's claims fundamentally different from the ones in the BPAI cases? It is clear that the claimed invention indeed "transforms" a musical work into a graphical representation. And the transformation is certainly cognizable to a human operator. But should music be considered a "physical item" or a "tangible object?" That's less clear. And it may depend on how the input music is represented.
Looking at dependent claims and reading the specification, it seems the inputted "electronic version" of the music could be generated on-the-fly from a live performance (claim 8), or as a MIDI-file (claim 6) (a MIDI-file is kind of an electronic version of a player-piano roll, describing how music is to be played). It may be that the Abele distinction differentiates these two types of input. That is, perhaps the transformation of a musical performance (i.e., the actual sounds heard) is a patentable transformation, while the transformation of a written score or MIDI-file (i.e., a description of music to be used in a performance) is not. Just a theory.
Or it could be that there is just no way that "sound" is considered physical or tangible, and these claims cannot meet the transformation prong of Bilski (assuming the BPAI's more recent interpretations are correct). That doesn't seem quite right, but it may be the inevitable result of this trend. Consider the discussion open.
In the meantime, I've contacted Mr. Haeker, who has agreed to share some more samples of his work in the next few weeks. I'll post links here when they arrive.
Recent Comments