LilyPond Music Notation inside Max/MSP

I often find it easier to understand what I'm hearing when I can see a visual representation of the music. This has become more important as I've been experimenting with generative music. I'm starting to give up full control over the composition process and it's not always clear what the results are without a lot of repeated listens.

Unfortunately, Max/MSP doesn't have much support for standard music notation. So I've been experimenting with generating notation inside Max/MSP using LilyPond, my ajm.ruby object, and the jweb object from Max 5. This isn't a great solution: it's slow and it can only display notation. You can't interact with the notation in any way, which is what I'd really want. But if you don't want to pay money or spend a ton of time writing your own notation software, this is potentially a viable solution for getting notation inside Max.

If you want to try this out, first you need to install:

Then:
  1. download this zip file which contains everything else you need to run this example.
  2. Unzip maxlily.zip somewhere on your Max file path
  3. Edit maxlily.rb and change the LILYPOND and VERSION values to find the lilypond command on your system
  4. Open maxlily.maxpat and click the evaluate button.
It can be slow! If you don't see errors in the Max window then some notation should eventually appear:

I'm not going to give a tutorial on LilyPond, so check out the documentation on their site if you want to play around with notating different things.

There's a bonus on OS X (maybe Leopard only?): jweb uses webkit, which has some nice controls for viewing pdf. If you click on the notation, controls will appear that allow you to zoom in or out, and open the pdf in Preview (where you can print).

So what's going on under the hood? The maxlily.rb file is Ruby code that does most of the work. Ruby is certainly not the only solution here, I just find it very easy to work with. This code could be ported to C or Java or whatever without much difficulty. If you are interested in doing that, let's take a look at the code in detail and see what it's doing:

First some constants are setup:

LILYPOND = '/Applications/LilyPond.app/Contents/Resources/bin/lilypond'
VERSION = '2.10.33'
FORMAT = 'pdf'  
As mentioned above, you will need to modify these to match the version and path to lilypond on your system. Format determines the output format. There are some other choices like png, but pdf seems to look the best.

These lines setup file paths:

THIS_DIR = Pathname.new($0).parent
INPUT = THIS_DIR + 'tmp.ly'
OUTPUT = THIS_DIR + 'output'
THIS_DIR is set to the folder containing the Ruby script. From there we set the location of two files we will need. INPUT is the .ly file that will be generated and sent to the lilypond command in order to produce our notation, and OUTPUT will be the final output file that is loaded into jweb.

Skipping down to the bottom of the program, note that write_to_file() and load_page() methods do what the names imply. write_to_file() writes some plaintext to a file. load_page() sends a url to the jweb object. For proper functioning inside Max, ajm.ruby needs the attribute @evaloutlet -1 so it won't output the script return value and confuse jweb.

That leaves the notate() method, which does all the real work. First we build a multiline string that will be written out to the .ly file for the lilypond command to interpret. Extra backslashes are needed for the usual character escaping. What we are trying to write to disk is:

\include "english.ly"
#{str}
\version "#{VERSION}"
Where #{str} is whatever is passed to the ajm.ruby object and VERSION is the constant defined at the top of the script. I prefer to use lilypond in english language mode, hence the first line (here are other options). The version line is recommended to be included in every .ly file so lilypond knows how to correctly interpret the file as new versions introduce new features.

That text is written to disk as an .ly file, and we tell jweb to display a loading page for user friendliness:

write_to_file(INPUT, source)
load_page(THIS_DIR + 'loading.html')
Next the actual lilypond command is constructed:
%Q["#{LILYPOND}" -f #{FORMAT} -o "#{OUTPUT}" "#{INPUT}"]
I'm using %Q[] to quote the string because I don't want to escape all the double quotes. Double quotes wrap all the paths in order to handle any spaces in the path.

Finally the command is executed, and we do some error checking on the results. If everything looks good we tell jweb to display the resulting file.

Now here's perhaps the most important detail. When ajm.ruby executes, it will cause Max to wait while it's doing it's processing. Running lilypond is a long-running command (5-10 seconds on my computer). If we didn't do anything special, Max would hang for that time and the UI would be unresponsive. That's very bad. So we have to do all the execution of the lilypond command in a background thread. Luckily that's really easy in Ruby: just wrap the code in

Thread.new { }

So that's that. This isn't an ideal solution to music notation in Max/MSP but it works. Next steps would be to write an automated converter between whatever representation is being used for music in the Max patch and the lilypond format. A lot of patches probably don't have an explicit music representation, or every patch might be different, so writing the conversion to lilypond may be more trouble than it's worth. Still, this patch can be a fun way to learn LilyPond syntax and play around with it in an interactive environment.



Adam Murray, 2008
contact the author...

Comments

escaping backslashes

The method that you are using works well for a piece that I am doing for generative music, but I wanted to note that because of the way the Ruby notate def, it errors on any \ that are in the lilypond syntax. For example:

\relative c' {
c d e f
g a b c
}

Doesn't compile correctly. if you add another backslash:

\\relative c' {
c d e f
g a b c
}

it then works. I don't know Ruby well at all, so I don't know the proper way to Regex the string for the slashes, or I would provide code to fix the bug.

As a side note, it would seem like you could do this with simple scripting executing terminal commands to lilypond, but I won't argue with something that works.

RE: escaping backslashes

Good catch. I don't think there is actually anything wrong with the Ruby script. There are some confusing interactions between Max, Java (ajm.ruby is Java), and Ruby. By the time we get to the Ruby script, the "\r" in "\relative" is being interpreted as a "return feed" character rather than a separate "\" and an "r". So this is a much deeper problem.

A potential workaround is to add this line to the top of the notate def:
str.gsub!("\r", "\\r")
But this will only fix the specific problem you mentioned. I think you will run into further issues if you need to use a backslash for other LilyPond commands. Probably the best solution for now is to use the extra backslash like you are doing.

Perhaps a regexp object could be used in the Max patch to replace "\" with "\\" but I can't seem to get that to work. I'll investigate if I can make some adjustments to ajm.ruby to prevent this, but the obvious solution (replace "\" with "\\" before passing things to Ruby) does not work either! Argh! The problem may ultimately lie with how textedit/Max handles backslashes.

And you are right, there's no particular reason to use Ruby for this, I was just testing out my ajm objects when I made this patch. Terminal scripting could work too.

Re: escaping backslashes

I have been further modifying your work because I myself am doing research for a Chance Music Piece to generate original readable notation in real-time for perfomance based off of things the computer has "learned" by analysis of people whom have previously played the "Mother" instrument. I have a Disklavier Upright Player Piano that has midi capabilities that will be used to "teach" it(This is the Mother instrument). I have accomplished Key Signature Modification and Time Signature, all of which were a real pain, so I am starting to second guess the process. Max Objects like coll and sprintf, possibly text though I have not tested yet, all use backslashes to indicate either escaping or "insertions", similar to the whole \r issue. Past that, the comma also has significance to max also, so when dealing with lilypond syntax(\relative c,,) it becomes cumbersome. I will keep investigating also, and try some your suggestions. I have been meaning to pick up Ruby, but never had a use for it in the stuff I do, but with this awesome little object you made, I now have an excuse. I will keep you up to date with my progress, maybe we can work out something fun and useful. This has lots of potential, and is a big hole in max IMHO, that needs to be filled.

Quick Update

After some testing, it seems that using the Text Object is the best way to go in terms of storing the generation of the lilypond syntax either to compile together for use with your object/script, or as a "dictionary" of sorts to translate midi->lilypond. It seems to not have "programming" traits like coll and sprintf, and treats backslash and comma normally. You can also send it commands for carriage returns and have it spit out specific lines, and write to file with a .ly extension, eliminating the need to pass the str into Ruby at all, just call the file from the script. I will let you know how further tests go.

PS sorry for the back-to-back comments... just thought I would pass along that tidbit, because it makes this whole approach feasible again.

RE: Quick Update

No problem on the back-to-back comments. Actually, thanks for giving updates on your progress. It sounds like you have a pretty good solution now and hopefully this discussion can benefit anyone else who stumbles on this article.

I meant to reply sooner... I was going to play around with the text object over the weekend and I didn't get around to it. In any case, it seems the text object is much better than textedit when it's important to process the text exactly as entered. Maybe I should update the ajm.ruby help file with some examples, or at least post a follow up article here.