The code of music

Going over the title, you may misunderstand the point of this blog post. So I will have to tell it as early as now. This is to producing music through Scilab. Yes! Some functions will be able Scilab to sing. I will just be simple but what we want to accomplish in this activity is to make the Scilab read the notes in a music sheet.

I am a frustrated pianist. I was taught to play piano when I was in elementary but things (like my laziness and impatience in learning.. well I'm just a kid then so I do have more significant matters to be taken care of :P) lead to another path and I wasn't able to learn much. Well, I regret missing that chance because I really have a passion for music. It just that I am not the best suited for it so let's just stay admiring these work of art. :)

At first, I really thought that it would be tedious in making this activity work. I listed below some of the things that I think what are needed to be considered.

(a) make the Scilab know the pitch of the note (if a C, D, E, etc);

(b) make the Scilab know the tempo of the note (or the duration, depends if a whole note, quarter note, etc.); which will both require to

(c) have the right filtered image

Better start losing sweat! The song that I chose is Old MacDonald, one simple nursery song because we don't want to make our life more difficult. Below is the music sheet for Old MacDonald in C Major [2]. Again this is simple because there were no flats and sharps :)

The music sheet was first cropped so we can handle on the first line for the mean time.
 

The image must be binarized first, if it is not a binary image. Then the color of the image was inverted. As I worked through my code, the task seems more problematic than I thought. The first problem that I encountered: problem (c) or filtering the image. I instantly thought that using a circle structuring element then applying open operator would isolate the notes. It pretty much did however, the half note would not be included. So I think of another way. It was to remove the vertical and horizontal line. Shown in below is the step by step cleaning of the image. First, removing the vertical, then horizontal lines, both using the TopHat() function. Unfortunately, there really parts of the note that would be removed in performing these steps so the CloseImage() function was used.

I will now be having my blobs and using the FilterBySize() would remove the unnecessary small blobs. We now have the following blobs.

We now arrive to problem (a) mentioned above. How will the Scilab know the ptich of the note? In the music sheet, the pitch was differentiated through the location of the note in the staff (the one with the five horizontal lines). Thus, those position should also be recognized by the Scilab. AnalyzeBlobs function in Scilab measures the location of the centroid of each blob. In Gimp, the range of pixel location of each line and space along the y-axis were determined. These were incorporated as conditions for the Scilab to know the right pitch. We should now face problem (b). Since, half notes are not solid circles, we can deduce that their areas are different from the quarter notes. So again, conditions were made to determine the tempo.

The whole process of making the Scilab read notes seems successful. Below is the output.

This task really has been tedious. I spend a lot of time in determining the location range because the you have to consider that there are different range for spaces and lines. Though once established correctly, you're okay. For the next line, however the next staff will be another problem because the pixel position or ranges of spaces and line would be different when another cropping of image was made. The practical thing to do is to work to match the pixel positions of spaces and lines of each staff. In addition, working on the tempo is really a killer. I am lucky that the first staff only contains only half and quarter notes. I able to distinguish between the two notes. There still whole, eighth and sixteenth notes and the dot beside the note. The eighth and sixteenth notes is notable for the one bar and two bars above their musical symbol, respectively. I plan on working on those by determining their position along the x-axis and if they are reasonably close to the blob representing the note, the tempo of the note will be known. Still, these need a lot of work to be accomplished. Maybe I'll try it when I had the time, including the rests in the music scores.

I am thankful for the initial code that Ma'am has provided us that make the Scilab produce different pitch. I also thank Aeriel Constantino and Phoebe Gallanosa for the useful discussion. I believe I deserve a 10/10 in this Activity because technically, I accomplished the task at hand.

__________

References

[1] Soriano, M. "Playing Notes by Image Processing." AP 186 Laboratory Manual. National Institute of Physics, University of the Philippines, Diliman. 2013.
[2] http://www.pianolessons4children.com/songs/old-mcdonald.php
[3] http://www.ukuleles.com/spreadsheets/Guitnote.XLS (for the frequency of other pitch)