How to model interpretations of rap music

How to model interpretations of rap music

I just started working on a website that will help people understand what rappers are talking about. Users will see the lyrics to a rap song and they'll be able to click certain lyrics to see an explanation. Here's a screenshot (you can also check out the site itself here):

(Original lyrics censored; click here to see them)

Anyway, my question is how to model these annotations in my application. Right now, I'm storing the lyrics and annotations as one big blob of HTML in this format:

    With the goons I spy    Stay in tune with ma    She like damn    This the realest since 'Kumbaya'    Kumbayay Killa Cam my lord   
    
    
  "Ma" refers to ladies, generally, and specifically also the woman singing the hook;  "Stay in tune" is a musical metaphor: he literally stays in tune with the singer and also in the sense that he has game.    
    
  Kumbaya is a campfire singalong.    
  
  

and then processing it with this method for output:

class Song < ActiveRecord::Base    include ActionView::Helpers      def annotated_lyrics      lyrics = read_attribute('annotated_lyrics')      return if lyrics.blank?        require 'hpricot'      doc = Hpricot lyrics        doc.at('.lyrics').inner_html = doc.at('.lyrics').inner_html.strip      doc.search("a[@href^='#note']").set('class', 'tooltip').each do |t|        t.inner_html = t.inner_html.strip      end      doc.search("div[@id^='note']").set('class', 'annotation').each do |a|        a.inner_html = auto_link(a.inner_html.strip, :all, :target => '_blank')      end      simple_format doc.html.strip    end  end  

and the rest I do with jQuery and the fantastic qTip plugin.

This works fine for display, but since my application doesn't know about the relationship between annotations and lyrics, it will be hard to, say, add an interface for updating an individual annotation inline (or at all, really).

On the other hand, I don't really know the best way to represent this in ActiveRecord. I suppose a song could "have_many" annotations, but how would I represent which lyrics were annotated? I could store the start and end word index, but this seems painful and sensitive to minor changes in the lyrics.

Answer by Mike Buckbee for How to model interpretations of rap music

Your first instinct to setup associations with a song having many annotations would definitely work. Two potential approaches to storing the start and stop annotation indexes:

1. Store the start and end line that the lyric occurred on (count the linebreaks in your lyric file)

or

1. Store the start and end word boundary (or just space) that denotes the annotation. This would at least let you correct most typos without breaking the annotation index.

Answer by pvoosten for How to model interpretations of rap music

1. Tokenize your lyrics, so that you can identify a word in the lyrics by using e.g. a line and word number. Another option is to use character positions for your annotations. In any case, as always, take care of the character encoding of the lyrics.
2. Further, never touch the lyrics anymore. Better not store them as html, but as xml or as plain text.
3. Don't annotate within lyrics. Use a model wherein you can attach a position in the lyrics to an annotation. Use stand-off annotation.

Stand-off annotation will allow you to add more features over time, such as letting many users annotate the same lyrics. Generating the HTML you store as a blob is easy to do from stand-off annotations.

You might be interested in the (xml) data models of annotation tools that are quite well known among linguists: e.g. MMAX2 and Callisto. These are easily convertible to database models.

Answer by Alfa07 for How to model interpretations of rap music

As for linking annotations and lyrics you can have several approaches:

1. Link as proposed above annotations to exact places in lyrics (eg. line numbers, words, characters).

2. Make dictionary phrases/words <-> annotation. Just before displaying you search dictionary and insert into page annotations. If speed or specificity is concern each entry in dictionary can be tagged by relevant songs. If you want your annotations to be robust to small changes in lyrics than while finding matches in lyrics for annotated phrase use Longest common subsequence metric.

3. Combine #1 and #2

Answer by Chris McCall for How to model interpretations of rap music

What about presenting the lyrics like this (with thanks to the People's Champ)?

Well it's that [grain grippa][1] from Houston, Tex  That bar sippa, that bar no plex  I'm straight up outta that [Swishahouse][2]  Where G. Dash write all the checks  So [check the neck, check the wrist][3]  I'm balla status from head to toe    [1]Referring to the wood grain steering wheel common to luxury cars  [2]Swisha House is the record label Paul Wall records for  [3]"Look at my watch and necklace because they are expensive"  

Just an idea, I was inspired by the markup used to add comments on this site.

So, for the database, create Lyric, LyricLine and Annotation tables. Annotations have LyricLineIds, StartChar and EndChar values and a Meaning or Description field. LyricLines are the text of each line, related to the Lyric entity by LyricIds. Lyrics store song info, language info, whatever.

This format should be pretty easy to generate off of the database and has the benefit of being more "human readable" than XML and editable in-place, so you can test it a lot easier before you have to develop a whole UI.

I have this question favorited, and look forward to watching the site progress. Interesting work!

Answer by Sixty4Bit for How to model interpretations of rap music

XML would be a great model also.

                      Well it's that 
grain grippa
Referring to the wood grain steering wheel common to luxury cars
 from Houston, Tex        That bar sippa, that bar no plex        I'm straight up outta that 
Swishahouse
Swisha House is the record label Paul Wall records for        Where G. Dash write all the checks        So 
check the neck, check the wrist
"Look at my watch and necklace because they are expensive"        I'm balla status from head to toe                    ...            ...              

Pretty easy to edit and update. Creating a UI for it probably wouldn't be difficult. If you were opening up the creation of records to the public, you might change

and

to and . But dd and dt are the HTML standard which is why I used them in the first place. This would enable you to use straight CSS to style it up with a little love from JavaScript to make it look awesome. (BTW The site is awesome.)

Fatal error: Call to a member function getElementsByTagName() on a non-object in D:\XAMPP INSTALLASTION\xampp\htdocs\endunpratama9i\www-stackoverflow-info-proses.php on line 72