Chris Mendez in Radio, ID3, Ruby, Rake, Homebrew, Audio, DevOps

Radio: From WAV to FLAC with ID3 using Echoprint

I'm currently learning about different ways to do music fingerprinting. In this article, I'm simply identifying how to do it using Echoprint. Here's how I did it.

I used Ruby Rake to run command line scripts. You can certainly type these commands inside of Terminal but this method allows me to better organize my scripts into something useful.

Step 0 - Download a sample wav file

Thank you Trent Reznor for providing us with content that's cool to work with 999,999 (Flac). We'll use this file to manipulate stuff.

Step 1 - Install Homebrew

If you are unfamiliar with Homebrew, read this article.

Step 2 - Install Rake

You will need Ruby rake for the rest of this tutorial. If you're on a mac, simply enter this into Terminal.

gem install rake

Step 2 - Create a Rakefile

Create a file called Rakefile and paste this code.

namespace :ffmpeg do
  desc %Q{ ›› Install FFMpeg Encoder }
  task :install do
    # https://gist.github.com/predakanga/2376835
    sh %{ brew install ffmpeg boost taglib }
  end
  
  desc %Q{ ›› Encode Wav to FLAC }
  task :encode, [:input, :output] do |task, args|
    input = args.input
    output = args.output
    sh %{ ffmpeg -i #{input} -c:a flac #{output} }
  end

  desc %Q{ ›› Read ID3 tag within a specific file (.wav,.mp3) }
  # Source http://stackoverflow.com/a/35166619
  task :probe, [:file] do |task, args|
    file = args.file
    sh %{ ffprobe -print_format json -show_entries stream=codec_name:format -select_streams a:0 -v quiet #{file} }
  end

  desc %Q{ ›› Strip metadata and remove artwork (without encoding) }
  # Source http://stackoverflow.com/a/20202233
  task :strip, [:input, :output] do |task, args|
    input = args.input
    # If no output is specified, the file will be overwritten
    output = args.output || input
    sh %{ ffmpeg -i #{input} -vn -codec:a copy -map_metadata -1 #{output} }
  end  
end

Install

This command simply tells homebrew to download some necessary libaries.

rake ffmpeg:install

A few more functions

Confirm whether the WAV file holds any metadata.

rake ffmpeg:probe[~/Desktop/01999999.wav]

Strip out any metadata if necessary.

rake ffmpeg:strip[~/Desktop/01999999.wav]

Step 3 - Convert WAV to FLAC

Convert WAV file to FLAC format.

rake ffmpeg:encode[~/Desktop/01999999.wav,~/Desktop/01999999.flac]

Step 4 - Identifying music through fingerprinting

Now that we've converted a WAV file to FLAC, the next step is to find the ID3 tag data. There are a bunch of excellent tools out there but for this example, I will use the EchoNest tool called EchoPrint Codegen.

Add this to your existing Rakefile. We'll keep things organized by wrapping these commands with a echoprint namespace.

namespace :echoprint do
  desc %Q{ ›› Download Echoprint Server Dependencies }
  task :server_dependencies do
    # Remove existing Github Repo
    sh %{ rm -rf #{destination}/echoprint-server }
    # Download the repo to your local ~/Desktop
    sh %{ git clone git://github.com/spotify/echoprint-server.git #{destination}/echoprint-server }
    
    # Install JQ library http://brewformulas.org/Jq
    sh %{ brew install jq }
    # Database manager http://fallabs.com/kyotocabinet/
    sh %{ brew install kyoto-cabinet }
    # Tokyo Tyrant replacement http://fallabs.com/tokyotyrant/
    sh %{ brew install kyoto-tycoon }
    sh %{ brew cask install caskroom/versions/java8 }
    sh %{ brew install solr }
    sh %{ solr start }
  end
  
  desc %Q{ ›› Download Echoprint Client Dependencies }
  task :client_dependencies do
    # Change directory to desktop
    Dir.chdir(destination) do
      sh %{ rm -rf #{destination}/echoprint-codegen }
      # Download this specific repo from within Github
      sh %{ git clone -b release-4.12 git://github.com/echonest/echoprint-codegen.git }
    end    
  end

  desc %Q{ ›› Install Echoprint Server }
  task :install_server => [:server_dependencies] do
    Dir.chdir(destination + '/echoprint-server') do
      sh %{ python setup.py install}
    end
  end
  
  desc %Q{ ›› Install Echoprint Client }
  task :install_cli => [:client_dependencies] do
    # Once the repo has been downloaded, we need to modify the Makefile
    Dir.chdir(destination + '/echoprint-codegen/src/') do
      file = "Makefile"
      # Remove this specific reference to vecLib. It's no longer needed.
      newdata = File.read(file).gsub(/-framework vecLib/, "")
      # Create a new make file with the updated string
      File.open(file, 'w') do |out|
        out << newdata
      end 
      # Build the executable file
      sh %{ make }
    end
  end

  desc %Q{ ›› Identify a sound sample. }
  task :identify, [:file, :start_time, :duration] do |task, args|
    file       = args.file
    start_time = args.start_time || 10
    duration   = args.duration || 30
    
    # Remove the files
    sh %{ rm -rf #{destination}/codegen_output.json #{destination}/codes.txt  }
    # Change directory into the Echoprint client
    Dir.chdir(destination + '/echoprint-codegen') do
      # Double check which directory you're in.
      #sh %{ ls -la }
      #Run the fingerprinting process and output the results in a json file.
      sh %{ ./echoprint-codegen #{file} #{start_time} #{duration} > #{destination}/codegen_output.json }
      # Get the fingerprint from codegen and decode it using JQ and echoprint-decode
      sh %{ cat #{destination}/codegen_output.json | jq -r '.[0].code' | echoprint-decode > #{destination}/codes.echoprint }
      sh %{ curl -s --data echoprint=`cat #{destination}/codes.echoprint` http://localhost:5678/query }
    end
  end
  
  def destination
    Dir.home + "/Desktop"
  end
end

Source

Step 5 - Install Echonest

Install Echonest's Echoprint by first downloading the Git repo and opening up the Makefile.

rake echoprint:install_server
rake echoprint:install_cli

Step 6 - Modify Makefile

Sadly, the repo is no longer up-to-date so we'll need to modify one thing within the Makefile. Find this section of code:

ifeq ($(UNAME),Darwin)
	libtool -dynamic -flat_namespace -install_name libcodegen.$(VERSION).dylib -lSystem -compatibility_version $(VERSION_COMPAT) \
		-macosx_version_min 10.6 -current_version $(VERSION) -o libcodegen.$(VERSION).dylib -undefined suppress \
	    $(MODULES_LIB) -framework vecLib -framework Accelerate
else
	$(CXX) -shared -fPIC -Wl,-soname,$(SONAME) -o $(LIBNAME).$(VERSION) $(MODULES_LIB) -lz
endif

And replace it with this:

ifeq ($(UNAME),Darwin)
  libtool -dynamic -flat_namespace -install_name libcodegen.$(VERSION).dylib -lSystem -compatibility_version $(VERSION_COMPAT) \
  -macosx_version_min 10.6 -current_version $(VERSION) -o libcodegen.$(VERSION).dylib -undefined suppress \
  $(MODULES_LIB) -framework Accelerate
else
  $(CXX) -shared -fPIC -Wl,-soname,$(SONAME) -o $(LIBNAME).$(VERSION) $(MODULES_LIB) -lz
endif

Specifically, what we're doing is getting rid of this parameter -framework vecLib.

Now run the make command within Terminal by typing make within the ~/Desktop/echoprint-codegen/src/ directory.

make

Process

That's it! You've now compiled and created the Echonest's Echoprint executable file. This will now allow you to do music fingerprinting using a single command! Woot Woot!

rake echoprint:identify[~/Desktop/01999999.wav]

Resources