2Do Consulting

Blog

Google News: Change Sports Scores Timezone

If you use news.google.com on a regular basis you may notice that the “Sports scores” section shows game times exclusively in Pacific Time.

With no option for non-Pacific Time users to customize the displayed timezone, many have voiced their displeasure on the product forum.

Until Google News is fixed, Google Chrome users can use Timezone Adjuster for Google News™. The extension automatically adjusts the times according to your system’s timezone settings.

The source is available on github.

Can’t Find Libpq-fe Header

Problem: Installing/upgrading your pg gem on a Mac using the Postgres.app.

1
2
3
4
5
6
7
8
9
ERROR:  Error installing pg:
  ERROR: Failed to build gem native extension.

checking for pg_config... no
No pg_config... trying anyway. If building fails, please try again with
 --with-pg-config=/path/to/pg_config
checking for libpq-fe.h... no
Can't find the 'libpq-fe.h header
*** extconf.rb failed ***

Solution: Specify the path to your pg_config.

1
gem install pg -- --with-pg-config=/Applications/Postgres.app/Contents/MacOS/bin/pg_config

Markup Chemical Formulas With Ruby

Are you storing chemical formulas in your database but don’t want it littered with HTML markup? Use this code in a helper.

1
formula.gsub(/\d+/) { |f| "<sub>#{f}</sub>" }

To go from this:

to this:

Much nicer!

Manipulating Data With Ruby

Here’s a little Sinatra app I made called CSV2Sinatra. It’s a simple solution for those times when you have data in a spreadsheet or CSV file and want to query or manipulate the data with Ruby.

Clone the app from Github:

git clone https://github.com/mcianni/csv2sinatra.git

Drop a CSV file into the csvs directory, and run…

rake csv:import

Run the Sinatra app with:

rackup

Visit localhost:9292 to see the imported table.

Example Table

Now that we’ve imported the data we can use that Ruby and DataMapper goodness to work with it. Load up irb, and require './app.rb' to interact with the data (or similarly put the same require statement in a script).

Each file in the csvs directory gets imported into its own table in the database. The column names are determined by the first row of data in the CSV file.

The class names are generated by capitalizing the the file names. You can get a list of the classes by calling DynamicClasses.list.

Examples

The image above shows data from the Nation Water Quality Monitoring Council. After importing the data, there is a table in the database named stations.

Here’s how I easily access the data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1.9.3p194 :001 > require './app.rb'

1.9.3p194 :002 > DynamicClasses.list
 => ["Station"]

1.9.3p194 :003 > Station.new.inspect
 => "#<Station @id=nil @organization_identifier=nil @organization_formal_name=nil 
@monitoring_location_identifier=nil @monitoring_location_name=nil 
@monitoring_location_type_name=nil @monitoring_location_description_text=nil 
@huc_eight_digit_code=nil @drainage_area_measure_measure_value=nil"

1.9.3p194 :004 > station = Station.first
 => #<Station @id=1 @organization_identifier=<not loaded>
@organization_formal_name=<not loaded>
@monitoring_location_identifier=<not loaded>
@monitoring_location_name=<not loaded>
@monitoring_location_type_name=<not loaded>
@monitoring_location_description_text=<not loaded> 
...

1.9.3p194 :005 > station
 => #<Station @id=1 @organization_identifier="USGS-AK" 
@organization_formal_name="USGS Alaska Water Science Center" 
@monitoring_location_identifier="USGS-15008000"
@monitoring_location_name="SALMON R NR HYDER AK" 
@monitoring_location_type_name="Stream"
@monitoring_location_description_text="QUARTER SECTIONS CHANGED FROM 
NESWSW 10/4/11" @huc_eight_digit_code="19010101"
@drainage_area_measure_measure_value="94.1" 
...

You can find documentation for DataMapper here.

Ruby - Convert Nested Hash to Query Parameters

Does your ruby script need to convert a nested hash to query params? It’s tempting to write your own, but just… don’t.

If Rails is available, we’ll require the minimum necessary- ‘active_support/core_ext/object/to_query’.

With Rails
1
2
3
4
5
6
7
8
9
10
require 'active_support/core_ext/object/to_query'
nested_data = { foo: { title: "Title 1", status: 3 },
                bar: { title: "Title 2", status: 6 } }
nested_data.to_query("items")
=> "items%5Bbar%5D%5Bstatus%5D=6&items%5Bbar%5D%5Btitle%5D=Title+2&
items%5Bfoo%5D%5Bstatus%5D=3&items%5Bfoo%5D%5Btitle%5D=Title+1"

CGI::unescape(nested_data.to_query("items"))
=> "items[bar][status]=6&items[bar][title]=Title 2&items
[foo][status]=3&items[foo][title]=Title 1"

Otherwise you can load up ‘rack/utils’.

With Rack
1
2
3
4
5
6
7
8
9
10
11
require 'rack/utils'
nested_data = { foo: { title: "Title 1", status: 3 },
                bar: { title: "Title 2", status: 6 } }
Rack::Utils.build_nested_query(nested_data, "items")
=> "items[foo][title]=Title+1&items[foo][status]&items[bar][title]
=Title+2&items[bar][status]"

Rack::Utils.escape(Rack::Utils.build_nested_query(nested_data, "items"))
=> "items%5Bfoo%5D%5Btitle%5D%3DTitle%2B1%26items%5Bfoo%5D%5Bstatus%5D
%26items%5Bbar%5D%5Btitle%5D%3DTitle%2B2%26items%5Bbar
%5D%5Bstatus%5D"

Using Ruby to Export NikePlus Running Data

Are you looking to export your Nike+ Data without writing code? The Nike+ Data Exporter is an implementation of this code.

Nike Plus is a slick exercise tracking app for your mobile device. It utilizes either the GPS in your device or a dongle that hooks on your shoe and pairs with your device. The app tracks a ton of data including all the basics like time elapsed, distance traveled, mile splits, estimated calories burned, and even the weather!

Nike Plus lets you review past data in the app as well as on the slightly more expansive companion website, NikePlus.com. Unfortunately, neither offer a way for us to export the data.

We’re going to use Ruby to authenticate our account on the service, download the complete set of data for our account, and then write the data out to a CSV file.

1
2
3
4
5
6
7
8
9
10
11
require 'net/https'
require 'JSON'

user     = '...'
email    = '...'
password = '...'

login_path = "/nsl/services/user/login?app=b31990e7-8583-4251-808f-9dc67b40f5d2&format=json&contentType=plaintext"
data_path = "http://nikeplus.nike.com/plus/activity/running/#{user}/lifetime/activities?indexStart=0&indexEnd=99999"
post_data = "email=#{email}&password=#{password}"
headers = {"Content-Type" => "application/x-www-form-urlencoded"}
Autheticate
1
2
3
4
5
6
7
8
9
10
11
url = URI("https://secure-nikeplus.nike.com")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
resp, data = http.post(login_path, post_data, headers)

unless JSON.parse(resp.body)['serviceResponse']['header']['success'] == 'true'
  puts "Could not login. Server returned the following error(s):"
  puts "\t" + JSON.parse(resp.body)['serviceResponse']['header']['errorCodes'].collect{|e| e['message']}.join("\n\t")
  exit
end

Before we make the get request we need to set all of the cookies we received after authenticating.

Set Cookies, Make and Handle Request
1
2
3
4
5
6
7
8
9
all_cookies = resp.get_fields('set-cookie')
cookies = all_cookies.collect{|c| c.split('; ')[0]}.join('; ')

#request data
url = URI(data_path)
http = Net::HTTP.new(url.host, url.port)

resp, data = http.get(data_path, {'Cookie' => cookies})
data = JSON.parse(resp.body)

The server responds with multidimensional JSON object; we call JSON.parse to convert it to a hash. In order to flatten the values from the multidimensional hash we use the following method (via Stackoverflow).

Code to flatten a multidimensional hash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
module Enumerable
  def flatten_with_path(parent_prefix = nil)
    res = {}

    self.each_with_index do |elem, i|
      if elem.is_a?(Array)
        k, v = elem
      else
        k, v = i, elem
      end
      # assign key name for result hash
      key = parent_prefix ? "#{parent_prefix}.#{k}" : k
      if v.is_a? Enumerable
        # recursive call to flatten child elements
        res.merge!(v.flatten_with_path(key))
      else
        res[key] = v
      end
    end

    res
  end
end

We write the data to a CSV file to complete our successful extraction of running data from NikePlus!

Write to CSV file
1
2
3
4
5
CSV.open("out.csv", "w") do |csv|
  data['activities'].each do |activity|
    csv << activity.flatten_with_path.values
  end
end

Export your own Nike+ Data using this demo.

Fork the code on Github

Measuring the Length of an NSBezierPath

Neither NSBezierPath nor its iOS cousin, UIBezierPath offer a method to find the length of a particular curve. I’m going to show you an easy way to quickly approximate the length of a curve stored in an NSBezierPath object.

Grab the code from this gist, or read on if you’re interested in the details.

NSBezierPath+Length.h
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
//
//  NSBezierPath+Length.h
//
//  Created by mcianni on 10/14/12.
//  Copyright (c) 2012 mcianni. No rights reserved.
//

#import <Cocoa/Cocoa.h>

@interface NSBezierPath (Length)
- (double)length;
@end

@implementation NSBezierPath (Length)

- (double)length
{
    NSBezierPath *flattened_path = [self bezierPathByFlatteningPath];
    int segments = (int)flattened_path.elementCount;
    NSPoint last_point;
    NSPoint point;
    double length;

    for (int i=0; i<segments; i++) {
        /*
         * Normally, elementAtIndex:associatedPoints: would set the variable 
         * passed to associatedPoints as an array of NSPoints with max 
         * size 3. However, since we've called bezierPathByFlatteningPath, 
         * we can assume we'll always get a single point. The other points 
         * are normally used as control points for bezier curves.
         */
        NSBezierPathElement e = [flattened_path elementAtIndex:i
                                              associatedPoints:&point];

        // e == 0 is a moveToPoint command
        if (e == 0) {
            last_point = point;
        }
        else {
            // distance formula
            double distance = sqrt(pow(point.x - last_point.x, 2) +
                                   pow(point.y - last_point.y, 2));
            length += distance;
            last_point = point;
        }

    }
    return length;
}
@end

The secret to this method is the use bezierPathByFlatteningPath on the curve we wish to measure. Once we’ve created a new NSBezierPath (or UIBezierPath) it’s easy to iterate through the path elements. As we go along we’ll check if we should moveToPoint or lineToPoint; if it’s the latter we’ll use the distance formula to calculate the difference between the current and the last point which we then add to the total.

Importantly, this is an approximation which depends on the precision of the flat line approximations of bezierPathByFlatteningPath. Apple documentation explains:

Flattening a path converts all curved line segments into straight line approximations. The granularity of the approximations is controlled by the path’s current flatness value, which is set using the setDefaultFlatness: method.

You can increase the granularity, if necessary, but if you require exact measurements you’re going to need to integrate each curve of the NSBezierPath.

Graphing ESPN.com’s MLB Power Ranking With Ruby

During the baseball season ESPN.com publishes a weekly ranking of all 30 teams in Major League Baseball. I’m going to show you how to scrape those rankings using Nokogiri and generate a graph using the Google Charts Javascript Library so we can get an overview of who is rising, who is falling, who is managing to hold the top positions and who is struggling to move up from the bottom positions.

First lets take a look at the HTML markup for the ESPN Power Rankings.

Select week markup

There’s a select box listing all weeks up to and including the current week for which they have published rankings. That’s where we’ll grab the urls to each weeks ranking.

Once we have the URLs to each weeks rankings we can iterate through the list, requesting each page, parsing it for the information we’re looking for (only rank and team for now). We’ll sleep after parsing each page so as to be polite. First things first, lets look at the markup for a ranking page.

Rankings markup

We’re interested in div.mod-content, specifically the table inside. The only rows we’re interested in have a class of oddrow or evenrow, so we’ll skip rows not matching that criteria. We’ll need to grab the rank and the team name. The rank will be in the first td inside the tr, we can get the team name using:

1
team = tr.search('td')[1].search('a')[1].text

Our complete code resembles the following.

ESPN MLB Power Rankings - espn_mlb_power_rankings.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
BASE_URL = "http://espn.go.com/mlb/powerrankings/_/week"
doc = Nokogiri::HTML(open(BASE_URL))
number_of_weeks = doc.css("div.mod-article-title select option")
                     .select{ |option|
                        option.text.to_i.to_s == option.text
                     }
                     .count

rankings = {} #will be a 2D array, first by week, then by rank

(1..number_of_weeks).each_with_index do |week|
  doc = Nokogiri::HTML(open("#{BASE_URL}/#{week}"))
  doc.css("div.mod-content table tr").each do |tr|
    next if tr['class'] != 'oddrow' and  # skip colhead
            tr['class'] != 'evenrow'     #
    rank = tr.search('td')[0].text.to_i
    team = tr.search('td')[1].search('a')[1].text
    (rankings[team] ||= []) << rank
  end
  sleep(rand(10))
end

Well that’s quite… busy. Let’s graph the rankings of the current top 10 teams.

Much better. At the end of the season we’ll update this post and take a look the ups and downs of the champions season.

Loading the Content for a Bootstrap Popover via AJAX

How to dynamically load content for a Twitter Bootstrap Popover.

A quick example follows below. If you want the full Ruby on Rails/Coffeescript example, skip this code and begin reading below.

Dynamically Loading Content for a Twitter Bootstrap Popover
1
2
3
4
5
6
7
8
9
10
11
12
13
$(document).ready(function() {
  $(".popover-trigger").click(function() {
    el = $(this);
    $.get("/path/to/resource", function(response) {
      el.unbind('click').popover({
        content: response,
        title: 'Dynamic response!',
        html: true,
        delay: {show: 500, hide: 100}
      }).popover('show');
    });
  });
});

Full Twitter Bootstrap/Ruby on Rails/Coffeescript Example

Let’s assume the following situation. We have a Ruby on Rails website that collects baseball player data. We have an index page that lists all the players and that links to individual pages for each player. Those individual pages have full season stats for the player. On the index page, we would like for the user to be able to quickly view a players recent stats without leaving the page. We don’t want to load all this data for every player whenever a user visits this page. Our solution is to use jQuery to make an asynchronous request when the user hovers over a “More Info” link that will appear next to each player’s name.

Our views and controller are moderately basic.

Lets say that we have an action in our controller that looks like this:

PlayersController
1
2
3
4
5
6
7
8
9
10
11
12
class PlayersController < ApplicationController
  before_filter :load_player

  def load_player
    @player = Player.find(params[:id])
  end

  def recent_stats
    @stats = @player.game_stats.last(5)
    render :layout => nil
  end
end

And that last_5_games responds with a view like:

last_5_games.js.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<%= content_tag :h4, @player.name %>

<div style="display:none;">
  <div class="player-name">Recent Stats <%= link_to @player %></div>
</div>

<table>
  <tr>
    <th></th>
    <th>Value</th>
  </tr>

  <% @stats.each do |stat| %>
  <tr>
    <td><%= stat.name %></td>
    <td><%= stat.value %></td>
  </tr>
  <% end %>
</table>

And that our view looks like:

index.html.erb
1
2
3
4
5
6
7
8
<table>
<% @player.each do |player| %>
  <tr>
    <td><%= player.name %></td>
    <td><%= link_to 'More Info', '#', :class => 'more-info' %></td>
  </tr>
<% end %>
</table>

Then our coffeescript will look like this:

Binding our handlers
1
2
3
4
5
6
7
8
9
10
11
$(".more-info").each (index) ->
  $(this).bind 'hover', ->
    el = $(this)
    player_id = $(this).attr('data-player-id')
    callback = (response) ->
      el.unbind('hover').popover({
        content: response,
        title: $(response).find(".player-name"),
        delay: {show: 500, hide: 100}
      }).popover('show')
    $.get("/players/#{player_id}/recent_stats", '', callback, '')

Some things to note:

  • Our request returned an html snippet instead of JSON. If we wanted to use JSON we would have formatted the response before setting it to content in our callback.
  • Part of our response is a hidden div with the player name and a link to another page. This lets us set the title of the popover when we get a response.

System Wide .gitignore

Adding *.DS_Store and *.swp to the .gitignore file in every project gets tedious, but it’s an easy fix.

  • Edit ~/.gitconfig and add the following line: [core] excludesfile = ~/.gitignore
  • Create ~/.gitignore and add excludes as necessary!