Programming | A Tempest of Thoughts

Archive for the ‘Programming’ Category

BetterSoftware 2010: Agile Tricks

Posted by Giovanni Intini | Filed under Programming

This week I attended Better Software 2010, where I gave a presentation about the making of the DoomBoard . I had a lot of fun and attended a lot of interesting presentations from great professionals.

Here’s my slides.

Agile tricks

View more presentations from intinig.

The unrestful programmer

Posted by Giovanni Intini | Filed under Programming, Ruby

Yesterday in Mikamai we had a Ruby Social Club meeting. I did a small presentation about the need to never stop learning. Here’s the slides:

New Techs

View more presentations from intinig.

The thrill of a new technology: CouchDB

Posted by Giovanni Intini | Filed under Programming

It has been a long time since I fell in love with a new technology (maybe Ruby on Rails when it first came out?), and the time I am spending with CouchDB lately has been a nice return to the pleasure of programming.

I’ll probably write some in-depth articles on Document based DBs later, but let me show you how complex operations on relational DBs can become really simple with a document based approach.

Let’s say you have a db of tagged objects, and you want to get a report with a “per tag” distribution of your objects. In a relational environment you would need at least three tables, objects, tags and a join table and a select operation involving two joins to get the results you needed:

SELECT tags.name, group_concat(people.name) 
FROM tags 
INNER JOIN tags_people ON tags.id = tags_people.tag_id
INNER JOIN people ON people.id = tags_person.id 
GROUP BY tags.name

This isn’t rocket science but I guess not everyone knows about group_concat

In a document based environment I would just have a document for each person, with a tags property that holds an array of tags. The map and reduce functions would look like this:

// map
function(doc) {
  if (doc['couchrest-type'] == 'Person' && doc.tags) {
    doc.tags.forEach(function(tag) {
      emit(tag, doc.name);
    }
  }
}
 
// reduce
function(keys, values, rereduce) {
  return values;
}

Using Javascript to manipulate data may seem counter intuitive, but as you can see we have a much simpler datastore (just people documents thrown in the db) and an easy way of manipulating them to extract the info we need.

I love CouchDB.

How I made Autotest RedGreen and Growl party together

Posted by Giovanni Intini | Filed under Programming, Ruby, Ruby on Rails, Testing

If you’re like me and like autotest as a sort of private continous integration system, and if you like pretty output, you will probably have used some kind of autotest + redgreen and/or growl.

Unfortunately I had strange quirks in my growl notifications. Sometimes they worked, sometimes they didn’t (I was using the standard :ran_command autotest hook at the time), so I switched to the newer :red and :green hooks, and only worsened the problem. No matter what the test output was, I always got a green notification.

I was able to trace the problem to the redgreen gem. Its colorized output wasn’t being recognized correctly by autotest and it kept thinking everything was ok.

A little ruby fiddling and this is my new improved (and working!) .autotest file:

# -*- ruby -*-
 
module Autotest::RedGreen
  Autotest.send(:alias_method, :real_ruby, :ruby)
  Autotest.send(:define_method, :ruby) do |*args|
      real_ruby + %[ -rrubygems -e "require 'redgreen'" ] 
  end
 
  # Clean the output so other modules can work correctly
  Autotest.add_hook :ran_command do |at|
    at.results.each do |r|
      r.gsub!("\033[31m", "")
      r.gsub!("\033[32m", "")
      r.gsub!("\033[33m", "")
      r.gsub!("\033[0m", "")
    end
  end
end
 
module Autotest::Growl
  AUTOTEST_IMAGE_ROOT = "~/.autotest_images"
 
  def self.growl(title, msg, img, pri=0, sticky="")
    system "growlnotify -n autotest --image #{img} -p #{pri} -m '#{msg.inspect} #{title}' #{sticky}"
  end
 
  Autotest.add_hook :red do |at|
    growl("FAIL", "#{get_results(at)}", "#{AUTOTEST_IMAGE_ROOT}/fail.png", 2)
  end
 
  Autotest.add_hook :green do |at|
    growl("Pass", "#{get_results(at)}", "#{AUTOTEST_IMAGE_ROOT}/pass.png")
  end
 
  private
  def self.get_results(at)
    results = [at.results].flatten.join("\n")
 
    if results.include? 'tests'
      output = results.slice(/(\d+)\s+tests?,\s*(\d+)\s+assertions?,\s*(\d+)\s+failures?(,\s*(\d+)\s+errors)?/)
    else
      output = results.slice(/(\d+)\s+examples?,\s*(\d+)\s+failures?(,\s*(\d+)\s+not implemented)?/)
    end
    output
  end
end
 
# Esclusioni
Autotest.add_hook :initialize do |at|
  %w{.hg .git .svn stories tmtags Rakefile Capfile README spec/spec.opts spec/rcov.opts vendor/gems autotest svn-commit .DS_Store }.each do |exception|
    at.add_exception(exception)
  end
 
  at.add_mapping(/spec\/defaults.rb/) do |f, _|
    at.files_matching %r%^spec/(controllers|helpers|lib|models|views)/.*\.rb$%
  end
end

Our setup had apache as webserver so I don’t need any special code to restart the application. In addition to this recipe, made to deploy to production I also wrote a couple of tasks that shine during development.

Drupal has the problem of storing most of the important stuff in the database, and if you do local development, showing the status of the work to your clients can be a chore. The following tasks allow you to easily dump the development db and import the dump to the staging server to show your progresses to your customers and allow testing the site:

# Callbacks
before 'deploy:start', 'drupal:db:import:production'
 
before 'deploy:restart', 'mikamai:permissions:fix', 'mikamai:production:symlink', 'drupal:configure:production'
before 'deploy:start', 'mikamai:permissions:fix', 'mikamai:production:symlink', 'drupal:configure:production'
before 'deploy:cold', 'drupal:db:dump:development'
 
# DB Stuff
 
set :mysqldump, "/opt/local/bin/mysqldump5" # your path to mysqldump
# local db credentials
set :local_db_user, "root"
set :local_db_password, ""
set :local_db_name, "database"
# remote db credentials
set :db_user, "user"
set :db_password, "secret"
set :db_name, "database"
 
namespace :drupal do
 
  namespace :configure do
    task :production do
      sudo "cp #{latest_release}/sites/default/settings.production.php #{latest_release}/sites/default/settings.php"
    end
 
    task :development do
      sudo "cp #{latest_release}/sites/default/settings.development.php #{latest_release}/sites/default/settings.php"
    end
  end
 
  namespace :db do
    namespace :dump do
      task :development do
        raise RuntimeError.new("failed dump") unless system "#{mysqldump} -u #{local_db_user} --password=#{local_db_password} #{local_db_name} > dump.sql"
      end
    end
 
    namespace :import do
      task :production do
        ENV["FILES"] = "dump.sql"
        deploy::upload
        run "mysql -u #{db_user} --password=#{db_password} #{db_name} < #{latest_release}/dump.sql"
      end
    end
  end
end

How I worked with thirty amazing people and lived to tell: Stacktrace

Posted by Giovanni Intini | Filed under Italy, Programming, Python, Random Stuff, Ruby, Ruby on Rails, Stacktrace

I’ve had the pleasure of working with thirty brilliant techies for the launch of Stacktrace, an Italian website about technology in its various forms.

Stacktrace was and idea of Antonio Cangiano. He felt that Italy missed an authoritative and original technology reference, and contacted about thirty of the best Italy has to offer, people I am very proud I could work with, to prepare the lightning quick launch of a wonderful site, full of original, well written and interesting content.

If you can understand Italian, even a bit of it, I strongly suggest you check Stacktrace. You’ll be glad you did.

A couple of capistrano 2 recipes libraries

Posted by Giovanni Intini | Filed under Capistrano, Nimboo, Programming, Rails, Ruby, Ruby on Rails

I kept playing with Capistrano 2 after my last article, and I’ve refactored quite a bit my recipes, finally moving them in their own subversion repository. This allows much quicker deployment with my new rails applications. Here’s how I do it:

$ rails my_new_application
$ cd my_new_application
$ capify .

Then I edit Capfile:

load 'deploy' if respond_to?(:namespace) # cap2 differentiator
 
load 'lib/recipes/site5' # This is my site5 recipe
load 'lib/recipes/medlar' # The general use recipes
 
load 'config/deploy'

The site5 and medlar namespaces hold default configuration values, define some callbacks and the following tasks:

cap deploy:medlar:rails:freezer:edge # Fetch Rails edge and puts it into sh... cap deploy:medlar:rails:freezer:stable # Fetch Rails stable and puts it into ... cap deploy:medlar:rails:link # Links Rails to application/vendor cap deploy:medlar:rails:update # Updates the fetched version of rails.

cap deploy:site5:kill_dispatch_fcgi # Kills Ruby instances on Site5
cap deploy:site5:link_public_html # Links public_html to current_path/pu...

Last but not least, here’s the simple, clean and elegant deploy.rb:

set :application, "my_new_application"
set :user, "the_username"
 
set :repository,  "repo_address"
set :deploy_to, "/home/#{user}/apps/#{application}"
 
role :app, "server.com"
role :web, "server.com"
role :db,  "server.com", :primary => true

Quite readable, isn’t it?

The recipes are available via anonymous subversion: https://svn1.hosted-projects.com/medlar/recipes/

Enjoy and let me know if you found them useful.

Rails and JavaScript: page.call gotchas

Posted by Giovanni Intini | Filed under Ajax, Javascript, Programming, Prototype.js, Rails, Ruby, Ruby on Rails

Ruby on Rails has wonderful out of the box javascript support, but sometimes implementing dynamic user interfaces is not so easy as it seems.

In an application I’m working on I have a list of people with a checkbox each. In the load event of the page I add a click handler to every checkbox using this javascript code:

  $$('.ConfermaInvitati').each(function(element) {
    Event.observe(element, 'click', clickHandler);
  });
 
// ... 
 
function clickHandler(event) {
  var e = Event.element(event);
  new Ajax.Updater(e.up(), 'my/invited/toggle', {
    parameters: { id: e.up().up().id },
    onLoading: function() { e.src = "/images/admin/spinner.gif"; },
    onSuccess: function() { new Ajax.Request('my/refresh', {}) }
  });  
}

This works fine until I add a new person via an Ajax call. That person won’t have a clickHandler because the element wasn’t on the page when I called the click handler. So I thought it was time to test page.call in the render :update block I had in the rails application.

I tried this code:

render :update do |page|
  # Do stuff that creates the new objects and adds it to the page
  # The data I need is in @invited
  page.call("Event.observe($$('##{@invited.permalink} .ConfermaInvitati').first()), 'click', clickHandler)")

Obviously that didn’t work, and it turned out I have to read documentation before doing fancy things

The rails docs told me that I had to use call passing the function name as the first argument and an array of parameters as the second argument, the problem is that call turns all the parameters into strings—this means I could not pass the clickHandler function to Event.observe.

I found the solution in <<. If you do page << "foo", foo will be evaluated as raw javascripts. This meant I was able to do

page << "Event.observe($$('##{@invited.permalink} .ConfermaInvitati').first()), 'click', clickHandler)

and finally have the functionality I was looking for. So remember, don’t page.call if you need to pass javascript variables to your functions.

Sudoless Rails Stack on OSX

Posted by Giovanni Intini | Filed under Apple, Mac, Productivity, Programming, Rails, Ruby, Ruby on Rails

More and more of my developer friends are switching to OSX as time goes by, and they keep asking me for directions on the best setup for rails development: how to install ruby, how to install the missing libraries, and so on.

My professional development life started on a PB Titanium running OSX 10.2, and continued through a PB Aluminium and a Macbook Pro. Each time I changed laptop I also reinstalled everything again, and each time I tried to come up with a better setup.

Now I’ve finally found a setup I’m comfortable with, and it has the following advantages:

Sudoless: everything runs from my user directory
Non-system-tampering: doesn’t touch files in the original osx installation
Crash-proof-easy-reinstall: you can just delete everything and reinstall without fear of rendering your system unstable
Fink based: uses everything it can use from the fink repositories

Now that I’ve sold you on my setup it’s time to explain how to implement it.

Step one: XCode and Developer Tools

If you don’t have XCode already installed you can install it from the disks you got with your Mac or, better, download the newest version from Apple Developer Connection. Once you got it installed you can proceed to the next step.

Step two: Fink

Go to the Fink Download Page, and get the package that works best on your Mac (intel or powerpc). Follow the installation instructions and install the base fink system. After you’ve done that add these lines to your _/.profile

# Dev Enviroment
 
LDFLAGS=-L/sw/lib
export LDFLAGS
 
CPPFLAGS='-I/Users/your_username/unix/include -I/sw/include'
export CPPFLAGS

_/Users/your_username/unix/include_ doesn’t exist yet, but we’ll create it when installing ruby.

Step three: Ruby from Sources

This is the first tricky part. Before OSX 10.4.6 the Ruby version shipped by Apple didn’t work with rails, so you were on your own. Now it works, but I prefer to have ruby in my home directory so I can mess with the sources and with the gem files and upgrade painlessly. So download the latest ruby sources and unpack them wherever you wish (I like /src).

Now you’re ready to go. First make sure you have readline and readline5-shlibs installed via fink so you can have a comfortable irb environment:

user$ fink install readline readline5-shlibs

After that it’s time for ruby:

user$ cd <sub>/src/ruby-1.8.6
user$ ./configure <del>-prefix=/Users/your_username/unix
user$ make && make install

The whole magic (and it’s not a big magic btw) is in -prefix. Installing ruby will create the /Users/your_username/unix path. Now it’ time to add the unix dir to $PATH. Edit /profile once again and add these lines:

PATH=/Users/your_username/unix/bin:/usr/local/mysql/bin:/usr/local/sbin:$PATH
export PATH

Step four: MySQL

Quite simple with the packages from mysql.com.

Step five: Last but not least, rubygems

Fetch the gem package from RubyForge and install it:

user$ cd src/unpacked-rubygems-directory
user$ ruby setup.rb

Step six: This is the real last step – Rails and a little hack

Now you’re free to install rails (and friends) using

user$ gem install rails --include-dependencies
user$ gem install mysql-ruby
user$ gem install capistrano --include-dependencies
user$ gem install mongrel --include-dependencies

The small hack I was talking about is a symlink:

user$ sudo mv /usr/bin/ruby /usr/bin/ruby-apple
user$ sudo ln -s /Users/your_username/unix/bin/ruby /usr/bin/ruby

This way applications that insist on using /usr/bin/ruby (TextMate’s RubyMate for example) will work fine.

Have fun with your self-made rails stack

Capistrano 2 Callbacks

Posted by Giovanni Intini | Filed under Capistrano, Programming, Rails, Ruby, Ruby on Rails

Converting the callbacks in my cap recipes to the new capistrano 2 format wasn’t as easy as I thought it would be. It turned out I had to use fully qualified task names in the after callback instead of the non-namespaced-names.

Here’s a sample of working callbacks:

namespace :deploy do
  after "deploy:setup", "deploy:sposivip:create_galleries", "deploy:sposivip:freeze_rails"
  after "deploy:update", "deploy:site5:link_public_html", "deploy:sposivip:link_rails", "deploy:sposivip:link_galleries"
end

Here I have two callbacks. The first one runs after setup, creating shared paths and freezing rails in the shared directory, so I avoid having a copy on rails in each release.

The second callback runs after deploy:update, so it will be called whenever I do a simple deploy or a cold deploy, it links back various directories in the shared path and it links the shared rails in vendor.

Capistrano 2 on Site5

Posted by Giovanni Intini | Filed under Capistrano, Programming, Rails, Ruby, Site5

I finally took the time to browse the capistrano 2 sources, and after reaching enlightenment I was able to write a deploy.rb file (yes I still use capify + deploy.rb instead of Capfile) that works really fine and really sweet on Site5.

Without further ado, I introduce you to deploy.rb extreme Site5 version

# Necessary to run on Site5
set :use_sudo, false
set :group_writable, false
 
# Less releases, less space wasted
set :keep_releases, 2
 
# The mandatory stuff
set :application, "YOUR_APP_NAME"
set :user, "SSH_USERNAME"
 
set :repository,  "URL_FOR_YOUR_REPOSITORY"
 
# SCM information
set :scm_username, "SCM_USERNAME"
set :scm_password, Proc.new { CLI.password_prompt "SVN Password: "}
 
# This is related to site5 too.
set :deploy_to, "/home/#{user}/apps/#{application}"
 
role :app, "SERVERNAME"
role :web, "SERVERNAME"
role :db,  "SERVERNAME", :primary => true
 
 
# In the deploy namespace we override some default tasks and we define
# the site5 namespace.
namespace :deploy do
  desc <<-DESC
    Deploys and starts a `cold' application. This is useful if you have not \
    deployed your application before, or if your application is (for some \
    other reason) not currently running. It will deploy the code, run any \
    pending migrations, and then instead of invoking `deploy:restart', it will \
    invoke `deploy:start' to fire up the application servers.
  DESC
  # NOTE: we kill public_html so be sure to have a backup or be ok with this application
  # being the default app for the domain.
  task :cold do
    update
    site5::link_public_html
    site5::kill_dispatch_fcgi
  end
 
  desc <<-DESC
    Site5 version of restart task.
  DESC
  task :restart do
    site5::kill_dispatch_fcgi
  end
 
  namespace :site5 do
    desc <<-DESC
      Links public_html to current_release/public
    DESC
    task :link_public_html do
      run "cd /home/#{user}; rm -rf public_html; ln -s #{current_path}/public ./public_html"
    end
 
    desc <<-DESC
      Kills Ruby instances on Site5
    DESC
    task :kill_dispatch_fcgi do
      run "skill -u #{user} -c ruby"
    end
  end
end

May your deploys be merry and bright and I wish you all your applications be white

« Older Entries

A Tempest of Thoughts