Archive for the ‘Ruby’ Category
Continous Integration with RunCodeRun
Posted by Giovanni Intini | Filed under Ruby, Ruby on Rails, mikamai
Last thursday MIKAMAI hosted a Ruby Social Club meeting. Here’s the slides for my presentation.
Updated Language Redirect Extension for Radiant
Posted by Giovanni Intini | Filed under Radiant, Ruby
Thanks to the great work of netzpirat, the good old Language Redirect Extension has been updated to work with Radiant 0.8.0.
Thanks netzpirat!
Apache Vhost Templating
Posted by Giovanni Intini | Filed under Ruby
In Mikamai our deployment platform of choice is Ubuntu Linux. I like a lot the way Apache is set up on Debian based distributions, with the sites-available directory, but nonetheless creating new virtual hosts is a royal PITA.
Today I finally solved the problem once and for all via a super simple ruby templating script. Here it is, it uses a nice gem, optiflags, to parse the commandline arguments:
#!/usr/bin/env ruby require 'rubygems' require 'optiflag' module MyOptions extend OptiFlagSet flag "d" do description "The domain name the vhost should serve" long_form "domain" end optional_flag "a" do description "Email of the admin. If not specified defaults to info@domain" long_form "admin" end optional_switch_flag "w" do description "Adds www to non www redirection" long_form "www_redirect" end and_process! end flags = MyOptions.flags admin = flags.a ? flags.a : "info@#{flags.d}" domain = flags.d quoted_domain = flags.d.gsub(/\./, "\\.") TEMPLATE=<<-EOT <VirtualHost *:80> ServerName #{domain} ServerAdmin #{admin} DocumentRoot /var/apps/#{domain} <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/apps/#{domain}> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/#{domain}.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/#{domain}.log combined </VirtualHost> EOT REDIRECTION=<<-EOT <VirtualHost *:80> ServerName www.#{domain} ServerAdmin #{admin} RewriteEngine On RewriteCond %{HTTP_HOST} ^www\\.#{quoted_domain} RewriteRule (.*) http://#{domain}/$1 [R=301,L] </VirtualHost> EOT puts TEMPLATE puts REDIRECTION if flags.w?
I use it like this:
$ vhgen -d domain.com -w > /etc/apache2/sites-available/my_vhost $ a2ensite my_vhost
The unrestful programmer
Posted by Giovanni Intini | Filed under Programming, Ruby
Yesterday in Mikamai we had a Ruby Social Club meeting. I did a small presentation about the need to never stop learning. Here’s the slides:
Combinatorics for fun and profit
Posted by Giovanni Intini | Filed under Ruby
During my programming for fun moments I tend to always encounter a problem that needs to find combinations without repetitions for a set of data (numbers, objects, strings, letters, ...).
I have never been able to solve that problem in a way that satisfied me, and the languages I used didn’t have libraries for combinatorics that had a way to generate combinations without repetitions.
Yesterday a good friend of mine introduced me to the Set Puzzle, and, like I did for Word Challenge I had to try and attack the problem from a programming angle.
To do that I had to find once and for all a reusable way to generate combinations without repetitions. I did this time, and SetSolver was born.
You can check it out on GitHub, but I can’t help myself and not post the combinatorics code on this blog
module ArrayExtensions def combinations_without_repetitions(k) combine(self, k) end private def combine(array, k) return [array] if k == array.size return array.collect {|e| [e]} if k == 1 results = [] array[0..(array.size - k)].each_with_index do |val, idx| results += combine(array[(idx+1)..-1], k - 1).collect {|e| [val, e].flatten} end results end end Array.class_eval do include ArrayExtensions end
Word Cheat, a simple anagram engine
Posted by Giovanni Intini | Filed under Ruby
Those of you who follow this blog since its inception will be aware of my obsession with word games, and specifically algorithms that solve word games.
Yesterday I was playing Word Challenge on Facebook, a simple game where you have to find all the words that can be composed with six random letters. After a couple of trials I decided to try and solve it with a ruby program, but since I didn’t want to use my previous code I decided to try a new approach to solve this problem.
First of all I took an extensive list of Italian words and filtered it to suit my needs, leaving only the words ranging from three to six letters.
Once I had my wordlist I had to build a data structure that could be used for fast anagrams retrieval, thus the WordHash class was born.
class WordHash attr_reader :words def initialize(wordlist) @words = {} wordlist.each do |word| if @words.has_key?(w = word.strip.signature) @words[w] << word.strip else @words[w] = [word.strip] end end end def get_anagrams(word) result = [] wordlist = @words.each do |k,w| result << w if word.contains?(k) end result end end
This class makes use of two helper methods I added to String, that I think should be really part of the Ruby standard library:
module StringExtensions def signature self.split('').sort.join end def contains?(search) !Regexp.new(search.signature.split("").join(".*")).match(self.signature).nil? end end String.class_eval do include StringExtensions end
The get_anagrams method is used to also get anagrams with a length less than that of the original word.
The whole project, complete with tests and a helper script is available on GitHub, feel free to contribute.
Also, big thanks to Stefano Cobianchi, who contributed with the contains? method, that I really couldn’t code
Two improvements to your Capfiles
Posted by Giovanni Intini | Filed under Capistrano, Drupal, Ruby
I have lately started using a pattern that has become quite common among capistrano users: setting the server names and locations in a task. Doing this allows you to have multiple deployment environments, like development, staging, production, and so on.
desc "deploy to development environment" task :development do set :deploy_to, "/var/apps/#{application}" role :web, "servername.mikamai.com", :primary => true role :app, "servername.mikamai.com", :primary => true role :db, "servername.mikamai.com", :primary => true set :user, "username" set :password, "secr3t" set :remote_mysqldump, "/usr/bin/mysqldump" set :db_user, "username" set :db_password, "secre7" set :db_name, "db_name" end
This technique has proven itself to be really useful, especially when clients start to ask for deployments on their test servers, and you still want to be able to deploy to your development servers.
While refactoring my Capfiles I also took the time to rewrite the drupal:db namespace, adding the much needed tasks that allow you dump the remote databases and download them to your development box.
namespace :db do namespace :dump do desc "Deletes old database dumps, leaves only the latest on the server" task :cleanup, :roles => :db do dumps = capture("ls -xt #{shared_path}/dumps").split.reverse run "cd #{shared_path}/dumps; rm #{dumps[0..-2].join(" ")}" end desc "Dumps the local database" task :local, :roles => :db do raise RuntimeError.new("failed dump") unless system "#{local_mysqldump} -u #{local_db_user} --password=#{local_db_password} #{local_db_name} > dump.sql" end namespace :remote do desc "Dumps the remote database" task :default, :roles => :db do filename = "#{Time.now.to_i.to_s}.dump.sql" run "cd #{shared_path}/dumps; #{remote_mysqldump} -u #{db_user} --password=#{db_password} #{db_name} > #{filename}" run "cd #{shared_path}/dumps; bzip2 #{filename}" end namespace :download do desc "Dumps and downloads the remote database" task :default do drupal::db::dump::remote::default latest end desc "Downloads the latest database dump" task :latest, :roles => :db do dumps = capture("ls -xt #{shared_path}/dumps").split.reverse get("#{shared_path}/dumps/#{dumps.last}", "./#{dumps.last}") end end end end
Legacy Path Handler, a Radiant Extension
Posted by Giovanni Intini | Filed under Radiant, Rails, Ruby, Ruby on Rails
We’re preparing to deploy the new Mikamai site (not up at the time of this post), that runs on the wonderful Rails-based RadiantCMS.
The VPS we’re deploying to runs on Phusion Passenger, and that means we can’t use mod_alias or mod_rewrite to 301-redirect the old URLs, already indexed by Google, to their new locations.
To solve this problem I wrote a little Radiant Extension, called LegacyPathHandler, that reads a simple list of URLs from a text file and does a 301 redirection on them before handling the control to Radiant’s default SiteController.
It works quite fine for us, but it has no specs/tests or documentation. Please feel free to contribute to the project if you feel you can improve it.
Deploying Drupal with Capistrano
Posted by Giovanni Intini | Filed under Capistrano, Drupal, PHP, Ruby
Mikamai, the company I work for, has just released Montalbano.tv, the companion site to one of the most successful TV shows in Italy.
I was the technical director of this Drupal based project, and while I was happy we chose Drupal, because it allowed us to deliver all the features they needed on time, I almost panicked when they told us the production setup would have two servers, both with database and web serving duties.
The database replication was standard MySql master-master setup, but I had to develop a strategy to keep the two code-bases on the two servers synchronized.
Being a Ruby programmer at heart, I selected the only tool that never fails me in circumstances like the one we had: Capistrano.
Unfortunately, while Capistrano is all easy to use with Rails, I had to write a custom Drupal-tailored Capfile.
Here it is, in its entirety, in case you ever need to deploy Drupal with cap (now I always deploy Drupal with cap, since I have the recipe ready
):
load 'deploy' if respond_to?(:namespace) # cap2 differentiator # Standard configuration set :user, "username" set :password, "password" set :application, "application.name" # I like to deploy the code in /var/apps # and then link it to the webserver directory set :deploy_to, "/var/apps/#{application}" # SCM Stuff configure to taste, just remember the repository # here I used github as main repository set :repository, "git@github.com:username/project.git" set :scm, :git set :branch, "master" set :repository_cache, "git_master" set :deploy_via, :remote_cache set :scm_verbose, true # Two servers, double fun # You really don't need app, web and db here, # but I used all of them just to be sure. # Usually only web is ok. role :app, "first.server.address.com" role :app, "second.server.address.com", :primary => true role :web, "first.server.address.com" role :web, "second.server.address.com", :primary => true role :db, "first.server.address.com" role :db, "second.server.address.com", :primary => true after 'deploy:setup', 'drupal:setup' # Here we setup the shared files directory after 'deploy:symlink', 'drupal:symlink' # After symlinking the code we symlink the shared dirs # Before restarting the webserver we fix all the # permissions and then symlink it to production before 'deploy:restart', 'mikamai:permissions:fix', 'mikamai:symlink:application' namespace :drupal do # shared directories task :setup, :except => { :no_release => true } do sudo "mkdir -p #{shared_path}/files" sudo "chown -R #{user}:#{user} #{deploy_to}" end # symlink shared directories task :symlink, :except => { :no_release => true } do sudo "ln -s #{shared_path}/files #{latest_release}" end end namespace :deploy do # adjusted finalize_update, removed non rails stuff task :finalize_update, :except => { :no_release => true } do sudo "chmod -R g+w #{latest_release}" if fetch(:group_writable, true) end task :restart do # nothing to do here since we're on mod-php end end namespace :mikamai do # symlinking to production namespace :symlink do task :application, :except => { :no_release => true } do sudo "rm -rf /var/www/montalbano" sudo "ln -s #{latest_release} /var/www/montalbano" end end # change ownership namespace :permissions do task :fix, :except => { :no_release => true } do sudo "chown -R www-data:www-data #{latest_release}" end end end
How I made Autotest RedGreen and Growl party together
Posted by Giovanni Intini | Filed under Programming, Ruby, Ruby on Rails, Testing
If you’re like me and like autotest as a sort of private continous integration system, and if you like pretty output, you will probably have used some kind of autotest + redgreen and/or growl.
Unfortunately I had strange quirks in my growl notifications. Sometimes they worked, sometimes they didn’t (I was using the standard :ran_command autotest hook at the time), so I switched to the newer :red and :green hooks, and only worsened the problem. No matter what the test output was, I always got a green notification.
I was able to trace the problem to the redgreen gem. Its colorized output wasn’t being recognized correctly by autotest and it kept thinking everything was ok.
A little ruby fiddling and this is my new improved (and working!) .autotest file:
# -*- ruby -*- module Autotest::RedGreen Autotest.send(:alias_method, :real_ruby, :ruby) Autotest.send(:define_method, :ruby) do |*args| real_ruby + %[ -rrubygems -e "require 'redgreen'" ] end # Clean the output so other modules can work correctly Autotest.add_hook :ran_command do |at| at.results.each do |r| r.gsub!("\033[31m", "") r.gsub!("\033[32m", "") r.gsub!("\033[33m", "") r.gsub!("\033[0m", "") end end end module Autotest::Growl AUTOTEST_IMAGE_ROOT = "~/.autotest_images" def self.growl(title, msg, img, pri=0, sticky="") system "growlnotify -n autotest --image #{img} -p #{pri} -m '#{msg.inspect} #{title}' #{sticky}" end Autotest.add_hook :red do |at| growl("FAIL", "#{get_results(at)}", "#{AUTOTEST_IMAGE_ROOT}/fail.png", 2) end Autotest.add_hook :green do |at| growl("Pass", "#{get_results(at)}", "#{AUTOTEST_IMAGE_ROOT}/pass.png") end private def self.get_results(at) results = [at.results].flatten.join("\n") if results.include? 'tests' output = results.slice(/(\d+)\s+tests?,\s*(\d+)\s+assertions?,\s*(\d+)\s+failures?(,\s*(\d+)\s+errors)?/) else output = results.slice(/(\d+)\s+examples?,\s*(\d+)\s+failures?(,\s*(\d+)\s+not implemented)?/) end output end end # Esclusioni Autotest.add_hook :initialize do |at| %w{.hg .git .svn stories tmtags Rakefile Capfile README spec/spec.opts spec/rcov.opts vendor/gems autotest svn-commit .DS_Store }.each do |exception| at.add_exception(exception) end at.add_mapping(/spec\/defaults.rb/) do |f, _| at.files_matching %r%^spec/(controllers|helpers|lib|models|views)/.*\.rb$% end end
Our setup had apache as webserver so I don’t need any special code to restart the application. In addition to this recipe, made to deploy to production I also wrote a couple of tasks that shine during development.
Drupal has the problem of storing most of the important stuff in the database, and if you do local development, showing the status of the work to your clients can be a chore. The following tasks allow you to easily dump the development db and import the dump to the staging server to show your progresses to your customers and allow testing the site:
# Callbacks before 'deploy:start', 'drupal:db:import:production' before 'deploy:restart', 'mikamai:permissions:fix', 'mikamai:production:symlink', 'drupal:configure:production' before 'deploy:start', 'mikamai:permissions:fix', 'mikamai:production:symlink', 'drupal:configure:production' before 'deploy:cold', 'drupal:db:dump:development' # DB Stuff set :mysqldump, "/opt/local/bin/mysqldump5" # your path to mysqldump # local db credentials set :local_db_user, "root" set :local_db_password, "" set :local_db_name, "database" # remote db credentials set :db_user, "user" set :db_password, "secret" set :db_name, "database" namespace :drupal do namespace :configure do task :production do sudo "cp #{latest_release}/sites/default/settings.production.php #{latest_release}/sites/default/settings.php" end task :development do sudo "cp #{latest_release}/sites/default/settings.development.php #{latest_release}/sites/default/settings.php" end end namespace :db do namespace :dump do task :development do raise RuntimeError.new("failed dump") unless system "#{mysqldump} -u #{local_db_user} --password=#{local_db_password} #{local_db_name} > dump.sql" end end namespace :import do task :production do ENV["FILES"] = "dump.sql" deploy::upload run "mysql -u #{db_user} --password=#{db_password} #{db_name} < #{latest_release}/dump.sql" end end end end