programming is hard

Managing Memcache with Capistrano

Capistrano rocks for deployment. If you haven't ever used it to deploy an app, you're seriously missing out. There's a lot of good recipes floating around online and this stuff changes so often that there’s probably something out there that’s better. However I figured I'd share how I'm currently managing our memcache daemons at work.
memcached, rails wtf?



We are using defunkt's cache_fu plugin right now to cache models in our system. It's pretty straightforward to use; an acts_as class method, and a config/memcached.yml file. The memcached.yml file lets you provide one or more servers that the memcache clients can connect to. Unfortunately there's no centralized way to manage those memcache daemons on your deploy hosts. With our good friend cap, some ruby, and a little time invested we can roll our own solution.
Whatcha Want?



The requirements for this task are dead simple.



use cap to get out to the remote hosts
the memcache daemon list comes from cache_fu’s yml file
write a script to start, stop, restart, get the status of, and commit mass genocide against(I want all processes kill -9'd right now).


It turned out to be a pretty simple hack; parse the yaml, get the host's ip address, see if the host is in the server list, if it is do whatever command line operation we were given. Check it out.

#!/usr/bin/env ruby
# this goes in your script/ directory
# it parses your memcached.yml file and hooks you up w/ some info
# it keeps you from having to mess w/ stale memcached daemons for whatever reason.
require 'yaml'

class MemcachedCtl
  attr_accessor :memcached, :memory, :pids, :servers, :ip_address, :ethernet_device
  def initialize
    env = ENV['RAILS_ENV'] || 'development'
    self.memcached = `which memcached`.chomp
    self.servers = [ ]
    self.pids    = { }
    self.ethernet_device = ENV['ETH'] || 'eth0'
    self.ip_address = get_ip_address || '0.0.0.0'
    self.memory = '128'
    
    config = YAML.load_file(File.expand_path(File.dirname(__FILE__) + "/../config/memcached.yml"))
    self.servers = [ config['defaults']['servers'] ].flatten rescue ['127.0.0.1:11211']
    self.servers = [ config[env]['servers'] ].flatten if config[env]['servers']
    self.servers.reject! { |server| host,port = server.split(/:/); self.ip_address == host }
    self.memory = config[env]['memory'] unless config[env]['memory'].nil?
    
    each_server do |host,port|
      `ps auwwx | grep memcached | grep '\\-l #{ip_address} \\-p #{port}' | grep -v grep`.split(/\n/).each do |line|
        self.pids[port] = line.split(/\s+/)[1]
      end
      self.pids[port] ||= 'Down'
    end
  end
  
  def execute(cmd)
    send(cmd)
  end
  
  def restart; stop; sleep 1; start; end
  
  def status
    each_server { |host,port| puts "Port #{port} -> #{pids[port] =~ /\d+/ ? 'Up' : 'Down'}" }
  end
  
  def kill
    each_server { |host,port| `kill -9 #{pids[port]} > /dev/null 2>&1` if pids[port] =~ /\d+/ }
  end
  
  def stop; kill; end
  
  def start
    each_server do |host,port|
      `#{memcached} -d -m #{memory} -l #{ip_address} -p #{port}`
      STDERR.puts "Try memcached_ctl status" unless $? == 0
    end
  end
    
  protected
  def each_server(&block)
    servers.each do |server|
      host,port = server.split(/:/)
      yield host, port
    end
  end
  
  def get_ip_address # this works on linux you might have to tweak this on other oses
    line = `/sbin/ifconfig #{ethernet_device} | grep inet | grep -v inet6`.chomp
    if line =~ /\s*inet addr:((\d+\.){3}\d+)\s+.*/
      self.ip_address = $1
    end
  end
end
###########################################################################
cmd = ARGV.shift
unless cmd.nil?
  MemcachedCtl.new.execute(cmd)
end

Get Your Cap On

So I named the script memcached_ctl and added it to my script/ directory in svn and redeployed. Now I can start poppin caps in our memcache daemons. If I throw the following code at the bottom of my config/deploy.rb I can see what’s up with my memcache daemons on my production hosts. If I were smart/motivated I’d make a gem and let you good people include these tasks, you know like you should be doing for mongrel_cluster.

# ====================================
# Memcached Server TASKS
# ====================================
%w(start stop restart kill status).each do |cmd|
  desc "#{cmd} your memcached servers"
  task "memcached_#{cmd}".to_sym, :roles => :app do
    run "RAILS_ENV=production #{ruby} #{current_path}/script/memcached_ctl #{cmd}"
  end
end