Automatic Application Deployment with Mina

By Jimmy Bonney | August 17, 2013

It has been a while I meant to write something about Mina, a “really fast deployer and server automation tool” as the team behind it describes it.

The concept behind Mina is to connect through SSH to your remote server and execute a set of Bash instructions that you define in a local deployment file (deploy.rb). There is only one SSH connection, making it faster than other tools that encapsulate each instruction inside their own SSH transaction.

Using such a tool limits the number of manual operations required to deploy a new version of the application on a remote server. All tasks to be executed are gathered in one place and this allows reducing the risk of forgetting something or typing the wrong command.

Mina homepage

Introduction

When deploying a new Rails application with Mina, a few manual operations are required:

Create the deployment file
Create a folder on the server
Run mina setup to create the folder structure
Edit the database.yml file to configure the DB, user and password
Run mina deploy to deploy the application

In this article, I’ll illustrate how to reduce the amount of manual work required in the steps 2 and 4 above. We’ll include tasks in Mina in order to create the folder remotely and to automatically populate the database.yml file as well as creating the database and user associated (we suppose that a MySQL database needs to be created but this is easily transposable to other DBMS).

Finally, we’ll add one additional task to deploy a new Apache Virtual file. Note that this task, as well as the creation of a new DB and user I discussed above require sudo rights on the remote host.

Mina Deployment File

The script below (a gist is also available) is a cleaned-up version of the deploy.rb file that I use to set up new hosts. It might therefore not be fully functional as-is but you should get the idea pretty easily.

require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
require 'mina/rvm'

# Usually mina focuses on deploying to one host and the deploy options are therefore simple.
# In our case, there is a number of possible servers to deploy to, it is therefore necessary to
# specify the host that we are targeting.
server = ENV['server']
# Since the same host can have multiple applications running in parallel, it is necessary to
# specify further which application we want to deploy
version = ENV['version']

# Set the repository (here on BitBucket)
set :repository, 'git@bitbucket.org:username/project.git'
# setting the term_mode to system disable the "pretty-print" but prevent some other issues
set :term_mode, :system

# Manually create these paths in shared/ (eg: shared/config/database.yml) in your server.
# They will be linked in the 'deploy:link_shared_paths' step.
set :shared_paths, ['config/database.yml', 'log']

# Optional SSH settings:
# SSH forward agent to ensure that credentials are passed through for git operations
set :forward_agent, true

##########################################################################
#
# Setup environment
#
##########################################################################

# This task is the environment that is loaded for most commands, such as
# `mina deploy` or `mina rake`.
task :environment do
  # Ensure that a server has been set
  unless server
    print_error "A server needs to be specified."
    exit
  end

  # Remote application folder
  set :deploy_to, "/home/username/project/#{version}"

  # Set the basic environment variables based on the server and version
  case server
  when 'qa'
    # The hostname to SSH to
    set :domain, 'qa-domain.com'
    # SSH Optional settings
    set :user, 'foobar'    # Username in the server to SSH to.
    # set :port, '30000'     # SSH port number.
    # Rails environment
    set :rails_env, 'qa'
  when 'prod'
    # The hostname to SSH to
    set :domain, 'prod-domain.com'
    # SSH Optional settings
    set :user, 'foobar'    # Username in the server to SSH to.
    # set :port, '30000'     # SSH port number.
    # Rails environment
    set :rails_env, 'production'
  end

  # For those using RVM, use this to load an RVM version@gemset.
  invoke :'rvm:use[ruby-1.9.3-p327@project]'
end

##########################################################################
#
# Create new host tasks
# Tasks below are related to deploying a new version of the application
#
##########################################################################

# Function extracted from http://blog.nicolai86.eu/posts/2013-05-06/syncing-database-content-down-with-mina
# allowing to read the content of the database.yml file
RYAML = <<-BASH
function ryaml {
  ruby -ryaml -e 'puts ARGV[1..-1].inject(YAML.load(File.read(ARGV[0]))) {|acc, key| acc[key] }' "$@"
};
BASH

# Execute all setup tasks defined below
desc "Create new folder structure + database.yml + DB + VirtualHost"
task :'setup:all' => :environment do
  queue! %[echo "-----> Setup folder structure on server"]
  invoke :setup
  queue! %[echo "-----> Setup the DB (create user / DB)"]
  invoke :'setup:db'
  queue! %[echo "-----> Setup Apache VirtualHost Configuration"]
  invoke :'setup:apache'
  queue! %[echo "-----> Deploy Master for this version"]
  invoke :deploy
  queue! %[echo "-----> Enable Apache host and restart Apache"]
  invoke :'apache:enable'
end

# Put any custom mkdir's in here for when `mina setup` is ran.
# For Rails apps, we'll make some of the shared paths that are shared between
# all releases.
task :setup => :environment do
  queue! %[mkdir -p "#{deploy_to}/shared/log"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/log"]

  queue! %[mkdir -p "#{deploy_to}/shared/config"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/config"]

  queue! %[touch "#{deploy_to}/shared/config/database.yml"]
  queue  %[echo "-----> Fill in information below to populate 'shared/config/database.yml'."]
  invoke :'setup:db:database_yml'
end

# Populate file database.yml with the appropriate rails_env
# Database name and user name are based on convention
# Password is defined by the user during setup
desc "Populate database.yml"
task :'setup:db:database_yml' => :environment do
  puts "Enter a name for the new database"
  db_name = STDIN.gets.chomp
  puts "Enter a user for the new database"
  db_username = STDIN.gets.chomp
  puts "Enter a password for the new database"
  db_pass = STDIN.gets.chomp
  # Virtual Host configuration file
  database_yml = <<-DATABASE.dedent
    #{rails_env}:
      adapter: mysql2
      encoding: utf8
      database: #{db_name}
      username: #{db_username}
      password: #{db_pass}
      host: localhost
      timeout: 5000
  DATABASE
  queue! %{
    echo "-----> Populating database.yml"
    echo "#{database_yml}" > #{deploy_to!}/shared/config/database.yml
    echo "-----> Done"
  }
end

# Create the new database based on information from database.yml
# In this application DB, user is given full access to the new DB
desc "Create new database"
task :'setup:db' => :environment do
  queue! %{
    echo "-----> Import RYAML function"
    #{RYAML}
    echo "-----> Read database.yml"
    USERNAME=$(ryaml #{deploy_to!}/#{shared_path!}/config/database.yml #{rails_env} username)
    PASSWORD=$(ryaml #{deploy_to!}/#{shared_path!}/config/database.yml #{rails_env} password)
    DATABASE=$(ryaml #{deploy_to!}/#{shared_path!}/config/database.yml #{rails_env} database)
    echo "-----> Create SQL query"
    Q1="CREATE DATABASE IF NOT EXISTS $DATABASE;"
    Q2="GRANT USAGE ON *.* TO $USERNAME@localhost IDENTIFIED BY '$PASSWORD';"
    Q3="GRANT ALL PRIVILEGES ON $DATABASE.* TO $USERNAME@localhost;"
    Q4="FLUSH PRIVILEGES;"
    SQL="${Q1}${Q2}${Q3}${Q4}"
    echo "-----> Execute SQL query to create DB and user"
    echo "-----> Enter MySQL root password on prompt below"
    #{echo_cmd %[mysql -uroot -p -e "$SQL"]}
    echo "-----> Done"
  }
end

# Create a new VirtualHost file
# Server name is defined by convention
# Script executes some sudo operations
desc "Create Apache site file"
task :'setup:apache' => :environment do
  # Get variable for virtual host configuration file
  fqdn = get_fqdn(server, version)
  fqdn_ext = external_fqdn(server, version)
  # Virtual Host configuration file
  vhost = <<-HOSTFILE.dedent
    <VirtualHost *:80>
      ServerAdmin user@your-website.com
      ServerName #{get_fqdn(server, version)}
      DocumentRoot #{deploy_to!}/#{current_path!}/public
      RailsEnv #{rails_env}
      <Directory #{deploy_to!}/#{current_path!}/public>
        Options -MultiViews
        AllowOverride all
      </Directory>
      PassengerMinInstances 5
      # Maintenance page
      ErrorDocument 503 /503.html
      RewriteEngine On
      RewriteCond %{REQUEST_URI} !.(css|gif|jpg|png)$
      RewriteCond %{DOCUMENT_ROOT}/503.html -f
      RewriteCond %{SCRIPT_FILENAME} !503.html
      RewriteRule ^.*$ - [redirect=503,last]
    </VirtualHost>
  HOSTFILE
  queue! %{
    echo "-----> Create Temporary Apache Virtual Host"
    echo "#{vhost}" > #{fqdn}.tmp
    echo "-----> Copy Virtual Host file to /etc/apache2/sites-available/ (requires sudo)"
    #{echo_cmd %[sudo cp #{fqdn}.tmp /etc/apache2/sites-available/#{fqdn}]}
    echo "-----> Remove Temporary Apache Virtual Host"
    rm #{fqdn}.tmp
    echo "-----> Done"
  }
end

# Enable the new Virtual Host and restart Apache
desc "Enable new Apache host file"
task :'apache:enable' => :environment do
  fqdn = get_fqdn(server, version)
  queue! %{
    echo "-----> Enable Apache Virtual Host"
    #{echo_cmd %[sudo a2ensite #{fqdn}]}
    echo "-----> Remove Temporary Apache Virtual Host"
    #{echo_cmd %[sudo service apache2 reload]}
  }
end

##########################################################################
#
# Deployment related task
#
##########################################################################

desc "Deploys the current version to the server."
task :deploy => :environment do
  deploy do
    # Put things that will set up an empty directory into a fully set-up
    # instance of your project.
    invoke :'git:clone'
    invoke :'deploy:link_shared_paths'
    invoke :'bundle:install'
    invoke :'rails:db_migrate'
    invoke :'rails:assets_precompile:force'

    to :launch do
      queue "touch #{deploy_to}/#{current_path}/tmp/restart.txt"
    end
  end
end

#########################################################################
#
# Helper functions
#
##########################################################################

#
# Get the main domain based on the server
#
# @return [String] the main domain
def main_domain(server)
  case server
  when 'qa'
    "qa-domain.com"
  when 'prod'
    "prod-domain.com"
  end
end

#
# Fully Qualified Domain Name of the host
# Concatenation of the version and the domain name
#
# @return [String] the FQDN
def get_fqdn(server, version)
  fqdn = "#{version}.#{main_domain(server)}"
  return fqdn
end

#########################################################################
#
# Libraries
#
##########################################################################

#
# See https://github.com/cespare/ruby-dedent/blob/master/lib/dedent.rb
#
class String
  def dedent
    lines = split "\n"
    return self if lines.empty?
    indents = lines.map do |line|
      line =~ /\S/ ? (line.start_with?(" ") ? line.match(/^ +/).offset(0)[1] : 0) : nil
    end
    min_indent = indents.compact.min
    return self if min_indent.zero?
    lines.map { |line| line =~ /\S/ ? line.gsub(/^ {#{min_indent}}/, "") : line }.join "\n"
  end
end

Deploying New Version

Once this is in place, deploying a new version of the application on the QA server is as easy as running:

mina setup:all server=qa version=special_version

Breakdown

This setup:all task will actually execute the following tasks:

setup: creates the folder structure used by Mina afterwards and call the setup:db:database_yml task.
setup:db:database_yml: asks the user to specify the name of the DB to create as well as the user and password that should be used. This will be written in the database.yml file.
setup:db: read the information from the database.yml file and execute the necessary SQL commands to create the DB. This will prompt for the MySQL root password in order to execute the commands.
setup:apache: create a Virtual Host file and place it in /etc/apache2/sites-available/ folder (Ubuntu specific). This will require the user to specify the sudo password from the remote host.
deploy: Deploy the new version of the application in the new folder structure that was created.
apache:enable: Enable the new Apache site and restart Apache service.

As one can see, once the user has defined the DB username and password, they are stored in the database.yml file and the remaining of the script actually reads this file in order to extract the necessary information. This is quite convenient and can be re-used in other tasks within the script, for instance to perform a backup of the database. This allows to easily share the deploy.rb file, without having any DB passwords stored in clear within the file.

Conclusion

Mina has been more and more helpful lately. The fact that it only requires a combination of Ruby and shell commands makes it easy to get started and it has been simplifying my life a lot by automating many tasks that used to be run manually. One additional thing that I should mention is that it is not only useful to deploy Rails applications. It can be used for other kind of applications as well and I believe that this article is a good illustration of this.

Sources

Mina documentation

Mina source code

syncing database content with mina

Reading YAML files in bash with ruby

sync down the content of a mysql database used by a ruby on rails application using mina

CREATING A MYSQL DATABASE AND USER ON THE COMMAND LINE, AND A BASH SCRIPT TO AUTOMATE THE PROCESS

Credits Image

Mina

It seems that other articles are somewhat related to this one. Similar articles are identified through a set of keywords so this might not be 100% accurate, but I hope this can interest you.

Backup a Website on Shared Hosting

Published on July 14, 2012

For the time being, comments are managed by Disqus, a third-party library. I will eventually replace it with another solution, but the timeline is unclear. Considering the amount of data being loaded, if you would like to view comments or post a comment, click on the button below. For more information about why you see this button, take a look at the following article.

Automatic Application Deployment with Mina

Introduction

Mina Deployment File

Deploying New Version

Breakdown

Conclusion

Sources

Credits Image

Related Articles

Backup a Website on Shared Hosting

Somewhat related articles

Backup a Website on Shared Hosting