Check Multiple Mercurial Repositories for Incoming Changes

Currently I have a whole bunch of Mercurial repositories in a directory. All of these are cloned from a central repository that the team pushes their changes to. I like to generally keep my local repositories up-to-date so that I can review changes. Manually running hg incoming -R some_directory on 20 different projects is a lot of work. So I automated it with a simple shell script.

This script will run incoming (or outgoing) on all of the local repositories and print the results to the console. Then I can manually sync the ones that have changed if I want.

I called this file hgcheckall.sh and run it like: ./hgcheckall.sh incoming

#!/bin/bash
 
# Find all the directories that are mercurial repos
dirs=(`find . -name ".hg"`)
# Remove the /.hg from the path and that's the base repo dir
merc_dirs=( "${dirs[@]//\/.hg/}" )
 
case $1 in
    incoming)
    for indir in ${merc_dirs[@]}; do
        echo "Checking: ${indir}"
        hg -R "$indir" incoming
    done
    ;;
    outgoing)
    for outdir in ${merc_dirs[@]}; do
        echo "Checking: ${outdir}"
        hg -R "$outdir" outgoing
    done
    ;;
    *)
    echo "Usage: hgcheckall.sh [incoming|outgoing]"
    ;;
esac

I guess the next major improvement would be to capture the output and then automatically sync the ones that have changed, but I haven’t gotten around to that yet.

Using Ruby Subversion Bindings to Create Repositories

Subversion has a default Web UI that is served up by Apache if you run Subversion that way. It is pretty boring and read-only. Then there are things like WebSVN that make it less boring, but still read-only. I got curious about what it would take to make something even less boring and NOT read-only. Why not allow me to create a new repository on the server? Why not allow me to create a new default directory structure for a new project in a repository all through a web interface?

Parent Path Setup

All of my repositories use the idea of the SVNParentPath. This makes Apache assume that every directory under a given path is an SVN Repository. That structure makes it easy to deal with multiple repositories and secure them with a single security scheme. Using that assumption then it is easier to write some code that will list existing repositories and create new ones in a known location.

SvnAdmin With Ruby Subversion Bindings

Subversion provides language bindings for a number of different languages (Java, Python, Perl, PHP and Ruby) in addition to the native C libraries. Using these bindings it becomes fairly easy to deal with Subversion. The only hiccup will be dealing with the apparent lack of documentation for the code. So be prepared to do some exploration, digging and reading of the code.

I chose to try this using Ruby because it was quick and easy and it was a language I was already familiar with.

First you need to know how to create a new repository and open an existing repository. Fortunately those are simple, one-line operations:

Svn::Repos.create(repos_path, {})
repos = Svn::Repos.open(repos_path)

There was nothing special (from what I could tell) that would allow you to determine if a repository already existed, so I just created a simple function using the Ruby File operations to determine if a directory already existed. This code would allow me to determine if I needed to create new repository or open an existing one:

def repository_exists?(repos_path)
   File.directory?(repos_path)
end

Now I have a repository open so I wanted to build a project structure using the default conventions I use for Subversions projects. My convention is to have a repository named after a client, the top-level directories are named for the client’s project and then each project has the standard trunk, branches and tags within that. Depending on the kind of work you do that convention may or may not make sense for you.

With that decided, I created the code to write that structure in a repository. The one thing I found is that interacting with the Subversion repository allowed you to do things within a transaction that would force all of the changes to be recorded as a single commit. I thought this was a good thing, so performed these operations as a transaction:

txn = repos.fs.transaction
 
# create the top-level, project based directory
txn.root.make_dir(project_name)
 
# create the trunk, tags and branches for the new project
%w(trunk tags branches).each do |dir|
  txn.root.make_dir("#{project_name}/#{dir}")
end
 
repos.commit(txn)

Finally I put all of those things together into a class. The class had the concept of being initialized to a base Parent Path so all of the operations would know to start from that location:

require "svn/repos"
 
class SvnAdmin
   def initialize(parent_path)
     @base_path = parent_path
   end
 
   # Provides a list of directory entries. path must be a directory.
   def ls(client_name, path="/", revision=nil)
     repos_path = File.join(@base_path, client_name)
     repos = ensure_repository(repos_path)
 
     entries = repos.fs.root(revision).dir_entries(path)
     entries.keys
   end
 
   def create(client_name, project_name)
     repos_path = File.join(@base_path, client_name)
     repos = ensure_repository(repos_path)
 
     txn = repos.fs.transaction
     txn.root.make_dir(project_name)
     %w(trunk tags branches).each do |dir|
       txn.root.make_dir("#{project_name}/#{dir}")
     end
 
     repos.commit(txn)
   end
 
   private
   def ensure_repository(repos_path)
     if ! repository_exists?(repos_path)
       Svn::Repos.create(repos_path, {})
     end
     repos = Svn::Repos.open(repos_path)
   end
 
   def repository_exists?(repos_path)
     File.directory?(repos_path)
   end
 
end

SvnAdmin from Rails

Now that I had some simple code to create new repositories or add a new project to an existing repository I decided to wrap it in a simple Rails application that would allow me to create repositories using a web-based interface.

To start with, I’m not going to use a database or any ActiveRecord classes in this project (which you might do if you wanted authentication or something else) so I disabled ActiveRecord in the config/environment.rb

config.frameworks -= [ :active_record ]

Then I created an ReposController to manage the Subversion repositories. This controller contains a couple of simple actions:

  1. An index action to list the existing repositories (directories)
  2. A new action to display a form to enter the client and project names
  3. A create action to use the SvnAdmin class to create a new repository and/or project
require "svnadmin"
 
class ReposController < ApplicationController
  layout 'default'
 
  def index
    @dirs = Dir.new(SVN_PARENT_PATH).entries.sort.reject {|p| p == "." or p == ".."}
  end
 
  def new
  end
 
  def create
    svn = SvnAdmin.new(SVN_PARENT_PATH)
    repos = params[:repos]
 
    respond_to do |format|
 
      begin
        svn.create(repos[:client_name], repos[:project_name])
        flash[:notice] = "Successfully created."
        format.html { redirect_to :action => "index" }
        format.xml  { head :ok }
      rescue
        flash[:error] = "Failed to create structure."
        format.html { redirect_to :action => "index" }
        format.xml  { render :xml => object.errors, :status => :unprocessable_entity }
      end
    end
  end
end

You can also easily create a route and a ProjectsController that allows you to see all of the projects within a repository.

The route in config/routes.rb is simply:

  map.connect ':repos/projects/',
      :controller => 'projects',
      :action => 'index'

And the ProjectsController looks up the :repos param to open the proper repository and list the top-level directories with it:

require "svnadmin"
 
class ProjectsController < ApplicationController
  layout 'default'
 
  def index
    repos_path = params[:repos]
    svn = SvnAdmin.new(SVN_PARENT_PATH)
    @projects = svn.ls(repos_path)
  end
end

Hopefully that will help you handle your Subversion administration. It should let you code up your conventions so that they are followed whenever a new repository is created or a new project is started.