Monthly Archive for February, 2008

Using paginating_find with has_finder

Published by alon on February 27, 2008 in Ruby (on Rails). 0 Comments

I’ve been enjoying using has_finder by Nick at Pivotal Labs and recently added pagination to the application I am providing volunteer time to for the San Francisco Bike Kitchen.

I like the semantics of paginating_find over will_paginate but am not married to that choice if anyone wants to try to convince me otherwise.

In any case, I wanted to paginate the finders created by has_finder. I wrote a simple ActiveRecord mixin that provides a class method ‘acts_as_paginated and is used in your model like:

class Visit < ActiveRecord::Base
  belongs_to :person
 
  acts_as_paginated :size=> 10
 
  has_finder :for_person, lambda { |person| {
    :conditions => { :person_id => person},
    :order => 'datetime DESC'
  } }
 
  has_finder :in_date_range, lambda { |from,to| {
    :conditions => [ "visits.datetime >= ? and visits.datetime <= ?", from, to ]
  } }
end

Client code looks like:

from, to = Date.new(2008,2,1), Date.new(2008,2,3)
visits = Visit.for_person(@person).in_date_range(from, to).paginate(:page => 3)

The mixin looks like:

# Provides Model.paginate(options={}) to allow
# the paginating_find plugin to be used with the has_finder plugin.
#
# Usage:
#   acts_as_paginated
#   acts_as_paginated :page => 2, size => 10
#
# Install:
#   Save this file as lib/acts_as_paginated.rb and require 'acts_as_paginated'
#   in environment.rb.
#
module HasFinder
module PaginatingFind #:nodoc:
 
def self.included(mod)
  mod.extend(ClassMethods)
end
 
module ClassMethods
  def acts_as_paginated(options={ :page => 1, :size => 20})
    cattr_accessor :paginate_defaults
    self.paginate_defaults = options
 
    self.class_eval do
      extend HasFinder::PaginatingFind::SingletonMethods
    end
  end
end
 
module SingletonMethods
  def paginate(args={})
  options = self.paginate_defaults.clone
    options[:page] = args[:page].to_i if args[:page]
      options[:size] = args[:size].to_i if args[:size]
        find(:all, :page => { :size => options[:size], :current => options[:page], :first => 1 })
      end
    end
  end
end
 
ActiveRecord::Base.send(:include, HasFinder::PaginatingFind)

It also works to paginate associations:

@person.visits.paginate(:size => 4)

markaby_scaffold Rails Plugin

Published by alon on February 20, 2008 in Ruby (on Rails). 0 Comments

We’ve been enjoying using Markaby on our recent rails projects. When bootstrapping with scaffolding there’s a common recipe of turning ERB views into Markaby and then refactoring views with Markaby helper functions.

One project wrote a rake task that converts ERB views to Markaby. Then Ingar created a plugin that re-implements the base Rails scaffolding to create Markaby views that output the same HTML as the base Rails scaffolding. I took his work one standard refactor further to introduce a MarkabyHelper with helper functions for the layout of form inputs and model views.

Usage

./script/plugin install http://svn.carbonfive.com/public/carbonfive/rails/plugins/markaby_scaffold/trunk/

Use it as you would regular Rails scaffolding:

./script/generate markaby_scaffold Post title:string body:text published:boolean

Prerequisites

markaby_scaffold requires that you install Markaby as a Rails plugin.

Gravy

Unfortunately, Markaby does not appear to be getting the love that it deserves. There is an issue using Markaby with Rails 2.0.2 for which Randy has submitted a patch. This plugin includes a monkey patch for Markaby and Rails 2.0.2 that provides this fix so you don’t have to apply it yourself.

Additionally, the generated MarkabyHelper includes support for using Markaby in your helpers, provides a quick start for refactoring the scaffolding layouts and includes test cases to get you started testing your Markaby helpers.

Introducing Java DB Migrations

Published by christian on February 6, 2008 in Java. 6 Comments

UPDATE: A new version of the Java DB Migrations framework has been release, check this post for details and the project documentation.

Here at Carbon Five we have the luxury of working on many projects, so anything we can do to make things easier will pay off in multiplicity across new projects. One of the things that we have to deal with on every project is maintaining a database schema over time. We’ve had a manual process of capturing changes in incremental db patch scripts for a while, but it was error prone and sometimes neglected. We’ve been doing more Ruby on Rails work and found Rails Migrations easy to work with and a real time saver. We wanted something that would make our lives easier when working on Java projects in the same way Migrations improve Rails development. With that manifest in mind, Alon and I collaborated on a simple Java database migration framework.

During development, it’s a big deal because each engineer has two instances of the database, one for unit tests and another for running the application. We need an easy way to create a new, up-to-date database and update existing databases. Once a project has launched, it’s a big deal because we need a way to migrate a database teeming with important production data to the latest version without losing critical information.

High Level Requirements

  • Initiate a migration from the command-line as a Maven plugin
  • Programmatically migrate a database during application startup
  • Convention over Configuration
  • Initially support migrations written in SQL

At a high level, the migration process looks like this:

  1. Query the database (table db_version) to find the current version.
  2. Determine the latest database schema version available.
  3. If the database is out of date, run each migration in order in its own transaction, updating the db_version for each migration.

We’d identified two usage patterns, the first is more akin to the Rails Migration model in that you explicitly migrate the database via the command line. The second is automatic migration when an application starts up, before Hibernate initializes or any other data access takes place. I’ll
discuss each in turn.

Migrating using Maven

This functionality is easy to enable in a mavenized project. First you add the Carbon Five public plugin repository:

pom.xml

...
<pluginRepositories>
    <pluginRepository>
        <id>c5-public-repository</id>
        <url>http://mvn.carbonfive.com/public</url>
    </pluginRepository>
</pluginRepositories>
...

And then you configure the migration plugin:

pom.xml

...
<plugin>
    <groupId>com.carbonfive</groupId>
    <artifactId>maven-migration-plugin</artifactId>
    <version>0.9-SNAPSHOT</version>
    <configuration>
        <defaultEnvironment>test</defaultEnvironment>
        <environments>
            <environment>
                <name>default</name>
                <driver>com.mysql.jdbc.Driver</driver>
                <username>dev</username>
                <password>dev</password>
            </environment>
            <environment>
                <name>test</name>
                <url>jdbc:mysql://localhost/myapp_test</url>
            </environment>
            <environment>
                <name>development</name>
                <url>jdbc:mysql://localhost/myapp_development</url>
            </environment>
        </environments>
    </configuration>
    <dependencies>
        <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.5</version>
        </dependency>
    </dependencies>
</plugin>
...

You’ll notice that we’ve got 2 environments configured. You can have as many as you need and you can specify which you want to migrate on the command line. If none are specified the default environment will be migrated. In this example we’re specifying the dependency on our JDBC driver so that the plugin has access to the code it needs to connect the database.

Lastly, you drop in your migration scripts into the src/main/resources/db/migrations directory, naming them using the pattern NNN_description.sql, where NNN is three digits indicating the script sequence. Some examples might be:

  • 001_create_users_table.sql
  • 002_add_default_users.sql
  • 003_add_lastvisit_column.sql

The description is optional and isn’t used for anything, it’s just there so that other developers can get an idea of what a script does without having to open it.

From the command line, you can run the migration plugin like this:

$ mvn migration:migrate

Note that he database must exist for the migrations to take place as we do not create missing databases (yet).

I’ve created a simple, complete sample that shows off this functionality, it’s on the C5 public subversion repository here. Check it out and then read the readme.txt at the top of the project.

Migrating from your Application

The other usage scenario is to auto-migrate during application startup. At the core of the framework, there’s an interface called MigrationManager which has two implementations: DataSourceMigrationManager and DriverManagerMigrationManager. Migration happens right after a datasource (of the javax.sql variety) is created.

Migrating from your application is as easy as instantiating one of these early in the startup cycle and invoking the migrate() method, something like this:

MigrationManager migrationManager = new
    DriverManagerMigrationManager(“com.mysql.jdbc.Driver”,
    “jdbc:mysql://localhost/myapp_test”, “dev”, “dev”);
migrationManager.migrate();

Of course this needs to happen before anything else in the application uses the database; we want to database to be updated completely before it’s used.

Spring is part of our standard development stack on our Java projects, and it’s easy to enforce these dependencies in Spring configuration. First we define a data source for the application:

<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
    <property name="driverClassName" value="org.h2.Driver"/>
    <property name="url" value="jdbc:h2:file:~/.h2/migration_sample2_test"/>
    <property name="username" value="dev"/>
    <property name="password" value="dev"/>
</bean>

And now we declare our MigrationManager instance. Note the ‘init-method’ attribute:

<bean id="migrationManager"
    class="com.carbonfive.db.migration.DataSourceMigrationManager"
    init-method="migrate">
    <constructor-arg ref="dataSource"/>
</bean>

And then you define something that’s going to use the defined DataSource. Note the ‘depends-on’ attribute:

<bean id="userService"
    class="com.carbonfive.migration.sample2.UserService"
    depends-on="migrationManager">
    <constructor-arg ref="dataSource"/>
</bean>

This is obviously a little contrived for the sake of example, but you get the point. In a typical application the thing that would depend on the datasource is a Hibernate SessionFactory.

You can visit the source code for this example on the C5 public subversion repository href="https://svn.carbonfive.com/public/christian/migration-sample2/trunk">here.

Best Practices

Here are a few of the things we’ve learned along the way:

  • Start using migrations early. Definitely start by time there’s more than one person on a project. I usually start off letting hibernate generate my schema while I’m experimenting with things, but as soon as I’m really working on features I’ll switch over to migrations.
  • All database changes are captured as a new migration.
  • Migration scripts cannot be changed once anyone has run them and make further changes in a new migrations.

Source Code Access

The Carbon Five db-support project which contains all of this migration goodness is available on the C5 public subversion repository at https://svn.carbonfive.com/public/carbonfive/db-support/trunk. It’s a maven project and should compile and pass its tests out of the box. I encourage you to look through the code and check out the tests.

Future

If you look through the code you’ll see some of what’s in store for this project. We’ve got initial support for writing migrations in Groovy and JRuby and we’re thinking about added Java support as well. We’re looking for feedback to drive the future direction of the project, so feel free to write us and let us know what you think.

Versioning your IDEA module meta-data (.iml)

Published by christian on February 6, 2008 in IntelliJ IDEA. 0 Comments

On one of our larger client projects, we’ve been tackling an array of improvements to the development environment. This is largely motivated by the fact that it’s a big project with lots of modules and engineers and time lost dealing with environment issues was expensive and frustrating. Everyone wants to be productive without having to wrestle with their tools or understand the ins and outs of things on the periphery, like build systems and IDE configuration.

We migrated to Maven 2 for builds partially because it’s convention driven, so when questions of how we should do something come up we can default to Maven best practices. Another reason is that it enabled us to stick to the Don’t Repeat Yourself (DRY) principle. We didn’t want to duplicate build meta-data by maintaining both command-line configuration (Maven pom.xmls) and IntelliJ IDEA configuration. Maven deals with this pretty well via the maven-idea-plugin.

The downside to running “mvn idea:module” every time there’s a pom change is that it eats at least a few minutes with all of the checks for sources and javadocs. To make matters worse, we always had to run it from the master directory so that intra-module dependencies were modeled correctly as IDEA module dependencies, not jar dependencies. Each developer would go through this routine when there was a change because we weren’t versioning our IDEA meta-data in subversion. When considering all of the engineer’s time, this process would consume over half an hour in total.

I wanted to change the process so that we still used the maven-idea-plugin to generate the meta-data (staying true to DRY), but then commit the generated files to subversion so that only one person would have to go through the pain of re-generating (presumably whoever made the changes to the poms) and the rest of the team could continue without much disruption. The problem is that each generated .iml file contains absolute paths which are specific to their machine.

After a bit of sleuthing, I discovered an IDEA option called Path Variables which is available in the general settings section. There isn’t much to tell you what it does, so it’s easily overlooked. You can specify one or more variables and IDEA will automatically replace file paths in project (.ipr) and module files (.iml) with the provided variables. Everyone on the development team added a Path Variable called “M2_REPO” which points to their local maven repository (~/.m2/repository).

add path variable

After running “mvn idea:module” we just open the project in IDEA and it will do the replacing behind the scenes. Now the module meta-data can be versioned and shared.

It’s a small feature, but it’s helping us streamline everyone’s day-to-day.

IDEA 7 includes native Maven support which is getting better with each point release. The process described herein works great with the native plugin as well.

Google Talking with JRuby and Smack

Published by alon on February 3, 2008 in Ruby (on Rails). 1 Comment

We’ve kicked off a new project that extends well beyond standard webapp frameworks and tools including significant instant messaging integration. Portions of the project will use Ruby on Rails for web functionality. I know that the open source Jabber software from Jive Software is excellent quality and feature rich. Would JRuby be a good tool for leveraging these Java libraries in a Ruby environment?

Here’s a toy chat client and parrot bot I wrote for the Google Talk service using JRuby and the Java Smack API from Jive’s open source portal ignite realtime.

Continue reading ‘Google Talking with JRuby and Smack’