JRubyGems allows you to package RubyGems on your Java application classpath. It removes the file system dependencies on locally installed gems that get awkward when using JRuby and dependent gems in a Java environment.

Usage

Use JRubyGems as a replacement for RubyGems. Instead of

require 'rubygems'
gem 'activerecord'

use

require 'jrubygems'
gem 'activerecord'

Behind the scenes JRubyGems requires RubyGems and injects behavior specific to finding gems on the Java classpath if they are not already installed.

You make JRubyGems available to your application by putting the jar on your application classpath.

Get It

Source is available from Subversion at https://svn.carbonfive.com/public/carbonfive/jruby/jrubygems/trunk/. The tests in ruby/test/test_jrubygems.rb illustrate JRubyGems’ behavior.

The jar at http://mvn.carbonfive.com/public/com/carbonfive/jruby/jrubygems/0.3/jrubygems-0.3.jar is all you need to try it out in your own scripts. With JRuby you can require jar files that are in your load path to get them on your classpath and make Ruby files in the jar available on your load path. So if I have a folder lib/ in my load path with the file lib/jrubygems-0.3.jar, I can use JRubyGems directly from a JRuby script with:

require 'jrubygems-0.3.jar'
require 'jrubygems'

Most Java applications using embedded Ruby will have their own classpath management strategy. Just make sure the JRubyGems jar gets on the classpath of your application. If you are using Maven, you can use JRubyGems from the Carbon Five repository:

<repositories>
  <repository>
    <id>c5-public-repository</id>
    <name>Carbon Five Public Repository</name>
    <url>http://mvn.carbonfive.com/public</url>
  </repository>
</repositories>
 
<dependencies>
  <dependency>
    <groupId>com.carbonfive.jruby</groupId>
    <artifactId>jrubygems</artifactId>
    <version>0.3</version>
  </dependency>
</dependencies>

If you are using Maven you can also require your JRuby dependency with:

<dependency>
  <groupId>org.jruby</groupId>
  <artifactId>jruby-complete</artifactId>
  <version>1.1RC1</version>
</dependency>

Packaging Gem Dependencies

JRubyGems expects a gem to be available on your application classpath as a .gem file (the distribution package for a gem) under the classpath location /gems. This means you can have a gems/ folder on your classpath with a bunch of .gem files in it, a jar with all your .gem files under /gems or a jar per .gem file with each file under /gems in each jar. (A limitation in the JDK’s ClassLoader.getResources method which only returns file system locations for a passed-in empty String prevented me from allowing .gem files in jar roots.)

For our migration application, I built a jar for each gem dependency and deployed it to our central Maven repository. My Maven build references these dependencies in my project POM just as I would a jar dependency.

<dependency>
  <groupId>com.carbonfive.jruby.gems</groupId>
  <artifactId>activerecord</artifactId>
  <version>1.15.6</version>
</dependency>

I just grabbed the .gem files from my local JRuby installation’s gem cache dir at jruby-1.1b1/lib/ruby/gems/1.8/cache/ to build these jars for now.

I like that Raven takes exactly the opposite approach – managing jar dependencies as gems.

Background

Christian and I have a side project at Carbon Five to create a standard mechanism for managing database migrations for our Java projects. The first implementation defines a migration as a SQL script to be run against the target database. I thought it would be cool to support ActiveRecord migrations too, especially for providing an easy way to do the data migrations that accompany a schema change.

My prototype of this idea went well (more on that some other time) but I quickly ran into an issue with the gem dependency for ‘activerecord-jdbc-adapter’ and its dependencies ‘activesupport’ and ‘activerecord’. The issue is that my Java project was nicely portable but the gems create a dependency on the runtime system having specific gems installed. If I install the gems locally, I also have to install them on other runtime systems like our continuous integration server.

I found another example of a JRuby user running into this problem when reading about this example of using the RedCloth Ruby library to format text in a Spring/Java application. Note the “ugly underbelly” footnote.

In the Ruby world, this is the way things work. Large Ruby applications (especially Rails applications) often rely on system services like cron to do their work. These applications have their own mechanisms for setting up new runtime systems. In the Java world, we expect to package our application with all of its dependencies and deploy it to little more than a JVM and webapp container.

Java has the classpath abstraction to address minimizing file system dependencies while Ruby uses the file system directly and extensively. I’ve seen a couple approaches for addressing this mismatch between Java and Ruby. All of them involve taking some set of resources packaged in a Java archive (jar, war), extracting it to a file system location at runtime and configuring Ruby to use the resources at that location. The “jruby-complete” JRuby distribution unpacks the core Ruby libraries into ~/.jruby/. GoldSpike, the Rails plugin for packaging Rails applications as war files, bundles gems in WEB-INF and configures GEM_HOME at runtime to use the gems from the unpacked war.

I decided to take a similar approach with JRubyGems – bundle gem dependencies in the application classpath and install them on the local file system on demand if they are not already available. As I expect happens with many JRuby projects, it took a little Java and a little Ruby and works quite well.

How It Works

The main flow for RubyGems to load a gem that has not yet been loaded for an application is:

1. Kernel::gem
2. Kernel::activate_gem_with_options
3. Gem.activate
4. Gem.source_index.find_name
5. Gem.activate each dependency of this gem first
6. Finish activating this gem

JRubyGems replaces the Gem.activate implementation to insert a couple steps that:

1. Check if the gem is locally installed
2. Search the Java classpath for the gem source if not installed
3. Install the gem locally
4. Continue with the base Gem.activate implementation

This ensures that gems and their transitive dependencies are installed and loaded.

The classpath searching is implemented in Java.

Release History

0.3 2008-01-28

• Upgrade to use JRuby 1.1RC1 and RubyGems 1.0.1 (bundled with 1.1RC1)
• Added test for gem that bundles a jar in lib/ (hpricot)

0.2 2008-01-07

• Resolved issues with transitive dependency installation
• Force install of classpath gems to avoid dependency errors
• Additional logging of install locations

0.1 2008-01-04

Initial release.

Looking forward, I want to be sure that we take maximum advantage of our existing knowledge and new learning in both Java and Ruby. I have high hopes for JRuby as a valuable bridge that spans both languages and gives developers the flexibility to use the strengths of the two. Sun appears to feel the same way as they have hired the two core JRuby developers, Charles Nutter and Thomas Enebo, to work on JRuby full time.

In preparation for more specific posts, here are some light musings on JRuby.

JRuby 1.0

Released mid-2007 JRuby 1.0 was focused on Ruby compatibility. The idea is that any Ruby script or application you can run with the native C Ruby interpreter, you can run with JRuby. The success of this effort can be observed with Ruby on Rails applications now running on JRuby. Of course, there are some limitations.

In addition to running pure Ruby, JRuby can access the Java environment in which it is running. You get the Ruby niceties that you would expect like optional conversion of CamelCase method names to underscore_separated.

list = java.util.ArrayList.new
list.add("one")
list.add_all [2, "three"]

Any Java classes in the Java classpath are available to your JRuby script. Additionally, you can ‘require foo.jar’ to make it available on your classpath at runtime.

You can also extend Java classes in JRuby.

class MyList < java.util.ArrayList
  def say_hi
    "hi"
  end
end

Internally, many core Ruby classes have been rewritten to delegate to core Java classes.

JRuby 1.1

JRuby 1.1 is presently in beta. The top priorities for JRuby 1.1 are performance and Java integration. Ruby is not fast. JRuby 1.0 was worse. Performance was not a priority for 1.0 and the consensus is that there are a lot of easy performance wins that it is now time to take advantage of. In his blog, Charles Nutter states:

Straight-line execution is now typically 2-4x better than Ruby 1.8.6. Rails whole-app performance ranges from just slightly slower to slightly better, though there have been reports of specific apps running many times faster.

Java Integration

Java integration refers to features for the using Ruby from Java rather than Java from Ruby as I discuss above. When I first started looking at JRuby Christian and I were working on a Java library to manage database migrations ala RoR and some other neat tricks we’ve seen along the way. (More on that later.) I thought it would be cool if you could write migrations using ActiveRecord’s migration features and invoke them from Java. I found myself looking for the easy means to invoke Ruby from Java. Some of this is in place with more slated for future JRuby versions.

The Spring Framework provides JRuby integration through which you can expose Ruby classes as Spring-managed beans. This example of using the RedCloth Ruby library to format text in a Java application is a pretty slick example of how this works.

This post does point out one issue that I ran into with using Ruby from Java – RubyGems. Gems are Ruby libraries available from central repositories and installed on the system running Ruby. For Java applications, this dependency on system-installed resources is lame. To address this issue, I’ve spent some time working on a solution for packaging Ruby gem dependencies in Java applications. More on JRubyGems coming soon.

What We Did

The project was fairly small; we pair-programmed almost all of the development, totaling about 10 person weeks of work. The application replaces a hybrid FileMaker/Excel solution our client was using to collect and analyze research data on corporate practices. The application is used by a small number of users (<10) but all day, every day.

Development

We did our development on OS X using IntelliJ IDEA 7 beta with the Ruby plugin. While we suffered bleeding edge instability issues with some plugin versions, IDEA and the Ruby plugin were great tools providing support for rake, RDoc, unit testing, and running our development server. The PDF version of Agile Web Development with Rails was our bible for reference information.

We used Atlassian Bamboo for continuous integration since we are already using it for our Java projects. The ci_reporter and rails_rcov Rails plugins provide test and code coverage information to Bamboo’s statistics database.

Deployment

We hosted our staging server at Carbon Five running a pack of Mongrels with Apache mod_proxy_balancer. Our production server is hosted at Joyent (aka TextDrive).

We used capistrano and capistrano-ext to manage deployment to multiple targets (staging and production).

REST

We jumped straight in to using the Rails 2.0 RESTful support. To our surprise we found the scaffolding created by ‘script/generate scaffold_resource Foo’ to be very helpful from UI to database including a starting point for building out our tests. We used nested resources and custom actions. We used content-type/format detection to return different views for the same actions.

If you are going to use REST on Rails you have to understand Rails routes. Of all aspects of the Rails framework, understanding request routing and the finer details of the generated helper methods caused us the most head scratching moments. Writing unit tests for our routes.rb file was a helpful tool for answering questions.

Without a doubt, REST is the way to go in Rails. It creates a sense of context within your application and a taxonomy of the actions that can take place. The helper methods for generating paths simplify refactoring and reorganizing.

Plugins

In general we found the quality of available plugins to be high and their functionality rich. Plugins provide a range of functionality from extensions to ActiveRecord, mixins, rake tasks, and generators for migrations, models and controllers. Things are changing fast so it is important to survey the available options. Sometimes the most discussed plugin is not the best to use – just the one that has been available the longest.

We used Piston to manage our plugin dependencies. The standard Rails practice of using svn:externals linked to the trunk of a plugin’s development seemed sketchy to us. In fact, the trunk of the restful_authentication plugin had recent changes that caused it to fail in anything but Edge Rails. We used Piston to lock the plugin to the last working version. It’s strange to me that plugin developers do not seem to use tags and branches to manage releasing their plugins to the world.

The has_many_polymorphs plugin extends Rails’ associations with advanced support for polymorphic associations. We used this plugin to implement tagging features in place of the older, less flexible acts_as_taggable plugin. The plugin also provides a generator for specifically implementing tagging.

The userstamp plugin automatically updates ‘created_by’ and ‘updated_by’ attributes of model objects with the logged in user.

The acts_as_versioned plugin provides versioning of a subset of our model objects. We have a complex hierarchy for research information that can change through time. Research from a point in time needs to match the hierarchy at that time. The acts_as_versioned plugin manages a table of versions for each model being versioned with mixed-in methods on the model objects for accessing historical versions.

The restful_authentication plugin provides user authentication. The authorization plugin provides role and instance-based authorization. The authorization plugin implements a lightweight DSL – a technique that seems to be favored by many Ruby developers. The authorization plugin exposed us to the practice of using static class members as thread-local instances. We couldn’t quite stomach having User.current_user statically defined so implemented it with a thread-local instance:

class User < ActiveRecord::Base
 
  def self.current_user=(user)
    Thread.current[:user] = user
  end
 
  def self.current_user
    Thread.current[:user]
  end
 
end

We wrote our own acts_as_notable plugin using the core plugin generator and the HOWTO on the Rails wiki. Our plugin mixes-in the ability to add notes to any model object. This was our first exposure to the powerful meta-programming tools ‘class_eval’ and ‘instance_eval’. We later realized that we could have used the has_many_polymorphs plugin for this but the exercise implementing this plugin was valuable anyway.

Testing

We used the core Rails support for unit, functional and integration tests though, honestly, we didn’t write any integration tests. We did not use RSpec or Mocha so did not get into behavior-based testing for Rails.

In general we found the testing support in Rails to be great. Being able to make assertions on your HTML output with ‘assert_select’ is great. Even simply having exceptions raised in output rendering when we broke a view with a refactor was great. Coming from the Java world where JSP makes it very difficult to get HTML output outside of the container, controller (functional) testing was a breath of fresh air.

Rails data fixtures are pretty cool but also a pain in the butt. In Java land, we use DBUnit to create data fixtures that model an application scenario for a suite of tests. With the core Rails support for fixtures, you are stuck maintaining a monolithic fixture context with related information spread across multiple files. Our application is a small one but even so our fixtures quickly became hard to manage and understand. Adding a version dimension to our model data didn’t help.

One approach we have seen for maintaining fixtures in to write a suite of tests that assert the integrity of your fixture data. Keep your fixtures as lean as possible while representing all possible application states. Yuck. We are interested in using fixture scenarios on our next project.

Rake

Randy dropped a database.rake file into lib/tasks/ to give us some utilities for managing our databases. While in early development, we would occasionally modify a historical migration and recreate our database from scratch using db:reset.

We also created a project rake file that defined tasks for importing data into our system. The import could have been a migration but seemed better as a task that could be done or redone at any time.

Migrations

We like migrations a lot. We like the Rails DSL for defining migrations, using model objects in migrations, and explicit schema versioning. In fact, we liked migrations so much that we implemented a similar strategy on our Java projects over a year ago.

We did debate a couple issues. First, is it ever okay to modify historical migrations, for example, to add a column to a model. I felt that if a build has been pushed to our staging server, we should never modify a historical migration. We should have to migrate our staging data forward just like our production data. In that way we get to stage our migrations too. In reality, it turned out that it was only really important to be disciplined about this once we had a production system with data that we could not lose.

The other issue we debated was how much time to spend on our ‘down’ migrations. There are plenty of scenarios where you could roll back the schema but lose data. What do you do in this case? In reality, our deployment architecture and roll out plans were so simple that down migrations did not really seem relevant for this application. In the end we did test that our migrations could go up and down and preserved data where it was easy but did not spend much time on it. Do people implement migration tests?

Prototype for AJAX

Both Randy and I are very comfortable writing good object-oriented JavaScript for AJAX behaviors. For this project, we decided to try to use the Rails Prototype helper methods to best understand its strengths and weaknesses. For this project we use AJAX for inline page updates and found the Prototype helper support to be fine. I liked the consistent syntax for expressing AJAX calls, form posts and HTML links. Randy thought it would have been easier to use Prototype directly than learn the thin Rails wrappers.

What We Liked

From the discussion above:

• RESTful Rails
• Migrations
• HTML assertions in functional tests
• capistrano
• Mixins
• Agile Web Development with Rails

Also, we like Ruby. It is terse yet semantically expressive. We often found ourselves writing six lines of code that we would reduce down to one intuitive and easy-to-read line. Ruby has a great shape-changing ability that makes it easy to create lightweight DSLs for different contexts. Rails makes great use of this ability in database migrations and RJS templates. We liked mixins and found them far more useful than inheritance for our purposes.

What We Didn’t Like

Dependency Management

As mentioned above, we used Piston to manage our plugin dependencies. Where possible we also installed Gems in vendor/. In many cases, you need Gems to be built and installed for your system so you cannot check them in.

We ran into one issue where the plugins we we needed for our continuous integration server had gem dependencies that caused our application to not load in any environment, even if we did not need those plugins or gems for that environment. The issue was the way that rake tasks in the plugins loaded their dependencies. We ended up having to modify the plugins to not load if the gems were not available.

This dependency on system-installed software was foreign and awkward coming from Java land. Rails applications often depend on other system software like cron and ImageMagik. You can imagine multiple Rails applications running on one server having gem or software version conflicts. It seems that virtualization solutions could be very helpful here as it is for LAMP-based applications. We worked on a Python project that included a VMWare image of the entire development environment checked in to source control. It turned out to be a slick solution to the system dependency problem.

Foreign Keys

Rails does not have built in support for foreign keys. In your migrations, you have to execute SQL to create and remove foreign keys. Plenty of folks in the Rails community seem to think they are a good idea but there are issues. One issue we ran in to was that ‘db:test:prepare’ which replicates your database schema for testing does not replicated foreign keys so you can create scenarios that succeed in tests but fail in production.

On the other hand, not having foreign keys when testing made fixtures a bit easier to manage since load order did not matter.

We ended maintaining that foreign keys are important but that not having them in test was okay.

ActiveRecord Queries

Coming from Hibernate, we found the query-building facilities of ActiveRecord to be limited. Very quickly you end up doing a bunch of string-munging to build dynamic queries. has_finder looks like it may address some of these issues.

Refactoring

After years using IDEA for Java development, it was a bit odd to be back in the world of search-and-replace for rename refactors. The IDEA Ruby plugin support for refactoring is improving but still needs work to be generally useful.

The most important lesson we learned in regards to refactoring is to be diligent about naming classes, methods and variables to be consistent with the model names and inflections of your application. Don’t abbreviate. Avoid model objects where the plural is the same as the singular because renaming it later will be harder than if the forms are different.

After using a good refactoring tool to work on Java applications going on four and five years old, I am bit worried about what its going to be like to work on Rails applications that have been around that long. Hopefully the tool support will be there but it will never be as good for a scripting language as it is for Java. Basic refactors like renaming are only going to take longer as a Rails application grows. Keep your test coverage high and refactor regularly or you may find yourself in the regrettable circumstance where the cost of refactoring prevents you from doing it.

Looking Forward

On our next project we are looking forward to:

• SOLR for searching
• RSpec or Mocha for behavior-based testing
• fixtures-scenarios
• Debugging in IDEA
• Markaby for page markup
• has_finder for query building
• Digging into the ActiveRecord source
• Page caching