I created a basic project configuration in TeamCity 6 to checkout my iOS project from git.

I set the artifact paths for this configuration to

project_name.acceptance.app*
ad_hoc/*.png

This will eventually allow TeamCity to collect the icons, project_name.acceptance.app.ipa, and project_name.acceptance.app.plist files needed to perform an over the air install of the ad hoc build.

Finally I created the following custom script build step (available as https://gist.github.com/949831).

#!/bin/bash

# https://gist.github.com/949831
# http://blog.carbonfive.com/2011/05/04/automated-ad-hoc-builds-using-xcode-4/

# command line OTA distribution references and examples
# http://nachbaur.com/blog/how-to-automate-your-iphone-app-builds-with-hudson
# http://nachbaur.com/blog/building-ios-apps-for-over-the-air-adhoc-distribution
# http://blog.octo.com/en/automating-over-the-air-deployment-for-iphone/
# http://www.neat.io/posts/2010/10/27/automated-ota-ios-app-distribution.html

project_dir=`pwd`

# Configuration
environment_name="staging"
keychain="ci_keys"
keychain_password="super secret"
workspace="MyApp.xcworkspace"
scheme="Ad Hoc"
info_plist="$project_dir/MyApp-Info.plist"
environment_plist="$environment_name.plist"
environment_info_plist="$environment_name-Info.plist"
product_name="My App $environment_name"
mobileprovision="$project_dir/ad_hoc/MyAppStaging.mobileprovision"
provisioning_profile="iPhone Distribution: My Company, LLC"
build_number="%env.BUILD_NUMBER%"
artifacts_url="http://my_ci_server.example/artifacts/$build_number"
display_image_name="Icon-57.png"
full_size_image_name="Icon-512.png"

function failed()
{
    local error=${1:-Undefined error}
    echo "Failed: $error" >&2
    exit 1
}

function validate_keychain()
{
  # unlock the keychain containing the provisioning profile's private key and set it as the default keychain
  security unlock-keychain -p "$keychain_password" "$HOME/Library/Keychains/$keychain.keychain"
  security default-keychain -s "$HOME/Library/Keychains/$keychain.keychain"

  #describe the available provisioning profiles
  echo "Available provisioning profiles"
  security find-identity -p codesigning -v

  #verify that the requested provisioning profile can be found
  (security find-certificate -a -c "$provisioning_profile" -Z | grep ^SHA-1) || failed provisioning_profile
}

function describe_sdks()
{
  #list the installed sdks
  echo "Available SDKs"
  xcodebuild -showsdks
}

function describe_workspace()
{
  #describe the project workspace
  echo "Available schemes"
  xcodebuild -list -workspace $workspace
}

function increment_version()
{
  cd "MyApp"
  agvtool -noscm new-version -all $build_number
  cd ..
}

function set_environment()
{
  #copy the info plist for the selected environment into place
  cp -v "MyApp/$environment_info_plist" $info_plist || failed environment_plist
  #copy the environment settings plist into place
  cp -v "MyApp/$environment_plist" "MyApp/environment.plist" || failed environment

  #extract settings from the Info.plist file
  info_plist_domain=$(ls $info_plist | sed -e 's/\.plist//')
  short_version_string=$(defaults read "$info_plist_domain" CFBundleShortVersionString)
  bundle_identifier=$(defaults read "$info_plist_domain" CFBundleIdentifier)
  echo "Environment set to $bundle_identifier at version $short_version_string"
}

function build_app()
{
  local devired_data_path="$HOME/Library/Developer/Xcode/DerivedData"

  #get the name of the workspace to be build, used as the prefix of the DerivedData directory for this build
  local workspace_name=$(echo "$workspace" | sed -n 's/\([^\.]\{1,\}\)\.xcworkspace/\1/p')
  #build the app
  echo "Running xcodebuild > xcodebuild_output ..."

#  disabled overriding PRODUCT_NAME, setting applies to all built targets in Xcode 4 which renames static library target dependencies and breaks linking
#  xcodebuild -verbose -workspace "$workspace" -scheme "$scheme" -sdk iphoneos -configuration Release clean build PRODUCT_NAME="$product_name" >| xcodebuild_output
  xcodebuild -verbose -workspace "$workspace" -scheme "$scheme" -sdk iphoneos -configuration Release clean build >| xcodebuild_output

  if [ $? -ne 0 ]
  then
    tail -n20 xcodebuild_output
    failed xcodebuild
  fi

  #locate this project's DerivedData directory
  local project_derived_data_directory=$(grep -oE "$workspace_name-([a-zA-Z0-9]+)[/]" xcodebuild_output | sed -n "s/\($workspace_name-[a-z]\{1,\}\)\//\1/p" | head -n1)
  local project_derived_data_path="$devired_data_path/$project_derived_data_directory"
  #locate the .app file

#  infer app name since it cannot currently be set using the product name, see comment above
#  project_app="$product_name.app"
  project_app=$(ls -1 "$project_derived_data_path/Build/Products/Release-iphoneos/" | grep ".*\.app$" | head -n1)

  # if [ $(ls -1 "$project_derived_data_path/Build/Products/Release-iphoneos/$project_app" | wc -l) -ne 1 ]
  if [ $(ls -1 "$project_derived_data_path/Build/Products/Release-iphoneos/" | grep ".*\.app$" | wc -l) -ne 1 ]
  then
    echo "Failed to find a single .app build product."
    # echo "Failed to locate $project_derived_data_path/Build/Products/Release-iphoneos/$project_app"
    failed locate_built_product
  fi
  echo "Built $project_app in $project_derived_data_path"

  #copy app and dSYM files to the working directory
  cp -Rf "$project_derived_data_path/Build/Products/Release-iphoneos/$project_app" $project_dir
  cp -Rf "$project_derived_data_path/Build/Products/Release-iphoneos/$project_app.dSYM" $project_dir

  #rename app and dSYM so that multiple environments with the same product name are identifiable
  echo "Retrieving build products..."
  rm -rf $project_dir/$bundle_identifier.app
  rm -rf $project_dir/$bundle_identifier.app.dSYM
  mv -f "$project_dir/$project_app" "$project_dir/$bundle_identifier.app"
  echo "$project_dir/$bundle_identifier.app"
  mv -f "$project_dir/$project_app.dSYM" "$project_dir/$bundle_identifier.app.dSYM"
  echo "$project_dir/$bundle_identifier.app.dSYM"
  project_app=$bundle_identifier.app

  #relink CodeResources, xcodebuild does not reliably construct the appropriate symlink
  rm "$project_app/CodeResources"
  ln -s "$project_app/_CodeSignature/CodeResources" "$project_app/CodeResources"
}

function sign_app()
{
  echo "Codesign as \"$provisioning_profile\", embedding provisioning profile $mobileprovision"
  #sign build for distribution and package as a .ipa
  xcrun -sdk iphoneos PackageApplication "$project_dir/$project_app" -o "$project_dir/$project_app.ipa" --sign "$provisioning_profile" --embed "$mobileprovision" || failed codesign
}

function verify_app()
{
  #verify the resulting app
  codesign -d -vvv --file-list - "$project_dir/$project_app" || failed verification
}

function build_ota_plist()
{
  echo "Generating $project_app.plist"
  cat << EOF > $project_app.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>items</key>
  <array>
    <dict>
      <key>assets</key>
      <array>
        <dict>
          <key>kind</key>
          <string>software-package</string>
          <key>url</key>
          <string>$artifacts_url/$project_app.ipa</string>
        </dict>
        <dict>
          <key>kind</key>
          <string>full-size-image</string>
          <key>needs-shine</key>
          <true/>
          <key>url</key>
          <string>$artifacts_url/$full_size_image_name</string>
        </dict>
        <dict>
          <key>kind</key>
          <string>display-image</string>
          <key>needs-shine</key>
          <true/>
          <key>url</key>
          <string>$artifacts_url/$display_image_name</string>
        </dict>
      </array>
      <key>metadata</key>
      <dict>
        <key>bundle-identifier</key>
        <string>$bundle_identifier</string>
        <key>bundle-version</key>
        <string>$short_version_string $build_number</string>
        <key>kind</key>
        <string>software</string>
        <key>subtitle</key>
        <string>$environment_name</string>
        <key>title</key>
        <string>$project_app</string>
      </dict>
    </dict>
  </array>
</dict>
</plist>
EOF
}

echo "**** Validate Keychain"
validate_keychain
echo
echo "**** Describe SDKs"
describe_sdks
echo
echo "**** Describe Workspace"
describe_workspace
echo
echo "**** Set Environment"
set_environment
echo
echo "**** Increment Bundle Version"
increment_version
echo
echo "**** Build"
build_app
echo
echo "**** Package Application"
sign_app
echo
echo "**** Verify"
verify_app
echo
echo "**** Prepare OTA Distribution"
build_ota_plist
echo
echo "**** Complete!"

That is quite a few functions but the bottom of the script steps through them in what is hopefully an understandable sequence.

1. Unlock the keychain containing the private key and provisioning profile used to sign the ad hoc build. Necessary since the TeamCity user’s login keychain may be locked when this build runs.
2. List the available sdks on the build machine (unnecessary but I found it helpful when debugging build settings).
3. List the schemes found in the workspace to be built (again purely for debugging).
4. Copy a plist containing application settings to “environment.plist” which will be copied into the app bundle when built and used to define application behavior, for example it contains the url of the server this build should communicate with.
5. Build the app using the specified workspace and scheme. Copy the resulting app and dSYM to the project directory so TeamCity can easily find them as build artifacts.
6. Sign the app using the specified mobile provisioning profile and create a “.ipa” package.
7. Verify that the app was successfully signed.
8. Create a plist to allow over the air installation of the app.

Once run any device which has been added to the mobile provisioning profile used to sign this build can install the app just by visiting (using an appropriate %system.teamcity.buildType.id% for the TeamCity build configuration).

itms-services://?action=download-manifest&url=http://teamcity.example.com:8111/guestAuth/repository/download/%system.teamcity.buildType.id%/.lastSuccessful/project_name.acceptance.app.plist

Possible issues:

• Access to the keychain will present a security dialog the first time this build runs so it was necessary for me to sign into the TeamCity user’s account using VNC and allow access to that keychain.
• I found that builds occasionally failed to correctly create the CodeResources symlink so I recreate it manually. When this link was broken the ipa would fail verification.
• An iOS device won’t be able to install the app if the artifacts require authenticating to the TeamCity server so I enabled guest access. Alternately I could have exposed those artifacts through some other service and created a nice installation guide page which links to them if I didn’t want to allow guest access to my TeamCity server.

For additional resources for building over the air distribution builds take a look at:

Mike Nachbaur’s posts on “How to Automate your iPhone App Builds with Hudson” and “Building iOS Apps for Over the Air Ad Hoc Distribution“.
Vincent Daubry’s “Automating Over The Air Deployment for iPhone“.
Basil Shkara’s “Automated OTA iOS App Distribution“.

I also don’t want to spend time manually updating documentation every time I change the API and don’t want to risk wasting other developers’ time by providing out of date or inaccurate documentation.

Instead I hacked together a rake task to covert my API controller specs into documentation using a custom rspec formatter. Now I can automatically update the documentation as part of our CI build process.

This example is available as https://gist.github.com/934902

First I wrote a rake task to generate my API documentation. This runs my API controller specs after adding some additional files to the ruby load path and requiring them to patch ActionController::TestCase::Behavior and load a custom rspec formatter.

require 'rubygems'
require 'rspec/core/rake_task'

namespace :project_name do
  namespace :api do
    desc 'Generate API request documentation from API specs'
    RSpec::Core::RakeTask.new('generate_docs') do |t|
      t.pattern = 'spec/controllers/api/**/*_spec.rb'
      t.rspec_opts = ["-Ilib/tasks/api_docs", "-rinstrument_specs", "-rapi_formatter", "--format APIFormatter"]
    end
  end
end

I patched ActionController::TestCase::Behavior to capture the requests and responses from process (called by get, put, post, and delete) and store them in rspec’s example metadata.

require 'action_dispatch'
require 'action_controller'

module InstrumentSpecs
  module ::ActionController::TestCase::Behavior
    alias_method :process_without_logging, :process

    def process(action, parameters = nil, session = nil, flash = nil, http_method = 'GET')
      version, route = described_class.to_s.match(/::(Vd)::(S+)Controller/)[1..2].collect(&:downcase)
      curl = "curl "
      parameters.each {|key, value| curl << "-d "#{key}=#{CGI.escape(value)}" " } if parameters.present?
      curl << "-X #{http_method} "
      curl << "https://api.host.example/#{version}/#{route} "
      example.metadata[:curl] = curl

      response = process_without_logging(action, parameters, session, flash, http_method)
      example.metadata[:response] = "#{response.status} #{response.body}"

      response
    end
  end
end

Finally I added a custom rspec formatter to output the requests and responses as markdown.

require 'rspec/core/formatters/base_formatter'

class APIFormatter < RSpec::Core::Formatters::BaseFormatter
  def initialize(output)
    super(output)
  end

  def example_group_started(example_group)
    indented_message("#{example_group.description}n", example_group)
  end

  def example_group_finished(example_group)

  end

  def example_started(example)
    indented_message("#{example.description}n", example)
  end

  def example_passed(example)
    indented_message("#{example.metadata[:curl]}n")
    indented_message("#{example.metadata[:response]}nn")
  end

  def example_failed(example)

  end

  def example_pending(example)

  end

  private

  def indented_message(message, context = nil)
    if context.present?
      if context.respond_to?(:example_group)
        (context.example_group.ancestors.count + 1).times do
          output << '#'
        end
      else
        context.ancestors.count.times do
          output << '#'
        end
      end
    else
      output << '    '
    end
    output << message
  end
end

#Api::V1::AccountsController
###index
###without an API key
####behaves like an unauthenticated user
#####responds with success status
curl -X GET https://api.host.example/v1/accounts
200 {"error":{"type":"Unauthorized","message":"Authentication is required to access this resource"}}
...


Now I can generate a description of my API based on my specs complete with examples of hitting the API using curl and the expected response status and body. As long as I name my rspec assertions and contexts well this should be legible and always up to date since I can regenerate the docs and push them to a github wiki page every time my specs pass on a continuous integration server.

I have been able to run test suites from the command line but only for “logic” tests. From the iOS Development Guide:

Xcode offers two types of unit tests: logic tests and application tests.

Logic tests.
These tests check the correct functionality of your code in a clean-room environment; that is, your code is not run inside an application. Logic tests let you put together very specific test cases to exercise your code at a very granular level (a single method in class) or as part of a workflow (several methods in one or more classes). You can use logic tests to perform stress-testing of your code to ensure that it behaves correctly in extreme situations that are unlikely in a running application. These tests help you produce robust code that works correctly when used in ways that you did not anticipate. Logic tests are iOS Simulator SDK–based; however, the application is not run in iOS Simulator: The code being tested is run during the corresponding target’s build phase.

Application tests.
These tests check the functionality of your code in a running application. You can use application tests to ensure that the connections of your user-interface controls (outlets and actions) remain in place, and that your controls and controller objects work correctly with your object model as you work on your application. Because application tests run only on a device, you can also use these tests to perform hardware testing, such as getting the location of the device.

I am able to create a set of logic tests as a separate build target and scheme which will successfully run as part of an xcodebuild build.

1. Create a new “LogicTests” unit test build target.
2. Leave the “Test Host” build setting blank.
3. Set the “Test After Build” build setting to “No”.
4. Do not write tests which require an application to be present (ie no window or application delegate available).
5. Create a new “LogicTests” scheme which includes the “LogicTests” build target in its run action.
6. Run the “LogicTests” scheme using xcodebuild.

Scheme for running logic tests

> /Developer_Xcode4/usr/bin/xcodebuild -workspace WrappingScrollView.xcworkspace -scheme logictests -sdk iphonesimulator4.3 -configuration Debug clean build

…

Run test suite logictests
Test Suite ‘logictests’ started at 2011-04-06 07:44:57 +0000
Run test case testExample
Test Case ‘-[logictests testExample]‘ started.
/Users/Jonah/Desktop/WrappingScrollView/WrappingScrollViewDemo/logictests/logictests.m:30: error: -[logictests testExample] : Unit tests are not implemented yet in logictests
Test Case ‘-[logictests testExample]‘ failed (0.000 seconds).

Test Suite ‘logictests’ finished at 2011-04-06 07:44:57 +0000.
Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.000) seconds

That’s not a great solution but it at least allows me to run some tests on a continuous integration server. Hopefully I can find a way to run application tests through xcodebuild as well.

In case it is of any help to other developers struggling with the difference between command line and Xcode builds. The “Run Script” build phase of a unit test build target just runs “${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests”. The RunUnitTests script in turn invokes a platform specific script like “${SYSTEM_DEVELOPER_DIR}/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests” which ultimately calls RunTestsForBundle in “${SYSTEM_DEVELOPER_DIR}/Tools/RunPlatformUnitTests.include”. Hopefully it will prove possible to use these scripts to match the behavior seen when running tests from within Xcode to automatically run application tests.

> /Developer_Xcode4/usr/bin/xcodebuild -help
Usage: xcodebuild [-project ] [[-target ]…|-alltargets] [-configuration ] [-arch ]… [-sdk [|]] [=]… []…
xcodebuild -workspace -scheme [-configuration ] [-arch ]… [-sdk [|]] [=]… []…
xcodebuild -version [-sdk [|] [] ]
xcodebuild -list [[-project ]|[-workspace ]]
xcodebuild -showsdks

The workspace and scheme names should be known from the project. The default configuration options are “Debug” and “Release”. The “-showsdks” option lists available SDKs.

> /Developer_Xcode4/usr/bin/xcodebuild -showsdks
Mac OS X SDKs:
Mac OS X 10.6 -sdk macosx10.6

iOS SDKs:
iOS 4.3 -sdk iphoneos4.3

iOS Simulator SDKs:
Simulator – iOS 3.2 -sdk iphonesimulator3.2
Simulator – iOS 4.0 -sdk iphonesimulator4.0
Simulator – iOS 4.1 -sdk iphonesimulator4.1
Simulator – iOS 4.2 -sdk iphonesimulator4.2
Simulator – iOS 4.3 -sdk iphonesimulator4.3

Available build actions are listed in the xcodebuild man pages. Unfortunately these do not match the actions available in a scheme and the command line does not provide a mechanism for testing, running, profiling, or analyzing builds.

> man xcodebuild
…

buildaction …
Specify a build action (or actions) to perform on the target. Available build actions are:

build Build the target in the build root (SYMROOT). This is the default build action.

archive Archive a scheme from the build root (SYMROOT). This requires specifying a workspace and scheme.

installsrc Copy the source of the project to the source root (SRCROOT).

install Build the target and install it into the target’s installation directory in the distribution root (DSTROOT).

clean Remove build products and intermediate files from the build root (SYMROOT).

Given all of these configuration options it is then possible to perform builds from the command line.

> /Developer_Xcode4/usr/bin/xcodebuild -workspace WrappingScrollView.xcworkspace -scheme WrappingScrollViewDemo -sdk iphonesimulator4.3 -configuration Debug clean build
Build settings from command line:
SDKROOT = iphonesimulator4.3

=== CLEAN NATIVE TARGET JCFWrappingScrollView OF PROJECT JCFWrappingScrollView WITH CONFIGURATION Debug ===
Check dependencies

…

** CLEAN SUCCEEDED **

=== BUILD NATIVE TARGET JCFWrappingScrollView OF PROJECT JCFWrappingScrollView WITH CONFIGURATION Debug ===
Check dependencies

ProcessPCH /Users/Jonah/Library/Developer/Xcode/DerivedData/WrappingScrollView-fqscqfdtpoqunuanehtbgpseirdj/Build/PrecompiledHeaders/JCFWrappingScrollView-Prefix-dkkjrdaonxokgpbkgjfmpmcpadoz/JCFWrappingScrollView-Prefix.pch.gch JCFWrappingScrollView/JCFWrappingScrollView-Prefix.pch normal i386 objective-c com.apple.compilers.llvmgcc42
cd /Users/Jonah/Desktop/WrappingScrollView/JCFWrappingScrollView
setenv LANG en_US.US-ASCII

…

** BUILD SUCCEEDED **

Hopefully that is sufficient to automatically produce ad-hoc builds on a continuous integration server.

Running unit tests with GTM
I have been using GTM to run iPhone unit tests as part of the build process for a “Unit Tests” build target (see Alon’s post on our iPhone Unit Testing Toolkit). Running these tests with a TeamCity build agent just requires a command line build runner which calls the `xcodebuild` command and specifies the project, built target, and configuration to build.

Ad-hoc builds
Ad-hoc builds run with a similar command:

Now my ad-hoc builds are produced automatically, always up to date with my working code changes, and available as an easy download from my TeamCity server. TeamCity will also save the artifacts for each build so I can easily retrieve the dSYM for a particular ad-hoc build to symbolicate a crashlog for that build on any developer’s machine.

Future improvements
If necessary I can run tests on actual iPhone hardware by changing the sdk (`-sdk iphoneos4.0`) but I am not yet sure how I could specify which one of multiple devices would run the tests. It would be nice to eventually be able to choose between multiple devices so I can deliberately run some tests on a device with a specific OS version or feature set.
With my current configuration TeamCity can only report a pass/fail status for iPhone tests. I need to consider modifying the GTM test runner script or switching to GHUnit or Cedar to get per-test results and then be able to build up a history of individual test failures.
I also need to add a build configuration which uses agvtool to automatically increment the project’s build number as part of every ad-hoc build.

Much like shared github access I need to be able to specify which ssh key a particular project’s connections to heroku should use. Since I want to use client side checkouts I can’t specify a private key within TeamCity but I do know that all my compatible build agents are local so I can add a new entry to `~/.ssh/config` and define a host which will allow me to connect to Heroku with a particular key.

# Heroku for #{new_project}
Host heroku-#{new_project}
HostName heroku.com
User git
ForwardAgent yes
IdentityFile /home/teamcity/.ssh/teamcity_#{new_project}_rsa


Add a ssh key to the Heroku account
Now I need to add this new key to the Heroku account for this project by installing the heroku gem in my project’s gemset and using the `heroku` command line to add the key. Heroku stores the account credentials used in `~/.heroku` and the user account running my build agents might need to connect to many different heroku accounts. I could switch my heroku credentials, upload a key, and have different projects deploy using different accounts. However since I find it useful to be able to run the heroku command line (see below) I found it preferable to have a single `teamcity` heroku account added to every project as a collaborator.
If you find it necessary to switch between Heroku accounts frequently this tool from Aeonscope might be useful: managing multiple heroku accounts

Now I can add a key to the current project’s Heroku account.
heroku keys:add ~/.ssh/teamcity_#{new_project}_rsa

Deploying to Heroku
Now I can add a simple TeamCity build configuration which pushes code to Heroku once a particular revision passes the tests.

A project's deployment build configuration

I found that I need to add a delay between turning on maintenance mode for the app and pushing to heroku. Attempting to push immediately fails with the error `Heroku push rejected, your slug is currently being compiled. Please try again later.` Even with a significant delay I still see this error occasionally and have to rerun the deploy build but at this point I don’t know of a way to make sure heroku can handle a new revision before attempting to push changes.

Now I can use TeamCity to automatically deploy changes to Heroku as soon as my project’s tests pass.

1. Create a key for github access
> ssh-keygen -t rsa -C "#{project_name}"
Upload the public key to github as a deploy key for the project so that the build agent can pull from the github repository.

2. Setup rvm and a gemset for the project
> rvm install ruby-#{version}-r#{patch level}
> rvm use ruby-#{version}-r#{patch level}
> rvm gemset create #{project name}
> rvm gemset use #{project name}
> gem install bundler

This could probably all be automated as part of the project’s .rvmrc file.

3. Setup a ssh config for the project
The keys used by many of the services we interact with are unique and tied to an account so we need to have TeamCity use different ssh keys when connecting to projects under different accounts. Add a new entry to ~/.ssh/config to alias a host to use a specific ssh key

# GitHub for #{new_project}
Host github-#{new_project}
HostName github.com
User git
ForwardAgent yes
IdentityFile /home/teamcity/.ssh/teamcity_#{new_project}_rsa


Connections to `github-#{new_project}` will now default to use `teamcity_#{new_project}_rsa` rather than the default `id_rsa`. By specifying the key in the ssh config file we’re able to use client side checkouts which will be useful when deploying using git. For projects able to use server side checkouts we can specify a private key to use in TeamCity’s VCS settings.

4. Create a new TeamCity project
On projects with multiple build steps using the same build runner we could create a template to avoid repeating these settings in each build phase. Unfortunately TeamCity 5.1 templates do not support running multiple types of build runners so projects with both command line (`bundle install`) and rake runners (`rake spec`) require some repetition.

Client checkouts have a full git repository while a server side checkout performs a git pull and then copies the resulting files but not the full git work-tree to the build agents leaving the agents unable to easily run git commands. A client side checkout is therefore preferable when using `git push` to deploy a project.

If using a template set the `rake.tasks`, `rake.opts`, and any other configuration parameters the template uses. Each build configuration based on the template can specify its own configuration parameters but cannot override the rake task and command line parameters directly.

To use the project’s RVM ruby and gemset add the following environment variables with appropriate paths:

Now I can finally build a set of build configurations for a project which will all use the project’s RVM settings to isolate it from other projects on the TeamCity server.

JetBrains appears committed to adding RVM support to TeamCity and added the `RVM Gemset` property to rake tasks in TeamCity 5.1.3 so hopefully we’ll see additional RVM support in future releases.

Engine Yard will eventually allow us to push a branch to our cloud environment from git (ie “git push engineyard master”) but until that is available the best option seems to be to trigger a github post receive hook directly.

To that end I rewrote an existing script to trigger post receive hooks as a rake task; http://gist.github.com/271433

require 'rubygems'

require 'json'

begin

  require 'restclient'

rescue Exception

end

namespace :ey do
  # Based on http://gist.github.com/114954

  # Discussed at https://cloud-support.engineyard.com/discussions/questions/83-deploying-to-solo-instance-from-automated-build

  desc 'Trigger a GitHub Post Receive Hook to deploy the app'

  task :deploy do

    post_receive_hook 'preview'

  end
  desc 'Trigger a GitHub Post Receive Hook to deploy and migrate the app'

  task :deploy_and_migrate do

    post_receive_hook 'preview', true

  end
  def post_receive_hook(envname, migrate=false)

    token = `git config --get cloud.token`.chomp
    if token.empty?

      token = ENV['cloud.token']

      end
    if token.empty?

      puts < [{"id" => commit, "message"=>"[deploy #{envname}#{migrate ? ' migrate' : ''}]"}],

        "ref" => "refs/heads/master",

        }
    puts "Triggering a deploy for #{commit} on #{envname}"

    begin

      response = RestClient.post("https://cloud.engineyard.com/github/#{token}",

                                 :payload => payload.to_json)

      puts "Successfully triggered the deploy"

    rescue RestClient::RequestFailed => e

      puts "Could not deploy your changes"

      puts e.response.code

      puts e.response.body

      exit 1

    end

  end

end

[/sourcecode]
In setting this up I discovered a couple of potential stumbling points:

The environment must have been deployed via the EY dashboard before you will be able to trigger a deploy using the post receive hook API.
The branch specified in the API call (and therefore the git branch you should be working in if running that rake task) must match the branch deployed using the EY dashboard.
If you make an API call to deploy the 'master' branch EY will attempt to deploy branch 'HEAD'.
If the branch specified in the API call does not match the branch specified on the dashboard the API will report success but not redeploy the environment.

The folks at Engine Yard are working to correct some of those issues and now I've got a project which automatically pushes changes to the cloud every time our tests cycle.
]]>
			http://blog.carbonfive.com/2010/01/07/automatically-deploying-to-engine-yard-cloud/feed/
		0
		
		
		
		http://blog.carbonfive.com/2009/03/04/continuous-integration-and-build-promotion/
		http://blog.carbonfive.com/2009/03/04/continuous-integration-and-build-promotion/#comments
		Wed, 04 Mar 2009 17:24:33 +0000
		Christian Nelson
				
		

		http://blog.carbonfive.com/?p=340
		Continue reading →]]>
			We have a build server and we practice continuous integration on all of our projects.  In fact, it’s pretty much the first thing we set up after version control.  We’re feedback junkies.  It became especially apparent while working on a client project last year where we used their development infrastructure.  They had a build server running, but the problem was that it took too long to get feedback.  I was lost and jonesing.  They had a monolithic build that took about 45 minutes to run its course and give me the affirmation I was seeking.

While setting up my current project, I decided to take a different approach when configuring our build server and continuous integration.  I had a few goals in mind:

Super fast feedback
“Promoted” builds get pushed to the acceptance server
Minimize issues with the build infrastructure

I split our build into three plans:

Continuous Integration Build Plan
This resets the unit test database (using c5-db-migrations), compiles the project, runs all of the unit tests, and if there are no errors or failures, produces a war.  The war is installed locally using Maven so that it’s accessible to other processes in a known location.  This build is very fast and is triggered on every subversion commit.  The command used for this build plan is:
mvn db-migration:reset clean install
Functional Tests Build Plan
This resets the functional test database, deploys the war built during the continuous integration build to tomcat using Cargo, runs all of the functional tests, and shuts down tomcat.  This build is triggered on every successful continuous integration build (i.e. as a dependent build).  A very short script performs the work of this build:
mvn -Pdev db-migration:reset

cd functional-tests; mvn -Pdev clean test-compile cargo:start surefire:test cargo:stop
And here’s the cargo-maven-plugin configuration:
&lt;plugins&gt;
...
    &lt;plugin&gt;
        &lt;groupId&gt;org.codehaus.cargo&lt;/groupId&gt;
        &lt;artifactId&gt;cargo-maven2-plugin&lt;/artifactId&gt;
        &lt;version&gt;1.0-beta-2&lt;/version&gt;
        &lt;configuration&gt;
            &lt;wait&gt;false&lt;/wait&gt;
            &lt;configuration&gt;
                &lt;deployables&gt;
                    &lt;deployable&gt;
                        &lt;groupId&gt;com.acme&lt;/groupId&gt;
                        &lt;artifactId&gt;acme-web&lt;/artifactId&gt;
                        &lt;type&gt;war&lt;/type&gt;
                        &lt;properties&gt;
                            &lt;context&gt;acme-web&lt;/context&gt;
                        &lt;/properties&gt;
                    &lt;/deployable&gt;
                &lt;/deployables&gt;
            &lt;/configuration&gt;
            &lt;container&gt;
                &lt;containerId&gt;tomcat6x&lt;/containerId&gt;
                &lt;zipUrlInstaller&gt;
                    &lt;url&gt;http://www.apache.org/dist/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip&lt;/url&gt;
                &lt;/zipUrlInstaller&gt;
            &lt;/container&gt;
        &lt;/configuration&gt;
    &lt;/plugin&gt;
&lt;/plugins&gt;

Deploy to Acceptance Build Plan
This step doesn’t build anything or run any tests, but is a little more complicated than the others because it’s interacting with a another machine: our dedicated project acceptance server.  We scp the war (built during the continuous integration phase) to our acceptance server.  Next, we shutdown tomcat and clean a few things up (logs, old webapp, and work).  Then we migrate the database (no reset because we care about the data).  Last, we bring tomcat back up with the new war.  This build is triggered on every successful functional test build.  Here’s the script that’s run by the build server:
ARTIFACT_NAME=./ROOT.war

# SCP Application and Scripts
WAR=`find ~/.m2/repository/com/acme/acme-web -name &quot;*-*.war&quot; | sort | tail -1`
echo &quot;Copying $WAR to acceptance server.&quot;
scp $WAR acme@acme-acceptance:$ARTIFACT_NAME
scp ./bin/*_as.sh acme@acme-acceptance:.

# Shutdown and Clean Tomcat
ssh acme@acme-acceptance sh ./shutdown_as.sh

# DB Migration
mvn -Pdev db-migration:migrate -Djdbc.host=acme-acceptance

# Install Application
ssh acme@acme-acceptance mv $ARTIFACT_NAME ./apache-tomcat/webapps/

# Startup
ssh acme@acme-acceptance sh ./startup_as.sh

Conclusion
This has worked out really well for the project and we get feedback very quickly.
You may wonder why we broke functional tests into their own plan.  I find that functional tests can be a little less stable than unit tests (especially if selenium is involved), and they run much more slowly.  I’ve seen cases where flaky functional tests caused a team to start ignoring build results because it was usually a problem with the infrastructure, not with the code.  So, the decision was somewhat defensive and in retrospect, probably unnecessary.
We’ve spent less time maintaining our build plans than in the past as well.  At some point, our build server MySQL instance crapped out and even though all of the databases were deleted, our builds all ran successfully when the build server came back up because they start with a database reset, which creates the target database and migrates it to the latest schema.
A single war is promoted as it passes a greater level of testing, and is eventually deployed to the acceptance server if all of the tests pass.  While we’re saving a little time by not rebuilding the archive for each plan, that’s not the only thing I like about it.  It just feels a little more right and it completely eliminates the chance that something about the artifact changes as it makes its way through the pipelines.  The same war can make its way from the first CI build all the way to production.  This is possible because we include default configuration for the application which matches our development environments, and then provide a mechanism for externalizing application configuration for the one-off environments.
I think most modern build server software provides everything you need to do something like this as it’s rather straight forward.  However, for those who are curious, we have been using Bamboo for the last year or two and recently installed TeamCity so that we can give it a proper try.  Both are great products, and if you’re on a smallish team, TeamCity is completely free (and is superior to the open source alternatives, IMHO).
How are you using your build server?
]]>
			http://blog.carbonfive.com/2009/03/04/continuous-integration-and-build-promotion/feed/
		2