Happy JS hacking!

Outside-in development doesn’t require any specific tool or language. This article will demonstrate it in Rails, using two popular testing tools: Cucumber and RSpec.

Start with a High-level Specification

Starting with a high-level specification requires you to have a clear understanding about what you want to achieve. If it’s still unclear, now is the time to have a conversation with the client. After establishing a clear goal, we can use Cucumber to turn a plaintext story into executable code.

Our sample story will be from a news site. The feature is a JSON API endpoint for news articles.

This story can be copied directly into a Cucumber feature.

features/api/v1/articles.feature

Let’s run this feature to figure out what to do next.

Fantasy Coding

Cucumber successfully parsed our feature but it needs definitions for all of our steps. Let’s implement these steps writing code that we wish already existed.

features/step_definitions/api/v1/articles_steps.rb

In our two Given steps, we establish a context consisting of published and unpublished articles. These two factories don’t exist yet; we just wrote the code we wish we had. This is a major benefit of developing from the outside in. By writing code that doesn’t even exist, you’ll end up creating ideal objects and interfaces.

In our When step, we exercise our application by first setting a proper HTTP header and then making an HTTP GET request to a non-existent URL. Again, this URL doesn’t exist, it’s just where we would expect the articles to be.

Finally in our Then step, we verify the response. Here we begin to discover our domain model; imagining an Article class with a #published? instance method.

Let Cucumber Guide the Way

With our steps defined, a rerun of Cucumber fails in our Given step because our factories don’t exist yet.

Let’s define these two factories.

spec/factories/articles.rb

Cucumber guides us toward our next step.

Our factories are being automatically mapped to a non-existent Article class. We can use the Rails model generator to create this class. We’ll also specify the title and published attributes we referenced in our Then step.

With our setup passing, Cucumber now fails at our API request.

Drill Down to the Controller

We need to add a route for our API endpoint.

config/routes.rb

With the routes in place, Cucumber tells us we’re missing a constant. This particular error message is from the Rails #namespace method we used in our routes file. Namespacing our controller will fix this issue. We can use the Rails controller generator to create a namespaced controller class.

Cucumber fails again, this time looking for an #index action in our controller.

At this point, some outside-in practitioners will also drop down a level with respect to testing and use RSpec to spec out the controller. Since we’re new to outside-in development, I’m going to skip this step and go straight to the implementation. We’ll look at the value of controller specs and when it makes sense to write them later on.

app/controllers/api/v1/articles_controller.rb

Now we fail because our #index action needs a template. Let’s create an empty template just so we can finally get a non-infrastructure related failure, i.e., a logic error, from Cucumber.

Drill Down to the View

With the routing and request handling boilerplate out of the way, we finally get a “legitimate” failure from Cucumber. At this point, some outside-in practitioners will drop down a level with respect to testing and use RSpec to spec out the view. Like controller specs, I’m going to skip this step for simplicity. We’ll discuss when a view spec makes sense later on. For now, let’s update our blank view to actually render some JSON (we’ll use the jbuilder Gem to construct the JSON).

app/views/api/v1/articles/index.json.jbuilder

Again, we write this view imagining an “articles” instance variable; ideally, a collection containing our articles. This helps us avoid setting up unnecessary state in our action.

Cucumber fails again, this time with a long stacktrace (abbreviated below) originating in jbuilder.

This failure is because the instance variable doesn’t exist yet. Let’s update our Api::V1::ArticlesController#index action to find all published articles.

app/controllers/api/v1/articles_controller.rb

We’ve kept our controller thin and decided to not directly test it. By keeping controller logic to a minimum, skipping controller tests isn’t a significant risk.

Cucumber now guides us to our domain model.

Drill Down to the Model

Despite skipping controller and view specs, I do feel it’s beneficial to drill down a layer in our tests and directly test the model. Model tests will shorten our testing feedback loop and allow us to specify at a level closer to the code. Skipping model tests and relying on Cucumber, keeps the feedback loop too large, slowing you down.

spec/models/article_spec.rb

RSpec will now be our guide.

app/models/article.rb

app/models/article.rb

With Article.published specified, we can return to Cucumber.

Jump Back Up to Cucumber

Cucumber is now passing and our story is complete.

The complete code for this example can be found on github.

Change Your Perspective

I use the above approach on every feature I write. At each layer, I find myself getting lazier and lazier, delaying the hard work until the very end. At that point, I’m in the domain model, the heart of the application, and where the majority of logic should be. By taking a client perspective at each layer, the resulting objects remain simple, have expressive interfaces, and logic naturally finds its home.

You can begin developing from the outside in at every one of your application’s interfaces. The example above demonstrated a JSON API. Traditional HTML interfaces can easily be tested using capybara. And if you’re developing a command line interface, perhaps for a Ruby gem, take a look at aruba.

Do I Need to Test at Every Layer?

The RSpec book suggests writing tests at each layer, i.e., view specs, controller specs, helper specs, and model specs. I’ve tried this approach several times but I usually feel all the lower level specs, except model specs, aren’t worth it. They do shorten the testing feedback loop, but their reliance on stubbing and mocking to achieve true isolation makes refactoring and maintenance difficult. I also keep the logic to such a minimum in these objects, e.g., controllers, that the additional fine-grained unit tests don’t provide that much benefit.

There is nothing wrong with not unit testing each part of your application. Don’t dogmatically insist that everything be unit tested. Oftentimes a higher-level integration test will sufficiently exercise (albeit indirectly) a particular piece of code. The tradeoff here is that your testing feedback loop will be large. You’ll need to execute a full-stack Cucumber test just to see if a change you made, perhaps in a controller or a view, passes your failing test. Occasionally, I’ll use a lower level view or controller test to handle an edge case but this is a pretty rare occurrence. This is just my personal style. I would recommend trying out testing at every level, especially view and controller tests, to see how it feels and if it’s beneficial for you and your team.

Give It A Try

Outside-in development often feels strange to newcomers. Most developers prefer to start with the “important” part of an application, i.e., the domain model, and work their way outwards. Thinking like a server and not a client can lead to overengineering by implementing more than you need. Your resulting objects and their interfaces will also be less than optimal, or at least take longer to get quite right.

Like most things in Rails, the tools for developing outside-in are easy to setup and configure. If Cucumber or RSpec aren’t your thing, adapt your favorite tool and give outside-in development a try.

Monkey-patching is extending or modifying the behavior of code at runtime without changing its original source code. You can monkey-patch any code, it doesn’t matter whether it’s your own code or not. This is distinctly different than traditional sub-classing because you are not creating a new class, instead, you are reopening an existing class and changing its behavior.

Monkey-patching is possible in Objective-C by using categories. In fact, the definition of a category practically matches that of monkey-patching:

"A category allows you to add methods to an existing class—even to one for which you do not have the source."

In this series of posts, we’ll use categories to add and change methods, to add new instance variables and properties, and introduce swizzling, a technique that allows us to extend and preserve existing functionality. TL;DR »

Category Basics

To modify an existing class specify the category in both its interface and implementation definitions:

Interface Definition

Implementation

Adding Simple Methods

The most basic usage of categories is to add a new method to an existing class.

Suppose in our application we want to output dates relative to the current time, e.g., “13 minutes ago”, “4 hours ago”, “just now”, etc. Traditional object-oriented solutions would have us introducing a new class that either extends NSDate (e.g., creating a RelativeDescriptionDate subclass with a timeAgoInWords instance method) or is a standalone helper/utility class (e.g., [NSDateHelper timeAgoInWordsFromDate:myDate]).

But with categories, we can reopen the NSDate class and simply add a new instance method:

NSDate+Formatting.h

NSDate+Formatting.m

Now every NSDate object will have the new method available to it. The following code:

Will print out the following on the console:

The Dangers of Simply Overriding Methods

We can take this a step further and instead of adding new behavior we’ll override existing behavior. Continuing with our example, what if we wanted the default description of a NSDate object to include the time ago in words? We could simply do the following:

However, this is strongly discouraged for two reasons.

1. Other frameworks may rely on the expected behavior of the original method. We now have to go through the trouble of re-implementing that behavior, in addition to the new functionality we wanted to introduce, or risk strange side effects and possibly even crashing out.
2. If multiple categories implement the same method, the last one loaded wins! The load order is consistent within an application, but it’s arbitrary, out of our hands, and fragile. For all we know, our implementation could itself be overwritten by an internal framework category!

Because of these reasons, this blunt approach to overriding methods should only be used for the simplest of cases. Later in this series, we’ll explore how swizzling allows us to override a method while preserving all implementations.

Including Your Monkey-Patches

Categories are not automatically “picked up” in a project. Any code that relies on the behavior will need to #import the necessary header files:

However, including the same set of headers over and over again is redundant. We should first create a single header file that imports all of our most frequently used categories:

We can then import this single header file into a prefix header that is added to all source files. XCode projects often have a .pch file in the Supporting Files group for this very purpose.

Next Up

While adding and overriding classes is straightforward, there is one very big caveat when using Categories; you cannot add new instance variables to a class. We’ll take a look at working around this limitation in the next post.

tl;dr

• Use Objective-C categories to add functionality to existing classes without subclassing.
• Avoid simple overrides with categories as it can cause problems with other frameworks.
• Use prefix headers to easily import your extensions.

I've been working on a post here for C5 and in order to make my points, I was referencing gists hosted by GitHub. When those gists were shown as embeds, the line numbers, and thus the clarity of my documentation, was lost. We use Markdown for the creation of the texts so I needed something I could write inside of a Markdown post that would be applied as CSS.

Thanks to these tips:

1. http://www.whyhat.com/2012/01/line-numbers-on-embedded-gists
2. http://stackoverflow.com/questions/524696/how-to-create-a-style-tag-with-javascript

I was able to come up with the following:

Designing with services and a service layer is popular in the Java J2EE/Spring community. However, it’s not common in the Rails world. Is this because of the Ruby community’s general backlash against the complexity of Java? Or has the rise of HTTP-based JSON APIs made this architecture obsolete? To answer these questions, let’s take a deeper look at services and the benefits of a service layer.

What is a Service?

In Domain-Driven Design, Evans defines a service as an operation offered as an interface that stands alone in the model. In other words, a service is an action, not a thing. And instead of forcing the operation into an existing object, we should encapsulate it in a separate, stateless service.

It’s not always clear what logic constitutes a service. Taken to the extreme, domain models could be stripped entirely of their domain logic, and turned into nothing but simple data structures (an AnemicDomainModel). A good heuristic is to look for large, domain objects doing too much work and try to break their multiple responsibilities into separate services. At other times it may be your own intuition telling you that some logic just doesn’t feel to be in the right place. To get started, it’s helpful to classify services into different types.

Types of Services

Evans defines three types of services:

• Application
• Domain
• Infrastructure

Application Services

An application service is a service that provides non-infrastructure related operations that wouldn’t come up when discussing the domain model outside the context of software, i.e., in its natural environment.

Exporting account transactions as a CSV file in a banking app is an example of an application service because a file format, CSV, has no meaning in the domain of banking.

Application services improve the cohesiveness of your domain model by preventing “software” implementation details from leaking into it.

Domain Services

A domain service scripts a use-case involving multiple domain objects. Forcing this logic into a domain object is awkward because these use-cases often involve rules outside the responsibility of any single object.

Continuing with the banking app, a funds transfer would be an example of a domain service. Transferring funds doesn’t quite fit in either of the involved account objects, or perhaps a customer object, so it becomes a standalone service.

An alternative implementation, could be an instance method on Account, taking the amount and destination account as parameters. However, transferring money between accounts feels like the responsibility of another object, not a core responsibility of an account. Domain services like this one really help maintain the core responsibilities of your domain objects.

Another example of a domain service is a sitewide search across multiple domain models. Since it searches across multiple domain models it doesn’t conceptually fit in any one particular model. Instead, it should live in a separate search service.

Simple CRUD operations are not domain services. For example, I would consider the following an anti-pattern:

Sure, this does make the UserController independent of the domain model, but this type of flexibility is premature. Instead, the controller should interact directly with the domain model when creating a new User. Domain services should be short scripts, not simple 1-liners that delegate to an almost identical domain model interface.

Infrastructure Services

An infrastructure service encapsulates access to an external system. Infrastructure services are often used by application and domain services.

E-mailing and message queuing are two examples of infrastructure logic that has no place in a domain model.

Infrastructure services make your domain model more reusable by separating out non-domain, “software” concerns.

What is a Service Layer?

In PoEAA, Stafford defines a service layer as an application’s boundary and set of available operations from the perspective of interfacing client layers. In other words, it’s your application’s API.

Let’s take a look at some of the benefits of a service layer to see if they are useful in the Rails world.

Supporting Multiple Clients

In most web apps, the API mainly consists of simple CRUD use-cases. Encapsulating your application’s API, allows it to be reused across multiple client controllers, for example, an HTML and a Thrift client. The resulting controllers are thinner, and more cohesive.

I see this reuse across multiple clients as less and less of a benefit because of the popularity of HTTP-based JSON APIs. Most apps will typically have an HTML client and a few additional clients, for example, an iPhone and a console app, but the same controllers will serve both HTML and JSON. In a way, the controllers have become the services because all clients talk to the app over HTTP.

This reuse was more valuable when you could have non-HTTP based clients, for example, an Adobe Flash app using Hessian.

Encapsulating Application Logic

Stafford separates business logic into domain and application logic. Domain logic being purely the problem domain. Application logic dealing with technical responsibilities, for example, coordinating a workflow or sending an email.

In order to build a more cohesive and reusable domain model, application logic should be handled by infrastructure services, possibly in a service layer. Putting application logic into a domain model that’s used by multiple clients may have unwanted side effects. For example, if you want HTML clients to be emailed after user registration but don’t want your JSON API clients to be notified at all. So instead of emailing users from the domain model…

…let your controller ask a service to do it.

The Rails community definitely has gone back and forth on where to put application logic. Creating a service layer for it is too much though. Instead, put it the appropriate respond_to block in your controllers.

Applying Cross-Cutting Concerns

Java web apps using the Spring Framework, commonly use services to apply cross-cutting concerns, for example, transaction management and security.

Transaction management is a common justification for a service layer, specifically transaction management spanning multiple transactional resources. For example, in the Spring Framework, it’s possible to atomically write to a database, send an email, and enqueue a message.

In a Rails app, ActiveRecord provides automatic, and if necessary manual, transaction management. I can’t say I’ve ever thought much about transaction management across multiple transactional resources, but I see no reason for it not to belong in a controller. I’m not entirely sure why Spring discouraged transaction management at the controller level. Maybe for easier testing.

When Should I Use A Service Layer

The only reason to use a service layer is if you have to support multiple clients using different protocols. HTTP and JSON have become the de facto client communication protocol and format, eliminating the need for a service layer. The additional indirection and flexibility of a service layer should only be considered when you have to add a client that doesn’t speak HTTP. However, it’s still beneficial to use Evans-like services from your controllers in order to encapsulate domain, application, and infrastructure logic and preserve the cohesiveness of your domain model. Always remember to only build for today, not what could happen tomorrow.

While it wasn’t as trivial as a traditional server-side implementation in Rails, it did turn out relatively clean. This is a long post, and if you make it through it, let me know what you think.

Server-side Setup

Our server-side is a stock Rails 3.1 app. We’ll go with a classic blog app consisting of a single Post model to CRUD.

We’ll use the backbone-on-rails Ruby gem to add backbone.js and underscore.js (backbone.js’s only dependency) to our Rails app.

The backbone.js and underscore.js files are not copied into your Rails app’s directory. They’re both automatically loaded by the backbone-on-rails gem (the jquery-rails Ruby includes jquery.js and jquery_ujs.js the same way).

Let’s use the scaffold generator from backbone-on-rails to generate our client-side Backbone.js objects.

We’ll be using CoffeeScript, the Rails default client-side language, throughout our Backbone app. CoffeeScript’s cleaner, Ruby-like syntax will help make the code more readable and understandable.

Listing Posts

We’ll start our app by implementing a “Read” action, the homepage. The homepage will be a list of all our posts. The backbone-on-rails scaffold generator has already created for us a router, an index view and template, a collection, and a model. To wire up the homepage we’ll need to modify our router.

Backbone routers are like controllers in Rails. They map routes to actions. The empty route represents the default action, in our case, the homepage.

app/assets/javascripts/routers/posts_router.js.coffee

In our index action, we create a Posts collection, a view for the collection, and then tell the collection to fetch its posts from the server. We don’t tell the view to render itself because the fetch is asynchronous. We’ll see later how the view uses an event handler to render itself after the fetch.

The Posts collection is configured to make server-side requests to our Rails app at “/posts” and to contain Post models.

app/assets/javascripts/collections/posts.js.coffee

Our Post model is as simple as it gets. It’s just a Backbone.Model subclass and contains no custom logic.

app/assets/javascripts/models/post.js.coffee

Before we take a look at the PostsIndex view, let’s first quickly go over the Rails layout used on the server-side.

app/views/layouts/application.html.erb

The only change we’ve made to this default Rails layout is the addition of a single “#app” <div>. This <div> will act as the container for our app. Each Backbone view will replace its contents.

Now we can take a look at the Backbone view.

app/assets/javascripts/views/posts/posts_index.js.coffee

On initialization, the view binds its render method as a handler to its collection’s reset event (this is the Posts collection we created in the Blog.Routers.Posts#index action). The Posts collection will trigger a reset event after successfully fetching its posts from the server. When the event is fired, the view will render its template and create and render another view for each Post in its collection.

Here’s the template used by the PostsIndex view:

app/assets/templates/posts/index.jst.eco

This table will be populated by the view used for each post in the collection:

app/assets/javascripts/views/posts/posts_item.js.coffee

And its template:

app/assets/templates/posts/item.jst.eco

There’s one last thing we need to do before we can view our homepage in a browser: we have to bootstrap our app’s router.

app/assets/javascripts/blog.js.coffee

An onDOMReady event handler invokes an init function that creates a PostsRouter and tells Backbone to start monitoring all hashchange events. All our client-side pages will use hash fragments as URLs. Backbone will listen for hash fragment changes and route them using the routes specified in our PostsRouter.

Finally we can now start our Rails server and hit our app at http://localhost:3000/posts. Unfortunately, we don’t have any posts yet. So let’s move on to creating a post.

Creating a Post

At the bottom of our existing home template is a link to create a new post:

app/assets/templates/posts/index.jst.eco

This means we need to add another route and action to our PostsRouter.

app/assets/javascripts/routers/posts_router.js.coffee

In our new action, we create a Post model, then create a Posts collection and bind an anonymous handler to its add event. A collection will fire an add event whenever a new model is added to it. Our add event handler performs the equivalent of a server-side redirect by navigating the router to the “empty” router (the homepage). Finally we create and render a view to submit a new post.

app/assets/javascripts/views/posts/posts_new.js.coffee

The element (PostsNew#el) used in our new post view is the global “#app” <div>. The new post view will replace the contents of this <div> with its own template containing a form to submit a new post.

The view also registers a handler to the submit event from the <form> in its template. This handler stops the default form submission behavior and then asks its posts collection to create a new post. This will result in an HTTP POST request to the server and, if successful, the collection will fire an add event. This is the same event our PostsRouter, in its new action, bound an anonymous “redirect” handler to.

And here is the new post view’s template:

app/assets/templates/posts/new.jst.eco

Great, we can now create posts. However, we can’t let people create posts without a title and a body. Let’s see what we can do to validate our new posts.

Validating a Post

Adding some client-side validation will help improve the responsiveness of our UI. We’ll still validate posts on the server-side, but for the user’s sake, I feel this duplication is worth the tradeoff.

Backbone models include a validate method that can be used to perform custom validations. However, as Rails developers we’re used to ActiveRecord‘s high-level, declarative approach to validation. And thanks to the jQuery validation plugin, we can have declarative validation on the client-side too.

First, download the plugin (after downloading and unzipping it, remember to restart your Rails server).

Next, add the plugin to our application’s JavaScript manifest file.

app/assets/javascripts/application.js

Now we can add some validation to our new posts.

app/assets/javascripts/views/posts/posts_new.js.coffee

The jQuery validate plugin allows you to specify per-attribute validation rules. Here we’re requiring a title and a body. The validate method will stop the form submission when validation fails. This will also prevent the new post view’s form submission handler, #create, from executing as well.

Go and give it a try in the browser. Submitting a post without a title and body will now fail and display an error message(s).

Viewing a Post

Ok, so far we can display a list of posts, create a post, and validate a post. Next up is: viewing an individual post.

On our home page, each post row included the following link for viewing the post:

app/assets/templates/posts/item.jst.eco

Let’s add a route and action to our router for this post page (i.e., #show).

app/assets/javascripts/routers/posts_router.js.coffee

This newest route includes a named parameter, :id. Backbone will automatically pass the named parameter’s value as an argument to our #show action.

Currently in Backbone, fetching an individual model isn’t very elegant. Above is one of the cleanest ways. You first create a model with the requested id, then add the model to a collection, and finally, tell the model to fetch itself.

Our #show action also creates an instance of the following PostsShow view for the individual post.

app/assets/javascripts/views/posts/posts_show.js.coffee

During initialization, the view adds a handler to its Post‘s change event. The Post will fire a change event after it has been successfully fetched.

The element (PostsShow#el) for this view is the same global “#app” <div> container that was used by the post list and new post views. And like those two views, an individual post view will also replace the contents of this <div>.

And here’s the posts show view’s template:

app/assets/templates/posts/show.jst.eco

The “back” link goes back to our homepage.

Client-side MVC

Backbone.js brings the power and maintainability of MVC to the client-side. Routers can be used like Rails server-side controllers to create responsive, single-page apps. Views are also like controllers, but instead of responding to url changes, they respond to DOM level events, such as link clicks and form submissions. Templates contain the app’s markup, they’re what Rails calls views. Models and collections round out Backbone.js, providing traditional class-based, client-side domain modeling.

We’ve only covered the first two letters in CRUD, but it was a lot! I hope this article can give you some direction in implementing the rest of them. The client-side JavaScript MVC space is growing rapidly. Backbone’s quick and easy Rails integration makes it an easy sell but there are other options. Be sure to research other frameworks before evaluating what works best for you.

In this case, I wanted to use the nifty wkhtmltopdf utility to create a PDF. I was able to call the binary just fine with Homebrew on OSX but found that I had to use a custom binary checked into git for our production environment on Heroku. So I created a configuration variable to store where wkhtmltopdf could be found in the current environment.

First, we define a default value for all environments in config/application.rb:

module Configurator
  class Application < Rails::Application
    # By default, let OSX resolve the path to the binary
    config.wkhtmltopdf = "wkhtmltopdf"
  end
end

Then we override the default setting as necessary in config/environments/:

Configurator::Application.configure do
  # Settings specified here will take precedence over those in config/application.rb

  # Point Heroku explicitly to the binary we need to use
  config.wkhtmltopdf = "#{Rails.root}/bin/wkhtmltopdf"
end

Lastly, we access the configuration element in our code:

  cmd = [Configurator::Application.config.wkhtmltopdf, url, tmpfile.path]

Yes, that’s it. Just use Rails’s environment support and config to store your own configuration elements. They’re trivial to set, trivial to access and require no third-party gems or custom text files.

How does Agile help us to do better work for our clients?

To answer that question, I think it’s important to reflect on what makes building a product hard. Here’s a clue: Everything.

There are technical challenges, but those are the least of your worries in practically every case (especially early on). There are product questions: who are you building for? what features do your customers really want? how can we validate our hunches in a meaningful way? what incarnation of those features will be both useful and a delight to use? There’s time pressure. And to add fuel to the fire, the team is made of people. You know… with ideas, egos, personalities and feelings. It’s amazing successful products happen, period.

We wanted a process that’s effortless when it’s working awesome while also giving us signals when something is broken. We’re not interested in process just for process’s sake. It turns out that Agile can help navigate these challenges and gives us pretty clear warning signals.

At the Core

In trying to distill out the most important parts of our Agile process, I came up with my take on the essential ingredients. I’m not saying other aspects aren’t important too, just that these make a strong foundation.

• Small units of work
• Optimize for value
• Iteration
• Product owner involvement
• Feedback
• Honesty and trust

If any one of these is missing or broken, the project has a much higher risk of running into trouble. That doesn’t necessarily mean complete failure, but it does mean the project and overall experience will fall short of its potential.

Small units of work keep the team focused and allow for greater flexibility when planning. We write user stories and every one of them captures single workflow framed from the user’s perspective. A story should be whole, meaning it delivers value to the user on its own. Stories normally take a couple of minutes to a couple of days to deliver. When they’re small it’s easy to move them around and make tactical decisions during planning. It’s also easier to avoid needless yak-shaving when the stories are small.

Small units of work aren’t enough, they must deliver Value. We typically measure value in terms of user-facing functionality. We know that some amount of infrastructure, architecture, meetings, etc needs to happen to build a product. However, that’s not what first draws a customer to a product. It has to solve a customer’s problem. Getting everyone on the same page about value is crucial, otherwise folks may be optimizing for different things.

Software products evolve over time through the addition of new features, the enhancement of existing features, and removal of unused features. It’s inherently Iterative. We start the process of iteration from week one. By the time a product sees its first ray of light, the team has executed the process of planning, designing, building, and deploying many times. The structure of the process is in place and the team has enough experience with it that it can be repeated confidently.

Knowing that there’s always an opportunity to iterate on a feature allows us to build the right-sized version of the feature for where we are in the lifecycle of the product. Not every feature needs to have all of the bells and whistles when it’s born. The time saved can be spent on something that is more valuable.

Every one of our projects has a Product Owner who’s part of the client team. Our goal is to guide and empower the product owner so that they can direct the team’s focus on what’s important. Some product owners are very involved and others minimally so. The low bar of involvement includes two important activities: prioritization of stories and acceptance of the stories once they’re complete. Without these, there’s no one at the helm and the chance that the client isn’t going to be happy with the engagement goes up a ton.

There’s internal and external Feedback. External feedback on the product can be collected via a wide array of techniques. Until ideas are validated by customers, they’re just assumptions. Building features on top of untested assumptions drives up the chance that we’re building a dud. It might be high-quality code with an extensive set of automated tests, but who cares if no one wants to use it. Building a product without integrating external feedback is like betting everything on your first hand of poker when you haven’t yet learned the rules.

Internal feedback comes from the team and improves the process itself. Reflecting on how it’s going and making changes to better suit the reality on the ground is critical to keeping the process a good fit throughout the lifetime of the product. Otherwise, the process starts getting in the way when things are going awesome or it starts failing to give signals when it should.

It should go without saying: Transparency and trust are key to building a team that can overcome the challenges of building a product. If there’s bad news, we want to deliver it early so that we’re in the best position to do something about it and reap the benefits. Without trust, it’s hard to stay psyched about the people with whom you’re working. When every action comes from a place of honesty and your intentions true, trust grows naturally.

Signals

Here are a couple of the common signals that suggest something’s up:

Low velocity – Velocity is the rate at which the team is completing stories that deliver user-facing value. Really low velocity can be a sign that stories are too big, there are inefficiencies in how work gets done, or the team is spending a lot of time on “infrastructure”.

Erratic velocity – Often the result of inconsistent planning. When the backlog is well thought out, things move fast. When the team gets to stories that are poorly written or vetted, productivity grinds to a halt.

Slow acceptance – The product owner isn’t providing the team with the feedback it needs to know it’s doing a good job. Sure, developers move on to the next story, but they may be building on building blocks that miss the mark. It can be a real drag on morale and velocity.

Big bug fix session before a release - Rather than finding and fixing bugs continuously, the team is delivering stories that are buggy. That debt has to be paid down before the software gets into the hands of users. This creates a false sense of velocity which is knocked down periodically before a release.

Others: Running out of stories, stories are frequently blocked, need for big refactors.

The more experience you have the easier it is to pick up on the signals and figure out what to do about them.

What about the Developer Practices?

I’m leaving developer practices out of this conversation deliberately. Testing, continuous integration, pairing, sustained schedule, etc are all second nature for us. There’s enough content there for a another article (or several).

Conclusion

My experience has shown that if the process includes the above components, we’re able to rapidly build valuable features. When something isn’t right and we’re being perceptive, it becomes apparent quickly and we’re able to reflect and make the changes necessary to fix the problem. In the end, Agile lets us welcome feedback that inspires course correction while also making it really hard to overlook issues when they come up.

Our talk is going to cover product design, process and technology. We want to share the whole experience of building a product, not just the technical details.

Product Design Topics

• Design and development process
• Usability testing
• Prioritizing features to match a budget

Technical Topics

• Overview of the architecture and technical stack (node.js, express, jade, socket.io, flash, etc)
• Using development spikes to reduce technical risk
• Testing node.js
• Lessons learned

After our talk we’ll turn it around and open the floor for those in attendance to share their relevant experience, in 5-10 minute mini-talks.

Additional details and RSVP: http://c5-the-ovee-story.eventbrite.com/

The good folks at Instagram posted a fun challenge on their engineering blog recently:

"Your challenge, if you choose to accept it, is to write a simple script that takes a shredded image in as input and outputs an unshredded and reconstituted image. That is, imagine if you took an image, divided it into an even number of columns and shuffled those columns randomly to produce a shredded image. Then, take that image into the script and output the original image."

(The original post is here: Instagram Engineering Challenge: The Unshredder).

This struck me as a great opportunity to hack out some code in Go (rev 60.1) as part of my continuing exploration of the language and its packages.

The problem

Here are the input and target output images:

The shredded version

The unshredded version

They offer the following tips:

1. Don’t overthink it. Use of really complex algorithms isn’t needed. Our solution WITH the bonus exercise comes in at just over 150 lines of python.
2. Think about how you would quantify whether or not two shreds ‘fit’ together by using pixel data
3. Assume you’re using the source image, or other normal photographs without edge-case patterns.
4. There are edge cases where the script we wrote with our approach will not work because of repeating patterns. This is OK in your script as well. Don’t worry about special cases – focus on making the sample images work that we’ve provided.

Great. The simplest approach that occurred to me was this:

1. Assume the left-most shred is in the right place.
2. Search the other shreds, find the one that fits best to the right of the first one, then stick it there.
3. Repeat until you run out of shreds.
4. Figure which shred is actually the left-most one, then reorder to that configuration by rotating the ring of shreds.

As it turns out, this works. My solution is on Github here: https://github.com/joncooper/instashred. They've run out of t-shirts, though , so no soup for me.

Here's a quick tour of the standard library's packages that I used:

image

The image.Image interface from the image package provides what the docs describe in their typically awesome and laconic manner as "a finite rectangular grid of Colors drawn from a ColorModel."

Basically, the image.Image interface defines a read-only grid of pixels expressed as image.Color, upon which you can call RGBA() to get out a tuple containing alpha-premultiplied channel intensities.

At first the read-only nature of image.Image made me scratch my head a bit. But it ends up being pretty neat. You can do things like create a procedural image which can then be sampled, like this:

The standard library also implements image.Tiled, which acts like an infinitely (okay, just extremely large) large tiling of a source image. See the source here--it's just pretty.

image/png

Image file support is minimal but useful at the moment. There are image subpackages that implement decoding/encoding for JPEG and PNG, and decode only for BMP, GIF and TIFF.

Here's an example for how you write an image.Image to a PNG file:

image/draw

This package implements image compositing functions and little else. In particular, it doesn't implement vector art staples like line, circle, rectangle, etc.

It's deals solely with compositing images by copying image.Color pixels from one image.Image and pasting them into a draw.Image, optionally using a mask and a Porter-Duff compositing operator.

See the golang blog's post on this package here: The Go image/draw package.

The primary operation I used is image.Draw, which has the following function signature:

func Draw(dst Image, r image.Rectangle, src image.Image, sp image.Point, op Op)

This copies a region the size of r from src into dst with the top left corner at sp.

Here's an example combining tying the last three together:

An aside on Go interfaces

The relationship between image.Image and draw.Image is extremely representative of idiomatic Go. In Go, a type doesn't declare that it implements an interface, it just does so, and it gets checked statically at compile time.

A draw.Image is just like an image.Image, but you can draw into it. The interface requirements are a superset of those defined in image.Image. Taking a look at the source to draw.Image reveals the neat way that interface inheritance happens in Go:

Wrap

This implementation was enjoyable; I am really warming up to Go.

So far, the standard libraries are uniformly sensible and in many cases beautiful. It's clearly an opinionated language and set of packages, but so far, I seem to agree with those opinions.

Cheers,
Jon