Roll Your Own Asset Pipeline with Gulp

Jeff Dickey ·

I’ve found myself using Gulp for just about everything involving HTML/CSS/JS these days. It’s super fast, quick to write scripts for and flexible. I’m at a point now where I have a ton of projects I can just cd into, run gulp and be up and running. It’s the best solution I’ve found for delivering static assets whether for local development or production.

If you haven’t used a tool like this before, you might be thinking that your cobbled together scripts (or Asset Pipeline) work just fine. The value of using Gulp is not within its ability to concatenate files, minify files or any of that. Gulp’s true value lies in its ability to compose these things well. Using disparate components might make it easy to concatenate JS, but mixing that in with Rev will immediately compound the complexity of your home-grown scripts. It’s value compared to Asset Pipeline is that it is still flexible.

If you’ve been on Rails’ Asset Pipeline, or on a totally different stack, Gulp should offer you a great solution to static asset compilation. Here’s how:

Rails Asset Pipeline Woes

One of my favorite parts of Rails has been the Asset Pipeline. The ability to just start throwing assets into the application and have rails spit out CDN-ready assets has been super helpful. After using it for a few years now though, I’m feeling the pain that many of you are. Asset pre-compilation errors, slow deploys, inflexibility, tying yourself to the Rails stack. It’s rough.

Enter Gulp

Gulp is a build system. It’s like Grunt, Make, Rake, and the like. It’s easy to use for the person running it. While it does have a slight learning curve, you’ll find it a super useful tool for all kinds of tasks. It’ll be the fastest weapon in your toolbox for asset compilation (speed for both time to develop and runtime).

CSS Compilation

I imagine just about everyone these days is using a CSS preprocessor. So let’s see how to do this with Gulp.

First, create a new app folder and add an empty package.json to store your node.js dependencies (I’m assuming you have Node installed here). Then we install Gulp and its LESS compiler:

Alright, now create your gulpfile.js:

This file just defines a single task less that will compile the css/app.less file into dist/

And a basic LESS file at css/app.less:

Now if we run gulp less we’ll see our compiled css in dist/app.css. Great!

Watching for Changes

This is great for building the assets once (perhaps in a deploy), but for local development it would be a pain to have to type gulp less every time we wanted new assets. gulp.watch() will be our helper here. Rewrite your gulpfile like so:

Note a couple things:

  • We watch for any less file but only compile app.less. This is because with LESS you would likely @import whatever other files you need. This strategy will not work for JavaScript (but we’ll cover that later).
  • The watch task depends on less. This ensures that the first run will have the assets there already.

Now, run a gulp watch, make an edit to your file and see gulp recompiling the stylesheets each time:


✗✗✗ myapp:(gh-pages) ✗ gulp watch
[gulp] Using gulpfile ~/src/myapp/gulpfile.js
[gulp] Starting 'watch'...
[gulp] Finished 'watch' after 6.17 ms
[gulp] Starting 'less'...
[gulp] Finished 'less' after 24 ms
[gulp] Starting 'less'...
[gulp] Finished 'less' after 2.73 ms

Now we have Gulp compiling our assets each time we touch the file: great! On my projects I like to include livereload to refresh the browser automatically on updated files. (it’s not as hard to setup as it sounds, really!)

Concatenating JavaScript

Now that we’ve mostly reimplemented asset pipeline for CSS, it’s time to look at JavaScript. JavaScript, unlike LESS does not have the convenient @import to support specifying what files to import, so we will just concatenate all of them into one file. (If you want something like that, look into Browserify, but that’s out of scope for this guide).

First install gulp-concat:

npm install --save-dev gulp-concat

Then add this script task to your gulpfile.js:

And the following JS file to src/users.js (not the most interesting code in the world):

console.log('user.js is called');

Now a gulp scripts or a gulp watch will concatenate your JS as well!

Minification

Minification is easy mode. Just pipe the stream JS through Uglifier (make sure you npm install) and set the compress flag on less:

Rev

This is my favorite part of using Gulp. Rev will give you that friendly app-ef62e7.js filename output that Asset Pipeline is famous for. The reason for it is you can cache it forever. New requests will just point to new files. CDNs love this. To get the files to have the hash is pretty easy with Rev.

Now the filename has the hash appended to it! Note the digest I generate as well; that looks like the following:

Here is where a bit of custom code would come into play. When you generate your index.html (or wherever else you reference the CSS/JS) you will have to swap out the URL for the one in the digest file. Should only be a matter of parsing this JSON file in your framework of choice, or having Gulp rewrite your index.html to replace the CSS/JS include with the correct filename.

Note for Angular.js users

Most of my front-end apps are built with Angular. Two issues crop up when using angular with this method: ordering of the module getters/setters and minification, both of which can break Angular’s dependency injection.

If you’ve used Angular, you know there is a way to ‘get’ the module, and a way to ‘set’ the module. angular.module('appname', []); vs angular.module('appname');. You must call the setter once before the getter, then can use the getter as much as you want.

To make this work with my concat strategy, first create a module.js for each of your modules, then edit the gulpfile’s script task like so:

Also note that ngmin is thrown in there to solve the minification/DI issue.

There are a lot of places you can go with this. It is trivial to deploy this to Google Pages, S3, or the public folder of a web app. You could put all of your static assets into its own Git repo, or make this part of your deploy automation. The point here is not to dictate how your app should be built, but that Gulp can offer you flexibility to compose these tools in a workflow that works for you.

So next time you need asset pre-compilation, take a look at Gulp and let me know what you think.