• Read more about Back to Code: Deploying with Heroku and Git

As I dip my toes back into programming waters after a several-year hiatus, I'm finding that I'm pretty rusty and things have changed quite a lot in the Rails world. As I run across sticky problems or new ways of doing things, I'll be noting them and writing the occasional blog post. I promise there will be nothing earth-shattering in any of these posts, and these posts will likely serve more as a reminder to myself than a public service of any sort, but the occasional useful insight may accidentally slip its way in.

One great development since I've done any serious programming is Heroku. I spent way too many hours hacking away at the command line trying (often in vain) to get a Rails app deployed, and the ease of using Heroku is just awesome. I've been coding on some side project lately, and contributing on some other people's Heroku projects, as well. Despite the extreme ease of use, each time I start working on a new project, I seem to end up googling for the same set of information.

I often have multiple Heroku apps using the same base git repo. For example, most production apps I host on Heroku also have a QA app that I'll deploy to before releasing a new version to production. Fortunately, it's very easy to add multiple remotes to a project. I tend to give the remotes explicit names like heroku_production or heroku_staging just to give myself that last little reminder of what environment I'm deploying to.

> heroku git:remote -a your_production_app_name heroku_production
> heroku git:remote -a your_staging_app_name heroku_staging

But half of the time I've forgotten or done something that doesn't follow this pattern, so I have to re-determine what I used. You can view your git remotes with:

> git remote -v

If you aren't already using a similar pattern and would like to rename a git remote, it's as easy as:

> git remote rename origin destination

Now that it's all set up correctly, it's time for a deployment. Deployment on Heroku is dead simple - just push to the correct remote git repo.

Use the git remote corresponding to the environment you wish to deploy from, and specify that you're deploying from master:

> git push herokou_staging master

Deploying from a different branch or a tag is pretty dead-simple, too, though I'm embarrassed to admit how often have to look this up. I'm going to blame it on my stopping coding around the time everyone started moving from svn to git. (Yes, I realize that makes me a bit of a dinosaur)

> git push heroku_staging your_branch_or_tag_name:master

That's all for now. Stay tuned for more blog posts in this series.

• Read more about Tips And Tricks For Deploying Rails 4 Apps On Heroku

Recently, I deployed a client project to Heroku, using the Heroku instance as an internal demo server. The project is based on Ruby 2.0 and Rails 4. I was surprised to find that the deployment to Heroku is not as smooth as it was before.

Here are the tricks and tips I learned to help you overcome the learning curve and get your Rails 4 apps deployed to Heroku.

Rails 4 Specific Tricks

• Heroku forces you to specify your ruby version in Gemfile, it's supported by bundler from 1.2 version, so just add below line into your Gemfile:

#specify ruby version to enable the deploy to Heroku instance. ruby '2.0.0' 

Rails4 gets rid of plugin system. In order to use features such as static asset serving and logging on Heroku, you should add the following gems to your Gemfile:

gem 'rails_log_stdout',           github: 'Heroku/rails_log_stdout' gem 'rails3_serve_static_assets', github: 'Heroku/rails3_serve_static_assets' 

Use puma as the web server, you want to add a Procfile file and add below line in it:

web: bundle exec puma -p $PORT 

Below http basic auth doesn't work as what I did before for non-rails4 app on Heroku.

config.middleware.insert_after(::Rack::Lock, "::Rack::Auth::Basic", "Production") do |u, p|   [u, p] == [USER, PASSWORD] end 

# Error message when pushing to Heroku... -----> Preparing app for Rails asset pipeline        Running: rake assets:precompile        rake aborted!        No such middleware to insert after: Rack::Lock 

Actually, Rails 4 is thread safe by default, you do not have the overhead of including the Rack::Lock middleware in each request. So just use Rack::Auth::Basic directly as there is no Rack::Lock middleware any more when you using a web server such as puma:

config.middleware.use '::Rack::Auth::Basic' do |u, p|   [u, p] == [USER, PASSWORD] end 

Heroku Related Tips

In my case, I need to transfer PG database between local and Heroku. Firstly, I need to find the URL to my db on Heroku:

$ heroku config:get DATABASE_URL postgres://bla:sth@ec2-17-21-12-11.compute-1.amazonaws.com:5432/123 

Then transfer from the Heroku db to your local db (switch -t to and -f from versus):

$ heroku plugins:install https://github.com/ddollar/heroku-pg-transfer $ heroku pg:transfer -t postgres://localhost/dbname -f postgres://bla:sth@ec2-17-21-12-11.compute-1.amazonaws.com:5432/123 

Since the heroku-pg-transfer tool utilizes postgres's native pg_dump facility it is a much more predictable and resilient tool. Check out a related blog for more details.

There is another needs in my case, I have to deploy development branch to Heroku, but Heroku default is to deploy master branch to their instance, use below command to specify a branch for deployment.

# deploy a specific branch to Heroku git push heroku other-barnch:master 

Resources

https://devcenter.heroku.com/articles/rails4
http://blog.remarkablelabs.com/2012/12/rails-4-is-thread-safe-by-default-rails-4-countdown-to-2013

• Read more about The Problem with PaaS

I'm a huge fan of Heroku. I mean I'm a huge fan of Heroku. Their platform is much closer to exactly how I would want things to work than I ever thought I would get. However in the past few weeks Heroku has had a number of serious outages...enough to the point where I started thinking that maybe we needed to start working out a backup plan for when our various Heroku-hosted applications were down. That's when I realized a big problem, and it's not just a problem with Heroku but with any Platform-as-a-Service:

The moment you need to have failovers or fallbacks for a PaaS app is the moment that it loses 100% of its value.

Think about it: to have a backup for a Heroku app, you're going to need to have a mirror of your application (and likely its database as well) running on separate architecture. You will then need to (in the best case) set up some kind of proxy in front of Heroku that can detect failures and automatically swap over to your backup architecture, or (in the easiest case) have the backup architecture up and ready to go and be able to flip a switch and use it.

The backup architecture is obviously going to have to be somewhere else (preferrably not on EC2) to maximize the chance that it will be up when Heroku goes down which leaves you with the glaring problem that if you have to mirror your apps architecture on another platform, all of the ease of deployment and worry-free infrastructure evaporates. This leaves you with two options:

1. Put your faith in your PaaS provider and figure that they will (in general) be able to do a better job of keeping your site up than you could without hiring a team of devops engineers.
2. Scrap PaaS entirely and go it on your own.

A "PaaS with fallback" simply doesn't work because it's easier to mirror your architecture across multiple platforms than you control than it is to mirror it from a managed PaaS to a platform you control.

Don't Panic

Note that I'm not telling anyone to abandon Heroku or the PaaS concept; quite the opposite. My personal decision is to take choice #1 and trust that while Heroku may have the occasional hiccup (or full-on nosedive) they are still providing high levels of uptime and a developer experience that is simply unmatched.

Heroku has done a great job of innovating the developer experience for deploying web applications, but what they need to do next is work on innovating platform architecture to be more robust and reliable than any other hosting provider. Heroku should be spread across multiple EC2 availability zones as a bare minimum and in the long run should even spill over into other cloud providers when necessary.

If they can nail reliability the way they've nailed ease-of-use even the most skeptical of developers would have to take a look. If they could say with confidence "Your app will be up even if all of EC2 is down" that's yet another powerful selling point for an already powerful system.

The Third Option

There is actually a third option: if your PaaS is available as open source then you will be able to run their architecture on someone else's systems, giving you a backup that is at least a middleground between the ease of PaaS and the reliability of Do-it-Yourself. The two current players in this arena are Cloud Foundry and OpenShift.

While Heroku currently has them beat for developer experience (in my opinion) and the addon ecosystem makes everything just oh-so-easy, it might be worth exploring these as a potential middleground. Of course, if Heroku would open source their architecture (or even a way to simply get an app configured for Heroku up and running on a third-party system with little to no hassle) that would be great as well.

In the end I remain a die-hard fan of PaaS. It's simply amazing that, merely by running a single command and pushing to a git repo, I can have a production environment for whatever I'm toying with available in seconds. After the past few weeks, however, I am spending a little more time worrying about whether those production environments will be up and running when I need them to be. And that's the problem with PaaS.

• Read more about Lessons Learned: Natively Compiling Tidy HTML for Heroku

Recently I was working on a project and wanted to be able to utilize Tidy to clean up some HTML output. I added the tidy_ffi gem to my project and voila, it worked! Or, to be more specific, it worked locally.

Once I pushed to Heroku I started running into trouble, namely that libtidy.so, the dynamically linkable native library that tidy_ffi depends upon, wasn't found. Uh oh.

Getting My Hands Dirty To Get Tidy

Before yesterday I knew about Heroku buildpacks in theory. I also knew that I really didn't want to have to use one to solve this problem. My first clue came in the form of a Stack Overflow post in which someone used Tidy on a Bamboo-stack application but was having trouble migrating it. Aha! Surely this will solve my problem.

So I rolled up my sleeves to do the kind of low-level work I usually try to avoid (while still trying to avoid as much of it as possible). I used heroku run bash to shell into a fresh Bamboo app that I created and then used SCP to copy out the libtidy.so file there. I added it to my repo, followed the instructions on the StackOverflow post, and pushed. And the app came crashing down.

Sha Sha

As it turns out, since the post was authored Bamboo and Cedar have diverged in their precise Linux installations. The versions are the same, the git SHAs are different. C'est la vivre. Now we turn to more complex solutions.

I knew that I would need to compile Tidy myself, but how? As it turns out, Heroku offers a tool called vulcan that allows you to create a build server in the cloud and compile binaries that are compatible with Heroku (because they're compiled on Heroku). After a few hiccups, I had my build server up and running, but now I needed to build from source!

Tidy is an old project. I mean, it's a really old project. It uses CVS as its versioning system. Unable to check out using CVS as per the project's instructions (I'm still not sure why this didn't work, but it probably has something to do with the fact that it was CVS), I instead downloaded a tarball from browsing the CVS repo on SourceForge.

Once I had the source, it was time to build it. Tidy doesn't have a typical structure for a buildable library, but after some experimentation I finally figured out the necessary incantations:

vulcan build -v    -s ~/Downloads/tidy    -p /tmp/tidy    -c "sh build/gnuauto/setup.sh && ./configure --prefix=/tmp/tidy --with-shared && make && make install" 

Some notes about what's going on here:

• -s ~/Downloads/tidy was the directory into which I downloaded Tidy's source.
• -p /tmp/tidy sets the prefix on the Heroku filesystem. Since Heroku apps can only write to /tmp this needed to be inside /tmp.
• -c "..." I used the same prefix when configuring for make to build to the right directory. --with-shared gets Tidy to compile the .so files and not just .a files.

Once this command was run, vulcan downloaded a tarball containing the files I needed. Woohoo! I added this to my repo as lib/native/libtidy.so and I was ready to rock and roll!

Getting Up and Running

Further experimentation and frustration ensued trying to get everything just right but here's the Ruby code that finally got things working:

require 'tidy_ffi' if ENV['RACK_ENV'] == 'production'   TidyFFI.library_path = "/app/lib/native/libtidy.so"   require 'tidy_ffi/interface'   require 'tidy_ffi/lib_tidy' end 

Here we set the library path manually and throw in some extra requires that didn't autoload properly for some reason in production. After all that work my app was able to Tidy HTML like a champ!

So Long, Comfort Zone

While I'm comfortable plumbing the depths of any Ruby application I'm not actually well versed in solving problems like this. It was a chance to step outside my comfort zone and figure something out through trial, error, patience, and frustration. Knowing a little about how vulcan works is going to have me feeling more confident the next time I need a native library that isn't available by default on Heroku.

If you want to use Tidy on Heroku, you don't have to go through quite the same ardor that I did because you can download the Heroku-compatible libtidy.so file directly! Just add it to your repo, link it using the Ruby above, and have fun tidying up!

• Read more about Using NPM with Heroku Node.js

The Node Knockout is this weekend right now and I've been trying to teach myself Node and get ready in a variety of ways. One of the most important (and least clear) aspects of preparation was figuring out how to properly vendor the latest Node libraries for use in Heroku. I've got NPM up and running locally, but as of its latest release it has no built-in support for vendoring. Here's how I managed.

Updated Method

Since this post, the maintainers of NPM have released an updated version with a new command: bundle. Here's how to use it (make sure you have NPM >= 0.1.27 for this to work).

First, create a package.json file in your project's root directory, and populate it with dependencies like so:

{   "name":"yourproject",   "version":"0.0.1",   "dependencies":{     "express":"1.0.0rc2",     "ejs":""   } }

If you don't need a specific version, just specify a blank string. Now that you've created a package.json, you need to run bundle:

npm bundle ./vendor 

This will bundle the dependencies specified into the vendor directory of your project. Now you're almost done! The last step is to add the vendor directory to your load paths:

require.paths.unshift('./vendor') 

That's it! You can now require the dependencies you've specified like you would if NPM were installed normally (i.e. without special functions or directory specifiers). Happy Node-ing!

Old Method

Warning: I am not an experienced Node developer and I may be doing this completely wrong. However, I am happy to publish this anyway with the hope that more learned Javascripters will correct my naive ways.

NPM's Command Options

NPM provides a few helpful command options that let you specify the directory of installation when you install a library. By using the --root and --binroot options you can use a folder inside your local project instead of the default NPM root. For example, in my project I created a vendor folder and then, to install Express, specified:

npm install express --binroot ./vendor --root ./vendor 

Now while this seems like it might do exactly what we want, it's not quite perfect. Rather than install everything in vendor, it installs the important stuff in vendor/.npm/.cache. This isn't the end of the world but can make for some pretty ugly require statements.

The Require Blues

So now you've got your packages all nice and installed inside your project directory (probably a lot of support files that you don't need, as well, but the Node slugs on Heroku tend to be small anyway so no biggie). Now you need to include them in your application. To do this, I wrote a quick function, vrequire that adds the necessary load path and then requires the library all at once:

vrequire = function(lib) {    require.paths.unshift("vendor/.npm/" + lib + "/active/package/lib");   return require(lib);  }

Now if I call vrequire("express") it will load up from my vendored library. If I deploy my app to Heroku, it's able to find everything it needs and (cross your fingers) launch and work correctly!

Hopefully some people find this little guide useful, and I look forward to someone pointing me to the "real" way to do this stuff! If you want to poke around the exploratory code I've been creating to play with Node, you can find it on GitHub (but don't expect much!).