Dave Konopka

RSpec run fails, returns exit code 0

My team recently setup CI for a few Rails projects using TeamCity at work. When new commits hit a project’s repo if an RSpec test run passes the CI setup deploys the code to a Heroku site used by our QA team. One way or another some specs started failing on a project but the code still deployed to Heroku. That’s not how it’s supposed to work.

TeamCity looks at each command’s exit code to determine if a CI step fails or succeeds. If the exit code is 0 everything is copacetic and it continues on to the next step. Anything other than 0 means shit’s broke so it stops the process and logs failure.

It turns out the command rspec spec was returning exit code 0 on the CI server even when specs were failing.

After some research I found this Ruby bug which seemed to explain the behavior: at_exit bug with exception handling

The suggested fix solved our problem. I put the monkey patch into place in the project’s spec_helper.rb file and rspec spec exit codes are coming back “1” with failing specs and “0” with no fails on the server. TeamCity now stops short of sending broken code off to Heroku.