RSpec run fails, returns exit code 0
Let's work together!
Are you hiring for remote engineering leadership positions? I help teams thrive through iterative improvement. Here's my resume. Let's talk.
I'm also available for DevOps project work. Are you fighting fires instead of serving your customers? Hire me to identify the roadblocks and make improvements so you can get back to focusing on your customers.
My team recently setup CI for a few Rails projects using TeamCity at work. When new commits hit a project’s repo if an RSpec test run passes the CI setup deploys the code to a Heroku site used by our QA team. One way or another some specs started failing on a project but the code still deployed to Heroku. That’s not how it’s supposed to work.
TeamCity looks at each command’s exit code to determine if a CI step fails or succeeds. If the exit code is 0 everything is copacetic and it continues on to the next step. Anything other than 0 means shit’s broke so it stops the process and logs failure.
It turns out the command
rspec spec was returning exit code 0 on the CI server even when specs were failing.
After some research I found this Ruby bug which seemed to explain the behavior: at_exit bug with exception handling
The suggested fix solved our problem. I put the monkey patch into place in the project’s spec_helper.rb file and
rspec spec exit codes are coming back “1” with failing specs and “0” with no fails on the server. TeamCity now stops short of sending broken code off to Heroku.
March 09, 2013