My team recently setup CI for a few Rails projects using TeamCity at work. When new commits hit a project’s repo if an RSpec test run passes the CI setup deploys the code to a Heroku site used by our QA team. One way or another some specs started failing on a project but the code still deployed to Heroku. That’s not how it’s supposed to work.
TeamCity looks at each command’s exit code to determine if a CI step fails or succeeds. If the exit code is 0 everything is copacetic and it continues on to the next step. Anything other than 0 means shit’s broke so it stops the process and logs failure.
It turns out the command
rspec spec was returning exit code 0 on the CI server even when specs were failing.
After some research I found this Ruby bug which seemed to explain the behavior: at_exit bug with exception handling
The suggested fix solved our problem. I put the monkey patch into place in the project’s spec_helper.rb file and
rspec spec exit codes are coming back
1 with failing specs and
0 with no fails on the server. TeamCity now stops short of sending broken code off to Heroku.
if defined?(RUBY_ENGINE) && RUBY_ENGINE == "ruby" && RUBY_VERSION >= "1.9" module Kernel alias :__at_exit :at_exit def at_exit(&block) __at_exit do exit_status = $!.status if $!.is_a?(SystemExit) block.call exit exit_status if exit_status end end end end