How to check for failures in the RSpec test suite?

Question

I experiment with RSpec and consider a system that changes random seed only when the test suite passes. I am trying to implement this in the after(:suite) block, which is executed in the context of the RSpec::Core::ExampleGroup .

While RSpec::Core::Example has an “exception” method that allows you to check if any of the tests worked, there seems to be no similar method for RSpec::Core::ExampleGroup or any accessor for the list of examples . So, how can I check if the tests passed or failed?

I understand that this is possible using custom formatting, which keeps track of whether any tests have passed, but it seems that a bad idea for the formatting process affects the actual running of the tests.

Answer 1

I poked at the RSpec source code and realized that the following would work. Just put this code in spec_helper.rb or some other file that loads when you run the tests:

 RSpec.configure do |config| config.after(:suite) do examples = RSpec.world.filtered_examples.values.flatten if examples.none?(&:exception) # change the seed here end end end 

The RSpec.world.filtered_examples hash associates group examples with an array of examples from this group. Rspec has functions for filtering specific examples, and this hash, apparently, contains only those examples that were actually running.

---

One alternative way to configure your system would be to check the rspec process return code. If it is 0, all tests are passed and you can change the seed.

In a shell script, you can define a command that modifies the seed and runs:

 rspec && change_seed 

If your project has a Rakefile, you can configure something like this:

 task "default" => "spec_and_change_seed" task "spec" do sh "rspec spec/my_spec.rb" end task "spec_and_change_seed" => "spec" do # insert code here to change the file that stores the seed end 

If the specifications do not work, then the rake "spec" task will fail and it will not continue to modify the seed.