SimpleCov with multiple applications - or, in short, how does Simplecov work?

Question

I am trying to configure SimpleCov to create reports for 3 applications that share most of their code (models, controllers) with a local stone, but the specifications for the code that each application uses are inside each. / spec, not the stone itself.

For a clearer example. When I run the exec rspec spec package inside app_1, which uses common models from the local gem, I want to get (accurate) reports for all the specifications that this app_1 has inside. / spec.

The local gem also has some models that belong exclusively to app_2, inside the namespace, so I want to skip the report for these files when I run the test suite inside app_1.

I am trying to achieve this with the following code in app_1 / spec / spec_helper.

# This couple of lines are needed to generate report for the models, etc. inside the local gem. SimpleCov.adapters.delete(:root_filter) SimpleCov.filters.clear SimpleCov.adapters.define 'my_filter' do root = SimpleCov.root.split("/") root.pop add_filter do |src| !(src.filename =~ /^#{root.join("/")}/) end add_filter "/app_2_namespace/" end if ENV["COVERAGE"] == "true" SimpleCov.start 'rails' end 

This works until some questions arise.

Why do I get 85% coverage of the model that is inside the gem, but the specification is inside app_2 (I run the spec inside app_1).

The first problem was when I tried to improve this model, so I clicked on the report for it and saw which lines were detected, and I tried to fix them by typing tests for them in app_2 / spec / namespace / my_model_spec.rb.

But it didn't matter, I tried a more aggressive test and I deleted all the content in the spec file, but somehow I still got 85% coverage, so my_model_spec.rb is not related to the coverage results my_model.rb. Kind of unexpected.

But since this file was on app_2, I decided to add a filter to the SimpleCov.start block in app_1 spec_helper, for example:

 add_filter "/app_2_name_space/" 

I then moved to the app_2 folder and began to configure SimpleCov and see what results I would get here. And they were weirder.

For the same model, I got 100% coverage, I did the same test as the empty my_model_spec.rb file, and still got 100%. So this is really f ** ed, or I don’t understand anything.

How it works? ( with the Ruby 1.9 coverage module , you say it’s good when I run the example locally in the official documentation, which I get different results, so I think there is an error or outdated documentation there)

 ruby-doc: {"foo.rb"=>[1, 1, 10, nil, nil, 1, 1, nil, 0, nil]} locally: {"foo.rb"=>[1, 1, 10, nil, nil, 1, 0, nil, 1, nil]} 

I hope that in the reports there are no positive results for lines that are evaluated somewhere in the application code, no matter where.

I think the expected behavior is that the results for the model, for example, are related to spec, the same for controllers, etc.

This is true? If so, why am I getting these strange results.

Or do you think that the structure of my applications might be corrupted with SimpleCov and Coverage?

Thanks for taking the time to read this, if you need more details, just ask.

Answer 1

Regarding your confusion with a model that is 100% covered, because I'm not sure I understand correctly: for Coverage (and therefore SimpleCov) there is no way to find out if your code was executed from the specification or “elsewhere”. Let's say , I have a foo method and a bar method that calls foo. If I refer to the bar in my specifications, of course foo will also be shown as covered.

Regarding your common problem: I think it should be possible for the lighting to be communicated. Just because the source code is at some other point than your project root should not lead to loss of coverage reports.

Two things in your basic configuration: removing the base adapter (line 2) does not really matter, since the adapters are mostly distinguished configuration fragments, and at this point you have already completed it (since it starts when Simplecov loads). Filter reset should be sufficient.

In addition, the custom adapter that you define is not used. Please refer to the README on how to configure the adapters correctly, but I think you will be fine using this in the SimpleCov configuration block when you start the coverage launch:

 SimpleCov.start 'rails' do your_custom_config end 

What you probably want, but this is a consolidated reach report for all of your apps. To do this, you first need to define a command_name for each of your specification sets, inside your configuration block, for example: command_name 'App1 Specs' .

You will also need to define a central coverage_path that will store your coverage reports in your application packages. Let's say you have ~/projects/my_project/app[1-3] , then the meaning in my_project/coverage might make sense. This will result in your test suite results being combined into a single report, for example, when using SimpleCov with Cucumber and RSpec. The merge has a default timeout of ~ 10 minutes, so you may need to set this higher value using merge_timeout 3600 in your configuration (these are seconds). For the specifics of these configuration parameters, re-check the README and SimpleCov :: Configuration documentation. These things are described in detail here.

So, to summarize, each of your applications should look something like this:

 require 'SimpleCov' SimpleCov.start 'rails' do reset_filters! command_name 'App1 Spec' coverage_path File.dirname(__FILE__) + '../../coverage' # Assuming this is in my_project/app1/spec/spec_helper.rb merge_timeout 3600 end 

Next, you can add filters to reject all non-design stones along the way, and you should be up and running.