Grails integration tests fail (apparently) in random and unique ways

Question

We are writing integration tests for our Grails 2.0.0 application using Fixtures and Buid-Test-Data .

During testing, it was found that the integration test is not performed at a specific time and passes at a different time. Running the "test-app" sometimes leads to the passage of all tests, and sometimes leads to the failure of some of our tests.

When tests fail, they are caused by a unique constraint that is violated during the insertion of an instance of a domain class. This indicates that there are still entries in the test database. I run H2 db and definitely got 'dbCreate = "create-drop" in my DataSource.groovy.

Dirty Grails 2.0 Integration Test? seems to indicate a significant problem of sample contamination in Grails. Are there any solutions? I pressed Grails-8530 ?

[Edit] The pollution seems to be caused by unit tests. We somehow proved this by removing unit tests and successfully launching the "test-app" several times.

Answer 1

When I encounter such errors, I like trying to find the unit test (s) that are causing the problem. This may seem complicated, because it seems that failure sometimes happens.

1) I would look at recently added universes. If this problem has just begun, then this is a good place to look.

2) Metaclassing seems to be good in that you are causing similar errors, so I would look for a metaclassification that wasn’t configured / broken correctly. There are not many problems with 2.0, as with <= 1.3.7, but it can be a problem.

3) I wrote a plugin that runs your tests in random order. This may not help you solve your current problem. But what can help you is to print all your tests so that you can take what it gives you and run grails test-app <pasted list of unit tests> IntegrationTestThatIsFailing , then start removing unit tests to find the culprit ( s). ( http://grails.org/plugin/random-test-order ). I found a mistake in this with 2.0 that I did not have time to fix it yet (integration tests fail when approving for the visualized name of the view), but it should still print your test names for you (which is better than doing it yourself: )

Answer 2

Tests of fact integration with violation of restrictions due to existing records resemble a situation that I once encountered with functional tests (selenium), performed in an unpredictable order, some of them do not clean the database properly. Of course, the situation with functional tests is different, since it is more difficult to restore the state of a database (a test test cannot cancel a transaction in another jvm).

Although integration tests usually roll back transactions, you can still break this behavior if your code controls the transactions (commits) explicitly.

First, I would try to force an order, as Jarred pointed out in 3). Assuming you can then reproduce the behavior, I would then check for transactional behavior. Setting the org.hibernate.transaction logging level for debugging should show where transaction boundaries are located.

Sorry, you do not yet have a good explanation why the destruction of unit tests helps to get rid of the symptoms, in addition to the general "possible problems with metaclassification." :)