Too much end-to-end testing?

Question

I am really really disappointed with this whole situation, and here's why:

I have inherited a completely untested legacy system that allows you to synchronize many different client databases and one base database (with a different schema). The system was only partially completed when it was provided to me, and many defects prevented it from working normally ~ 90% of the time.

This system also has six different types of allowed synchronizations, each of which synchronizes different (sometimes overlapping) tables, since the databases can be quite large, so clients can first prioritize the most important tables depending on their state.

I started with a few end-to-end tests, installed the wizard locally and several client databases with specific data, and then called different synchronization methods and checked the correct data found in the correct databases in the correct format.

I was pressed for a while, and since this system has at least a hundred different ways of transferring data from one database to another and only a few thousand lines of code, I just continued to do more and more, ending with tests, mostly 1-2 to the defect that existed when I took over the project. I ended up with 16 unit tests (TDD'd from the code I added) and 113 end-to-end tests, many of which were directly based on previous defects.

I have finished the system and it has been working for several months without incident.

Recently, we decided to convert the client database into a new database, and when I ran my tests (which worked all the time on the CI server all the time) with the new database, about 100 out of 113 failed. (Of course, all tests pass, of course).

I fix unsuccessful end-to-end tests, and to be honest, most of them failed for one or two simple reasons (for example, new database rounding dates in different ways), but I'm upset by the fact that my tests were so fragile. while they correctly failed, I only needed one or two to tell me this, and not 100. The problem is that there is not much unit test code, because most of them just select data from one table for dates, then selecting the same data from another database, the union Thread them, then inserting / updating accordingly.

I could not finish this system without these tests, but the pain in maintaining them basically leads me to this question: any suggestions on how I should act or what I could do better? Did I spend too much time writing these end-to-end tests for the first time? I read the Effective Work with Heritage Code, but I felt that there really wasn’t a good answer for the pain that I experienced, except: “just refactoring and write more unit tests” that I feel really In fact, very little code and a lot of database conversion are not very suitable for the unique nature of this system.

Answer 1

You can create database proxy classes by making sure that they are the only classes that speak to the real database. Use dependency injection to unit test your entire logic code without talking to a real database. Create as few end-to-end tests as possible to ensure that proxies can read / write to the database correctly.

End-to-end tests are usually fragile in nature . Therefore, create as many of them as possible, and if you need to create a lot, you can create an abstraction layer for customizing fixtures and statements. Duplication in test cases is just as difficult to maintain as duplication in code.

Running legacy code efficiently is a good start, and I recommend xUnit Test Patterns, which is basically a unit testing bible with a lot of useful tips, including a section on database testing.

Edit: TDD is all about isolating logic. I mean, control flow operators, regular expressions, algorithms, math, should be located outside your proxies, where you can easily unit test them easily. You had 113 tests that make me suspect that there is logic that can be extracted and verified on the module.

If you create SQL commands, you can use the Builder template to make sure the commands are created correctly, and if you need to change the SQL dialect, there is only one place to make changes.

Checking the code may mean that you need to reorganize somewhat aggressively. The hard part will determine how much it costs, based on the importance and durability of the project.