Separation devices

Question

I write unit tests for a project (written in PHP using PHPUnit) that have their entire environment (loaded components, events, configuration, cache, single individuals, etc.) stored in an object that all components use to interact with each other with a friend (using an intermediary template).

To speed up the execution of unit tests, I separate the environment object and some other objects (for example, in my test case for the view object [as in V from MVC], the view manager object [which acts as a factory for view objects and is responsible for the actual rendering]) among the tests in the same test case (using PHPUnit setUpBeforeClass() and static properties).

Despite the fact that, as far as I know, the objects that I share should not affect the integrity of the tests (in the case of views, for example, the environment object and the view manager object are common, but a separate view object created for each test - which is an object which is actually being tested in a test case), it just feels more and more wrong for me.

I would prefer that each test use a completely isolated environment and in no way could affect other tests in one test case. However, this would make the tests run much slower, and it seems like a great price for not being able to pinpoint the flaw and basically just "feeling wrong."

What do you think? Can you identify any flaws so that I can convince myself that it costs more time to complete? Or am I just reacting, and is all this fine?

Answer 1

As with writing “normal” code, when you write test cases, it’s great to rely on knowing how binding objects work.

If this factory method is documented as generating new instances every time, then I can only see the other side when creating the factory method again every time, especially if creating a factory is expensive in itself.

This helps keep in mind the key goal when writing unit tests. You want to find out within 5-10 minutes if you broke the assembly. So you can go to lunch, go to a meeting, go home, etc. After you get "everything is clear." If you know that some part of the instrument is reused without creating interactions, you should use this knowledge to make your tests even more comprehensive during this 5-10 minute window. I understand the purist momentum here, but it doesn't buy anything in terms of test independence, and it unnecessarily limits what your test suite will do for you.

Answer 2

I share your feelings, so maybe I’ll just formulate my goals and my decision when I come across this problem:

• Devs should have a test suite that works really fast
• At least individual test cases should run in less than a second
• I really want to be sure that I have no interdependencies in my test cases.

I assume that you have a continuous integration server running. If it weren’t for the cronjob, but thought about setting up jenkins , it is really really easy .

---

For normal use:

Just share with us as many devices as you need to get the speed you need. It may not be very pretty, and there may be better solutions along the way, but if you have something expensive, just create it once.

I would suggest helper methods getFoo() { if(!self::$foo) .... create ... return $foo;} over setUpBeforeClass , because this can facilitate the exchange, but mainly because of the next point.

One night:

Run your test package with -process isolation and in this bootstrap recreate the full database and that's it.

It can work for 6 hours (disable code coverage for this!), But how is it done. Your lights will be recreated for each test case, as the new php process and static vars do not exist.

---

Using this method, you can be sure that you did not create dependents once a day. Thats good enough to remember what you did (and you can work with --filter and --process isolation if you need to fix something).

Answer 3

This may be blasphemy, but if you can still manage with a significant code coverage threshold, and you can also ensure that state pollution cannot exist (and this is your real problem - you will make sure that every test will not be done with the data left after test before), I see no problem in leaving things as they are.

Let the tests run quickly, and when an error is detected (which is inevitable when testing integration), you have a reason to spend time localizing one specific test or set of tests. However, if you have a toolbox that works in the general case, I would leave it as it is.