How to automate integration testing that requires multiple computers?

Question

How to automate integration testing that requires 2 or more PCs (distributed application)? What is your strategy for conducting integration testing (or performance testing) in cases where multiple machines are involved?

Example

We need to integrate-test our client / server application. To simulate a live system, we need to deploy the client on one machine, and the server on another. Then we measure the TCP transmission rate.

Answer 1

There are ways to do this, but none of them are built into any of the frameworks that I know of.

Listed below are three paths I have used in the past:

• use VMWare Server / ESX. What we did most recently is to actually create VM images for the server and client machine with a mounted second disk (data storage). Then we create and unit test our software, before testing the performance, we deploy the virtual machine, and then deploy the code to the data disk. After that, we deploy a set of test scripts on machines and run them (via Powershell). This works very well, has good playback capabilities and allows us to give test servers to other teams / clients to evaluate them. The disadvantage is that it is very resource intensive.
• Dedicated server and client test suites. We had two different source repositories: one for the server and one for the client. Then we went through the assembly as described above, but one at a time, deploying the server (and testing it against the old client), deploying the client (and testing it on the old server), and then deploying both and testing the combination. This worked pretty well, but it required some manual testing for certain scenarios and could become cumbersome if we needed to test several changes in the server or client changes at the same time.
• Only a test against production. We only updated the client server OR , and then we updated this part and tested it against the current production setup. Of course, the disadvantage of this is that we had to deploy much more slowly and make incremental changes in one system or another, deploy, test and release, and then make changes to another component. Rinse and repeat.

If you have the resources, I highly recommend No. 1. It is more difficult to install it initially, but it pays for itself very quickly, and as soon as its configuration is repeated for other products (as long as they follow a relatively similar deployment scheme).

Answer 2

It depends on your setup. For example, I needed to check out a group of web services that my team created / modified. During the test, we deployed the application to one machine as a manufacturer and used SoapUI to create several thousand transactions across many threads (from 1 to 100 threads, as I recall). In this way, we guaranteed a response and a service level agreement (SLA).