Offer a test automation framework for my scenario

Question

I am evaluating several test automation circuits and would appreciate some input. We have a large business application for processing loans, and it has three endpoints: web interface, soap and basic http / post. My requirements are given below.

Our QA is very often used for data-based testing (all input comes from excel, so the expected result for statements). Basically, each row in an excel sheet contains both input and the corresponding expected results. We want our QA to simply edit Excel worksheets. Our development team will be responsible for providing the necessary test libraries / devices. Given the scenario described above, I would like to use one single automation system that can read these excel sheets and check all the endpoints of my application, i.e. web ui, soap and http / post. Please let me know which test automation framework fits my needs. Rate some pointers and tutorials to get you started.

Answer 1

I looked at several suggested frameworks, but the final choice I made was to write my own domain libraries (DSL, if you like) on top

• Spock
• Geb
• Groovy and
• HTTP Builder
• Sonar and hudson

We currently perform 14,000+ (and growing) regression tests every night as part of CI.

Answer 2

Robotframework allows you to use test data coming from external files. I have not tried using excel sheets, but I did it with CSV formatted data as well as JSON. In the worst case, you may have to export Excel data in a format other than python, but this can be done transparently for people who support the spreadsheet.

In short, you can create a keyword in python (or any JVM language if you use jython) that reads in your spreadsheet and converts it into data used in test cases.

Although, if absolutely all of your tests are derived from this data file, you do not need a framework. Just write a short script in any language that iterates over each line and calls a function that does the equivalent of curl or wget. The framework is useful if you need a way to present test cases, but you already have it.

However, the advantage of using a robot - even if you have only one test that says “check the use of each row of the table” - is that you get a good report that the robot has and can easily integrate it with jenkins / hudson .

Answer 3

Fit / Fitness is recommended. This is a good way to let business analysts run tests on their own.

Answer 4

I do not know a tool that will allow you to test the entire endpoint with data that is stored in a spreadsheet. You will need to look at a custom test runner that is built on an existing test runner so that data can be read from spreadsheets.

However, I would focus on choosing a tool that matches the endpoint you are trying to test. For the web interface, I would recommend selenium or HtmlUnit . You will need to look at alternative methods for retrieving data from a spreadsheet in these tests. But they will use the user interface as close to the user as possible.

To check the end point of the soap, I would suggest just writing unit tests with JUnit and mocking the backend. You can test the server system separately with unit tests to ensure that it works correctly. Depending on the architecture of the application, this may not be for you. You can set them as integration tests and run them through JUnit and acknowledge the returned message. Again you will need to write a library that will receive the data and expected values ​​from the spreadsheet.

Hope this helps you.

Answer 5

I can recommend soapUI ( http://www.soapui.org/ ) or the JMeter application ( http://jmeter.apache.org/ ).

With SoapUI, I made functional tests for my (soap-) webservice. You can automate these tests.

With Apache JMeter, I conducted automated performance tests in the web interface. Test data was also based on an excel sheet.

Regards, Patrick

Answer 6

You can take a look at Twought from Thoughtworks. This is a functional testing IDE. QA / BAs can use natural language to identify test cases. Twist uses Selenium and JUnit under the hood, so the generated test cases can be versioned.

I agree with @stuartf that you will not find a solution out of the box that will load Excel data and argue against it. You can use a POI or some other excel reader to load data into a test framework.