Test environment for testing black box regression

Question

I am looking for a tool for regression testing a set of equipment that we are building.

The current concept is that you create an input file (text / csv) for a tool that defines the input data for the system under test. The tool then captures the system exits and writes the inputs and outputs to the output file.

The output is in the same format as the original input file, and can be used as input for the next tool runs, while the measured outputs correspond to the values ​​from the previous run.

The results of the two runs will not be exact matches; there are some temporary differences depending on the state of the battery or depending on the other internal state of the equipment.

We would have to write our own interfaces to transfer commands from the tool to the equipment and to capture the output of the equipment.

This is a relatively simple task, but I am looking for an existing tool / package / library to avoid re-creating the lessons from the wheel / theft.

Answer 1

I recently built such a system on top of git ( http://git.or.cz/ ). Basically, write a program that takes all your input files, sends them to the server, reads the output, and writes it to the set of output files. After the first run, copy the output files to git.

For future launches, your success is determined by whether the git repository is clean after completion:

test 0 == $(git diff data/output/ | wc -l)

git , , , . .

Answer 2

, , . :

• . , (RS232), .
• . . - , ( PASS/FAIL ). Perl Python .
• , . , . . , . , diff script.

Answer 3

. - / , .

:

x = read_input_file(ifilename);
y1 = read_expected_data(ofilename);
send_input_file_to_server();
y2 = read_output_from_server();
checkequal(y1, y2)