How to automate non-interactive tests on Microsoft Surface

Question

We have a set of tests (test cases for the native library) that we want to run on Microsoft Surface (ARM version). This is easy to do manually from Visual Studio. However, we would like to automate the process so that we can compile, install, run, and collect output from the command line (for example, from Jenkins CI).

IOS has Fruitstrap, which emulates Xcode to perform a similar task for iOS. Is there a Microsoft tool / third-party tool that can help us do this for Surface?

The question of automatic testing of the GUI (the best way to test the Microsoft Surface application ) is not relevant to this case, because we do not have a graphical interface and the problem is to download and run the application on the surface from the node not interactively.

Edit: There is a good video on how to use the Google Test platform in VS2012 http://www.youtube.com/watch?v=o-Gi6REeGN0 , but our problem is that we want to run tests on Microsoft Surface instead of the host .

Answer 1

I used psexec to run tests in specific environments before, and it seems that psexec works on Surface:

http://technet.microsoft.com/en-us/library/ee692107(v=surface.10).aspx

I'm not sure if you can run mstest.exe or nunit.exe on the surface, so if I didn’t make my own light test environment that would create some kind of easily parsed output (possibly xml) with the test results and put it to the .exe file.

So your jenkins script assembly will have these steps:

[Copy SurfaceTests.exe to \\mySurfaceDevice\c$\testfolder] psexec.exe \\mySurfaceDevice c:\testfolder\surfacetests.exe > c:\testfolder\output.xml [Copy \\mySurfaceDevice\c$\testfolder\output.xml to jenkins folder] 

then you have to configure jenkins to analyze your output. If you research the nunit or mstest search results, you can probably create something similar in your surfacetests.exe file and configure the jenkins as if they were created by one of them. I have no experience with Jenkins, but in cruisecontrol.net it is pretty easy to make xslt files that parse the xml output and appear in build reports.

If the psexec approach does not work, I would make a small wcf service host that always worked on the device. This service has one method (RunTests ()) that executes your surfacetests.exe file whenever it is called. Or even better, maybe just run the tests from the assembly stored on disk (you will have to do some reflection to achieve this) and return the results in the method call. Here's an article that explains how to create a service that will run in Windows storage applications

http://www.c-sharpcorner.com/uploadfile/7e39ca/simple-wcf-service-in-windows-store-apps/

Good luck. It seems like an interesting task. Please specify how you decided to solve this.