We usually don’t test too much. We once had the rule of "80% verification", but it turned out that this does not make us better or faster. Most of the codes and data structures we use are developed quite protectively, so it is rarely possible to do harm that cannot be undone. Our users prefer fast turnaround times of up to 100% uptime.
We have two app installs: my app.appspot.com and my app-test.appspot.com . The entire code base is a designer to ensure that app-test.appspot.com never changes state on external systems.
sometimes we copy data from app.appspot.com to app-test.appspot.com . This can become messy because the ID generation counters for the data store are not updated, but it works quite well.
We are developing on both systems. Frontend development is mainly app.appspot.com on app.appspot.com , and backend experiments are done on app-test.appspot.com .
We have three branches: master, rc and production.rc is updated from master and production from rc. rc is deployed daily on rc.app.appspot.com or processes them. production takes place weekly at production.app.appspot.com (which is also available through a different application name.
Developers are usually divided into dev- whoami .app.appspot.com for experimentation. We use the development server very little because we need a lot of data from the data warehouse.
Now for testing: we mainly use acceptance tests. We have a small structure called resttest_dsl , which we use to describe such tests:
client.GET('/').responds_access_denied() client.GET('/', auth='user').responds_html() client.GET('/admin').responds_access_denied() client.GET('/admin', auth='user').responds_access_denied() client.GET('/admin', auth='admin').responds_html() client.GET('/artikel/').responds_with_html_to_valid_auth() client.GET('/artikel/?q=Ratzfratz', auth='user').responds_html() client.GET('/api/ic/v3/a/14600/03/zukunft.json').responds_with_json_to_valid_auth() client.GET('/kunden/SC50313/o/SO1194829/', auth='user').responds_html() client.GET('/api/masterdata/artikel/v2/artnr/14600/bild=s210').redirects_to('...')
hostname and credentials are default values, but can be overwritten by environment variables. Most of the errors that we have ever fixed have a regression test. We use a Makefile to manage all the material. Eg.g:
deploy: appcfg.py update -V dev-`whoami` -A app . TESTHOST=dev-`whoami`.app.appspot.com make resttest open http://dev-`whoami`.app.appspot.com/
Deployment always comes from the central git repository as follows:
deploy_production: rm -Rf tmp mkdir tmp (cd tmp ; git clone git@github.com :user/app.git) (cd tmp/app ; git checkout production ; make dependencies) (cd tmp/app ; git show-ref --hash=7 refs/remotes/origin/production > version.txt) appcfg.py update -V "v`cat tmp/app/version.txt`" -A app tmp/app (cd tmp/huWaWi ; TESTHOST="v`cat version.txt`".app.appspot.com make resttest) appcfg.py update -V production -A app tmp/app appcfg.py backends -V production -A app tmp/huWaWi app
First we deploy the tagged version of the current version of AppEngine. Then we run resttest.py against this newly deployed version. On failure, m make stops execution. If the failure did not occur, the production version is deployed.
We also run credential pep8 , pyflakes and pylint checks the verification of the source code.
In general, we have very simple analytical tests, but they work a lot against production code and data. For us, this catches most of the mistakes we make with relatively little effort.