31 May 2017

Execution Time of Test Suites

Your tests should be fast. Really fast. Like all your unit tests should run in 10 seconds fast. 10 seconds is actually an eternity. That is almost long enough for me to loose focus and walk away. But if there are a lot of tests I can see how 10 seconds is reasonable.

Crazy talk? I think not. There are some caveats of course, but in general, if your system is up configured, the actual execution of your unit test suite should be in the vicinity of 10 seconds. For that mater, your Integration and Acceptance tests should be really fast too. 

In the end, everything should run in 10 minutes. 10 minutes is really too long, more than enough time to get coffee and become distracted by the PingPong table. But 10 minutes will do.

If you can't get your tests running this fast you need to seriously examine what you are doing. Some things you might investigate are;

Are you unit tests really unit tests? Or have they integrated with something?
Is your setup overly complicated? 
Do your integration tests or acceptance tests keep rebuilding the system?
Have you maximized your ability to run test suites in parallel?
Have you failed to modularize your application?

I think there will be several more posts on this topic. But let me lay out this maxim for you; No matter how fast your tests are, you should always be trying to make them faster.