Ask DevOps: Three Ways to Make Automated Tests Faster


A lot of us have encountered an app that suddenly won’t work, hangs, or won’t open at all.  And we immediately think that the app developer slacked off and and made a really crappy app.

But app development isn’t as easy as one may think. You’d have to deal with numerous libraries and code segments that if meshed together could cause the app to fail.  Aside from that, developers must deal with numerous tests in order to see if their apps would run smoothly, and these tests are often time consuming.

Developers deal with automated tests that ran several times within the day and this is quite normal but as I’ve mentioned earlier, time consuming.  The slowness of these tests depends on the artifacts produced from the build that are copied over to the test servers are very large–more than 1 GB in size; the artifacts are sometimes copied across continents; there are multiple versions of the build artifacts that must be copied to different test servers after the build; the database(s) for the product must be loaded with a large amount of test data, which can take a long time to copy and load; and many products have separate test teams writing test automation so testers who are not developers tend to write tests that run through the UI, and those tests are usually slower than developers’ code-level unit tests.

Running builds and test often is important so developers would know if something breaks or if something doesn’t go well with the other parts.  Developers need to find the perfect balance in order to ensure continuous integration and delivery.  So it’s important for them to get feedback under five minutes, anything beyond that is too slow and could greatly affect their process.

So here are some tips for developers to make automated tests run quickly:

Fail Quickly

Often times, tests are delivered in alphabetical order or in the order they were created.  The problem here is, if it’s in a particular order, the slowest test could be delivered at the start, which would make the whole process really long and pointless when you get to a test that fails at the end.  So it would be a waste of time.

In order to make the process faster, you need to run a smoke test which ensures that the infrastructure and services your tests will need are available–simply put, check if everything you need is ready and available.  This would only take a few seconds to a few minutes.

Run small sets of test that often fail, this would only take a few minutes, if it runs longer than 5 minutes, you should reduce the number of tests in a set.  After that, run the tests whose codes haven’t changed for a long time then run the tests that never fail for last.  So it would seem like you’re just checking if everything’s in order.  And run the slowest or most time consuming tests lastly.  This method will help you determine problems much faster since you’re running the tests that often fail at the beginning.

Run in parallel

Another step to make automated tests faster is by running test sets parallel to one another, which means conducting tests simultaneously.  But you have to take note that you need to deploy many virtual machines in the process and this might not be the cheapest workaround.

You can also configure VMs faster if you use saved snapshots or images to compare the end result from what you started with.  Or you can create database backups with a lot of test data a;ready pre-loaded rather than relying on SQL statements or running code to populate the database.

Break up tests into smaller groups

Testing builds is a process of elimination.  Take for example testing Christmas lights.  There are about 50 to a hundred bulbs in a string of lights.  Before you put them up, you need to test them if they work.  Sometimes, the lights won’t work, so you’d need to know if the bulbs are busted or if the wires are screwed. Same thing goes with applications.

Applications can be divided into smaller components so you can test and build them separately, then once they’re all perfect separately, you can test them with each other, so it would be easy to identify what went wrong.  It’s basically a process of elimination.  Then you need to configure the sequence of the tests that automatically run based on the codes that changed.

There are a lot more tips available here, so if you’re a developer looking into making your process faster and far more effective, check it out.