The size of our testing team at Audacia has grown dramatically in recent years, tripling in size since 2019. This growth has led to a large increase in available resources. QAs are now dedicated to one or two projects and have time to look at Automated regression, security and performance testing.
In the past, manual testing has been the main focus within the company. The size of the testing team meant that we did not have available resources to dedicate to non-functional testing. However, recent growth has allowed us to change the way that the software testing team works.
We have adopted an automation first approach to testing. This means that we are writing automation tests from the inception of the project. Taking this action allows an entire regression test to be run automatically. By saving time on manual testing, we were able to look at two key areas of non-functional testing; Performance and Security.
A focus on Performance in QA software testing services
Imagine a scenario where a project is nearing completion and the client drops the bombshell that performance is of paramount importance to their business and customers.
This non-functional requirement was not mentioned during the scoping of the system requirements. The developers have created the application under the assumption that performance is not a high priority. The test team has accepted the application as functional.
In a typical project with no test automation, the real user load is only tested at the point of going live. We may be aware of the expected number of users, but have not tested the system with that volume.
On the first day the system goes live, it fails because it cannot handle a large number of users. This failure renders the application unusable for the client.
Who is at fault for this?
The short answer is: everyone. Someone should have questioned this at all points of the development process.
Continuous Performance Testing - Living in a Perfect World
Ideally, the performance of an application should be considered at all points of the software development lifecycle: Design, Implementation and Testing. However, this is an ideal that is rarely achieved due to the time and budget constraints of software development.
Where this is not possible, we aim to effectively test the application so we have an awareness of performance bottlenecks in the system.
Even if it is considered, performance testing is not usually actioned until the end of a project due to time constraints. It may be deemed time consuming or unessential.
Integrating Performance Testing into QA Software Testing Lifecycles
At Audacia we aim to run key forms of testing at all points in a project's life. This includes Manual, Security, Performance and Accessibility. This process starts with scoping out the non-functional requirements and continues with testers planning & writing a full stack of tests.
Since taking an automated first approach to testing, we have seen an improvement in the overall quality in products. Time saved on regression and other manual testing has allowed software testers to spend more time on non-functional tests. These are becoming a core activity within the test team and give us more confidence in our products.
Creating performance tests at the beginning of the project means that we are aware of potential performance issues much earlier.
Locating and fixing performance issues through the project encourages the whole team to value performance. Having that mindset in place makes performance a top consideration when creating products. With this consideration, everyone has greater confidence in their work when the project goes live.
Holistic QA Software Testing Strategy
It can also be easy to value performance too highly. This would be the case if your software testing team only considers page load times for a public facing application.
While this approach will improve performance it does not capture the whole user experience. The software testing team is also left vulnerable to unexpected performance issues.
We Should Take a Holistic Approach to QA Software Testing
Further Reading: 5 Best Practices
For example, alongside page load times, we should test how easy/quickly a user can get meaningful data. It would not be acceptable for the page to load in a few seconds, but requests to get meaningful data take over a minute.
Manufacturing Chaos into QA Software Testing Processes
Your software testing team could integrate performance testing into all points of your lifecycle. It is also possible to consider all predictable failures and protect against these failures. This procedure will help prevent your application roll-out from failing or any other unforeseen issues.
Chaos by its very definition is the inability to predict future events.
How Do We Plan for the Unexpected?
Inspired by Netflix, we manufacture our own chaos and include it in the software lifecycle.
Netflix created a number of open-source tools such as Chaos Monkey that prepare software developers and QA for working in an artificially chaotic environment. The overall purpose of these tools is to randomly shut down infrastructure such as servers, Load balancers etc.
While this may seem strange, at first, it means that project teams are constantly battling unforeseen circumstances. This changes the way that the team thinks; they inherently create a more resilient application by creating added protection against chaos.
Netflix tools work best for solutions with a large infrastructure i.e. globally used software. Using these principles, we can then adapt them to suit a software company like ours. Something that is relevant to project size and scope.
On small projects this could be putting a small module in the software that randomly fails API requests at random intervals.
Working in a predictably chaotic environment creates a mind-set that can anticipate chaos and create enhanced protection against it.
Expect the Unexpected
The software industry is chaotic due to ever-changing requirements and priorities. A software tester can learn from this reality by implementing ‘chaos’ into their own procedures. If you manufacture your own chaos it allows your team to be better prepared for actual unplanned events.
This model will encourage your team to have a changed mindset and improve their quality assurance skills. Long term this leads to a better product being delivered to the client.
Audacia is a software development company based in the UK, headquartered in Leeds. View more technical insights from our teams of consultants, business analysts, developers and testers on our technology insights blog.
Technology Insights
