By Manoj Chaudhary, CTO and SVP of Engineering
To check the functionality, reliability, and security of their application programming interfaces (APIs), IT teams conduct a range of different tests, including functional, governance, chaos, penetration, performance, stress, and longevity assessments. Below, we outline common challenges teams face in performing these tests and offer our recommendations for addressing them.
Challenge: Getting through the initial setup
Creating, developing, and getting testing infrastructure up and running can be a time-consuming endeavor and it may be difficult to motivate your team throughout the process.
Our recommendation
Explaining how the testing process pays off over the long term can be key in motivating a testing team. It’s also important to note how critical this phase is in API testing overall. In doing the initial setup, think carefully about how you will be testing the API and what those tests require in your infrastructure design.
Challenge: Generating and managing large volumes of test data
APIs with many parameters require a significant amount of data (and in some cases, data with high cardinality) for effective testing. Creating, maintaining, and ensuring that this data is reusable can pose a challenge for testers.
Our recommendation
The key to generating and setting up test data is understanding the request and response structures and the associated API calls. Use data management tools that allow you to understand these connections, then mask, generate, and subset new data that fits your testing needs.
Challenge: Clearly understanding objectives for governance testing
Critical for APIs built for multi-tenant cloud applications, governance testing is performed to help protect infrastructure from overuse, noisy neighbors, and attacks. APIs typically have governance rules for their usage (e.g., storage policies, rate limits, and payload size), but an absence of knowledge and understanding related to both API architecture logic and these rules often results in uncertainty about testing objectives.
Our recommendation
Create well-defined and well-written governance rules; build utilities for your team to tweak them as needed to make testing easier and more efficient.
Challenge: Performing chaos testing effectively
In 2010, Netflix development and operations teams created this form of testing as they started to move traditional infrastructure to Amazon Web Services (AWS) cloud infrastructure. Chaos testing, or chaos engineering, is a highly disciplined approach that tests system integrity by proactively simulating and identifying failures before they lead to unplanned downtime or a poor user experience. The process involves:
- Testing that a system works in a defined steady state. First, testers must identify a measurable system output that indicates normal working behavior. Creating the infrastructure and tools for defining and measuring a steady state can be complex, depending on the architecture and scope of the applications involved.
- Testing to determine that a system’s steady state will hold. After establishing a steady state, testers determine whether it will continue in control, experimental, and real-world conditions.
- Testing for minimal impact on users. Testers break or disrupt service to determine negative impact to users.
- Introducing chaos. After establishing that a system is working in a defined steady state, that the steady state is holding up, and that the “blast radius” is contained, testers run their chaos-testing applications to see how the system behaves under particular stress conditions or circumstances (e.g., server crashes, malfunctioning hardware, or severed network connections).
Our recommendation
Build hooks in your application and infrastructure that allow your testing team to create chaos and easily get relevant metrics for the blast radius.
Challenge: Engaging the right team for penetration testing
In penetration testing, a tester with limited working knowledge of the specific API will attack it to assess the threat vector from an external perspective. Attacks can target the entire API or certain functions and processes.
Our recommendation
An internal team can perform initial penetration testing, but a third-party penetration tester should perform the follow-up deep penetration testing.
