A presentation at OSCON in in Portland, OR, USA by Taylor Barnett
BETTER API TESTING WITH OPENAPI SPECIFICATION TAYLOR BARNETT — @TAYLOR_ATX
LEAD COMMUNITY ENGINEER @ STOPLIGHT TAYLOR@STOPLIGHT.IO @TAYLOR_ATX HI! 👋
[COMPANY] [BLEEPING]
@taylor_atx
Image by Paul Downey, CC by 2.0
SINGLE SOURCE OF TRUTH ☝ @taylor_atx
SINGLE SOURCE OF TRUTH ▸ What does everything (humans and machines) look to? ▸ Messy, outdated design docs? ▸ Something that everyone agrees to - a contract @taylor_atx
OPENAPI SPECIFICATION @taylor_atx
Standard, structured approach for describing APIs that is both human and machine readable @taylor_atx
Image by OpenAPI Initiative
OPENAPI SPECIFICATION ▸ Development contract ▸ Prototyping and mocking ▸ Client SDKs and libraries ▸ Testing ▸ Server stubs @taylor_atx
OPENAPI SPECIFICATION ▸ Simplifies describing an API ▸ Standardizes terminology ▸ “API Fastness” @taylor_atx
OPENAPI SPECIFICATION ▸ Good tutorial to walk through the specification: ▸ https://apihandyman.io/openapi/ @taylor_atx
“OPENAPI IS A BRIDGE TO UNDERSTANDING AND BEING ABLE TO COMMUNICATE AROUND USING HTTP AS A TRANSPORT, AND OUR GREATEST HOPE FOR HELPING [PEOPLE] LEARN THEIR HTTPS AND 123S.” Kin Lane (@apievangelist) @taylor_atx
DESIGN-FRIST APIS ✍ @taylor_atx
DESIGN-FIRST APIS ▸ Consistency for users ▸ Important for critical APIs ▸ Code-first can be expensive ▸ Gain the benefits of using OpenAPI @taylor_atx
CONTRACT TESTING @taylor_atx
Tests that ensure the API implementation meets the standards and definitions described in a contract @taylor_atx
CONTRACT TESTING ▸ Popularity has increased with the growth of microservices ▸ Some kind of contract available to all parties ▸ Enables independent deployments while not breaking other people’s code ▸ What it is not testing: ▸ Service availability ▸ Load tolerance ▸ Deployment integrity @taylor_atx
OPENAPI SPECIFICATION API RESPONSE VALIDATION SERVER PASS OR FAIL @taylor_atx
BUT I ALREADY TEST MY APIS… 🤔 @taylor_atx
(SOME) DIFFERENT TYPES OF TESTING ▸ Integration/Functional Tests ▸ Unit Tests ▸ End-to-end Tests @taylor_atx
(SOME) DIFFERENT TYPES OF TESTING UI TESTING CONTRACT TESTING INTEGRATION TESTING UNIT TESTING @taylor_atx
EXAMPLE: BROKEN SCHEMA 💔 @taylor_atx
curl —request GET —url http://api.taylorbar.net/hello-world 200 👍 @taylor_atx
@taylor_atx
Test Greetings Scenario (0.009s) Hello World, GET http://api.taylorbar.net/hello-world (0.009s) ✗ hello: hello is required ✓ ✗ output.status (eq) 2xx 1 of 2 tests failed 😔 @taylor_atx
API SPEC { “ello”: “world” } @taylor_atx
Test Greetings Scenario (0.012s) Hello World, GET http://api.taylorbar.net/hello-world (0.011s) ✓ output.body (validate.contract) against JSON Schema ✓ output.status (eq) 2xx ✓ 2 tests passed 😁 @taylor_atx
Todo CRUD Scenario (0.831s) Create Todo, POST http://todos.stoplight.io/todos?apikey=123 (0.244s) ✗ user: user is required (root): Must validate all the schemas (allOf) ✓ output.status (eq) 201 Get Todo, GET http://todos.stoplight.io/todos/61904 (0.13s) ✗ user: user is required (root): Must validate all the schemas (allOf) ✓ output.status (eq) 200 Update Todo, PUT http://todos.stoplight.io/todos/61904?apikey=123 (0.137s) ✗ user: user is required (root): Must validate all the schemas (allOf) ✓ output.status (eq) 200 Delete Todo, DELETE http://todos.stoplight.io/todos/61904?apikey=123 (0.126s) ✓ output.status (eq) 204 ✓ output.body (validate.contract) against JSON Schema Make Sure Deleted, GET http://todos.stoplight.io/todos/61904 (0.192s) ✓ output.status (eq) 404 ✓ output.body (validate.contract) against JSON Schema ✗ 3 of 10 tests failed 😔 @taylor_atx
API SPEC { “completed”: null, “completed_at”: null, “created_at”: “2018-07-18T18:33:19.341Z”, “id”: 61907, “name”: “feed Yoda”, “updated_at”: “2018-07-18T18:33:19.341Z” } @taylor_atx
Todo CRUD Scenario (0.902s) Create Todo, POST http://todos.stoplight.io/todos?apikey=123 (0.277s) ✓ output.status (eq) 201 ✓ output.body (validate.contract) against JSON Schema Get Todo, GET http://todos.stoplight.io/todos/61908 (0.141s) ✓ output.status (eq) 200 ✓ output.body (validate.contract) against JSON Schema Update Todo, PUT http://todos.stoplight.io/todos/61908?apikey=123 (0.145s) ✓ output.status (eq) 200 ✓ output.body (validate.contract) against JSON Schema Delete Todo, DELETE http://todos.stoplight.io/todos/61908?apikey=123 (0.129s) ✓ output.status (eq) 204 ✓ output.body (validate.contract) against JSON Schema Make Sure Deleted, GET http://todos.stoplight.io/todos/61908 (0.209s) ✓ output.status (eq) 404 ✓ output.body (validate.contract) against JSON Schema ✓ 10 tests passed 😁 @taylor_atx
EXAMPLE: SDKS 🔥 @taylor_atx
@taylor_atx
ADD CONTRACT TESTING INTO CI/CD PIPELINE @taylor_atx
SDKS ▸ Helps maintainers ▸ Ensures API and SDK/libraries parity ▸ Go see Elmer Thomas’ talk at 5:05pm on “Managing SDKs and their communities in multiple programming languages” @taylor_atx
EXAMPLE: DOCS 📄 @taylor_atx
@taylor_atx
WHAT ABOUT PACT TESTING? @taylor_atx
SINGLE SOURCE OF TRUTH PITFALLS @taylor_atx
SINGLE SOURCE OF TRUTH PITFALLS ▸ Forgetting the “why” ▸ Failing to version ▸ “version” is in the Info Object ▸ Bad communication @taylor_atx
COMMUNICATION @taylor_atx
@taylor_atx
@taylor_atx
THEY BECOME THE CODIFICATION OF A SET OF DISCUSSIONS ABOUT WHAT A SERVICE API SHOULD LOOK LIKE, AND WHEN THEY BREAK, THEY BECOME A TRIGGER POINT TO HAVE CONVERSATIONS ABOUT HOW THAT API SHOULD EVOLVE. Sam Newman, “Building Microservices: Testing” @taylor_atx
TAYLOR@STOPLIGHT.IO @TAYLOR_ATX
If you search on the internet for unexpected API behavior, you’ll soon discover that no one likes when an API doesn’t work as expected. When you take into account the increasing number of APIs and continuous development and delivery of the services built on top of them, it’s no surprise that APIs can move away from their initially expected behavior. This is why expanding API test coverage is critical for success. For years, we have created unit and functional tests for our APIs, but where do we go from there?
Taylor Barnett explains how to contract test APIs (i.e, writing tests to ensure that an API or microservice works as described in an agreement) based on a single source of truth—in this case, an OpenAPI Specification document. Contract testing is useful for testing the accuracy of API implementations in server code, SDKs, client libraries, and even third-party APIs, which is especially helpful when other services rely on an API.
Taylor details the differences between testing an SDK and the implementation of an API and other use cases and outlines how contract testing can lead to faster development, fewer breaking code changes, and more accurate implementations. Along the way, she shares real-life contract testing examples and techniques to implement within your API projects.
