BETTER API TESTING WITH OPENAPI SPECIFICATION TAYLOR BARNETT — @TAYLOR_ATX
A presentation at OSCON in July 2018 in Portland, OR, USA by Taylor Barnett
BETTER API TESTING WITH OPENAPI SPECIFICATION TAYLOR BARNETT — @TAYLOR_ATX
LEAD COMMUNITY ENGINEER @ STOPLIGHT TAYLOR@STOPLIGHT.IO @TAYLOR_ATX HI! 👋
[COMPANY] [BLEEPING]
@taylor_atx
Image by Paul Downey, CC by 2.0
SINGLE SOURCE OF TRUTH ☝ @taylor_atx
SINGLE SOURCE OF TRUTH ▸ What does everything (humans and machines) look to? ▸ Messy, outdated design docs? ▸ Something that everyone agrees to - a contract @taylor_atx
OPENAPI SPECIFICATION @taylor_atx
Standard, structured approach for describing APIs that is both human and machine readable @taylor_atx
Image by OpenAPI Initiative
OPENAPI SPECIFICATION ▸ Development contract ▸ Prototyping and mocking ▸ Client SDKs and libraries ▸ Testing ▸ Server stubs @taylor_atx
OPENAPI SPECIFICATION ▸ Simplifies describing an API ▸ Standardizes terminology ▸ “API Fastness” @taylor_atx
OPENAPI SPECIFICATION ▸ Good tutorial to walk through the specification: ▸ https://apihandyman.io/openapi/ @taylor_atx
“OPENAPI IS A BRIDGE TO UNDERSTANDING AND BEING ABLE TO COMMUNICATE AROUND USING HTTP AS A TRANSPORT, AND OUR GREATEST HOPE FOR HELPING [PEOPLE] LEARN THEIR HTTPS AND 123S.” Kin Lane (@apievangelist) @taylor_atx
DESIGN-FRIST APIS ✍ @taylor_atx
DESIGN-FIRST APIS ▸ Consistency for users ▸ Important for critical APIs ▸ Code-first can be expensive ▸ Gain the benefits of using OpenAPI @taylor_atx
CONTRACT TESTING @taylor_atx
Tests that ensure the API implementation meets the standards and definitions described in a contract @taylor_atx
CONTRACT TESTING ▸ Popularity has increased with the growth of microservices ▸ Some kind of contract available to all parties ▸ Enables independent deployments while not breaking other people’s code ▸ What it is not testing: ▸ Service availability ▸ Load tolerance ▸ Deployment integrity @taylor_atx
OPENAPI SPECIFICATION API RESPONSE VALIDATION SERVER PASS OR FAIL @taylor_atx
BUT I ALREADY TEST MY APIS… 🤔 @taylor_atx
(SOME) DIFFERENT TYPES OF TESTING ▸ Integration/Functional Tests ▸ Unit Tests ▸ End-to-end Tests @taylor_atx
(SOME) DIFFERENT TYPES OF TESTING UI TESTING CONTRACT TESTING INTEGRATION TESTING UNIT TESTING @taylor_atx
EXAMPLE: BROKEN SCHEMA 💔 @taylor_atx
curl —request GET —url http://api.taylorbar.net/hello-world 200 👍 @taylor_atx
@taylor_atx
Test Greetings Scenario (0.009s) Hello World, GET http://api.taylorbar.net/hello-world (0.009s) ✗ hello: hello is required ✓ ✗ output.status (eq) 2xx 1 of 2 tests failed 😔 @taylor_atx
API SPEC { “ello”: “world” } @taylor_atx
Test Greetings Scenario (0.012s) Hello World, GET http://api.taylorbar.net/hello-world (0.011s) ✓ output.body (validate.contract) against JSON Schema ✓ output.status (eq) 2xx ✓ 2 tests passed 😁 @taylor_atx
Todo CRUD Scenario (0.831s) Create Todo, POST http://todos.stoplight.io/todos?apikey=123 (0.244s) ✗ user: user is required (root): Must validate all the schemas (allOf) ✓ output.status (eq) 201 Get Todo, GET http://todos.stoplight.io/todos/61904 (0.13s) ✗ user: user is required (root): Must validate all the schemas (allOf) ✓ output.status (eq) 200 Update Todo, PUT http://todos.stoplight.io/todos/61904?apikey=123 (0.137s) ✗ user: user is required (root): Must validate all the schemas (allOf) ✓ output.status (eq) 200 Delete Todo, DELETE http://todos.stoplight.io/todos/61904?apikey=123 (0.126s) ✓ output.status (eq) 204 ✓ output.body (validate.contract) against JSON Schema Make Sure Deleted, GET http://todos.stoplight.io/todos/61904 (0.192s) ✓ output.status (eq) 404 ✓ output.body (validate.contract) against JSON Schema ✗ 3 of 10 tests failed 😔 @taylor_atx
API SPEC { “completed”: null, “completed_at”: null, “created_at”: “2018-07-18T18:33:19.341Z”, “id”: 61907, “name”: “feed Yoda”, “updated_at”: “2018-07-18T18:33:19.341Z” } @taylor_atx
Todo CRUD Scenario (0.902s) Create Todo, POST http://todos.stoplight.io/todos?apikey=123 (0.277s) ✓ output.status (eq) 201 ✓ output.body (validate.contract) against JSON Schema Get Todo, GET http://todos.stoplight.io/todos/61908 (0.141s) ✓ output.status (eq) 200 ✓ output.body (validate.contract) against JSON Schema Update Todo, PUT http://todos.stoplight.io/todos/61908?apikey=123 (0.145s) ✓ output.status (eq) 200 ✓ output.body (validate.contract) against JSON Schema Delete Todo, DELETE http://todos.stoplight.io/todos/61908?apikey=123 (0.129s) ✓ output.status (eq) 204 ✓ output.body (validate.contract) against JSON Schema Make Sure Deleted, GET http://todos.stoplight.io/todos/61908 (0.209s) ✓ output.status (eq) 404 ✓ output.body (validate.contract) against JSON Schema ✓ 10 tests passed 😁 @taylor_atx
EXAMPLE: SDKS 🔥 @taylor_atx
@taylor_atx
ADD CONTRACT TESTING INTO CI/CD PIPELINE @taylor_atx
SDKS ▸ Helps maintainers ▸ Ensures API and SDK/libraries parity ▸ Go see Elmer Thomas’ talk at 5:05pm on “Managing SDKs and their communities in multiple programming languages” @taylor_atx
EXAMPLE: DOCS 📄 @taylor_atx
@taylor_atx
WHAT ABOUT PACT TESTING? @taylor_atx
SINGLE SOURCE OF TRUTH PITFALLS @taylor_atx
SINGLE SOURCE OF TRUTH PITFALLS ▸ Forgetting the “why” ▸ Failing to version ▸ “version” is in the Info Object ▸ Bad communication @taylor_atx
COMMUNICATION @taylor_atx
@taylor_atx
@taylor_atx
THEY BECOME THE CODIFICATION OF A SET OF DISCUSSIONS ABOUT WHAT A SERVICE API SHOULD LOOK LIKE, AND WHEN THEY BREAK, THEY BECOME A TRIGGER POINT TO HAVE CONVERSATIONS ABOUT HOW THAT API SHOULD EVOLVE. Sam Newman, “Building Microservices: Testing” @taylor_atx
TAYLOR@STOPLIGHT.IO @TAYLOR_ATX
