A presentation at Eurostar Huddle Automation Deep Dive Webinar by Daniel Kocot
APIOps Daniel Kocot, Senior Solution Architect / Head of API Experience & Operations
Name: Daniel Kocot Role: Senior Solution Architect / Head of API Experience & Operations Email: daniel.kocot@codecentric.de Twitter: @dk_1977 LinkedIn: https://www.linkedin.com/in/danielkocot/
No Buzzword Bingo!
CALMS Model
Collaboration
Automation
Lean Principles and Processes
Measurement
Sharing
API first An API is the first (and often only) interface to users of an application An API comes first — before the implementation An API is described (documented) or self-descriptive
API Design (first)
OpenAPI / AsyncAPI
OpenAPI Spec Example
Specification Version
Tooling
GIT some kind of GitFlow no direct commits to main Branch every change to specification and the pipeline has to be a pull request
Developer Setup
IDE or Editor Eclipse JetBrains Products Visual Studio Code Stoplight Studio Apicurio Studio Insomnia
local validation Redocly-CLI Spectral
Spectral > npm install -D @stoplight/spectral > npx spectral lint news.yaml OpenAPI 3.x detected No results with a severity of ‘error’ or higher found!
Ruleset formats: - oas3.0 extends: - ‘spectral:oas’ rules: tags-have-description: description: Tags must have a description. message: Description of Tag is missing given: $.tags[*] recommended: true type: style then: field: description function: truthy
local mocking
Prism > npm install -D @stoplight/prism-cli > npx prism mock news.yaml -p 8080 [17:13:00] › [CLI] … awaiting Starting Prism… [17:13:01] › [CLI] ℹ info GET http://127.0.0.1:8080/news [17:13:01] › [CLI] ▶ start Prism is listening on http://127.0.0.1:8080
local testing
Contract testing Create a test suite based directly on the spec Using a BDD framework Create a test suite based on a postman collection
Portman > npm install -D @apideck/portman > prism mock specs/news.yaml -p 8080 | portman -l specs/news.yaml -n
Portman ================================================================= Local Path: specs/news.yaml Portman Config: portman-config.default.json Postman Config: postman-config.default.json Environment: .env Inject Tests: true Run Newman: true Newman Iteration Data: false Upload to Postman: false ================================================================= ✔ Conversion successful ================================================================= Run Newman against: ================================================================= newman News API ❏ news ↳ get News GET http://localhost:8080/news [200 OK, 384B, 85ms] ✓ [GET]::/news - Status code is 2xx ✓ [GET]::/news - Content-Type is application/json ✓ [GET]::/news - Response has JSON Body
Load testing Smoke Load Stress Soak
postman-to-k6 > npm install -D postman-to-k6 > mkdir k6 > npx postman-to-k6 post-collections/news-postman-collection.json -o k6/news-k6-script.js
k6 > prism mock specs/news.yaml -p 8080 | k6 run k6/news-k6-script.js
k6 /\ /\ / \ / / \ / \ / __________
|‾‾| /‾‾/ /‾‾/ | |/ / / / | ( / ‾‾\ | |\ \ | (‾) | |__| _\ ____/ .io
execution: local script: k6/news-k6-script.js output: scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop): * default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)
running (00m00.0s), 0/1 VUs, 1 complete and 0 interrupted iterations default ✓ [======================================] 1 VUs 00m00.0s/10m0s data_received…………..: data_sent………………: http_req_blocked………..: http_req_connecting……..: http_req_duration……….: http_req_failed…………: http req receiving :
502 B 26 kB/s 134 B 7.0 kB/s avg=1.31ms min=1.31ms avg=237µs min=237µs avg=12.92ms min=12.92ms 100.00% ✓ 1 ✗ 0 avg=110µs min=110µs
1/1 iters, 1 per VU
med=1.31ms max=1.31ms p(90)=1.31ms p(95 med=237µs max=237µs p(90)=237µs p(95 med=12.92ms max=12.92ms p(90)=12.92ms p(95 med=110µs
max=110µs
p(90)=110µs
p(95
OpenAPI
YAML / JSON YAML is more human-readable JSON is more machine-readable Parsing JSON is faster ;)
Converting YAML to JSON > npm install -g yaml2json > yaml2json specs/news.yaml
Structure https://openapi-map.apihandyman.io
Splitting the structure for reuse and better overview == Design Library
Hard splitting one file per object
Soft splitting Depending on the size of the whole document or the objects
Use of references with $ref local ‘#/components/schemas/myElement’ remote ‘myElement.yaml’ url ‘http://path/to/your/myElement.yaml’
Something is needed to rebundle the files to one > npx @redocly/redocly-cli > redocly bundle specs/news.yaml —output output/news.yaml
Use of OpenAPI Extensions/X-Objects to handle own or vendor needs x-vendor-… x-… Supported by: root level info paths operation parameters responses tags security schemes
From API description to configuration as code OpenAPI with Extensions AWS Cloudformation AWS CDK Azure ARM Templates Azure Bicep Pulumi
Example AWS Cloudformation - API Spec openapi: 3.0.0 info: title: API Gateway OpenAPI Example version: 1.0.0 paths: /api/posts: get: summary: List Posts operationId: listPosts requestBody: required: true content: application/json: schema: ‘$ref’: ‘#/components/schemas/CreatePostRequestBody’ responses: ‘200’: description: Retrieve the list of Posts content: application/json: schema: ‘$ref’: ‘#/components/schemas/ListPostsResponseBody’ x-amazon-apigateway-integration:
Example AWS Cloudformation - S3 Bucket Stack AWSTemplateFormatVersion: 2010-09-09 Resources: ArtifactBucket: Type: AWS::S3::Bucket Outputs: ArtifactBucket: Description: The name of the artifact bucket Value: !Ref ArtifactBucket Export: Name: !Sub ${AWS::StackName}-artifact-bucket
Example AWS Cloudformation - AWS API-Gateway Stack AWSTemplateFormatVersion: ‘2010-09-09’ Parameters: ProjectId: Type: String Default: experiment Bucket: Type: String Default: api-gateway-openapi-artifact-bucke-artifactbucket-1wmq2pswrxwjw OpenAPIS3Key: Type: String Default: openapi.yaml Resources: Api: Type: AWS::ApiGateway::RestApi Properties: Name: !Ref AWS::StackName Description: ‘An experimental API’ FailOnWarnings: true
Some gateways vendors have their own toolsets for CaC which have to be integrated in an existing toolchain For example: Kong decK Inso (Insomnia CLI) Tyk Tyk Sync
Deployable Infrastructure based on the definition Gateways Portals Hubs Registries
Transformation to automation within CI/CD API first GIT Process Well structured and formed API specification Automated Linting Automated Testing Automated Deployment of revelant infrastructure
Should we build a framework on our own for this?
Missing Parts
Building SDKs
Security OWASP API Top 10 Security Best Practices
Policy (as Code) OPA Sentinel
Wrap Up Posts on codecentric blog: https://blog.codecentric.de/en/author/daniel-kocot/ Posts on my blog: https://danielkocot.github.io Posts on Medium: https://medium.com/@daniel.kocot
Q&A
Thank you
References Photo on slide 7 by Alice Dietrich on Unsplash Photo on slide 13 by Danial Igdery on Unsplash
for free. You
can too.