What happened in Serverless in 2019

About me Chris Munns - munns@amazon.com, @chrismunns • Sr Manager/Principal Developer Advocate – Serverless • New Yorker (ehhh…ish.. kids/burbs/ya know?) • Previously: • AWS Business Development Manager – DevOps, July ‘15 - Feb ‘17 • AWS Solutions Architect Nov ‘11- Dec ‘14 • Formerly on operations teams @Etsy and @Meetup • Little time at a hedge fund, Xerox and a few other startups • Rochester Institute of Technology: Applied Networking and Systems Administration ’05 • Internet infrastructure geek © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Serverless applications – add events Event source Changes in data state Requests to endpoints Changes in Resource state © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Function Node.js Python Java C# Go Ruby Runtime API

Serverless applications – add any other services Event source Changes in data state Requests to endpoints Changes in Resource state © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Function Node.js Python Java C# Go Ruby Runtime API Services

Behind the curtain The Lambda service team is dedicated to making things you don’t even see or know about better: “a reduction of 20 ms jitter in the invoke duration for Lambda Functions across all runtimes..”– May 2019

April 2019 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Lambda states Serverless applications with Lambda often require other non-Lambda resources Creation/modification of those resources can impact function execution time The states lifecycle allows us to move that resource modification before execution reducing the pain involved Now: VPC resources are created/modified before your function is “Active” for executions © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Cold Starts and you A cold start is what occurs when a system needs to create a new resource in response to an event/request. For Lambda: • • Happens when new execution environments are needed to handle requests Typically <1% of all invokes for “production workloads” • • As measured by functions with consistent invokes over a period of time, aka not dev, excludes rarely invoked Can vary from <100ms to a > 1 second Why this variance? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available Yes Complete invocation Invoke handler warm start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available Yes Complete invocation Invoke handler warm start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. This is the path that most invocations end up taking in applications with somewhat consistent traffic patterns. A small <1% of requests would cause a cold-start and then ~99% would end on warmed environments

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available No Find available compute resource “full” cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available No Find available compute resource “full” cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Download customer code

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available No Find available compute resource Download customer code Start execution environment “full” cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available No Find available compute resource Download customer code Execute INIT Start execution environment “full” cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

INIT - Pre-handler code, dependencies, variables Executes at the initial start of an execution environment. • Import only what you need • Where possible trim down SDKs and other libraries to the specific bits required • • Pre-handler code is great for establishing connections, but be prepared to then handle reconnections in further executions REMEMBER – execution environments are reused • • • Lazily load variables in the global scope Don’t load it if you don’t need it – cold starts are affected Clear out used variables so you don’t run into left-over state © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Import sdk Import http-lib Import ham-sandwich Pre-handler-secret-getter() Pre-handler-db-connect() Function myhandler(event, context) { ….

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available Complete invocation Invoke handler No Find available compute resource Download customer code Execute INIT Start execution environment “full” cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available Yes Complete invocation Invoke handler warm start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Subsequent invokes could then follow the warm path

Concurrency and you A single Lambda execution environment can only process a single event at a time. • • Regardless of event source or invoke type Batches pulled from Amazon Kinesis Data Streams, Amazon SQS, or Amazon DynamoDB Streams count as a single event Concurrent requests will require new execution environments to be created • Limited in concurrency by burst rate per account per region © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Concurrency and you Time Request 1 Cold start Execution Request 2 Cold start Execution Request 3 Cold start Execution Request 4 Request 5 Cold start Cold start Execution Execution The first environment is now free and warm © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Concurrency and you Time Request 1 Cold start Execution Request 2 Cold start Execution Request 3 Cold start Execution Request 4 Request 5 Cold start Cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Request 6 Execution Execution Execution

Concurrency and you Time Request 1 Cold start Execution Request 2 Cold start Execution Request 3 Cold start Execution Request 4 Request 5 Cold start Cold start Request 6 Execution Request 7 Execution Request 8 Execution Execution Request 10 Execution Request 9 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cold start Ex

Concurrency and you Time Request 1 Cold start Request 2 Cold start Execution Request 3 Cold start Execution Request 4 Request 5 Request 6 Execution Cold start Cold start Execution Request 7 Execution Request 8 Execution Execution Request 10 Execution Request 9 Concurrency 3 5 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4 4 6 5 Cold start Ex

Lambda per function concurrency controls Concurrency a shared pool by default = 1000 per account per region Separate using per function concurrency settings • Acts as reservation Also acts as max concurrency per function • Especially critical for downstream resources like databases “Kill switch” – set per function concurrency to zero © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Introducing AWS Lambda Provisioned Concurrency Pre-creates execution environments all the way up through the INIT phase. • • • • • Mostly for interactive workloads that are heavily latency sensitive Greatly improved consistency across the full long tail of performance Little to no changes to your code or way you use Lambda Integrated with AWS Auto Scaling Adds a cost factor for per concurrency provisioned but a lower duration cost for execution • This could end up saving you money when heavily utilized © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. NEW!!!

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available Complete invocation Invoke handler No Find available compute resource Download customer code Execute INIT Start execution environment “full” cold start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Function configured with Provisioned Concurrency Find available compute resource Download customer code Execute INIT Start execution environment Provisioned concurrency start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Function lifecycle Request made to Lambda’s API Service identifies if warm execution environments is available Yes Complete invocation Invoke handler warm start © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. This becomes the default for all provisioned concurrency execution environments

Provisioned Concurrency – things to know • • Reduces the start time to your function handler to <100ms Can’t configure for $LATEST • • • • • Use versions/aliases Soft limit of 500 provisioned execution environment creation per minute No changes to function handler code performance Requests above provisioned concurrency follow on-demand Lambda limits and behaviors for cold-starts, bursting, pricing Still limited by overall account concurrency per limit region © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Provisioned Concurrency – things to know In order to maintain availability and help with scaling we provision more resources than you request • • You don’t pay for these extra execution environments But because they execute INIT, they could consume other resources from your account (DB connections, calls to other services) In order to keep execution environments fresh, we still reap them regularly but will pre-create execution environments behind the scenes • • You’ll see INITs in your logs every few hours without configuration changes or other events This won’t impact performance We give less CPU burst to Provisioned Concurrency than On-Demand during INIT and so code could take longer to execute © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Provisioned Concurrency – pricing https://aws.amazon.com/lambda/pricing/ : • Provisioned Concurrency is calculated from the time you enable it on your function until it is disabled, rounded up to the nearest 5 minutes. The price depends on the amount of memory you allocate to your function and the amount of concurrency that you configure on it. No free tier. • Duration is calculated from the time your code begins executing until it returns or otherwise terminates, rounded up to the nearest 100ms. The price depends on the amount of memory you allocate to your function. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Lambda API API provided by the Lambda service SDK clients 1. Lambda directly invoked via invoke API Used by all other services that invoke Lambda across all models Supports sync and async Can pass any event payload structure you want Lambda function © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Client included in every SDK

Lambda execution model Synchronous (push) Amazon API Gateway Asynchronous (event) Amazon SNS Amazon DynamoDB Amazon S3 Amazon Kinesis /order Lambda function © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Stream (Poll-based) reqs Lambda function changes AWS Lambda service function

HTTP APIs for Amazon API Gateway Achieve up to 67% cost reduction and 50% latency reduction compared to REST APIs. HTTP APIs are also easier to configure than REST APIs, allowing customers to focus more time on building applications. Reduce application costs by up to 67% Reduce application latency by up to 50% © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Configure HTTP APIs easier and faster than before

Synchronous APIs POST /order POST /invoice Who owns the retry? Does OrderFor how long?Invoice 201client service the ever know?201Etc.. service Client This effectively creates a “tight coupling” where failures become harder to recover from © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

If you don’t need a response, execute async Use the Lambda APIs to start an asynchronous execution • Built-in queue (Amazon SQS behind the scenes) • Automatic retries • Dead letter queue for failed events client = boto3.client(“lambda”) client.invoke_async( FunctionName=”test” InvokeArgs=json_payload ) © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Recent announcements for async event sources Amazon SQS FIFO as invoke source for Lambda Amazon SNS Dead Letter Queues (DLQs) For streamed events: • • • • • • NEW!!! MaximumRetryAttempts, MaximumRecordAgeInSeconds, BisectBatchOnFunctionError, On-failure destination Batch Window Parallelization Factor For async events: • • • MaximumRetryAttempts MaximumEventAgeInSeconds ICYMI: Serverless pre:Invent 2019 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Directed vs. Observable events I don’t care about this Joe, please create an invoice. Customer X just ordered a widget I’ll send an invoice Ok, here it is I’ll add that to the sales report Directed Observable Commands Events © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Event passing with Amazon EventBridge Amazon Simple Notification Service HTTPS server on premises Amazon Kinesis Data Firehose Amazon S3 Amazon Athena Amazon Elastic Container Service Task Lambda based service Amazon EventBridge AWS Step Functions Workflow Amazon Comprehend AWS Lambda © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker

Events with Amazon EventBridge • Order service Amazon EventBridge © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Your services can both produce messages onto the bus and consume just the messages they need from the bus Services don’t need to know about each other, just about the bus.

Introducing: RDS Proxy (Preview) Fully managed, highly available database proxy feature for Amazon RDS. Pools and shares DB connections to make applications more scalable, resilient to database failures, and secure. Pool and share app scaling availability DB failover times © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. data security access controls Fully managed compatible

Scaling Serverless apps with Amazon RDS Proxy Lambda functions RDS Proxy SQL / TLS Connection Pool Amazon RDS AWS Secrets Manager © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IAM

Two options for security: Control is in your hands Developers can use either IAM or native DB credentials to connect 1 n o ti p O 1: t tion p e es a St equ ntic R uthe n A ke To Amazon RDS Proxy SQL / TLS SQL / TLS Connect with native DB creds © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step 2: Connect with IAM token Amazon RDS 2 n o ti p O AWS IAM

Introducing Lambda Event Destinations For asynchronous invocations, capture Success or Failure record contains details about the request and response in JSON format • Contains more information than data sent to a DLQ • Can send both outcomes to same destination or • Can send success to one destination, failure to another • © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. NEW!!! AWS Lambda AWS Lambda Amazon SNS Amazon SQS Amazon EventBridge

AWS X-Ray Profile and troubleshoot serverless applications: • Lambda instruments incoming requests for all supported languages and can capture calls made in code • API Gateway inserts a tracing header into HTTP calls as well as reports data back to X-Ray itself © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. var AWSXRay = require(‘aws-xray-sdk-core‘); var AWS = AWSXRay.captureAWS(require(‘aws-sdk’)); S3Client = AWS.S3();

Recent announcements for X-Ray & Amazon CloudWatch CloudWatch ServiceLens • a “single pane of glass” monitoring tool CloudWatch Synthetics • Generate “canary” tests for your APIs and other services Embedded Metric Format for CloudWatch Logs CloudWatch Contributor Insights • Identify top talkers, common patterns in logs X-Ray Trace Maps • Map a single event though your distributed application X-Ray integration with CloudWatch Synthetics © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ICYMI: Serverless pre:Invent 2019

AWS Serverless Application Model (AWS SAM) AWS CloudFormation extension optimized for serverless Special serverless resource types: functions, APIs, tables, layers, and applications Supports anything AWS CloudFormation supports Open specification (Apache 2.0) https://aws.amazon.com/serverless/sam © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AWS SAM template AWSTemplateFormatVersion: ‘2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: GetProductsFunction: Type: AWS::Serverless::Function Properties: Handler: index.getProducts Runtime: nodejs8.10 CodeUri: src/ Policies: - DynamoDBReadPolicy: TableName: !Ref ProductTable Events: GetResource: Type: Api Properties: Path: /products/{productId} Method: get ProductTable: Type: AWS::Serverless::SimpleTable © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Just 20 lines to create: • Lambda function • IAM role • API Gateway • DynamoDB table

AWS SAM Command Line Interface (AWS SAM CLI) CLI tool for local development, debugging, testing, deploying, and monitoring of serverless applications Supports API Gateway “proxy-style” and Lambda service API testing Response object and function logs available on your local machine Uses open source docker-lambda images to mimic Lambda’s execution environment such as timeout, memory limits, runtimes Can tail production logs from CloudWatch Logs Can help you build in native dependencies https://aws.amazon.com/serverless/sam © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

New with SAM CLI Completely improved deploy experience! • Interactive mode • Config file for deployed stacks • ChangeSet visibility • Stack events streamed © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

FIN/ACK • We are constantly working to improve our services for you • • Tell us your pains! Lot’s of exciting customer stories happening • • Please give me feedback! Don’t take just my word for it: https://www.datadoghq.com/state-ofserverless/ There’s lots of places where we’re working to make serverless better for you. Keep your eyes open! • Check the ICYMI blog posts on the AWS Compute blog! © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. https://rebrand.ly/PhillyDevOps

What happened in Serverless in 2019

Slide 1

Slide 2

Slide 3

Slide 4

Slide 5

Slide 6

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Slide 17

Slide 18

Slide 19

Slide 20

Slide 21

Slide 22

Slide 23

Slide 24

Slide 25

Slide 26

Slide 27

Slide 28

Slide 29

Slide 30

Slide 31

Slide 32

Slide 33

Slide 34

Slide 35

Slide 36

Slide 37

Slide 38

Slide 39

Slide 40

Slide 41

Slide 42

Slide 43

Slide 44

Slide 45

Slide 46

Slide 47

Slide 48

Slide 49

Slide 50

Slide 51

Slide 52

Slide 53

Slide 54

Slide 55

Slide 56

Slide 57

Slide 58

Slide 59

Slide 60

Slide 61

Slide 62

Slide 63

Slide 64

Slide 65

Slide 66

Slide 67

Slide 68

Slide 69

Slide 70

Slide 71

Slide 72

Slide 73

Slide 74

Slide 75