Running a Serverless Lucene Reverse Geocoder

Running a Serverless Lucene Reverse Geocoder Alexander Reelsen alex@elastic.co @spinscale

Agenda ‣ What is serverless? ‣ Searching for Locations ‣ Demo ‣ How to execute java code faster

Serverless?

Serverless? ‣ FaaS (Function as a Service) ‣ Execution Environment as a Service ‣ Payment model: Pay per code runtime ‣ Not running? No bill! ‣ Configure memory size (also changes CPU power) ‣ Maximum function execution time ‣ Provider takes care of scaling functions

Examples ‣ Good: Short lived HTTP requests ‣ Good: Short running jobs ‣ Good: Event streaming & processing ‣ Good: Share nothing web applications ‣ Good: Parallelizable workloads ‣ Bad: Slack bots?

Providers? ‣ AWS Lambda, GCP Cloud Functions, Azure Cloud Functions, Cloudflare, IBM OpenWhisk, Google Cloud Run ‣ Faastruby, Binaris, Spotinst ‣ K8s: KNative, Fission, Kubeless, Nuclio, OpenFaas ‣ Docker: Fn, OpenFaas

Java? ‣ Not too well suited for short lived tasks ‣ JVM startup time ‣ JIT compiler ‣ Dependency initialisation ‣ Application initialisation

Location Tracker: Owntracks + Lambda + Kibana

https://github.com/spinscale/serverless-owntracks-kotlin

Location search

Reverse Geocoder ‣ Input: Latitude, Longitude ‣ Output: readable representation (City)

Search across points ‣ Each city gets indexed with a lat/lon pair ‣ Search for the next point to the supplied one ‣ Problem: Neighbours!

Point based search: Near neighbours

Point based search: Near neighbours

Search across shapes ‣ Each city gets indexed with a lat/lon pair ‣ Certain cities get indexed as a geoshape ‣ Run two searches: ‣ Lat/Lon within any shape ‣ Lat/Lon nearby any point

Geo and Lucene: BFF! ‣ LatLonPoint: two points, 4 bytes each ‣ LatLonShape: triangular mesh tesselation

Geo and Lucene: BFF! https://home.apache.org/~mikemccand/geobench.html

Geo and Lucene: BFF! https://home.apache.org/~mikemccand/geobench.html

Serverless Lucene ‣ Local execution, index part of the package ‣ Offline index creation ‣ Packaging index into code ‣ Index needs to be unpacked, using Lucene via classpath resources is tricky

Demo

Summary ‣ Works! ‣ Problem: Data quality, getting accurate shape data ‣ Problem: First invocation (up to 2s) ‣ JVM startup ‣ Lucene index opening

Faster startup & runtime

Enter GraalVM! ‣ A new compiler, supporting HotSpot and AOT compilation ‣ Graal compiler part of Java9 (experimental!) ‣ Graal JIT compiler part of Java10 (Linux 64bit only) ‣ Project Metropolis: Java-on-Java Hotspot implementation ‣ Truffle: Framework to implement other languages on top of graal (jruby replacement)

Enter GraalVM! ‣ AOT static compilation + SubstrateVM = executable binaries of java apps ‣ Using SubstrateVM ‣ Reflection!

Deployment model

AWS Lambda ‣ Deployment model: Zip archive in S3 bucket ‣ Execution: Download zip archive & execute ‣ Requires regular java start (AWS reduced JVM startup time) ‣ GraalVM can only be used with custom runtime

runtime flow

CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap AWS ENDPOINT CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap GET /2018-06-01/runtime/invocation/next AWS ENDPOINT CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap GET /2018-06-01/runtime/invocation/next AWS ENDPOINT Lambda-Runtime-Aws-Request-Id: 123 { “body”: “{ “foo”: “bar” }”, “requestContext”: “{ … }”, “headers” : { … } } CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap POST /2018-06-01/runtime/invocation/123/response { AWS ENDPOINT “body”: “{ “foo”: “bar” }”, “headers” : { … } } CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap POST /2018-06-01/runtime/invocation/123/response { AWS ENDPOINT “body”: “{ “foo”: “bar” }”, “headers” : { … } } HTTP OK 202 CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap POST /2018-06-01/runtime/invocation/123/error { AWS ENDPOINT “statusCode”: 500, “body” : “…” } CUSTOM RUNTIME

AWS_LAMBDA_RUNTIME_API=localhost:12345 _HANDLER=”my_handler” /bin/bootstrap POST /2018-06-01/runtime/init/error { AWS ENDPOINT “errorMessage” : “…”, “errorType” : “…” } CUSTOM RUNTIME

Google Cloud Run

Google Cloud Run ‣ Serverless done ‘right’? ‣ Docker container as a web application listening in $PORT ‣ Deployment model: docker push && gcloud beta run ‣ Easier to test ‣ Configurable concurrency ‣ Improved billing model due to concurrency

Demo

Summary

Summary ‣ Economics ‣ Billing model ‣ Runtime cost vs. development cost ‣ Break even vs. non serverless model ‣ Development ‣ Vendor lock-in ‣ Deployment model is important ‣ Cloud Run: Run serverless or as container ‣ Scalability strategies, base load container, increased load serverless? ‣ Observability

Discussion … ask all the things!

Links ‣ https://serverless.com/framework/docs/ ‣ https://www.openfaas.com ‣ https://cloud.google.com/knative/ ‣ https://kubeless.io/ ‣ https://fission.io/ ‣ http://fnproject.io/ ‣ https://nuclio.io ‣ https://openwhisk.incubator.apache.org/ ‣ https://www.graalvm.org ‣ https://openjdk.java.net/projects/metropolis/ ‣ https://github.com/oracle/graal/tree/master/substratevm ‣ https://en.wikipedia.org/wiki/Reverse_geocoding