A presentation at Search Meetup Munich in in Munich, Germany by Alexander Reelsen
Running a Serverless Lucene Reverse Geocoder Alexander Reelsen alex@elastic.co @spinscale
Agenda ‣ What is serverless? ‣ Searching for Locations ‣ Demo ‣ Searching with Lucene using a binary
Serverless?
Serverless? ‣ FaaS (Function as a Service) ‣ Execution Environment as a Service ‣ Payment model: Pay per code runtime ‣ Not running? No bill! ‣ Configure memory size (also changes CPU power) ‣ Maximum function execution time ‣ Provider takes care of scaling functions
Providers? ‣ AWS Lambda, GCP Cloud Functions, Azure Cloud Functions, Cloudflare, IBM OpenWhisk ‣ Faastruby, Binaris, Spotinst ‣ K8s: KNative, Fission, Kubeless, Nuclio, OpenFaas ‣ Docker: Fn, OpenFaas
Java? ‣ Not too well suited for short lived tasks ‣ JVM startup time ‣ JIT compiler ‣ Dependency initialisation ‣ Application initialisation
Location search
Reverse Geocoder ‣ Input: Latitude, Longitude ‣ Output: City (a readable representation)
Search across points ‣ Each city gets indexed with a lat/lon pair ‣ Search for the next point to the supplied one ‣ Problem: Neighbours!
Point based search: Near neighbours
Point based search: Near neighbours
Search across shapes ‣ Each city gets indexed with a lat/lon pair ‣ Certain cities get indexed as a geoshape ‣ Search twice: ‣ Lat/Lon within any shape ‣ Lat/Lon nearby any point
Geo and Lucene: BFF! ‣ LatLonPoint: two points, 4 bytes each ‣ LatLonShape: triangular mesh tesselation
Geo and Lucene: BFF! https://home.apache.org/~mikemccand/geobench.html
Geo and Lucene: BFF! https://home.apache.org/~mikemccand/geobench.html
Serverless Lucene ‣ Local execution, index part of the package ‣ Offline index creation ‣ Packaging index into code ‣ Index needs to be unpacked, using Lucene via classpath resources is tricky
Demo
Summary ‣ Works! ‣ Problem: Data quality, getting accurate shape data ‣ Problem: First invocation (up to 2s) ‣ JVM startup ‣ Lucene index opening
Faster startup & runtime
Enter GraalVM! ‣ A new compiler, supporting HotSpot and AOT compilation ‣ Graal compiler part of Java9 (experimental!) ‣ Graal JIT compiler part of Java10 (Linux 64bit only) ‣ Project Metropolis: Java-on-Java Hotspot implementation ‣ Truffle: Interpreter to implement other languages on top of graal (jruby replacement)
Enter GraalVM! ‣ AOT static compilation + SubstrateVM = executable binaries of java apps ‣ Using SubstrateVM ‣ Reflection!
Discussion … ask all the things!
Links ‣ https://serverless.com/framework/docs/ ‣ https://www.openfaas.com ‣ https://cloud.google.com/knative/ ‣ https://kubeless.io/ ‣ https://fission.io/ ‣ http://fnproject.io/ ‣ https://nuclio.io ‣ https://openwhisk.incubator.apache.org/ ‣ https://www.graalvm.org ‣ https://openjdk.java.net/projects/metropolis/ ‣ https://github.com/oracle/graal/tree/master/substratevm ‣ https://en.wikipedia.org/wiki/Reverse_geocoding
This presentation talks about the possibility of running a reverse geocoder based on lucene in a serverless environment. The talk features the serverless framework, lucene geo capabilities using points and shapes, and last but not least mentions graalvm to reduce start up times.
for free. You
can too.