Let’s dive into Terraform provider creation

A presentation at Paris Open Source Data Infrastructure in March 2023 in Paris, France by Horacio Gonzalez

Slide 1

Slide 1

Let’s dive into Terraform provider creation Aurélie Vache - Horacio Gonzalez 2023-03-06

Slide 2

Slide 2

Aurélie Vache @aurelievache DevRel at Conferences organizer Tech visual articles & books Sketchnoter … & ❤ Retrogaming https://www.youtube.com/c/AurelieVache https://dev.to/aurelievache/ Les Productions de MOA

Slide 3

Slide 3

Horacio Gonzalez @LostInBrittany Spaniard lost in Brittany, developer, dreamer and all-around geek

Slide 4

Slide 4

OVHcloud: A global leader Web Cloud & Telcom 30 Data Centers in 12 locations 1 Million+ Servers produced since 1999 Private Cloud 34 Points of Presence on a 20 TBPS Bandwidth Network 1.5 Million Customers across 132 countries Public Cloud 2200 Employees worldwide 3.8 Million Websites hosting Storage 115K Private Cloud VMS running 1.5 Billion Euros Invested since 2016 300K Public Cloud instances running P.U.E. 1.09 Energy efficiency indicator 380K Physical Servers running in our data centers 20+ Years in Business Disrupting since 1999 Network & Security

Slide 5

Slide 5

Warning Gophers, gophers everywhere!

Slide 6

Slide 6

Terraform De facto standard for IaC

Slide 7

Slide 7

Infrastructure as Code (IaC)

Slide 8

Slide 8

Terraform becoming the de facto standard

Slide 9

Slide 9

HashiCorp Terraform

Slide 10

Slide 10

Modular architecture: providers

Slide 11

Slide 11

Configuration packages: modules

Slide 12

Slide 12

Terraform registry

Slide 13

Slide 13

Writing Terraform providers Defining new Terraform resources

Slide 14

Slide 14

Provider SDK https://developer.hashicorp.com/terraform/plugin/framework

Slide 15

Slide 15

Installing Terraform providers $ terraform init Initializing provider plugins… - Finding terraform.local/local/myprovider versions matching “0.0.1”… - Installing terraform.local/local/myprovider v0.0.1…

Slide 16

Slide 16

Installing providers from registry $ vi provider.tf terraform { required_providers { thenamespace = { source = “thenamespace/myprovider” } } } If your provider is on the official registry at https://registry.terraform.io/providers/thenamespace/myprovider

Slide 17

Slide 17

Installing providers locally $ go build -o terraform-provider-myprovider $ mkdir -p ~/.terraform.d/plugins/terraform.local/local/myprovider/0.0.1/darwin_amd64 $ mv terraform-provider-myprovider ~/.terraform.d/plugins/terraform.local/local/myprovider/0.0.1/darwin_amd64

Slide 18

Slide 18

Installing providers locally $ vi provider.tf terraform { required_providers { thenamespace = { source = “terraform.local/local/myprovider” version = “0.0.1” } } }

Slide 19

Slide 19

Do I need a Terraform provider? If you have an API, you should have a Terraform provider

Slide 20

Slide 20

Letʼs create a provider! Step by step

Slide 21

Slide 21

What do we want? ● In a simple and easy Terraform provider ● Handle cute Gophers ● In Go, because providers are made in Go 😁

Slide 22

Slide 22

Everything begins with an API https://github.com/scraly/gophers-api

Slide 23

Slide 23

For the demos we will use Gitpod Automated, ephemeral developer environments in the web

Slide 24

Slide 24

Everything begins with an API $ task swagger.serve task: [swagger.serve] swagger serve -F swagger ./pkg/swagger/swagger.yml —no-open 2022/10/31 20:16:51 serving docs at http://localhost:38457/docs

Slide 25

Slide 25

Everything begins with an API

Slide 26

Slide 26

Everything begins with an API $ task run task: [run] GOFLAGS=-mod=mod go run internal/main.go 2022/10/30 20:22:05 Serving gophers API at http://[::]:8080 $ curl localhost:8080/gophers [{“name”:”5th-element”,”displayname”:”5th Element.png”,”url”:”https://raw.githubusercontent.com/scraly/gophers/main/5th-ele ment.png”}]

Slide 27

Slide 27

Gophers deserve to be seen https://github.com/LostInBrittany/gophers-api-watcher

Slide 28

Slide 28

Everything begins with an API $ curl -X POST localhost:8080/gopher -H “Content-Type: application/json” -d \ ‘{“name”:”yoda-gopher”,”displayname”:”Yodada Gopher”,”url”:”https://raw.githubusercontent.com/scraly/gophers/main/yoda-gopher. png”}’ $ curl -X DELETE localhost:8080/gopher?name=5th-element $ curl -X PUT localhost:8080/gopher \ -H “Content-Type: application/json” -d \ ‘{“name”:”yoda-gopher”,”displayname”:”Yoda Gopher”,”url”:”https://raw.githubusercontent.com/scraly/gophers/main/yoda-gopher. png”}’

Slide 29

Slide 29

Let’s create our provider! 1. Create the skeleton of our provider thanks to scaffolding https://github.com/hashicorp/terraform-provider-scaffolding-framework

Slide 30

Slide 30

Let’s create our provider! https://github.com/LostInBrittany/terraform-provider-gophers

Slide 31

Slide 31

Demo time!

Slide 32

Slide 32

Provider > Makefile

Slide 33

Slide 33

Demo time!

Slide 34

Slide 34

Some concepts to introduce…

Slide 35

Slide 35

Customizing provider definition

Slide 36

Slide 36

Test it! $ vi provider.tf terraform { required_providers { gophers = { source = “terraform.local/local/gophers” version = “0.0.1” } } } provider “gophers” { endpoint = “http://myawesomeurl.com” }

Slide 37

Slide 37

Adding datasource: gophers

Slide 38

Slide 38

Adding the schema “Translating” the Swagger into a Go schema

Slide 39

Slide 39

Test it! $ vi gophers_data.tf # List of available gophers data “gophers” “my_gophers” { } output “return_gophers” { value = length(data.gophers.my_gophers.gophers) >= 1 }

Slide 40

Slide 40

Adding datasource: gopher

Slide 41

Slide 41

Test it! $ vi gopher_data.tf # Display information about a Gopher data “gophers_gopher” “moultipass” { name = “5th-element” }

Slide 42

Slide 42

Adding resource: gopher

Slide 43

Slide 43

Test it! $ vi gopher_resource.tf resource “gophers_gopher” “x-files” { name = “x-files” displayname = “X Files” url } = “https://raw.githubusercontent.com/scraly/gophers/main/x-files.png”

Slide 44

Slide 44

Testing the provider locally $ go build -o terraform-provider-gophers $ mkdir -p ~/.terraform.d/plugins/terraform.local/local/gophers/0.0.1/darwin_arm64 $ mv terraform-provider-gophers ~/.terraform.d/plugins/terraform.local/local/gophers/0.0.1/darwin_arm64 $ make install

Slide 45

Slide 45

Testing the provider locally $ rm .terraform.lock.hcl && terraform init $ terraform apply

Slide 46

Slide 46

Testing the provider locally $ terraform destroy

Slide 47

Slide 47

Slide 48

Slide 48

OVHcloud Terraform Provider To easily manage OVHcloud products

Slide 49

Slide 49

OVHcloud Terraform Provider https://registry.terraform.io/providers/ovh/ovh/latest/docs

Slide 50

Slide 50

OVHcloud Terraform Provider https://github.com/ovh/terraform-provider-ovh

Slide 51

Slide 51

Best practices But we have learnt with our providers

Slide 52

Slide 52

Doc is not optional $ tfplugindocs generate Generate the doc of your provider. Based on the schema the provider exposes. https://github.com/hashicorp/terraform-plugin-docs

Slide 53

Slide 53

Write useful examples in your doc Examples in your documentation should be: ● Useful ● Up-to-date ● Working Users will copy paste your examples! 😉

Slide 54

Slide 54

And… test your doc! Use the doc preview tool https://registry.terraform.io/tools/doc-preview

Slide 55

Slide 55

Acceptance tests $ make testacc $ make testacc TESTARGS=”-run TestAccDataSourceGopher”

Slide 56

Slide 56

Have the simplest JSON structures

Slide 57

Slide 57

Provider is a reflection of your API client Think about API first design

Slide 58

Slide 58

Use the logs for debugging $ TF_LOG=INFO terraform plan

Slide 59

Slide 59

Set timeouts / retry Timeouts: &schema.ResourceTimeout{ Create: schema. DefaultTimeout (20 * time.Minute), Update: schema. DefaultTimeout (20 * time.Minute), Delete: schema. DefaultTimeout (20 * time.Minute), }, Timeout/retry par resource

Slide 60

Slide 60

Read the code See how other open source providers are written

Slide 61

Slide 61

The “3 P” rule Practice, practice, practice

Slide 62

Slide 62

One more thing… Or two or three

Slide 63

Slide 63

A handy cheat sheet https://github.com/scraly/terraform-cheat-sheet/

Slide 64

Slide 64

Thank you! https://bit.ly/tf-provider-posdi

Slide 65

Slide 65

We ❤ feedbacks https://bit.ly/vote-tf-provider-posdi