How to make a world-class integration testing library?

How to make a world-class integration testing library?

testcontainers.org

Oleg Šelajev Developer relations @shelajev github.com/shelajev oleg@atomicjar.com

Model: record Rating (String talkId, Integer value) {}

Downloads

Unique IPs

e2e Integration tests Unit tests By Unknown author - U.S. Army Photo, Public Domain, https://commons.wikimedia.org/w/index.php?curid=55124

https://labs.spotify.com/2018/01/11/testing-of-microservices/

Is this the goal?

You’re given tools but they don’t work reliably or are di cult to use to solve problems, you don’t know exist ffi fi slow, joyless, or signi cantly behind the state of the art

—version: ‘2’ services: zookeeper: image: con uentinc/cp-zookeeper:7.0.1 ports: - “32181:32181” environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 kafka: image: con uentinc/cp-kafka:7.0.1 ports: - “29092:29092” depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092 fl fl KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR :1

What does winning look like?

Replacing a hard problem with a hard solution is not success. Nor is replacing an easy problem with an easy solution.

Reliability

Trustworthyness Shifting failures left Errors making sense Sensible defaults Developers are people too!

System checks Checking the system… ✔ Docker version should be at least 1.6.0 ✔ Docker environment should have more than 2GB free disk space ✔ File should be mountable ℹ︎ ✔ A port exposed by a docker container should be accessible

During After

The problem Extra mile Port conflicts Port randomization Hard-coded scenarios Data-driven tests Slow tests Parallelization

Technology radar We think it’s a useful default option for creating a reliable environment for running tests. … Our teams have consistently found this library of programmable, lightweight and disposable containers to make functional tests more reliable. https://www.thoughtworks.com/en-us/radar/languages-and-frameworks/testcontainers

The simplest way JDBC driver magic: spring.datasource.url=jdbc:tc:postgresql:14-alpine: testcontainers/database With a schema initialization: jdbc:tc:mysql:5.7.34: init_mysql.sql databasename?TC_INITSCRIPT=file:src/main/resources/ With additional con g: / databasename?TC_TMPFS=/testtmpfs:rw / / / / / / / fi jdbc:tc:postgresql:9.6.8:

The ecosystem of modules KafkaContainer kafka = new KafkaContainer( DockerImageName.parse(“confluentinc/cp-kafka”).withTag(“5.4.3”) ) public ToxiproxyContainer toxiproxy = new ToxiproxyContainer(DockerImageName.parse(“shopify/toxiproxy:2.1.0”)) .withNetwork(network) .withNetworkAliases(“toxiproxy”); PostgreSQLContainer<?> postgreSQLContainer = new PostgreSQLContainer (DockerImageName.parse(“postgresql:14-alpine”));

< K3sContainer k3s = new K3sContainer( DockerImageName.parse(“rancher/k3s:v1.21.3-k3s1”)) .withLogConsumer(new Slf4jLogConsumer(log));

testcontainers.org

Developer Experience Community Technical excellence

• The technology

The generic API

< GenericContainer<?> redis = new GenericContainer (“redis:6-alpine”) .withExposedPorts(6379);

The generic API

If it’s not in the docs — it doesn’t exist!

The problem with docs (a) people will never read external docs; (b) people will rarely read inline docs/help strings/tooltips; (c) people will most likely only read the information that is shown to them

The IDE is your only friend!

Our clients’ clients love us

Private registries Local API: final PostgreSQLContainer<?> mysql = new PostgreSQLContainer ( DockerImageName.parse(“registry.mycompany.com/mirror/mysql:8.0.24”) .asCompatibleSubstituteFor(“mysql”) ); Con guration: TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX=registry.mycompany.com/mirror/ Programmatic API:

< fi public abstract class ImageNameSubstitutor implements Function<DockerImageName, DockerImageName>

Chaos engineering public Network network = Network.newNetwork();

< public GenericContainer<?> redis = new GenericContainer (DockerImageName.parse(“redis:6-alpine”)) .withExposedPorts(6379) .withNetwork(network);

Chaos engineering public Network network = Network.newNetwork(); public GenericContainer<?> redis = new GenericContainer (DockerImageName.parse(“redis:6-alpine”)) .withExposedPorts(6379) .withNetwork(network);

< public ToxiproxyContainer toxiproxy = new ToxiproxyContainer(DockerImageName.parse(“shopify/toxiproxy:2.1.0”)) .withNetwork(network) .withNetworkAliases(“toxiproxy”);

Chaos engineering final ToxiproxyContainer.ContainerProxy proxy = toxiproxy.getProxy(redis, 6379); proxy.toxics() .latency(“latency”, ToxicDirection.DOWNSTREAM, 1_100) .setJitter(100); • • • • • • bandwidth - Limit a connection to a maximum number of kilobytes per second. latency - Add a delay to all data going through the proxy. The delay is equal to latency +/- jitter. slicer - Slices TCP data up into small bits, optionally adding a delay between each sliced “packet”. slowClose - Delay the TCP socket from closing until delay milliseconds has elapsed. timeout - Stops all data from getting through, and closes the connection after timeout limitData - Closes connection when transmitted data exceeded limit.

Reusable containers Extending the lifecycle of the containers Local API: static KafkaContainer kafka = new KafkaContainer() .withReuse(true); fi Con guration:

Changing local development git clone https://github.com/shelajev/ primes-kafka.git && cd primes-kafka ./mvnw quarkus:dev Clone & Run!