iter8

Iter8

Metrics Driven Experiments

Kubernetes-friendly metrics-driven experiments and safe rollouts.

Use Cases

1. Load testing with SLOs
2. A/B(/n) testing for improving business value with each release of app/ML model
3. Safe rollout for multi-cluster and edge
4. Traffic mirroring experiments

The traffic mirroring use-case is achieved by using Iter8 along with a Kubernetes service mesh or ingress that supports mirroring.

Quick Start

1. Install Iter8

Install Iter8 using Go 1.16+ as follows.

go install github.com/iter8-tools/iter8@latest

You can now run iter8 (from your gopath bin/ directory)

2. Download experiment chart

Download the load-test experiment chart from Iter8 hub as follows.

iter8 hub -e load-test

This creates a local folder called load-test containing the chart.

3. Generate experiment.yaml

Generate the experiment.yaml file which specifies your load test experiment.

cd load-test
iter8 gen exp --set url=https://example.com

4. Run experiment

The iter8 run command reads the experiment.yaml file, runs the specified experiment, and writes the results of the experiment into the result.yaml file. Run the experiment as follows.

iter8 run

4. Assert outcomes

Assert that the experiment completed without any failures and SLOs are satisfied.

iter8 assert -c completed -c nofailure -c slos

5. Generate report

Generate a report of the experiment in HTML or text formats as follows.

HTML Report

iter8 report -o html > report.html
# open report.html with a browser. In MacOS, you can use the command:
# open report.html

The HTML report looks as follows.

Text Report

iter8 report -o text

Congratulations! 🎉 You completed your first Iter8 experiment.

Documentation

Iter8 documentation is available at https://iter8.tools.

Contributing

We are delighted that you want to contribute to Iter8! 💖

As you get started, you are in the best position to give us feedback on areas of our project that we need help with including:

• Problems found during setup of Iter8
• Gaps in our quick start tutorial and other documentation
• Bugs in our test and automation scripts

If anything doesn't make sense, or doesn't work when you run it, please open a bug report and let us know!

See here for information about ways to contribute, Iter8 community meetings, finding an issue, asking for help, pull-request lifecycle, and more.