iter8ctl

Iter8ctl

Iter8 command line utility for service operators to understand and diagnose their iter8 experiments.

Iter8ctl can be used with iter8-kfserving experiments.

Installation

GO111MODULE=on GOBIN=/usr/local/bin go get github.com/iter8-tools/[email protected]

The above command installs iter8ctl under the /usr/local/bin directory. To install under a different directory, change the value of GOBIN above.

Removal

rm <path-to-install-directory>/iter8ctl

Usage

Example 1

Describe an iter8 Experiment resource object present in your Kubernetes cluster.

kubectl get experiment sklearn-iris-experiment-1 -n kfserving-test -o yaml > experiment.yaml
iter8ctl describe -f experiment.yaml

Example 2

Supply experiment YAML using console input.

kubectl get experiment sklearn-iris-experiment-1 -n kfserving-test -o yaml > experiment.yaml
cat experiment.yaml | iter8ctl describe -f -

Example 3

Periodically fetch an iter8 Experiment resource object present in your Kubernetes cluster and describe it. You can change the frequency by adjusting the sleep interval below.

while clear; do
    kubectl get experiment sklearn-iris-experiment-1 -n kfserving-test -o yaml | iter8ctl describe -f -
    sleep 10.0
done

Sample Output

The following is the output of executing iter8ctl describe -f testdata/experiment10.yaml; the testdata folder is part of the iter8ctl GitHub repo and contains sample experiments used in tests.

$ ./iter8ctl describe -f testdata/experiment10.yaml
****** Overview ******
Experiment name: experiment-1
Experiment namespace: knative-test
Target: knative-test/sample-application
Testing pattern: Canary
Deployment pattern: Progressive

****** Progress Summary ******
Experiment stage: Completed
Number of completed iterations: 8

****** Winner Assessment ******
App versions in this experiment: [sample-application-v1 sample-application-v2]
Winning version: sample-application-v2
Version recommended for promotion: sample-application-v2

****** Objective Assessment ******
+--------------------------+-----------------------+-----------------------+
|        OBJECTIVE         | SAMPLE-APPLICATION-V1 | SAMPLE-APPLICATION-V2 |
+--------------------------+-----------------------+-----------------------+
| mean-latency <= 2000.000 | true                  | true                  |
+--------------------------+-----------------------+-----------------------+
| error-rate <= 0.010      | true                  | true                  |
+--------------------------+-----------------------+-----------------------+

****** Metrics Assessment ******
+-----------------------------+-----------------------+-----------------------+
|           METRIC            | SAMPLE-APPLICATION-V1 | SAMPLE-APPLICATION-V2 |
+-----------------------------+-----------------------+-----------------------+
| request-count               |              1022.565 |               514.445 |
+-----------------------------+-----------------------+-----------------------+
| mean-latency (milliseconds) |                 5.881 |                 4.702 |
+-----------------------------+-----------------------+-----------------------+
| error-rate                  |                 0.000 |                 0.000 |
+-----------------------------+-----------------------+-----------------------+

Contributing

Documentation and code PRs are welcome. When contributing to this repository, please first discuss the change you wish to make using Issues, Discussion, or Slack.