e2e

In Memory End-to-End tests

This directory contains end-to-end tests that do not require Kubernetes, and persist configuration in memory.

Note: All commands should be run from the root directory of the Gloo repository

Background

This is the most common type of end-to-end test, since it is the quickest to set up and easiest to debug. Additionally, since Gloo Edge may be run using various backing stores, these tests provide a single space to validate the translation of Gloo resources into Envoy resources, independent of where Gloo Edge is deployed. As a result, these test do not rely on Kubernetes, so if there is any Kubernetes behavior that needs to be tested, write a kubernetes end-to-end test instead.

How do the tests work?

1. Run the Gloo controllers in goroutines
2. Run Envoy either using a binary or docker container
3. Apply Gloo resources using in-memory resource clients
4. Execute requests against the Envoy proxy and confirm the expected response. This validates that the Gloo resources have been picked up by the controllers, were been translated correctly into Envoy configuration, the configuration was sent to the Envoy proxy, and the proxy behaves appropriately.

Example Test

We have an example test which outlines how these tests work. It also provides some examples for basic testing operations. If you are writing a new e2e test, we recommend that you review the example test first.

This was introduced in a pull request which includes other useful details about e2e test considerations.

CI

These tests are run by build-bot in Google Cloud as part of our CI pipeline.

If a test fails, you can retry it using the build-bot comment directives. If you do this, please make sure to include a link to the failed logs for debugging purposes.

Local Development

Setup

For these tests to run, we require Envoy be built in a docker container.

Refer to the Envoyinit README for build instructions.

Run Tests

The run-tests make target runs ginkgo with a set of useful flags. The following environment variables can be configured for this target:

Example:

ENVOY_IMAGE_TAG=solo-test-image TEST_PKG=./test/e2e/... make run-tests

Debugging Tests

Use WAIT_ON_FAIL

When Ginkgo encounters a test failure it will attempt to clean up relevant resources, which includes stopping the running instance of Envoy, preventing the developer from inspecting the state of the Envoy instance for useful clues.

To avoid this clean up, run the test(s) with WAIT_ON_FAIL=1. When the test fails, it will halt execution, allowing you to inspect the state of the Envoy instance.

Once halted, use docker ps to determine the admin port for the Envoy instance, and follow the recommendations for debugging Envoy, specifically the parts around interacting with the Administration interface.

Use INVALID_TEST_REQS

Certain test require environmental conditions to be true for them to succeed. For example, certain tests will only run on a Linux machine.

By setting INVALID_TEST_REQS=skip, you can run all tests locally, and any tests which will not run in your local environment will be skipped. The default behavior is that they fail.

Additional Notes

Notes on EC2 tests

Note: these instructions are out of date, and require updating

• set up your ec2 instance
  • download a simple echo app
  • make the app executable
  • run it in the background

wget https://mitch-solo-public.s3.amazonaws.com/echoapp2
chmod +x echoapp2
sudo ./echoapp2 --port 80 &

Notes on AWS Lambda Tests (test/e2e/aws_test.go)

In addition to the configuration steps provided above, you will need to do the following to run the AWS Lambda Tests locally:

1. Obtain an AWS IAM User account that is part of the Solo.io organization
2. Create an AWS access key
   • Sign into the AWS console with the account created during step 1
   • Hover over your username at the top right of the page. Click on "My Security Credentials"
   • In the section titled "AWS IAM credentials", click "Create access key" to create an acess key
   • Save the Access key ID and the associated secret key
3. Install AWS CLI v2
   • You can check whether you have AWS CLI installed by running aws --version
   • Installation guides for various operating systems can be found here
4. Configure AWS credentials on your machine
   • You can do this by running aws configure
   • You will be asked to provide your Access Key ID and Secret Key from step 2, as well as the default region name and default output format
     • It is critical that you set the default region to us-east-1
   • This will create a credentials file at ~/.aws/credentials on Linux or macOS, or at C:\Users\USERNAME\.aws\credentials on Windows. The tests read this file in order to interact with lambdas that have been created in the Solo.io organization