Skip to content
Zarf

Running Tests

Currently, we primarily test Zarf through a series of end-to-end tests. These tests are called in the test-*.yml workflows and undergo automatic execution against several K8s distros whenever a pull request is created or updated.

In addition, Zarf implements unit tests for specific functions where edge cases prove difficult to cover through end-to-end testing alone. Unit tests follow standard Go convention and are *_test.go files.

To run the end-to-end tests locally, you must meet the same prerequisites as those required for building and running Zarf, which include:

  1. GoLang >= 1.21.x
  2. Make
  3. Any clean K8s cluster (local or remote) or Linux with root if you want to use the Zarf-installed K3s cluster

There are several ways to run tests depending on your specific situation, such as:

Terminal window
# Note: You can prepend CI=true to these commands to force the --no-progress flag like CI does


# The default way, from the root directory of the repo. Will run all of the tests against your chosen k8s distro. Will automatically build any binary dependencies that don't already exist.
make test-e2e ARCH="[amd64|arm64]"


# To test against a Zarf-created cluster (on Linux with sudo)
APPLIANCE_MODE=true make test-e2e ARCH="[amd64|arm64]"


# If you already have everything build, you can run this inside this folder. This lets you customize the test run.
go test ./src/test/... -v -failfast -count=1


# Let's say you only want to run one test. You would run:
go test ./src/test/... -v -failfast -run TestFooBarBaz -count=1

Depends on:

  • A locally built Zarf binary.
  • A fresh cluster.
Terminal window
# The default way, from the root directory of the repo. This will automatically build any Zarf related resources if they don't already exist (i.e. binary, init-package, example packages):
make test-upgrade


# or


# If you are in the root folder of the repository and already have everything built (i.e., the binary, the init-package and the flux-test example package):
go test ./src/test/upgrade/...

Depends on:

  • A locally built Zarf binary.
  • A fresh cluster.

Note: For this test case, we deploy an ‘external’ Git server and container registry as pods running within the k8s cluster. These are still considered ‘external’ servers since they already existed inside the k8s cluster before zarf init command is executed

Terminal window
# The default way, from the root directory of the repo. This will automatically build any Zarf related resources if they don't already exist (i.e. binary, init-package, example packages):
make test-external


# or


# If you are in the root folder of the repository and already have everything built (i.e., the binary, the init-package and the flux-test example package):
go test ./src/test/external/... -v

When adding new tests, there are several requirements that must be followed, including:

  1. Tests cannot be run in parallel as they utilize the same K8s cluster.
  2. Each test should begin with the entries below for standardization and test setup/teardown:
func TestFooBarBaz(t *testing.T) {
    t.Log("E2E: Enter useful description here")
    e2e.setup(t)
    defer e2e.teardown(t)


    ...
}

The end-to-end tests are run sequentially and the naming convention is set intentionally:

  • 00-19 tests run prior to zarf init (cluster not initialized).
  • 20 is reserved for zarf init.
  • 21 is reserved for logging tests so they can be removed first (they take the most resources in the cluster).
  • 22 is reserved for tests required the git-server, which is removed at the end of the test.
  • 23-98 are for the remaining tests that only require a basic Zarf cluster without logging or the git-server.
  • 99 is reserved for the zarf destroy and YOLO Mode test.

There are several ways to run tests depending on your specific situation, such as:

Terminal window
# The default way, from the root directory of the repo. Will run all of the unit tests that are currently defined.
make test-unit


# If you already have everything built, you can run this inside this folder. This lets you customize the test run.
go test ./src/pkg/... -v


# Let's say you only want to run one test. You would run:
go test ./src/pkg/... -v -run TestFooBarBaz

When adding new unit tests, please ensure that the following requirements are met:

  1. The test must focus on a true unit, such as a single function or file.
  2. The code being tested must have a clearly defined interface, such as a public specification.
  3. The code being tested should be located within the src/pkg.

If all these requirements are met, then a unit test would be appropriate. If not, please consider writing an end-to-end test instead or modify your approach to meet these requirements.

To create a unit test, search for or create a file that ends with _test.go in the package of the file that requires testing, such as auth.go -> auth_test.go. Import the testing library and create test functions as necessary. In case you need to mock something out, determine the most suitable approach and if the mock can be used in multiple tests, consider placing it in ./src/test/mocks/. This will help enhance the efficiency and organization of the unit tests.