KCP Tests

This repository holds the kcp tests that tests against the publicly available interfaces (APIs, code, CLIs)

Prerequisites

• Git installed. See Installing Git

• Golang installed. See Installing Golang, the newer the better ( Ensure you install Golang from a binary release found here, not with a package manager such as dnf ）

• golint installed. See Installing golint

• Install kubelogin and KCP kubectl plugin ( Ensure your installed KCP kubectl plugin version is the same with the kcp service version )

  • If you would like to download the suitable version packages directly go to kcp release assets, download the zip file suitable for your OS, extract and copy the binaries to a place which is in your PATH and once copied run kubectl kcp --version to verify whether your client matches with the server
  • If you would like to build the plugin by your self you could follow the steps like below:

  $ git clone https://github.com/kcp-dev/kcp.git
  Check the same version with your kcp server in https://github.com/kcp-dev/kcp/tags and figure out its commit ID to checkout.
  $ git checkout <commit ID>
  $ make install WHAT=./cmd/kubectl-kcp && make install WHAT=./cmd/kubectl-workspaces && make install WHAT=./cmd/kubectl-ws

• Have the environment variable KUBECONFIG set pointing to your kcp service

  • If your KUBECONFIG has multiple contexts please specific target test environment by using environment variable E2E_TEST_CONTEXT ( In addition, if the environment variable E2E_TEST_CONTEXT doesn't exist, it will try to use kcp-stable-root context, if kcp-stable-root doesn't exist either, it'll use the current-context in KUBECONFIG instead).

    $ export E2E_TEST_CONTEXT="<specific test context>"

• Log in to the kcp service via SSO (Single Sign On)

  $ kubectl oidc-login get-token --oidc-issuer-url=<oidc issuer url> --oidc-client-id=<oidc client ID> --oidc-redirect-url-hostname=127.0.0.1

  • For kcp services deploy with redhat sso, there's another login method called login with offline token that could be used, you could get an offline token ( How to generate an offline token ) and log in to the kcp service by executing the following commands with out having to input credentials into the browser.

    $ export OFFLINE_TOKEN=<YOUR_OFFLINE_TOKEN>
    $ make login
    ./hack/login_with_offline_token.sh
    INFO: Log in to kcp service with "OFFLINE_TOKEN" succeed

• If you would like to test BYO cases please make sure the PCLUSTER_KUBECONFIG environment variable is exported otherwise else all the tests related to BYO will be skipped.

Contribution

Below are the general steps for submitting a PR to main branch. First, you should Fork this repo to your own Github account.

$ git remote add <Your Name> [email protected]:<Your Github Account>/kcp-tests.git
$ git pull origin main
$ git checkout -b <Branch Name>
$ git add xxx
$ git diff main --name-only |grep ".go$"| grep -v "bindata.go$" | xargs -n1 golint
  Please fix all golint error
$ git diff main --name-only |grep ".go$"| grep -v "bindata.go$" | xargs gofmt -s -l
  Please fix all gofmt error, running 'gofmt -s -d [file_path]' or autocorrect with 'gofmt -s -w [file_path]'
$ git add xxx
$ make build
$ ./bin/kcp-tests run all --dry-run |grep <Test Case Name>|./bin/kcp-tests run -f -
$ git commit -m "xxx"
$ git push <Your Name> <Branch Name>:<Branch Name>

And then there will be a prompt in your Github repo console to open a PR, click it to do so.

Include new test folder

If you create a new folder for your test cases, add the path to the include.go

Create go-bindata for new YAML files

If you have some new YAML files used in your code, you have to generate the bindata first. Run make update to update the bindata. For example, you can see the bindata has been updated after running the make update as follows:

$ git status
	modified:   test/extended/testdata/bindata.go
	new file:   test/extended/testdata/kcp/xxxx.yaml

Compile the executable binary

Note that we use the go module for package management, the previous go path is deprecated.

$ git clone [email protected]:kcp-dev/kcp-tests.git
$ cd kcp-tests/
$ make build
mkdir -p "bin"
export GO111MODULE="on" && export GOFLAGS="" && go build  -ldflags="-s -w" -mod=mod -o "bin" "./cmd/kcp-tests"
$ ls -hl ./bin/kcp-tests 
-rwxrwxr-x. 1 cloud-user cloud-user 106M Sep 13 14:41 ./bin/kcp-tests

Run the automation test case

The binary finds the test case via searching for the test case title. It searches the test case titles by RE (Regular Expression). So, you can filter your test cases by using grep.

Run automation test cases related to an area

If I want to run all workspaces test cases, and all of them contain the area/workspaces key word, I can use the grep "area/workspaces" to filter them, as follows:

$ ./bin/kcp-tests run all --dry-run | grep "area/workspaces" | ./bin/kcp-tests run -f -
"[area/workspaces] Author:pewang-Medium-[Smoke] Multi levels workspaces lifecycle should work [Suite:kcp/smoke/parallel/minimal]"
"[area/workspaces] Author:zxiao-Medium-[Serial] I can create context for a specific workspace and use it [Suite:kcp/smoke/serial]"
...

You can save the above output to a file and run it:

$ ./bin/extended-platform-tests run -f <your file path/name>

Run all smoke automation test cases

If you want to run all smoke test cases which has the [Smoke] label, you can do:

$ ./bin/kcp-tests run smoke

Run a single automation test case

If you want to run a single test case, such as g.It("Author:pewang-Medium-[Smoke] Multi levels workspaces lifecycle should work", you can do:

$ ./bin/kcp-tests run all --dry-run|grep "Multi levels workspaces lifecycle should work"|./bin/kcp-tests run --junit-dir=./ -f -

Debugging

Keep generated temporary workspaces

Sometime, we want to keep the generated workspaces for debugging, we could just set export DELETE_WORKSPACE=false, then these temporary workspaces will be kept.

...
Dec 18 09:39:33.448: INFO: Running AfterSuite actions on all nodes
...
1 pass, 0 skip (1m50s)
$ kubectl get ws
NAME                        TYPE        PHASE   URL
e2e-test-kcp-syncer-s6ktv   universal   Ready   https://<kcp-test-env-domain>/clusters/root:users:rp:pv:rh-sso-xxxx:e2e-test-kcp-syncer-s6ktv
e2e-test-kcp-syncer-a5spq   universal   Ready   https://<kcp-test-env-domain>/clusters/root:users:rp:pv:rh-sso-xxxx:e2e-test-kcp-syncer-a5spq
...