Testing¶
During local development, we may want to test the code in three different scenarios, largely corresponding to real-world deployment scenarios:
- Run code locally (possibly inside a virtual environment)
- Dockerize code and run it inside a docker container
- Run code inside a Kubernetes Pod
Running integration tests with either of these options requires access to a Kubernetes cluster to spawn worker pods.
Note
In CI, we perform static checks in a container (option 2 above), and run tests in a Pod (option 3 above)
Set up local development environment¶
- Make sure you have
make
installed - Run
make install
to install dependencies in your current python environment
You can now run make lint
, make test
etc. - however, many tests will require access
to a kubernetes cluster, as described below.
Set up a local Kubernetes cluster¶
We can set up a kubernetes cluster on the local machine using either minikube or kind
minikube
¶
Install
minikube
by following online instructions, or use:make kubectl-bootstrap
Start
minikube
with:minikube start
(recommended) to run containerized tests in a pod, make sure you have installed docker and run
eval $(shell minikube docker-env)
before building the Docker image(optional) if instead you want to run tests locally, you don’t need
docker
, but you will need to make it possible for your host to be able to talk to the pods on minikube - see section below.Create a namespace and role bindings for testing:
make k8s-deploy
(Optional) Configure network access to the minikube
pods¶
Note
This step may vary depending on your network configuration, and it may be easier to run containerized tests in-cluster instead
On Linux:
sudo ip route add 172.17.0.0/16 via $(minikube ip)
On OS X:
sudo route -n add -net 172.17.0.0/16 $(minikube ip)
If you get an error message like the following:
RTNETLINK answers: File exists
it most likely means you have docker running on your host using the same
IP range minikube is using. You can fix this by editing your
/etc/docker/daemon.json
file to add the following:
{
"bip": "172.19.1.1/16"
}
If some JSON already exists in that file, make sure to just add the
bip
key rather than replace it all. The final file needs to be valid
JSON.
Once edited, restart docker with sudo systemctl restart docker
. It
should come up using a different IP range, and you can run the
sudo ip route add
command again. Note that restarting docker will
restart all your running containers by default.
kind
¶
Make sure you have installed docker
Install
kind
by following online instructions, or use:make kind-bootstrap
Start
kind
:make kind-start
Create a namespace and role bindings for testing:
make k8s-deploy
Remember that local images will need to be pushed to
kind
nodes withmake push-kind
Build a docker image for Testing¶
Ensure you have installed docker
Build docker image:
make build
(if using
kind
) push image to cluster nodes:make push-kind
Run tests locally¶
Check code for formatting errors:
make lint
(Optional) run
kubectl config use-context <context>
, wherecontext
is eitherkind-kind
orminikube
Run tests with
make test
Run tests in a container¶
Any make command, e.g. make lint
, can be executed in the pre-built container using:
make docker-make COMMAND=lint
Note
By default, local code is mounted in the docker container, so you don’t need to rebuild the image to see local changes to your code or tests.
Note
Tests requiring cluster access will not run without further setup, run them in a Pod insted - see below
Run tests in a pod¶
Similar to running tests in a docker container, simply run:
make k8s-make COMMAND=test