Test Contexts

This repo contains example test contexts to use on the test bench.

Structure of the test contexts

A test context is basically a collection of shell scripts that the test bench executes inside of virtual machines. The infrastructure_definition.json in combination with additional specification files tells the test bench how the virtual machines need to be defined and how it should calculate the environmental impacts. (The actual format and values used here are work in progress.)

The shell scripts contain a few 'variables' that will be replaced by the test bench. Currently, they are used to specify a hostname that will be replaced with the actual IP, or to define a file path that will be mounted into the vm. These variables are always surrounded by double curly braces. (E.g: ip_a='{{eco_digit_ip_0}}' will be replaced to ip_a='172.16.0.2'. eco_digit_ip_0 corresponds to the id in the infrastructure_definition.json.)

The shell scripts are sorted into different folders. sut will be executed first, in the install phase. us will be executed afterwards in the usage phase. Each shell script also has to follow a naming scheme containing the device id (0, 1, ...). The script can be omitted if the device doesn't need to do anything in this phase.

All files inside a context will be mounted to all virtual machines, so they can be accessed by each device using the {{eco_digit_path}} reference.

Overview of test contexts

• Firewall Check: Minimal test context with small dependencies that runs two machines and determines wether they can reach each other, and also checks that they can't access hosts that they should not reach. (Useful if the host is inside a private network.) It is used mainly to check that the test bench is working correctly.
• Teastore ServerClient: Test context with an actual complex containerized web server GitHub: TeaStore and a small cypress test on a client device that put's an item in a basket and places the order. It has a relatively large footprint on the install phase, as it installs Docker, Node + Cypress and the teastore containers.
• Teastore with Locust: Uses the same TeaStore as above, but instead of Cypress it uses Locust to generate load on the server. This test context uses the same machine as server and client, and is used to validate our calculations with Cloud and Edge devices. (The actual applicationvariant for the testbench is inside the folder validation-teastore_with_locust, which contains instructions and start-locust.sh-script necessary for running this benchmark on the real devices.)