Integration Testing with TigerGraph
- Blog >
- Integration Testing with TigerGraph
CI/CD with Docker, and GitHub Actions for a TigerGraph-backed Gradle application.
This blog was originally posted on Medium CodeX
Integration testing helps us verify that our applications work as expected when interacting with third-party software as they would in production. When integration testing involves the inclusion of a database, then this brings in its own array of configuration and set up.
Giraffle is a popular Gradle plugin used to deploy schema creation, loading jobs, and queries that have applications in CI/CD. There already exist articles on how to use Giraffle for deployment, but in the following, we cover a simple and free way of running integration testing using localized TigerGraph Docker containers within GitHub Actions.
What will be covered:
- Creating a simple Gradle project via the CLI
- Setting up a basic Docker Compose script for TigerGraph
- Using the Avast Docker Compose plugin to start TigerGraph during a Gradle verification task
- Running GSQL scripts on start-up before tests begin running
- Using GitHub Actions to run the integration tests as part of CI/CD
The Docker image used is the one created and discussed in this article, while the Gradle plugin used to coordinate spinning up the container is the Avast plugin.
The final code example of this guide can be found on my GitHub.
Creating a Simple Gradle Project
In the directory in which you want to create the project, run the following command from the command line. In this example, we will be using JDK 11 and Gradle 6.7.1.
gradle init \
--type java-application \
--test-framework junit-jupiter \
--dsl groovy
You will be prompted for the project name and the source package — both of which are up to you.
The resulting project will be a simple Java application with JUnit 5 as the testing framework and Groovy as the DSL for build.gradle
. A class called App
and a test class called AppTest
will be generated under src/main
and src/test
respectively.
Starting TigerGraph in a Gradle Task
First, let’s create a simple Docker Compose script that will create our Docker instance. The TigerGraph image used allows us to run GSQL scripts on startup. Let’s place the following script under src/test/resources/docker/tigergraph.yml
.
version: '3'
services:
tigergraph:
image: dbakereffendi/tigergraph:3.0.5
ports:
- "14022:22"
- "9000:9000"
- "14240:14240"
ulimits:
nofile:
soft: 1000000
hard: 1000000
Second, let’s include the Avast plugin in our build.gradle
. At the time of writing the latest version is 0.14.0. This plugin will run our TigerGraph Docker Compose script during our integration testing.
plugins {
id "com.avast.gradle.docker-compose" version "0.14.0"
}
Now let’s create a verification task that will run the integration tests in our build.gradle
. In this example, we will separate our JUnit unit tests from the integration tests by including IntTest
in the class name.
test {
useJUnitPlatform()
exclude "**/*IntTest*"
testLogging { events "FAILED", "SKIPPED" }
}task intTest(type: Test) {
useJUnitPlatform()
description = "Execute integration tests."
group = "verification"
include "**/*IntTest*"
testLogging { events "FAILED", "SKIPPED" }
doFirst { dockerCompose.exposeAsEnvironment(intTest) }
}
The Avast plugin then needs to be told where our Docker Compose script lies and which Gradle task it should be associated with.
dockerCompose {
tigerGraphSetup {
useComposeFiles = ["src/test/resources/docker/tigergraph.yml"]
isRequiredBy(project.tasks.intTest)
}
}
At this point, running ./gradlew intTest
will
- Set up our database by executing
docker-compose
on our Docker Compose script - Run all tests with
IntTest
in the class name - Tear down the container and its associated volumes
Creating a Schema and Installing Queries Before Tests
To configure our test TigerGraph instance to resemble the schema and queries as they are in production, we will need to run GSQL scripts before the tests are executed.
The queries will be executed in alphabetical order which is why it is convenient to simply prefix the scripts with the order in which they should be executed.
Creating Our GSQL Scripts
Let us start by creating our first GSQL script and placing it under src/test/resources/config/1-schema.gsql
. We will simply use the schema and queries as they are in GSQL 101.
DROP ALL CREATE VERTEX person ( PRIMARY_ID name STRING, name STRING, age INT, gender STRING ) CREATE UNDIRECTED EDGE friendship ( FROM person, TO person ) CREATE GRAPH social (person, friendship)
We can then create a separate script for our queries and name it src/test/resources/config/2-queries.gsql
.
USE GRAPH social CREATE QUERY hello(VERTEX p) FOR GRAPH social { Start = {p}; Result = SELECT tgt FROM Start:s-(friendship:e) ->person:tgt; PRINT Result; } CREATE QUERY status() FOR GRAPH social { INT status = 0; PRINT status; } INSTALL QUERY ALL
Enabling a Health Check
The status
query is arbitrary but should be our last query as it will be used by our Docker health check to indicate when all of the queries have been installed successfully. This works because the TigerGraph queries are installed sequentially.
The Avast plugin will only run the tests once the container is healthy. Let’s adjust our Docker Compose script to include a health check and bind our scripts to /docker-entrypoint-initdb.d
.
version: '3' services: tigergraph: image: dbakereffendi/tigergraph:3.0.5 ports: - "14022:22" - "9000:9000" - "14240:14240" volumes: - ../config/1-schema.gsql:/docker-entrypoint-initdb.d/1-schema.gsql - ../config/2-queries.gsql:/docker-entrypoint-initdb.d/2-queries.gsql healthcheck: test: ["CMD-SHELL", "curl --fail http://localhost:9000/query/social/status || exit 1"] interval: 30s timeout: 10s retries: 5 ulimits: nofile: soft: 1000000 hard: 1000000
Now running ./gradle intTest
will additionally set up a graph schema and install the queries before running our integration tests. This can be extended further to include and run loading jobs.
Creating an Integration Test
Let’s create a test class that we can run during integration testing that will call our TigerGraph instance. Java 11 gives us some decent tools for making HTTP requests which is how we will communicate with the database.
Loading Some Test Data
Our database needs some dummy data to test on. We can create a simple JSON payload in which we can insert the data via TigerGraph’s built-in endpoints. Create a file src/test/resources/data/payload.json
with the following content.
{ "vertices": { "person": { "Dave": { "name": { "value": "Dave" }, "age": { "value": 23 }, "gender": { "value": "Male" } }, "Fred": { "name": { "value": "Fred" }, "age": { "value": 63 }, "gender": { "value": "Male" } } } }, "edges": { "person": { "Dave": { "friendship": { "person": { "Fred": {} } } } } } }
This is appropriate for such a small example, but for larger datasets, another way that test data can be loaded is via a loading job that ingests a CSV file.
Setting Up Our Test Environment
Now create a test class called TigerAppIntTest
next to AppTest
. As a reminder, we will use the fact that IntTest
is in the class name to differentiate this as an integration test.
We will start by inserting our test data in the database during a @BeforeAll
method and clear the database at the end of our tests during a @AfterAll
method.
import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.nio.file.Path; import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertTrue; public class TigerAppIntTest { private static final String DB_ENDPOINT = "http://localhost:9000"; @BeforeAll static void setUpAll() { var client = HttpClient.newHttpClient(); var payload = TigerAppIntTest.class.getClassLoader().getResource("data/payload.json"); HttpRequest request; HttpResponse response; try { request = HttpRequest.newBuilder(URI.create(DB_ENDPOINT + "/graph/social")) .header("accept", "application/json") .POST(HttpRequest.BodyPublishers.ofFile(Path.of(payload.getPath()))) .build(); response = client.send(request, HttpResponse.BodyHandlers.ofString()); // Test the response assertNotNull(response.body()); assertTrue(response.body().contains("\"results\":[{\"accepted_vertices\":2,\"accepted_edges\":1}]")); } catch (Exception e) { System.out.println("Error: " + e.getMessage()); e.printStackTrace(); } } @AfterAll static void tearDownAll() { var client = HttpClient.newHttpClient(); HttpRequest request; try { request = HttpRequest.newBuilder(URI.create(DB_ENDPOINT + "/graph/social/vertices/person")) .header("accept", "application/json") .DELETE() .build(); client.send(request, HttpResponse.BodyHandlers.ofString()); } catch (Exception e) { System.out.println("Error: " + e.getMessage()); e.printStackTrace(); } } }
Testing a Query
Now that our test environment is set up we can test the hello
query. The following test does this.
@Test void helloQueryTest() { var client = HttpClient.newHttpClient(); HttpRequest request; HttpResponse response; try { request = HttpRequest.newBuilder(URI.create(DB_ENDPOINT + "/query/hello?p=Dave")) .header("accept", "application/json") .GET() .build(); response = client.send(request, HttpResponse.BodyHandlers.ofString()); // Test the response assertNotNull(response.body()); assertTrue(response.body().contains("\"results\":[{\"Result\":[{\"v_id\":\"Fred\",\"v_type\":\"person\",\"attributes\":{\"name\":\"Fred\",\"age\":63,\"gender\":\"Male\"}}]}]")); } catch (Exception e) { System.out.println("Error: " + e.getMessage()); e.printStackTrace(); } }
Now running ./gradlew intTest
will, once the database is set up, run the methods in the following order: setUpAll
, helloQueryTest
, and tearDownAll
.
Running TigerGraph Integration Tests using GitHub Actions
An advantage of using my dbakereffendi/tigergraph
images are that they are small with the resource constraints on CI/CD services in mind. That being said, from downloading the image to installing the schema and queries, the integration testing phase can still end up being quite time-consuming.
To start running our tests on GitHub Actions create the following file in the root of your project .github/workflows/ci.yml
.
name: CI on: push: branches: [ main ] pull_request: branches: [ main ] jobs: test: name: Test on JDK 11 and Ubuntu 20.04 runs-on: ubuntu-20.04 steps: - uses: actions/[email protected] - name: Set up JDK 11 uses: actions/[email protected] with: java-version: 11 - name: Grant execute permission for gradlew run: chmod +x gradlew - name: Compile classes run: ./gradlew classes - name: Run unit tests run: ./gradlew test - name: Run TigerGraph integration tests run: ./gradlew intTest
This will run unit and integration tests when you push or have a pull request to themain
branch. Adjust your configuration for the branch name, OS, or Java version of your choice, but what is important here are the Gradle commands.
Conclusion
Using Docker, GitHub Actions, and Gradle we have managed to run integration testing for a TigerGraph-backed application for free with a localized TigerGraph instance.
This can be extended to run loading jobs, deploy to production using Giraffle, or run multiple third-party Docker services during testing using the Avast plugin.
As a reminder, a fully working example of everything discussed in this article can be found on my GitHub repository.