Integration Testing with TigerGraph

Blog >
Integration Testing with TigerGraph

CI/CD with Docker, and GitHub Actions for a TigerGraph-backed Gradle application.

This blog was originally posted on Medium CodeX

Integration testing helps us verify that our applications work as expected when interacting with third-party software as they would in production. When integration testing involves the inclusion of a database, then this brings in its own array of configuration and set up.

Giraffle is a popular Gradle plugin used to deploy schema creation, loading jobs, and queries that have applications in CI/CD. There already exist articles on how to use Giraffle for deployment, but in the following, we cover a simple and free way of running integration testing using localized TigerGraph Docker containers within GitHub Actions.

What will be covered:

Creating a simple Gradle project via the CLI
Setting up a basic Docker Compose script for TigerGraph
Using the Avast Docker Compose plugin to start TigerGraph during a Gradle verification task
Running GSQL scripts on start-up before tests begin running
Using GitHub Actions to run the integration tests as part of CI/CD

The Docker image used is the one created and discussed in this article, while the Gradle plugin used to coordinate spinning up the container is the Avast plugin.

The final code example of this guide can be found on my GitHub.

Creating a Simple Gradle Project

In the directory in which you want to create the project, run the following command from the command line. In this example, we will be using JDK 11 and Gradle 6.7.1.

gradle init \
    --type java-application \
    --test-framework junit-jupiter \
    --dsl groovy

You will be prompted for the project name and the source package — both of which are up to you.

The resulting project will be a simple Java application with JUnit 5 as the testing framework and Groovy as the DSL for build.gradle. A class called App and a test class called AppTest will be generated under src/main and src/test respectively.

Starting TigerGraph in a Gradle Task

First, let’s create a simple Docker Compose script that will create our Docker instance. The TigerGraph image used allows us to run GSQL scripts on startup. Let’s place the following script under src/test/resources/docker/tigergraph.yml.

version: '3'
services:
  tigergraph:
    image: dbakereffendi/tigergraph:3.0.5
    ports:
      - "14022:22"
      - "9000:9000"
      - "14240:14240"
    ulimits:
      nofile:
        soft: 1000000
        hard: 1000000

Second, let’s include the Avast plugin in our build.gradle. At the time of writing the latest version is 0.14.0. This plugin will run our TigerGraph Docker Compose script during our integration testing.

plugins {
id "com.avast.gradle.docker-compose" version "0.14.0"
}

Now let’s create a verification task that will run the integration tests in our build.gradle. In this example, we will separate our JUnit unit tests from the integration tests by including IntTest in the class name.

test {
    useJUnitPlatform()
    exclude "**/*IntTest*"
    testLogging { events "FAILED", "SKIPPED" }
}task intTest(type: Test) {
    useJUnitPlatform()
    description = "Execute integration tests."
    group = "verification"
    include "**/*IntTest*"
    testLogging { events "FAILED", "SKIPPED" }
doFirst { dockerCompose.exposeAsEnvironment(intTest) }
}

The Avast plugin then needs to be told where our Docker Compose script lies and which Gradle task it should be associated with.

dockerCompose {
tigerGraphSetup {
useComposeFiles = ["src/test/resources/docker/tigergraph.yml"]
        isRequiredBy(project.tasks.intTest)
}
}

At this point, running ./gradlew intTest will

Set up our database by executing docker-compose on our Docker Compose script
Run all tests with IntTest in the class name
Tear down the container and its associated volumes

Creating a Schema and Installing Queries Before Tests

To configure our test TigerGraph instance to resemble the schema and queries as they are in production, we will need to run GSQL scripts before the tests are executed.

The queries will be executed in alphabetical order which is why it is convenient to simply prefix the scripts with the order in which they should be executed.

Creating Our GSQL Scripts

Let us start by creating our first GSQL script and placing it under src/test/resources/config/1-schema.gsql. We will simply use the schema and queries as they are in GSQL 101.

DROP ALL

CREATE VERTEX person (
    PRIMARY_ID name STRING,
    name STRING, age INT,
    gender STRING
)

CREATE UNDIRECTED EDGE friendship (
    FROM person,
    TO person
)

CREATE GRAPH social (person, friendship)

We can then create a separate script for our queries and name it src/test/resources/config/2-queries.gsql.

USE GRAPH social

CREATE QUERY hello(VERTEX p) FOR GRAPH social {
  Start = {p};
  Result = SELECT tgt
           FROM Start:s-(friendship:e) ->person:tgt;
  PRINT Result;
}

CREATE QUERY status() FOR GRAPH social {
  INT status = 0;
  PRINT status;
}

INSTALL QUERY ALL

Enabling a Health Check

The status query is arbitrary but should be our last query as it will be used by our Docker health check to indicate when all of the queries have been installed successfully. This works because the TigerGraph queries are installed sequentially.

The Avast plugin will only run the tests once the container is healthy. Let’s adjust our Docker Compose script to include a health check and bind our scripts to /docker-entrypoint-initdb.d.

version: '3'
services:
  tigergraph:
    image: dbakereffendi/tigergraph:3.0.5
    ports:
      - "14022:22"
      - "9000:9000"
      - "14240:14240"
    volumes:
      - ../config/1-schema.gsql:/docker-entrypoint-initdb.d/1-schema.gsql
      - ../config/2-queries.gsql:/docker-entrypoint-initdb.d/2-queries.gsql
    healthcheck:
      test: ["CMD-SHELL", "curl --fail http://localhost:9000/query/social/status || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 5
    ulimits:
      nofile:
        soft: 1000000
        hard: 1000000

Now running ./gradle intTest will additionally set up a graph schema and install the queries before running our integration tests. This can be extended further to include and run loading jobs.

Creating an Integration Test

Let’s create a test class that we can run during integration testing that will call our TigerGraph instance. Java 11 gives us some decent tools for making HTTP requests which is how we will communicate with the database.

Loading Some Test Data

Our database needs some dummy data to test on. We can create a simple JSON payload in which we can insert the data via TigerGraph’s built-in endpoints. Create a file src/test/resources/data/payload.json with the following content.

{
  "vertices": {
    "person": {
      "Dave": {
        "name": { "value": "Dave" },
        "age": { "value": 23 },
        "gender": { "value": "Male" }
      },
      "Fred": {
        "name": { "value": "Fred" },
        "age": { "value": 63 },
        "gender": { "value": "Male" }
      }
    }
  },
  "edges": {
    "person": {
      "Dave": {
        "friendship": {
          "person": {
            "Fred": {}
          }
        }
      }
    }
  }
}

This is appropriate for such a small example, but for larger datasets, another way that test data can be loaded is via a loading job that ingests a CSV file.

Setting Up Our Test Environment

Now create a test class called TigerAppIntTest next to AppTest. As a reminder, we will use the fact that IntTest is in the class name to differentiate this as an integration test.

We will start by inserting our test data in the database during a @BeforeAll method and clear the database at the end of our tests during a @AfterAll method.

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.file.Path;

import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;

public class TigerAppIntTest {

    private static final String DB_ENDPOINT = "http://localhost:9000";

    @BeforeAll
    static void setUpAll() {
        var client = HttpClient.newHttpClient();
        var payload = TigerAppIntTest.class.getClassLoader().getResource("data/payload.json");

        HttpRequest request;
        HttpResponse response;
        try {
            request = HttpRequest.newBuilder(URI.create(DB_ENDPOINT + "/graph/social"))
                    .header("accept", "application/json")
                    .POST(HttpRequest.BodyPublishers.ofFile(Path.of(payload.getPath())))
                    .build();
            response = client.send(request, HttpResponse.BodyHandlers.ofString());
            // Test the response
            assertNotNull(response.body());
            assertTrue(response.body().contains("\"results\":[{\"accepted_vertices\":2,\"accepted_edges\":1}]"));
        } catch (Exception e) {
            System.out.println("Error: " + e.getMessage());
            e.printStackTrace();
        }
    }
    
    @AfterAll
    static void tearDownAll() {
        var client = HttpClient.newHttpClient();

        HttpRequest request;
        try {
            request = HttpRequest.newBuilder(URI.create(DB_ENDPOINT + "/graph/social/vertices/person"))
                    .header("accept", "application/json")
                    .DELETE()
                    .build();
            client.send(request, HttpResponse.BodyHandlers.ofString());
        } catch (Exception e) {
            System.out.println("Error: " + e.getMessage());
            e.printStackTrace();
        }
    }
  
}

Testing a Query

Now that our test environment is set up we can test the hello query. The following test does this.

@Test
  void helloQueryTest() {
      var client = HttpClient.newHttpClient();

      HttpRequest request;
      HttpResponse response;
      try {
          request = HttpRequest.newBuilder(URI.create(DB_ENDPOINT + "/query/hello?p=Dave"))
                  .header("accept", "application/json")
                  .GET()
                  .build();
          response = client.send(request, HttpResponse.BodyHandlers.ofString());
          // Test the response
          assertNotNull(response.body());
          assertTrue(response.body().contains("\"results\":[{\"Result\":[{\"v_id\":\"Fred\",\"v_type\":\"person\",\"attributes\":{\"name\":\"Fred\",\"age\":63,\"gender\":\"Male\"}}]}]"));
      } catch (Exception e) {
          System.out.println("Error: " + e.getMessage());
          e.printStackTrace();
      }
  }

Now running ./gradlew intTest will, once the database is set up, run the methods in the following order: setUpAll, helloQueryTest, and tearDownAll.

Running TigerGraph Integration Tests using GitHub Actions

An advantage of using my dbakereffendi/tigergraph images are that they are small with the resource constraints on CI/CD services in mind. That being said, from downloading the image to installing the schema and queries, the integration testing phase can still end up being quite time-consuming.

To start running our tests on GitHub Actions create the following file in the root of your project .github/workflows/ci.yml.

name: CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    name: Test on JDK 11 and Ubuntu 20.04
    runs-on: ubuntu-20.04

    steps:
      - uses: actions/checkout@v2

      - name: Set up JDK 11
        uses: actions/setup-java@v1
        with:
          java-version: 11
          
      - name: Grant execute permission for gradlew
        run: chmod +x gradlew
      - name: Compile classes
        run: ./gradlew classes
      - name: Run unit tests
        run: ./gradlew test
      - name: Run TigerGraph integration tests
        run: ./gradlew intTest

This will run unit and integration tests when you push or have a pull request to themain branch. Adjust your configuration for the branch name, OS, or Java version of your choice, but what is important here are the Gradle commands.

Conclusion

Using Docker, GitHub Actions, and Gradle we have managed to run integration testing for a TigerGraph-backed application for free with a localized TigerGraph instance.

This can be extended to run loading jobs, deploy to production using Giraffle, or run multiple third-party Docker services during testing using the Avast plugin.

As a reminder, a fully working example of everything discussed in this article can be found on my GitHub repository.