---
title: "Workload Management"
description: "Overview of workload management in TigerGraph."
component: "tigergraph-server"
version: "4.3"
module: "system-management"
html_url: "/tigergraph-server/4.3/system-management/workload-management"
canonical_url: "https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management"
markdown_url: "https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management.md"
---

[View as HTML](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management)

# Workload Management

This page explains the different methods available to manage system workloads in TigerGraph.

In distributed or replicated clusters, users have multiple processing units at their disposal. Depending on their needs, they can configure [query routing schemes](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management.md#_query_routing_schemes) such as round-robin or CPU load-aware routing. For replicated clusters, queries can also be [directed to specific replicas](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management.md#_specify_replica_to_run_query_on).

## Workload Queues

Workload Queues help manage queries by routing them to predefined queues with specific configurations. Each queue can have properties like the maximum number of concurrent queries and delay queue size, preventing system overload. Queries can be assigned to queues based on user roles and permissions.

In 3.10, **workload queues** were introduced. Multiple independent queues can be configured, allowing application designers to decide how specific operations are handled.

Workload queues manage the following types of operations:

* Running an installed query.
* Interpreting ad-hoc queries.
* Executing heavy built-in queries, commonly used for "Explore Graph" in GraphStudio.

### Configure Workload Queues

Workload queues help control how queries are executed within the system by defining limits such as the maximum number of concurrent queries and the maximum size of the delay queue. Administrators can configure workload queues by creating, modifying, or deleting them as needed.

Each queue has the following properties:

| Field | Description | Data type |
| --- | --- | --- |
| `description` | The description of the queue. | `STRING` (< 256 characters) |
| `isDefault` (optional) | The flag to indicate if the queue is the default queue. Must be set to `true` for exactly one queue. | `BOOL` |
| `maxConcurrentQueries` | The maximum number of concurrent queries allowed in the queue. | `UINT` in the range of `(0, 131072)` |
| `maxDelayQueueSize` | The maximum number of queries that can be queued in the delay queue. | `UINT` in the range of `[0, 131072)` |

#### maxConcurrentQueries and maxDelayQueueSize

`maxConcurrentQueries` and `maxDelayQueueSize` are enforced on per machine level. More specifically, it puts a limit on how many requests ONE `GPE` process can handle. For example, in a TigerGraph cluster with `4` nodes (there will be `4` `GPE` processes), the total number of qureies allowed for a `WorkloadQueue` is `4*maxConcurrentQueries`. Similarly, the total number of queries can be put into the corresponding delay queue is `4*maxDelayQueueSize`. TigerGraph internally would try to evenly distribute queries evenly among the nodes, hence, the `WorkloadQueue` from each `GPE` would be filled at a similar pace.

Before making changes, it's a good practice to retrieve the current configuration using the **GET Workload Queue** command. After reviewing, you can update or apply new configurations using the **PUT Workload Queue** command. These commands are available both as GSQL commands and as REST endpoints.

> [!NOTE]
> Use a JWT token with `-H "Authorization: Bearer <JWT-token>"` for RESTPP APIs in workload management to ensure secure authentication. `username:password` is supported only for GSQL APIs, not RESTPP.

**GSQL Commands**

* **Retrieve Current Configuration:**

GET WORKLOAD QUEUE

* **Update Configuration:**

PUT WORKLOAD QUEUE FROM "/path/to/queue.json"

**REST Endpoints**

* **Retrieve Current Configuration:**

GET /gsql/v1/workload-manager/configs

* **Configure Workload Queues**

POST /gsql/v1/workload-manager/configs

**Examples**

1. **Retrieve Current Configuration:**

curl -X GET -H "Authorization: Bearer <token>" \
  <hostname>:<nginx-port>/gsqlserver/gsql/workload-manager/configs

1. **Modify Workload Queue Configuration:**

curl -X POST -H "Authorization: Bearer <token>" \
<hostname>:<nginx-port>/gsqlserver/gsql/workload-manager/configs \
-d '{"isEnabled":true,"queues":{"OLTP":{"description":"OLTP queries","isDefault":false,"maxConcurrentQueries":100,"maxDelayQueueSize":200}}}'

**Response Status Codes**

**For GET Workload Queue**

| Status Code | Description |
| --- | --- |
| 200 | The queue configurations were successfully retrieved. |
| 403 | The user does not have the `READ_WORKLOAD_QUEUE` privilege. |
| 500 | Server error occurred while processing the request. |

**For PUT Workload Queue**

| Status Code | Description |
| --- | --- |
| 200 | The queue configurations were successfully uploaded. |
| 400 | The payload is ill-formed. |
| 403 | The user does not have the `WRITE_WORKLOAD_QUEUE` privilege. |
| 409 | Conflict in the configuration, such as multiple queues set as default. |
| 500 | Server error occurred while processing the request. |

**maxConcurrentQueries and maxDelayQueueSize**Each workload queue has two critical properties to manage system resources:

1. **maxConcurrentQueries:** Defines the maximum number of queries that can run simultaneously in a queue. This helps prevent resource contention and ensures that the system does not become overloaded.
2. **maxDelayQueueSize:** Specifies the maximum number of queries that can wait in a queue when the current workload exceeds the concurrent query limit. If this limit is reached, new queries are rejected until the queue has capacity.

> [!NOTE]
> Both properties are enforced at the per-machine level in a cluster. Setting the values too high may degrade performance. It's recommended to keep `maxConcurrentQueries` below three times the number of physical CPU cores in a machine.
> 
> Any configuration changes require a GPE restart to take effect.

### List Workload Queues

Displays a list of all workload queues available to the current user, including their configurations and permissions.

**GSQL Command**

```gsql
LIST WORKLOAD QUEUE
```

**REST Endpoints**

```none
GET /restpp/workload-manager/queue
```

**Example Request**

```bash
curl -X GET -H "Authorization: Bearer <token>" \
  <hostname>:<nginx-port>/restpp/workload-manager/queue
```

**Example Response**The response will include the information available to the general users.

```json
[
  {
    "id": "AdHoc",
    "description": "Ad-hoc queries",
    "isDefault": true
  },
  {
    "id": "OLTP",
    "description": "OLTP queries"
  }
]
```

**Response Status Codes**

| Status Code | Description |
| --- | --- |
| 200 | The queue info has been retrieved successfully. |
| 403 | The user doesn't have the privilege `READ_DATA`. |

### Grant/Revoke Workload Queue Access

You can grant or revoke workload queues to a user based on its user name, groups, and/or roles.

**GSQL Command**

```gsql
# GRANT
GRANT WORKLOAD QUEUE <queue_name> TO USER <user1>, <user2>

# REVOKE
REVOKE WORKLOAD QUEUE <queue_name> FROM USER <user1>, <user2>
```

**REST Endpoint**

```none
POST /gsql/v1/workload-manager/permission
```

**Request Body**The request body expects a JSON object with the following schema:

```json
{
  "OLTP": {
    "granted": {
      "USER": []
      "GROUP": ["*"]
      "ROLE": ["r1", "r2"]
    }
  }
}
```

The request body must have the following fields at the top level:

| Field | Description | Data type |
| --- | --- | --- |
| `action` | `GRANT` or `REVOKE` (case insensitive) | `STRING` |
| `queue` | The ID of the queue to be granted or revoked. | `STRING` |
| `user` (optional) | The list of the user names to be granted/revoked. | `STRING` or `STRING[]` |
| `group` (optional) | The list of the group names to be granted/revoked. | `STRING` or `STRING[]` |
| `role` (optional) | The list of the role names to be granted/revoked. | `STRING` or `STRING[]` |

> [!NOTE]
> * You can use the wildcard " \* " to grant/revoke the queue to all users, groups, or roles.
> * Note that " \* " must be the only entry in the list when available.

**Example Request**

Grant the queue `OLTP` to the user `u1` and `u2`:

```bash
curl -X POST -H "Authorization: Bearer <JWT-token>" \
  <hostname>:<nginx-port>/gsqlserver/gsql/workload-manager/permission \
  -d '{"action": "grant", "queue": "OLTP", "user": ["u1", "u2"]}'
```

Revoke the queue `scheduled_jobs` from all users and the role `r1`:

```bash
curl -X POST -H "Authorization: Bearer <JWT-token>" \
  <hostname>:<nginx-port>/gsqlserver/gsql/workload-manager/permission \
  -d '{"action": "revoke", "queue": "scheduled_jobs", "user": "*", "role": ["r1"]}'
```

**Response Status Codes**

| Status Code | Description |
| --- | --- |
| 200 | The queue has been granted/revoked successfully. |
| 400 | The payload is ill-formed so none of the given entities could be granted/revoked. |
| 403 | The user doesn't have the privilege `WRITE_WORKLOAD_QUEUE` |

> [!NOTE]
> Unlike REST API, the GSQL commands don't allow you to specify USER, GROUP, and ROLE in a command. You must use separate commands for each entity type.

### Show Workload Queue Permissions

The `SHOW WORKLOAD QUEUE` command lists detailed information about workload queues, including their permissions, descriptions, and limits. It is primarily used to inspect queue settings and permissions.

**GSQL Command**To show the permission info of all queues:

```gsql
SHOW WORKLOAD QUEUE
```

To show the permission info of a specific queue, for example `OLTP`:

```gsql
SHOW WORKLOAD QUEUE OLTP
```

**REST Endpoint**

```none
GET /gsql/v1/workload-manager/permission
```

**Example Request**To retrieve the permission info of the queue `OLTP`:

```bash
curl -X GET -H "Authorization: Bearer <JWT-token>" \
  localhost:14240/gsql/v1/workload-manager/permission?id=OLTP
```

**Example Response**

The response will be the combination of configs and permission, e.g.

```json
{
  "OLTP": {
    "description": "OLTP queries",
    "isDefault": false,
    "maxConcurrentQueries": 100,
    "maxDelayQueueSize": 200,
    "granted": {
      "USER": [],
      "GROUP": ["*"],
      "ROLE": ["r1", "r2"]
    }
  }
}
```

**Response Status Codes**

| Status Code | Description |
| --- | --- |
| 200 | The queue info has been retrieved successfully. |
| 403 | The user doesn't have the privilege `READ_WORKLOAD_QUEUE`. |

> [!NOTE]
> * Use `SHOW WORKLOAD QUEUE` to inspect queue configurations and access permissions.
> * This command focuses on **visibility of queue settings**, unlike `GET WORKLOAD QUEUE`, which exports configurations.

### Check Queue Status

You can use the following API to check the status of the workload queues for monitoring purposes.

```none
POST /restpp/workload-manager/queuestatus
```

Return the status of the given workload queue on each GPE instance.

**Request Body**

| Field | Description | Data type |
| --- | --- | --- |
| queuelist (optional) | The list of the ID of the WorkloadQueue. If not specified, all queues will be shown. | `STRING[]` |
| mode (optional) | `stats` or `verbose` (case-sensitive). If not specified, `stats` will be used. | `STRING` |

For `mode` field, if `stats` is specified, response only gives the numbers of queries waiting and running. If `verbose` is specified, the response will include the the request Ids of the queries that are waiting and running.

If Request Body is not provided, response is generated as if both fields are using the default values.

**Example Request**

```bash
curl -X POST -H "Authorization: Bearer <JWT-token>" \
  <hostname>:<nginx-port>/restpp/workload-manager/queuestatus \
  -d '{"queuelist": ["AdHoc"], "mode": "verbose"}'
```

**Example Response**

```json
{
  "version": {
    "edition": "enterprise",
    "api": "v2",
    "schema": 0
  },
  "error": false,
  "message": "Completes",
  "WorkloadQueueStatusByInstances": [
    {
      "version": {
        "edition": "enterprise",
        "api": "v2",
        "schema": 0
      },
      "error": false,
      "message": "",
      "results": {
        "GPE_2_1": [
          {
            "WorkloadQueueName": "AdHoc",
            "maxConcurrentQueries": 1,
            "maxDelayQueueSize": 2,
            "runningQueries": [
              "196702.RESTPP_1_1.1707799387957.N"
            ],
            "delayQueries": [
              "65630.RESTPP_1_1.1707799387958.N"
            ]
          }
        ]
      }
    },
    {
      "version": {
        "edition": "enterprise",
        "api": "v2",
        "schema": 0
      },
      "error": false,
      "message": "",
      "results": {
        "GPE_1_1": [
          {
            "WorkloadQueueName": "AdHoc",
            "maxConcurrentQueries": 1,
            "maxDelayQueueSize": 2,
            "runningQueries": [
              "94.RESTPP_1_1.1707799387957.N"
            ],
            "delayQueries": [
              "131167.RESTPP_1_1.1707799387959.N"
            ]
          }
        ]
      }
    }
  ],
  "code": "REST-0000"
}
```

### Use Cases

Suppose we have configured the following workload queues that are the output of the `SHOW WORKLOAD QUEUE` command:

```json
{
  "OLTP": {
    "description": "OLTP queries",
    "isDefault": true,
    "maxConcurrentQueries": 100,
    "maxDelayQueueSize": 100,
    "granted": {
      "USER": [],
      "GROUP": ["g1", "g2"],
      "ROLE": []
    }
  },
  "scheduled_jobs": {
    "description": "Scheduled jobs",
    "maxConcurrentQueries": 5,
    "maxDelayQueueSize": 0,
    "granted": {
      "USER": ["u1"],
      "GROUP": [],
      "ROLE": ["r1"]
    }
  },
  "AdHoc": {
    "description": "Ad-hoc queries",
    "isDefault": false,
    "maxConcurrentQueries": 10,
    "maxDelayQueueSize": 10,
    "granted": {
      "USER": [],
      "GROUP": ["g3"],
      "ROLE": ["r2"]
    }
  }
}
```

**Running a Query**

When running a query, you can specify the workload queue to run the query on. If the queue is not specified, the query will be routed to the default queue. To specify the queue in the GSQL shell, you can use the `-queue` option, e.g.

RUN QUERY -queue AdHoc q1()

or you can use the HTTP header `Workload-Queue`:

```bash
curl -X POST -H "Authorization: Bearer <JWT-token>" \
  -H "Workload-Queue: AdHoc" \
  <hostname>:14240/restpp/query/ldbc_snb/q1"
```

If the given queue is not granted to the current user, the query will be rejected with the error code `REST-14000` and return `HTTP 422 Unprocessable Entity`.

For example, if the user `tigergraph` who does not belong to the group `g3` or holds the role `r2` tries to run a query on the queue `AdHoc`, the query will be rejected.

> [!NOTE]
> If the queue is full of capacity, the query will be rejected.

## Other Query Workload Management Methods

### Limit number of concurrent heavy queries

> [!NOTE]
> This configuration is deprecated as of TG 3.10.0 and will be removed in a future release. This is ignored once the [workload queue](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management.md#_workload_queue) feature is enabled.

TigerGraph has a few built-in queries that are memory-intensive, here referred to as "heavy". These queries tend to be invoked by applications such as GraphStudio. You can set a limit of how many of these heavy queries are allowed to run concurrently by configuring the parameter `RESTPP.WorkLoadManager.MaxHeavyBuiltinQueries` with the [gadmin config command](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/management-commands.md#_gadmin_config).

For example, to set the maximum number of heavy built-in queries to 10, run the following command:

```console
$ gadmin config set RESTPP.WorkLoadManager.MaxHeavyBuiltinQueries 10
```

You must [restart the RESTPP service](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/manage-services.md#_start_stop_or_restart_a_service) for the change to take effect.

### Limit number of concurrent queries

> [!NOTE]
> This configuration is deprecated as of TG 3.10.0 and will be removed in a future release. This is ignored once the [workload queue](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management.md#_workload_queue) feature is enabled.

You can use the `RESTPP.WorkLoadManager.MaxConcurrentQueries` parameter to set a limit of how many queries are allowed to be running concurrently. The count of these queries **does not include** the built-in heavy queries.

For example, to specify that there can only be 50 concurrent queries at a time, excluding the heavy built-in queries, change the value of the configuration parameter to 50 with the [gadmin config command](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/management-commands.md#_gadmin_config):

```console
$ gadmin config set RESTPP.WorkLoadManager.MaxConcurrentQueries 50
```

If the maximum number of concurrent queries is reached, newly submitted queries are placed in a delay queue, and begin to run as the currently running queries finish. If the queue is at capacity, newly submitted queries are rejected. and you need wait until there is capacity to run the query again. You can adjust the size of the queue with the configuration parameter `RESTPP.WorkLoadManager.MaxDelayQueueSize`.

For example, to specify that a maximum 20 queries may remain in the queue, run the following command:

```console
$ gadmin config set RESTPP.WorkLoadManager.MaxDelayQueueSize 20
```

You must [restart the RESTPP service](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/manage-services.md#_start_stop_or_restart_a_service) for the change to take effect.

### Specify replica to run query on

On a distributed cluster, you can specify on which replica you want a query to be run through the [Run Query REST endpoint](https://www.tigergraph.com/docs/tigergraph-server/4.3/API/built-in-endpoints.md#_run_an_installed_query_post).

For example, to run the query on the primary cluster, use the `GSQL-REPLICA` header when running a query and set its value to 1:

Specify that the query run on the primary cluster

```bash
curl -X POST -H "GSQL-REPLICA: 1" -d '{"p":{"id":"Tom","type":"person"}}'
"http://localhost:14240/restpp/query/social/hello"
```

## Query Routing Schemes

In a distributed or replicated cluster, REST++ automatically routes queries to different GPEs, in order to spread the workload.

> [!NOTE]
> If [GSQL-REPLICA](https://www.tigergraph.com/docs/tigergraph-server/4.3/system-management/workload-management.md#_specify_replica_to_run_query_on) header is used when invoking a query, this header overrides the routing scheme for that query.

RESTPP supports three query routing modes:

* **Round Robin** (Mode 0): Routes queries across GPEs in round-robin order.
* **CPU Load Aware** (Mode 1): Routes queries to the least-loaded GPE based on CPU usage.
* **Local GPE** (Mode 2): Routes every query to the GPE running on the same node as the receiving RESTPP instance.

### Round Robin routing

The default query routing scheme is round-robin. The first query is managed by GPE 0, the next query by GPE 1, and so on. After the last GPE, the cycle returns to GPE 0.

Version 3.9.3 adds a system configuration parameter `RESTPP.CPULoadAware.Mode` to enable system administrators to select other query routing schemes:

* Mode = 0 (default): Round-Robin
* Mode = 1: CPU Load Aware

### CPU Load Aware Query Routing

When this query routing mode is selected, REST++ tries to direct incoming queries to the GPEs that are currently less busy.

Specifically, the system periodically polls CPU usage data to find a GPE whose CPU usage percentage is below`RESTPP.QueryRouting.TargetSelectionCPUThreshold` (default 50).

If no GPE satisfies the CPU threshold condition, REST++ falls back to the default behavior (round-robin selection).

Example: Change CPU Load Threshold and Enable CPU Load Aware routing

```console
$ gadmin config entry RESTPP.QueryRouting.TargetSelectionCPUThreshold 40
$ gadmin config entry RESTPP.QueryRouting.Mode 1
```

### Local GPE Routing

When `RESTPP.QueryRouting.Mode` is set to `2` (LocalGPE), REST++ forwards each query directly to the GPE co-located on the same node as the receiving RESTPP instance.

This removes the network hop between RESTPP and a remote GPE, reducing query latency for high QPS workloads. It is most beneficial in multi-node or HA deployments where RESTPP would otherwise forward requests to a remote GPE.

You can target a specific GPE by directing your requests to the RESTPP instance running on that GPE's node.

Example: Enable Local GPE routing

```console
$ gadmin config set RESTPP.QueryRouting.Mode 2
$ gadmin config apply
$ gadmin restart RESTPP -y
```

After restarting RESTPP, all incoming queries handled by that RESTPP instance are forwarded to its local GPE.
