Load from Data Lakehouse

A data lakehouse combines the scalability and flexibility of data lakes with the data management capabilities of data warehouses. TigerGraph supports loading data from lakehouse platforms through Kafka Connect-based connectors and Apache Spark integrations.

Supported Lakehouse Integrations

TigerGraph currently supports the following lakehouse integrations:

Lakehouse Platform Integration Method Details

Apache Iceberg

Kafka Connect

Load Iceberg tables stored in S3 or MinIO through the Iceberg connector and Iceberg REST Catalog. See Load from Apache Iceberg.

Apache Iceberg

Spark Connector

Read Iceberg tables into Apache Spark and write the data to TigerGraph using the Spark Connector. See from Spark.

DeltaLake

Spark Connector

Read DeltaLake tables into Apache Spark and write the data to TigerGraph using the Spark Connector. See from Spark.

Apache Hudi

Spark Connector

Read Hudi datasets into Apache Spark and write the data to TigerGraph using the Spark Connector. See from Spark.

Loading Workflow Overview

The overall loading workflow for lakehouse integrations is similar to other TigerGraph loading methods:

  1. Define your graph schema.

  2. Configure the external data connection.

  3. Create a loading job or Spark-based ingestion pipeline.

  4. Load data into TigerGraph.

  5. Monitor and manage loading operations.

For Kafka Connect-based integrations such as Iceberg, TigerGraph uses DATA_SOURCE objects and loading jobs.

For Spark-based integrations, TigerGraph uses the Spark Connector to write Spark DataFrames into TigerGraph.