Skip to content
START FOR FREE
START FOR FREE
  • SUPPORT
  • COMMUNITY
  • CONTACT US
Menu
  • SUPPORT
  • COMMUNITY
  • CONTACT US
MENUMENU
  • Products
    • The World’s Fastest and Most Scalable Graph Platform

      LEARN MORE

      Watch a TigerGraph Demo

      TIGERGRAPH CLOUD

      • Overview
      • TigerGraph Cloud Suite
      • FAQ
      • Pricing

      USER TOOLS

      • GraphStudio
      • Insights
      • Application Workbenches
      • Connectors and Drivers
      • Starter Kits
      • openCypher Support

      TIGERGRAPH DB

      • Overview
      • GSQL Query Language
      • Compare Editions

      GRAPH DATA SCIENCE

      • Graph Data Science Library
      • Machine Learning Workbench
  • Solutions
    • The World’s Fastest and Most Scalable Graph Platform

      LEARN MORE

      Watch a TigerGraph Demo

      Solutions

      • Solutions Overview

      INCREASE REVENUE

      • Customer Journey/360
      • Product Marketing
      • Entity Resolution
      • Recommendation Engine

      MANAGE RISK

      • Fraud Detection
      • Anti-Money Laundering
      • Threat Detection
      • Risk Monitoring

      IMPROVE OPERATIONS

      • Supply Chain Analysis
      • Energy Management
      • Network Optimization

      By Industry

      • Advertising, Media & Entertainment
      • Financial Services
      • Healthcare & Life Sciences

      FOUNDATIONAL

      • AI & Machine Learning
      • Time Series Analysis
      • Geospatial Analysis
  • Customers
    • The World’s Fastest and Most Scalable Graph Platform

      LEARN MORE

      CUSTOMER SUCCESS STORIES

      • Ford
      • Intuit
      • JPMorgan Chase
      • READ MORE SUCCESS STORIES
      • Jaguar Land Rover
      • Xbox
  • Partners
    • The World’s Fastest and Most Scalable Graph Platform

      LEARN MORE

      PARTNER PROGRAM

      • Partner Benefits
      • TigerGraph Partners
      • Sign Up
      TigerGraph partners with organizations that offer complementary technology solutions and services.​
  • Resources
    • The World’s Fastest and Most Scalable Graph Platform

      LEARN MORE

      BLOG

      • TigerGraph Blog

      RESOURCES

      • Resource Library
      • Benchmarks
      • Demos
      • O'Reilly Graph + ML Book

      EVENTS & WEBINARS

      • Events &Trade Shows
      • Webinars

      DEVELOPERS

      • Documentation
      • Ecosystem
      • Developers Hub
      • Community Forum

      SUPPORT

      • Contact Support
      • Production Guidelines

      EDUCATION

      • Training & Certifications
  • Company
    • Join the World’s Fastest and Most Scalable Graph Platform

      WE ARE HIRING

      COMPANY

      • Company Overview
      • Leadership
      • Legal Terms
      • Patents
      • Security and Compliance

      CAREERS

      • Join Us
      • Open Positions

      AWARDS

      • Awards and Recognition
      • Leader in Forrester Wave
      • Gartner Research

      PRESS RELEASE

      • Read All Press Releases
      TigerGraph Reports Exceptional Customer Growth and Product Leadership as More Market-Leading Companies Tap the Power of Graph
      March 1, 2023
      Read More »

      NEWS

      • Read All News

      Best paper award at International Conference on Very Large Data Bases

      New TigerGraph CEO Refocuses Efforts on Enterprise Customers

  • START FREE
    • The World’s Fastest and Most Scalable Graph Platform

      GET STARTED

      • Request a Demo
      • CONTACT US
      • Try TigerGraph
      • START FREE
      • TRY AN ONLINE DEMO

On “Benchmarking RedisGraph 1.0”

  • Yu Xu
  • November 19, 2018
  • Benchmark, blog, Graph Database Market
  • Blog >
  • On “Benchmarking RedisGraph 1.0”

Recently RedisGraph published a blog [1], comparing their performance to that of TigerGraph’s, following the tests [2] in TigerGraph’s benchmark report [3], which requires solid performance on 3-hop, 6-hop, and even 10-hop queries. Multi-hop queries on large data sets are the future of graph analytics.

After reviewing the report in depth with our engineering team, here are our key findings, including identifying fake news created by RedisGraph in their benchmarking process:

  • TigerGraph remains the fastest on queries with 2 or more hops. This applies to the larger dataset (a Twitter user-follower graph with 41.6M vertices and 1.5B edges). For the small dataset test, only 64M edges, RedisGraph is sometimes faster. We included this small dataset (0.9 GB raw data) in our benchmark because many of the other graph databases we tested could barely handle the larger Twitter dataset.
  • RedisGraph is faster on 1-hop queries. This is expected since Redis is specially designed for key-value lookups. In the real world, if you only need to do one hop, a key-value database or RDBMS is sufficient enough; you don’t need a graph product.
  • No Analytics Graph Query tests were reported. PageRank and Connected Components are tested in TigerGraph’s benchmark report but not in RedisGraph’s report. These types of queries are essential for finding hubs of influence or power, discovering communities, ranking, measuring similarity, and many other analytical tasks which are the reason that users choose a graph in the first place.
  • No Data Loading tests were reported. Data loading to a graph is the first step for graph analytics. Many times our customers and users have told us they rejected other graph databases based on their slow loading speed alone. Twitter’s dataset is only 29GB, a small dataset to TigerGraph. The larger the dataset, the bigger data loading and query performance gap between TigerGraph and other graph products, due to TigerGraph’s industry-first Native Parallel Graph design.
  • “Fake” new test: “Parallel requests benchmark”: RedisGraph added a new test. They repeated the 1-hop to 6-hop tests on their platform, allowed the queries to run in parallel by using “22 client threads running on the same test machine, which generated 300 requests in total.” However, they didn’t run parallel request tests on TigerGraph. They just took our average query times for a single query request and simply multiplied by 300.. We believe in fair tests and that is why we published the full steps and process for other people to follow in our report in github [2].

For queries starting from a single point, Redis has done well, and thank them for trying our benchmark.

If you are in the market for a graph database for analytics, we’d like to share with you some additional insights, with information drawn from Redis’ website and blog.

  • RedisGraph is not a distributed graph system [4],  You cannot use multiple machines to speed up your query performance. And, if your dataset is large, loading and querying the graph is impossible without a distributed graph system.
  • RedisGraph is pure in-memory. For users with terabytes or petabytes of data, the hardware cost can be prohibitively high for a pure in-memory system. TigerGraph still works very fast with some data stored on disk, by giving in-memory priority to the most critical elements for graph traversal and query processing. In fact, users can tell TigerGraph how they want to manage memory usage [5].   
  • RedisGraph’s execution is single-threaded. In their own words[1]:

RedisGraph was built from the ground up for extremely high parallelism, with each query processed by a single thread that utilizes the GraphBLAS library for processing matrix operations with linear algebra.”

Parallel processing is the preferred mode for graph processing, because parallel processing lets you access each of a vertex’s neighbors simultaneously. On large queries, a single thread will not scale well. TigerGraph’s Native Parallel Graph technology scales out to multiple machines for a single query execution AND scales up to multiple CPU cores on a single machine to speed up query performance for all workloads.

  • TigerGraph is designed for parallel computation.  TigerGraph’s capability for parallel computation of single queries is technically superior and well-suited for real-time scenarios, because it makes full use of available resources to complete the current workload. The number of threads for a single query is configurable, to fine tune the performance. If an application has a lot of data, or runs complex queries, TigerGraph’s built-in parallelism lets users either scale up (by using more threads, or running on a more powerful machine with more CPU cores), or scale out by simply adding more machines, to improve query performance.  Thinking about this in another way, Redis gives you only one way to speed up a query: to go to a more powerful machine. TigerGraph gives you two additional ways: increase the number of threads or scale out to more machines.
  • RedisGraph’s read and write operations block one another. From their blog [1]:

“RedisGraph enforces write/readers separation by using a read/write (R/W) lock so that either multiple readers can acquire the lock or just a single writer. As long as a writer is executing, no one can acquire the lock, and as long as there’s reader executing, no writer can obtain the lock.”

So RedisGraph may have parallelism in theory, but not when operations are being blocked.

In TigerGraph, as an enterprise graph database platform, reads don’t block writes, writes don’t block reads. Reads and writes work on their own ‘snapshots’ of the graph when they come to TigerGraph. Check out the details on our transaction documentation for more details.

In general, taking a shortcut by building a graph API on top of a key-value store has inherent and critical flaws which cause both query performance and data consistency issues. Please check out our blog “Building a Graph Database on a Key-Value Store?“, which is excerpted from our eBook “Native Parallel Graphs: The Next Generation of Graph Database for Real-Time Deep Link Analytics.”  

Redis as a key-value DB is a widely-adopted product. However, one size doesn’t fit all. As discussed in our blog, designing a graph API on top of a key-value store is an expensive and complex job which will not yield a high-performance result. While key-value stores excel at single key-value transactions, they lack ACID properties and the complex transaction functionality required by a graph update. Thus, building a graph database on top of a key value store opens the door to data inconsistencies, wrong query results, slow query performance for multi-hop queries, and an expensive and rigid deployment.  

Summary

  1. TigerGraph is the fastest on deep link analytics on large datasets. 
  2. RedisGraph is good for small datasets and simple 1-hop queries.
  3. RedisGraph does not support concurrent read and write queries.
  4. RedisGraph declined to show their performance for loading or for analytical queries like PageRank or community detection.
  5. With the inherent and critical limitations in its technical implementation, RedisGraph is neither a native graph nor a distributed graph.

It is great for the user community to see more graph offerings and more public benchmark results. TigerGraph will continue to lead the open and repeatable benchmarking efforts. Everything you need to reproduce our tests is available on GitHub [2].   

A Call to Action for RedisGraph

In the spirit of fair engineering tests that serve the interests of the user community, we ask Peter Cailliau, RedisGraph benchmarking author to do the following:

  1. Make all testing scripts public so potential users and third parties can repeat and verify any benchmark testing.  
  2. Update its benchmarking report with facts, not extrapolations, especially in the Section ‘Parallel Request Benchmark’. Do an apples-to-apples comparison, rather than just multiplying TigerGraph’s performance number from a different test by 300.  
  3. Update the benchmarking report with clear facts, not misrepresentations. Specifically:“It is important to note that TigerGraph applied a timeout of three minutes for the one and two-hop queries and 2.5 hours for the three and six-hop queries for all requests on all databases (see TigerGraph’s benchmark report for details on how many requests timed out for each database). If all requests for a given data set and given database timed out, we marked the result as ‘N/A’. When there is an average time presented, this only applies to successfully executed requests (seeds), meaning that the query didn’t time out or generate an out of memory exception. This sometimes skews the results, since certain databases were not able to respond to the harder queries, resulting in a better average single request time and giving a wrong impression of the database’s performance.”Peter Cailliau states that his presentation is “giving a wrong impression of a database’s performance” yet does it anyway.By not showing that some graph databases could not complete the harder queries, as TigerGraph’s report showed, they seem to be doing a favor to the competitors, by hiding their flaws and boosting their average query times. TigerGraph receives no such boost because TigerGraph completed all the tests. The effect is to make other graph databases appear to be closer in performance to TigerGraph than they really are. We used timeouts in our tests, a practice used in real-world applications, to abort tests which were running an unreasonably long time. TigerGraph received no benefit because the thresholds were always much longer than TigerGraph needed.   We think that Peter Cailliau should have also mentioned that TigerGraph never timed out.
  4. Correct a counting and computation error in its k-hop neighborhood query. According to [1],

“The k-hop neighborhood query is a local type of graph query. It counts the number of nodes a single start node (seed) is connected to at a certain depth, and only counts nodes that are k-hops away.

Below is the Cypher code:

MATCH (n:Node)-[*$k]->(m) where n.id=$root return count(m)  

Here, $root is the ID of the seed node from which we start exploring the graph, and $k represents the depth at which we count neighbors. To speed up execution, we used an index on the root node ID.“

However, this query will double-count some neighbors. The correct query should be

MATCH (n:Node)-[*$k]->(m) where n.id=$root return count(distinct m)  

The count (distinct m) query performs additional computation to remove duplicate counts.  Whatever the reason or limitation behind RedisGraph, we think the author needs to clarify this issue to the readers.  

Last Words

We’d like to thank RedisGraph for giving our benchmark test a try, and congratulate them on the results.  We also appreciate that they share our belief about the importance of real-time deep link analytics.

The graph market is growing rapidly. We welcome RedisGraph to the Graph World, and we feel honored that newcomers see us as the high-performance bar. We welcome any open and fact-based benchmarking, which we view as healthy and necessary for the whole graph market’s growth and user education.    

 

Yu Xu, CEO TigerGraph

[1] https://redislabs.com/blog/new-redisgraph-1-0-achieves-600x-faster-performance-graph-databases/

[2] https://github.com/tigergraph/ecosys/tree/benchmark/benchmark

[3] https://info.tigergraph.com/benchmark

[4] https://redislabs.com/blog/release-redisgraph-v1-0-preview/

[5] This feature is available in the Enterprise edition.

You Might Also Like

Utilizing Multi-edge for Temporal Search in a Sales Agent Hierarchy

Utilizing Multi-edge for Temporal Search in...

September 14, 2023
Advancing Entity Resolution for Fraud Detection with TigerGraph

Advancing Entity Resolution for Fraud Detection...

September 8, 2023
Reduce AML Investigation Costs with TigerGraph

Reduce AML Investigation Costs with TigerGraph

September 5, 2023

Introducing TigerGraph 3.0

July 1, 2020

Everything to Know to Pass your TigerGraph Certification Test

June 24, 2020

Neo4j 4.0 Fabric – A Look Behind the Curtain

February 7, 2020

TigerGraph Blog

  • Categories
    • blogs
      • About TigerGraph
      • Benchmark
      • Business
      • Community
      • Compliance
      • Customer
      • Customer 360
      • Cybersecurity
      • Developers
      • Digital Twin
      • eCommerce
      • Emerging Use Cases
      • Engineers
      • Entity Resolution
      • Finance
      • Fraud / Anti-Money Laundering
      • GQL
      • Graph Database Market
      • Graph Databases
      • GSQL
      • Healthcare
      • Machine Learning / AI
      • Podcast
      • Supply Chain
      • TigerGraph
      • TigerGraph Cloud
    • Graph AI On Demand
      • Analysts and Research
      • Customer 360 and Entity Resolution
      • Customer Spotlight
      • Development
      • Finance, Banking, Insurance
      • Keynote
      • Session
    • RulesEngine
    • Video
  • Recent Posts

    • Utilizing Multi-edge for Temporal Search in a Sales Agent Hierarchy
    • Advancing Entity Resolution for Fraud Detection with TigerGraph
    • Reduce AML Investigation Costs with TigerGraph
    • Using Graph Machine Learning to Improve Fraud Detection Rates
    • The Power Of Graph Machine Learning: Ethereum Blockchain Fraud Detection
    TigerGraph

    Product

    SOLUTIONS

    customers

    RESOURCES

    start for free

    TIGERGRAPH DB
    • Overview
    • Features
    • GSQL Query Language
    GRAPH DATA SCIENCE
    • Graph Data Science Library
    • Machine Learning Workbench
    TIGERGRAPH CLOUD
    • Overview
    • Cloud Starter Kits
    • Login
    • FAQ
    • Pricing
    • Cloud Marketplaces
    USEr TOOLS
    • GraphStudio
    • TigerGraph Insights
    • Application Workbenches
    • Connectors and Drivers
    • Starter Kits
    • openCypher Support
    SOLUTIONS
    • Why Graph?
    industry
    • Advertising, Media & Entertainment
    • Financial Services
    • Healthcare & Life Sciences
    use cases
    • Benefits
    • Product & Service Marketing
    • Entity Resolution
    • Customer 360/MDM
    • Recommendation Engine
    • Anti-Money Laundering
    • Cybersecurity Threat Detection
    • Fraud Detection
    • Risk Assessment & Monitoring
    • Energy Management
    • Network & IT Management
    • Supply Chain Analysis
    • AI & Machine Learning
    • Geospatial Analysis
    • Time Series Analysis
    success stories
    • Customer Success Stories

    Partners

    Partner program
    • Partner Benefits
    • TigerGraph Partners
    • Sign Up
    LIBRARY
    • Resources
    • Benchmark
    • Webinars
    Events
    • Trade Shows
    • Graph + AI Summit
    • Million Dollar Challenge
    EDUCATION
    • Training & Certifications
    Blog
    • TigerGraph Blog
    DEVELOPERS
    • Developers Hub
    • Community Forum
    • Documentation
    • Ecosystem

    COMPANY

    Company
    • Overview
    • Careers
    • News
    • Press Release
    • Awards
    • Legal
    • Patents
    • Security and Compliance
    • Contact
    Get Started
    • Start Free
    • Compare Editions
    • Online Demo - Test Drive
    • Request a Demo

    Product

    • Overview
    • TigerGraph 3.0
    • TIGERGRAPH DB
    • TIGERGRAPH CLOUD
    • GRAPHSTUDIO
    • TRY NOW

    customers

    • success stories

    RESOURCES

    • LIBRARY
    • Events
    • EDUCATION
    • BLOG
    • DEVELOPERS

    SOLUTIONS

    • SOLUTIONS
    • use cases
    • industry

    Partners

    • partner program

    company

    • Overview
    • news
    • Press Release
    • Awards

    start for free

    • Request Demo
    • take a test drive
    • SUPPORT
    • COMMUNITY
    • CONTACT
    • Copyright © 2023 TigerGraph
    • Privacy Policy
    • Linkedin
    • Twitter

    Copyright © 2020 TigerGraph | Privacy Policy

    Copyright © 2020 TigerGraph Privacy Policy

    • SUPPORT
    • COMMUNITY
    • COMPANY
    • CONTACT
    • Linkedin
    • Facebook
    • Twitter

    Copyright © 2020 TigerGraph

    Privacy Policy

    • Products
    • Solutions
    • Customers
    • Partners
    • Resources
    • Company
    • START FREE
    START FOR FREE
    START FOR FREE
    TigerGraph
    PRODUCT
    PRODUCT
    • Overview
    • GraphStudio UI
    • Graph Data Science Library
    TIGERGRAPH DB
    • Overview
    • Features
    • GSQL Query Language
    TIGERGRAPH CLOUD
    • Overview
    • Cloud Starter Kits
    TRY TIGERGRAPH
    • Get Started for Free
    • Compare Editions
    SOLUTIONS
    SOLUTIONS
    • Why Graph?
    use cases
    • Benefits
    • Product & Service Marketing
    • Entity Resolution
    • Customer Journey/360
    • Recommendation Engine
    • Anti-Money Laundering (AML)
    • Cybersecurity Threat Detection
    • Fraud Detection
    • Risk Assessment & Monitoring
    • Energy Management
    • Network Resources Optimization
    • Supply Chain Analysis
    • AI & Machine Learning
    • Geospatial Analysis
    • Time Series Analysis
    industry
    • Advertising, Media & Entertainment
    • Financial Services
    • Healthcare & Life Sciences
    CUSTOMERS
    read all success stories

     

    PARTNERS
    Partner program
    • Partner Benefits
    • TigerGraph Partners
    • Sign Up
    RESOURCES
    LIBRARY
    • Resource Library
    • Benchmark
    • Webinars
    Events
    • Trade Shows
    • Graph + AI Summit
    • Graph for All - Million Dollar Challenge
    EDUCATION
    • TigerGraph Academy
    • Certification
    Blog
    • TigerGraph Blog
    DEVELOPERS
    • Developers Hub
    • Community Forum
    • Documentation
    • Ecosystem
    COMPANY
    COMPANY
    • Overview
    • Leadership
    • Careers  
    NEWS
    PRESS RELEASE
    AWARDS
    START FREE
    Start Free
    • Request a Demo
    • SUPPORT
    • COMMUNITY
    • CONTACT
    Dr. Jay Yu

    Dr. Jay Yu | VP of Product and Innovation

    Dr. Jay Yu is the VP of Product and Innovation at TigerGraph, responsible for driving product strategy and roadmap, as well as fostering innovation in graph database engine and graph solutions. He is a proven hands-on full-stack innovator, strategic thinker, leader, and evangelist for new technology and product, with 25+ years of industry experience ranging from highly scalable distributed database engine company (Teradata), B2B e-commerce services startup, to consumer-facing financial applications company (Intuit). He received his PhD from the University of Wisconsin - Madison, where he specialized in large scale parallel database systems

    Todd Blaschka | COO

    Todd Blaschka is a veteran in the enterprise software industry. He is passionate about creating entirely new segments in data, analytics and AI, with the distinction of establishing graph analytics as a Gartner Top 10 Data & Analytics trend two years in a row. By fervently focusing on critical industry and customer challenges, the companies under Todd's leadership have delivered significant quantifiable results to the largest brands in the world through channel and solution sales approach. Prior to TigerGraph, Todd led go to market and customer experience functions at Clustrix (acquired by MariaDB), Dataguise and IBM.