Key Technologies of the Kunlun Sentian Database Kernel

Introduction

The Kunlun Santian Database is a new centralized database product developed by Klustron based on Huawei's Santian Engine. As a new product jointly launched by Klustron and Huawei Storage, it combines the powerful computing layer performance of the Kunlun Database with the excellent storage processing capabilities of the Santian Engine, achieving significant improvements in database functionality, performance, and stability across multiple scenarios. This presentation will mainly introduce some key core technologies of this new product, as well as new features to be developed on this basis in the future, aiming to give everyone a preliminary understanding of the Kunlun Santian Database while also providing a deeper insight into its core new technologies.

Key Quote from this Article: The Kunlun-CanTian database combines the powerful computing layer performance of the Kunlun database with the outstanding storage processing capabilities of the CanTian engine, achieving a significant improvement in database functionality, performance, and stability across multiple scenarios.

Background of the Kunlun Sentian Database

First, let's look at the common problems we often encounter when using common databases, such as MySQL, for high availability.

Usually, when we use MySQL, we use a Binlog-based master-slave replication architecture for high availability. In this architecture, we usually encounter the following problems:

• Performance bottleneck of the master node transaction group commit.

• • Threads of all transactions submitted concurrently are blocked;

• A single thread sequentially appends the already written Binlog events from each transaction's private cache to the Binlog file;

• Tests show that if Binlog is turned off, performance can double.

• Primary-backup delay: causes no primary to be available after the primary node crashes.

• • Typical causes: The table has no primary key or unique index, and later the hash mechanism was slightly improved;

• Typical causes: large-scale inserts, updates, and deletes within the same transaction;

• Typical causes: Insufficient concurrency on the primary node and the standby added later; large transactions (2 I/Os, sent only after writing).

• High recovery time: first the storage engine recovery, then MySQL server layer Binlog recovery

• • Scanning the last Binlog file may be very large.

And if we use MySQL MGR or Galera for high availability, we may encounter the problem of excessive overhead from multi-write conflict detection in the cluster, specifically:

• The table must have a primary key or a unique index, which will be used as the id after hashing to identify the row.

• A collection of row IDs that are added, deleted, or modified during the transaction execution;

• Before submitting the transaction, first send the collection of IDs for additions, deletions, and modifications to other nodes, and check for conflicts.

• • If there are conflicting transactions submitted concurrently, the transaction is rolled back, wasting computational overhead.

• If there is no conflict, distribute the transaction's Binlog through the Raft protocol;

• Significant performance overhead: a large number of concurrently submitted transactions queue and wait;

• Holding locks to complete the above operations causes a large number of concurrent running conflicting transactions to be blocked.

Therefore, in order to solve the problems described above, Klustron has teamed up with Huawei to launch the new Kunlun Stems solution. Its main technical points are:

• Both Klustore instances have their own buffer pool and redo log, sharing storage, similar to the Oracle RAC architecture;

• Concurrency control;

• • Page-level transaction row locks perform concurrency control for insert, delete, and update operations;

• Perform select concurrency control based on MVCC.

• Avoided the huge performance loss of conflict detection during transaction commit;

• During transaction runtime, redo logs are written, and commits are done in parallel without queuing.

• Resolved the consistency issue of MySQL InnoDB when performing select -> update in the same transaction.

• Assigned to Santen nodes by page hash, both nodes can be updated.

• crash recovery: simultaneous parallel recovery of two nodes;

• No PK or UK required for the table, to avoid performance bottlenecks from transaction queuing and long-lasting locks blocking a large number of conflicting transactions during submission.

• Update operations that can merge connection points generate Binlog for use by MySQL Binlog ecosystem tools.

As we introduced in the introduction, Kunlun Santian combines the powerful computing layer performance of the Kunlun database with the excellent storage processing capabilities of the Santian engine. Therefore, before taking a detailed look at the Kunlun Santian database, let's first briefly introduce the Kunlun KlustronDB distributed database and Huawei Santian engine.

First, a brief introduction to the core architecture of KlustronDB:

KlustronDB distributed computing-storage separation architecture

• Computing Layer (KlustronDB_server): Kluscomp instances composed of multiple PostgreSQL instances are responsible for accepting connection requests from verified application software clients, as well as receiving SQL query requests from established connections, executing the requests, and then returning the query results;
• Storage Layer (KlustronDB_storage): A storage shard (storage shard, abbreviated as shard) is composed of three or more MySQL 8.0 instances forming Klustore instances, and each shard stores a portion of user tables or table partitions;
• The metashard stores the metadata of the KlustronDB cluster, including the topology, node connection information, DDL logs, commit logs, and other cluster management logs.
• The cluster_mgr cluster is responsible for maintaining the correct cluster and node states, implementing functions such as cluster management, cluster logical backup and recovery, cluster physical backup and recovery, and horizontal elastic scaling.

Next, let's briefly learn about Huawei's Santen Engine:

Santen Engine is a storage engine launched by Huawei Storage, based on shared storage and supporting multi-point writing. The main features of Santen Engine are:

• Using global distributed caching technology to ensure cross-node data consistency, achieving a multi-read, multi-write cluster;
• Distributed MVCC technology greatly improves MVCC transaction commit performance;
• Using multi-active cluster high availability technology to achieve rapid node failure switching.

As shown in the figure below, the difference between the Pantheon engine on the right and the traditional MySQL one-master-two-slave architecture is that it supports dual-master concurrent writes and high availability through various internal management modules (cluster management, resource management, cache management, lock resource management, etc.).

02 The Architecture and Core Technologies of the Kunlun Santian Database

Architecture and Advantages of the Kunlun Sentian Database

• The Klustore instances of the Kunlun Santian database use the Santian engine.

• • No longer use other storage engines, including InnoDB; no need to enable Binlog, but Binlog can be generated;

• The catalog contains the Pantheon engine;

• Fully retain MySQL's SQL syntax, connection protocol, and transaction processing capabilities;

• Data is stored in multiple copies in the Santan storage system. Based on its high availability, the storage system can be deployed across multiple data centers;

• Santian can generate Binlogs that merge data updates from two master nodes.

• Ultimate performance.

• • Klustore instances do not need to output the Binlog, greatly improving the concurrency and performance of transaction processing;

• Avoided issues such as inherent master-slave delays in MySQL Binlog replication under special circumstances;

• Huawei high-performance storage system with high IO bandwidth and low latency: the fundamental guarantee of performance;

• Dual master nodes, better read-write scalability, transparent read consistency;

• Shorter RTO time (Samtian Engine high-speed parallel recovery vs. MySQL two-layer serial recovery)

Applicable Scenarios of the Kunlun Rise Database

The Kunlun Sentinel Database has the following characteristics:

• Achieve high performance with fewer constraints.

• • Long transactions or large transactions often occur;

• Frequently query and update tables without primary keys and unique indexes;

• Occasionally complex queries (multi-table joins, subqueries, window functions).

• Better write scalability when rewriting load.

• At the same time, achieving read consistency and read scalability.

• • MySQL read-write separation (reading from the replica) cannot achieve read consistency non-invasively.

• High-performance OLAP for BI and reporting workloads (dependent on KlustronDB Tornado).

• More reliable HA, the standby machine will not expire or fail due to master-slave delay.

• Faster crash recovery is needed (shorter RTO).

• Users who originally used Oracle RAC need a shorter RTO.

In reality, due to all of the above characteristics, the Kunlun Centenarian Database can break various restrictions and has broader support for scenarios under various conditions.

Kunlun Sentian Database - Advantages and Value:

• Fully retains KlustronDB's powerful features and ecological compatibility.

• • The ecological compatibility of KlustronDB_server fully utilizes PostgreSQL community technology resources;

• Supports JSON and vector (pgvector) data management;

• Supports both PostgreSQL and MySQL dual protocols and dual syntax;

• Supports machine learning (PostgresML, Python stored procedures with ML libraries);

• Various FDWs access various external data sources (object storage, databases);

• Advanced SQL features (MySQL does not have items 2 and 3);

• Stored procedures, triggers, [materialized] views, CTE;

• Multi-level fine-grained access control, data validation & constraints;

• OLAP-related query functions (window function, grouping-sets, cube, roll-up, etc).

• The high-performance distributed query processing technology system of KlustronDB_server.

• • Fully utilize hardware performance.

03 Kunlun Towering Database High-Performance Query Processing Technology

3.1 Distributed Parallel Query Execution Technology

In the computing layer of the Kunlun Santian database, we use the same computing layer architecture as the distributed database KlustronDB, providing distributed parallel query technology based on multiple Kluscomp instances.

The infrastructure for this distributed query, in detail, mainly has the following key points:

• Within the Kluscomp instance --- inherits from PostgreSQL's parallel query architecture.

• • Multi-process architecture (leader, worker) and Gather mechanism;
  • Parallel Append, Parallel Aggregate, Parallel Join；
  • Remote Plan；

• Parallelism within Kluscomp instances fully utilizes the existing parallel query framework.

• Parallel query capability between Kluscomp instances and Klustore instances -- newly designed and developed by KlustronDB.

• • It can work independently of Kluscomp instances as a parallel query framework, or collaborate with them.

• Parallel Remote Scan: Dividing into multiple processes and executing Remote Scan sub-tasks in parallel;

• Use multiple standby machines to enhance the scalability of read-only queries.

• Reduce query overhead;

• Support functions of Klustore instances -- KlustronDB completely redesigned and developed;

• Executing query subtasks of the same client connection across multiple backend connections (threads);

• Transaction snapshot sharing technology.

• Asynchronous communication between Kluscomp instances and Klustore instances.

• • Send SQL statements asynchronously on multiple connections without blocking to wait for query results;
• • Can be used to drive multiple shards to execute write commands (update, delete, insert) in parallel;

• Can be used to execute read-only queries (select) in parallel on a single or multiple nodes.

• Asynchronously send remote plan SQL statements.

• • Supported plan nodes: MergeAppend, Append, ModifyTable;

• After all are sent, wait together to collect the remote plan query results;

• Parallel Remote Scan: Read-only query task decomposition and multithreaded execution;

• Divide the remote scan task into multiple sub-tasks, each sub-task returning a sub-result set

• • Use the primary key or unique index for range partitioning;

• Asynchronously send across multiple connections so that Klustore instances can execute in parallel;

• Multiple worker processes can independently decompose tasks, send tasks, and collect and aggregate sub-results.

• KlustronDB version 1.2 and above are supported.

• Read-write separation, multiple standby machines execute SELECT.

• • Avoid affecting the performance of the primary node;

• Implement parallelism among multiple replicas in the same shard.

• Prepared statements are used between Kluscomp instances and Klustore instances.

• • Applicable for clients sending prepared statements or plain text statements;

• Avoid repeatedly parsing Klustore instances and optimizing query subtasks.

• Optimize LIMIT ... OFFSET performance: use fetch(N) to incrementally stream query results.

• • Stop midway to avoid unnecessary query execution and result transmission;

• Avoid reading and writing temporary tables to prevent a sudden increase in I/O load.

• Support functions of Kunlun-storage.

• • Multiple backend connections share the same snapshot.
• • Multiple threads execute multiple query sub-tasks in parallel, ensuring the consistency of query results within the same node.

• Use server-side read-only cursors to fetch results;

• • Avoid having Kluscomp instances materialize query results into temporary tables.

• Avoid caching query results in temporary tables on Klustore instances.

• Between fetches, other queries or other prepared statement fetches can be executed.

Next, we will illustrate parallel querying with a simple example:

Taking the two tables on the left of the above figure as examples for two join queries, the right side shows the execution plan when parallel query is not enabled. We can see that it simply splits the original query into several sub-queries according to the shards to execute them separately. The two queries require 250 and 247 milliseconds respectively.

After enabling parallel query, three worker threads simultaneously execute the execution plan, greatly improving execution efficiency. The query times are reduced to 92 and 71 milliseconds, less than half of the time of non-parallel execution, resulting in a very significant performance improvement.

3.2 JIT Code Just-In-Time Compilation Technology

Next, we will introduce the JIT code just-in-time compilation technology of the Kunlun Centian database.

First, let's look at why we adopt JIT technology. Everyone can take a look at the picture below.

From the flame graph above, we can see that the part of a simple query that executes expressions occupies a very high proportion of the entire SQL execution process. Therefore, if there is a way to improve the execution efficiency of expression calculations, it can greatly accelerate the SQL execution process. And this is exactly what Just-In-Time (JIT) compilation can provide us.

In the diagram below, we can see the difference between JIT and regular code execution.

JIT (Just-in-time compilation) is a technology in which code is compiled during runtime. It can analyze the code during execution and identify parts of the code that can be accelerated by immediate compilation. In these parts, the performance improvement brought by compilation or recompilation exceeds the overhead of compiling the code, thereby saving a significant amount of resources consumed during function calls and greatly improving code execution efficiency.

In the above figure, the left side shows the traditional code execution process, which requires three function calls to perform a simple expression calculation. However, after JIT analysis, this expression calculation can be completed with just a few simple assembly instructions. Therefore, JIT compiles this expression calculation into more efficient and simpler machine instructions, thereby speeding up the execution process.

The figure below shows a comparison of the SQL execution process before and after using JIT technology.

Currently, the data types supported by JIT in the Kunlun Santen database include:

• int, bigint, float, double, decimal
• date, time, datetime, timestamp
• varchar

Supported operators include:

• binary algebras：Plus , Minus , Multiply , Div
• binary comparators：GT , GE , LT , LE , EQ , NE, like
• logical operators：And , Or , Not
• unary operators：Abs
• others：Between . . .and , interval

After adopting JIT technology, TPC-H performance improved by 20% to 60%, and in the future, it could achieve a 10-fold performance increase! Moreover, both Kluscomp instances and Klustore instances have JIT capabilities.

3.3 Tornado High-Performance Vectorized Execution Engine

The Tornado high-performance vectorized execution engine is KlustronDB's latest SQL execution engine. It greatly accelerates the efficiency of SQL statement execution through vectorized execution technology.

First, let's take a look at what vectorization is. Vectorization is based on SIMD (Single Instruction Multiple Data) technology, which accelerates the execution of code by processing different data with the same instruction in batches.

History of SIMD

• 20First cited in the 1970s for the ILLIAC IV massively parallel computer;

• 20In fact, it was widely applied to consumer-grade computers in the 1990s:

• • 1996In the year, Intel launched the MMX instruction set extension for X86;

• 1999In [year], Intel launched the SSE instruction set extension, which fully covers MMX.

• ... Instruction set extensions such as SSE3, SSE4, AVX, and AVX2.

From the figure below, we can get a simple understanding of how SIMD processes the a b operation in batches.

At present, many analytical databases adopt vectorized execution technology, especially analytical databases such as ClickHouse, DukeDB, etc., which greatly accelerate the execution process of analytical computations.

The current architecture of Zeto's Tornado vectorized execution engine is as shown in the figure below:

First, Tornado needs to convert the data into columnar storage in memory, and then execute it in a pipeline manner using vectorized functions and operators.

In the figure below, we can compare the execution process of Tornado.

Next, let's look at an actual processing procedure through an example.

In the example above, Tornado first converts the values of a and b into column storage, then performs the operation on a and b in batches, and finally executes this SQL statement through the filter and aggregate operators.

In the image below, we can see how vectorized physical operators work using SIMD instructions.

04 Q & A

Q: Can the Kluscomp instances of the Kunlun Shentian database be multiple? What about the Klustore instances?

A: Yes, there can be multiple Kluscomp instances, but currently there can only be two Klustore instances. This is because, at present, the Santen engine only supports dual masters. In other words, the number of Kluscomp instances can be configured as needed, but there are only two Klustore instances, and both Klustore instances can write simultaneously.

END