Global MVCC mechanism
Global MVCC mechanism
Introduction
As a distributed database that can fully support strong consistency scenarios such as finance and securities, KlustronDB's global read consistency of data is an indispensable necessary feature. Global MVCC is a global consistency mechanism designed to solve the issue of read consistency in distributed environments. It obtains the snapshot of the current transaction by setting a global data version number for distributed transactions, thereby achieving global data read consistency.
KlustronDB's Global MVCC feature creates a snapshot of the current transaction by setting a global data version number, thereby achieving global data read consistency.
Why is Global MVCC needed?
Let's first take a look at the read consistency issues of distributed transactions, as shown in the figure below.
Distributed transaction GT1 being submitted
- Write into multiple shards (shard1 GT1.t1 & shard2 GT1.t2)
Submit in two stages
Running SELECT (GT2)
- Read the update of GT1.t1 in shard1
Failed to read the update of GT1.t2 in shard2
This has resulted in an inconsistency where only part of the transaction data can be read.
To solve this problem, KlustronDB implements Global MVCC, the principle of which is mainly to obtain the visible data of the current transaction by establishing a global snapshot.
03 Global MVCC Principles and Implementation
The implementation of Global MVCC requires corresponding modifications at the upper-level Kluscomp instances and metashard, as well as at the lower-level Klustore instances.
First is the upper-level Kluscomp instance part, as shown in the figure below. First, it is necessary to obtain and set a global version number for all distributed transactions, and then use the global version number to establish a global snapshot.
At the lower-level Klustore instances, support for global snapshots is implemented by modifying relevant parts of the MySQL InnoDB storage engine. As shown in the figure below, we mainly modified the transaction visibility determination process of InnoDB.
Global Visibility Determination Algorithm: Only for Updates of XA Transactions
First make a partial visibility judgment
Partially visible does not necessarily mean globally visible
- Anything less than local_xmin is definitely visible
Partially invisible does not necessarily mean completely invisible
- Items that have not been started when taking a snapshot must be invisible
Global version number comparison
What should I do if it is not visible globally?
- Generate older row versions using the undo log
After completing the above changes, let's compare the differences in the process before and after the modifications. As shown in the figure below, through the global version number and the global transaction snapshot, we can avoid transaction consistency issues.
Finally, let's analyze the performance cost of Global MVCC. Because some key processes of Global MVCC incur certain time and resource costs, there will be some performance loss. According to our testing and analysis, its performance loss is between 5% - 10%, which is within an acceptable range.
Comprehensive analysis is as follows:
Kluscomp instance:
No additional existing time overhead has been added
Allocate GVNO: Issue an integer along with the XA COMMIT statement and store it in tgvc in tgvc_cache
- Negligible
Obtain global snapshot: network transmission overhead
- Each SELECT statement (RC) or each transaction (RR) obtains it once
Get the current value from the metashard sequence select currval('global_mvcc_seq')
Allocate global snapshot: deliver an integer with the SELECT statement
- Negligible
Klustore instance:
tgvc management: negligible
Global MVCC visibility determination logic: integer comparison
- A small amount of READ waits to set the global version number: wait time is usually < 20ms
Covering index lookup: page header max_trx_id: the transaction that last updated this page
- Previously: If readview can see all rows on this page (rv.m_up_limit_id > max_trx_id), then return the index row directly.
Now: The above holds, and if max_trx_id > local_xmin, a table lookup must be performed in order to search.
- The proportion of table lookups has slightly increased
purge: retain undo log until global MVCC no longer needs to be advanced by the rise of global_xmin
04 Q&A
q1: In what scenarios is it necessary to enable Global MVCC?
a1: Scenarios that require high data consistency need to enable Global MVCC, such as finance and securities. Since enabling this feature may result in a certain degree of performance loss of 5% - 10%, it is still necessary to decide whether to enable it based on your own application scenario.