Introduction to KlustronDB Single Shard Feature
Introduction to KlustronDB Single Shard Feature
1 KlustronDB single-shard cluster application scenarios
For small to medium-sized applications, the KlustronDB single-shard architecture can be used to reduce the cost of system software and hardware. Compared with the MySQL master-slave architecture, the KlustronDB single-shard architecture not only improves system reliability, but also leverages KlustronDB's powerful computational and analytical capabilities to meet both OLTP and OLAP load requirements within one system. For the advantages of using a KlustronDB single-shard cluster, especially the important role of kunlun_proxysql, see this article.
2 KlustronDB single shard architecture
3 Configuration and Use
3.1 Use the cluster_mgr API to install a single-shard cluster
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"create_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"nick_name":"my_nick_name",
"ha_mode":"mgr",
"shards":"1",
"nodes":"3",
"comps":"1",
"max_storage_size":"20",
"max_connections":"6",
"cpu_cores":"8",
"innodb_size":"1024",
"dbcfg":"1",
"fullsync_level":"1",
"install_proxysql":"1",
"computer_user":"abc",
"computer_password":"abc"
}
}
' -X POST http://127.0.0.1:35001/HttpService/Emit
If the install_proxysql parameter is passed in as 1, cluster_mgr will automatically install the cluster in single shard mode.
3.2 Install a single-shard cluster using the Xpanel interface
Just choose to install kunlun-proxysql.
Precautions
- If you choose to install kunlun-proxysql, the cluster will be in single shard mode. In KlustronDB-1.1 version, adding or removing shards is not supported. In future versions, more storage shards can be added as needed, provided that:
- During the period when the user is using the single shard mode, they have been using the Kluscomp instance (KlustronDB_server) to execute DDL statements; or starting from version 1.3.2, if DDL is executed through kunlun_proxy, it is necessary to manually execute the DDL statements again through the Kluscomp instance, following the method in item 8 below.
- Users must ensure that once the cluster has more than one shard, kunlun-proxysql is no longer used. If it is used, some data will be inaccessible.
During installation, pass in computer_user/computer_password as the username/password to log in to kunlun-proxysql.
After successful installation, log in to the metashard kunlun_metadata_db/proxysql_nodes to view the machines and ports of the current cluster kunlun-proxysql.
Execute DDL statements through the Kluscomp instances of KlustronDB (KlustronDB_server), and do not execute DDL statements via kunlun-proxysql. This way, the Kluscomp instances are aware of the user-defined tables, allowing subsequent OLTP and OLAP SQL statements to be executed on these tables. If the DDL is in MySQL syntax, starting from KlustronDB version 1.2, you can directly connect to the Kluscomp instances using the MySQL protocol to execute MySQL DDL syntax; before KlustronDB 1.2, you need to use [the old method]](ddl2_kunlun.md).
When accessing data tables through kunlun-proxysql, the names of the databases where these tables are located are in the form of dbname_$$_schema_name. At this time, SQL statements need to use this kind of database name. This is because the names of databases created by KlustronDB_server on KlustronDB_Klustore instances are all in this format, where dbname is the database name in KlustronDB_server; schema_name is the name of the schema within it.
When installing a cluster, metashard shards and storage shards can be merged, using the same kunlun-storage shard, thereby further reducing resource overhead. That is, in a single-shard cluster configuration file, the same shard is used as both the metashard and the storage shard. Note that in this configuration, enabling Global MVCC is prohibited, and Global MVCC functionality is completely unnecessary for single-shard deployments.
Users can attach a KlustronDB cluster to an existing MySQL primary node to run as a standby machine, continuously replicating its data changes, and use KlustronDB's Kluscomp instances to execute OLAP data analysis SQL statements. This provides better performance than using a MySQL standby node for analysis. This KlustronDB cluster can be a single-shard cluster or a standard KlustronDB distributed cluster. For detailed steps, see this article.
Starting from version 1.3.2, users can directly connect to kunlun-proxy to execute DDL statements. After making a simple configuration on the Kluscomp instance (setting
skip_storage_ddl=true), the DDL statements can be executed again on the Kluscomp instance so that the Kluscomp instance can obtain the relevant metadata information. This allows the Kluscomp instance to read and write to the table.
When skip_storage_ddl=true, the Kluscomp instance will not execute the DDL as if sending the DDL statement to the Klustore instance because the database object (such as a table) already exists on the Klustore instance.
Because databases and their tables are directly created through kunlun_proxysql, users usually directly create databases like mydb, without considering creating similar ones by accessing Klustore instances through Kluscomp instances.
pgdb_$$_pgschema
the database. Therefore, in order to access this table on the Kluscomp instance, KlustronDB supports users creating a mapping relationship to achieve this. To do this, users need to establish a mapping from the Kluscomp instance's database and schema to the Klustore instance's database by inserting a row into the Kluscomp instance's metadata table pg_database_name_map. As described in this section, the inserted row should be ('pgdb', 'pgschema', 'mydb'), which allows the Kluscomp instance's pgdb.pgschema schema to be mapped to the Klustore instance's mydb database. Then, you can connect to the Kluscomp instance's pgdb database, execute USE pgschema to switch to pgschema, and then read and write tables in mydb on the Klustore instance.
For example, we first create table t1 in the mydb database on the Klustore instance, then connect to the pgdb database on the Kluscomp instance, and then execute USE pgschema; in the connection to switch the current path to pgschema. After that, we insert a row ('pgdb', 'pgschema', 'mydb') into the pg_database_name_map table on the Kluscomp instance. Then, execute set skip_storage_ddl=true, and in the same session, execute the CREATE TABLE statement for t1. After that, you can read and write the t1 table in any connection on the Kluscomp instance. If you need to ALTER TABLE t1 afterwards, it is best to operate through the Kluscomp instance (and be sure to set skip_storage_ddl=false in the session); if you execute ALTER TABLE t1 via kunlun_proxysql, then you also need to perform the same ALTER TABLE action on the Kluscomp instance after setting skip_storage_ddl=true again, in order to keep the metadata consistent between the Kluscomp instance and the Klustore instance.