KlustronDB ClusterManager API
KlustronDB ClusterManager API
0. Introduction to Use
KunlunClusterManager accepts requests via the HTTP protocol and executes corresponding tasks based on the type and parameters of the request. The interface response modes are divided into synchronous and asynchronous modes. In synchronous mode, after the client sends a request, the received response contains the execution result of the current interface (for example, when obtaining cluster information). Asynchronous mode is usually for tasks that take a longer time to execute. After sending a request, the client needs to use the get_status interface to obtain the execution status of the task, until the task succeeds or fails.
0.1 Assign Tasks
Interface Description
Assign cluster management tasks.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| job_id | string | 0 | For issuing tasks, no need to fill in job_id. This job_id will be automatically generated by clustermanager and returned in the response. |
| job_type | string | 1 | Task type, related to the specific task content |
| timestamp | string | 1 | Timestamp when the task is initiated |
| paras | json | 0 | Custom parameter section, related to the task type |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| error_code | string | 1 | Error code |
| error_info | string | 1 | Error information |
| status | string | 1 | Task status, possible values are accepted, started, ongoing, done, failed |
| job_id | string | 0 | job_id automatically generated by clustermanager. If the task type is an asynchronous task, the task progress and status can be queried using this job_id. |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"expand_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"dst_shard_id":"2",
"src_shard_id":"1",
"table_list":[
"sbtest.public.test1"
]
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
0.2 Get Task Execution Status
Interface Description
Get the execution status of the specified job_id task.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| job_id | string | 1 | ID of the task to be queried |
| job_type | string | 1 | Task type, current interface 'get_status' |
| timestamp | string | 1 | Timestamp when the task was initiated |
| paras | json | 0 | Custom parameter section, not required for get_status type interface, may be required for other task types. |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| error_code | string | 1 | Error code |
| error_info | string | 1 | Error information |
| status | string | 1 | Task status, possible values are started, ongoing, done, failed |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"expand_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"dst_shard_id":"2",
"src_shard_id":"1",
"table_list":[
"sbtest.public.test1"
]
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
1. Equipment Management
1.1 Reporting Machinery and Equipment
Interface Description
Register the initialized server device into the system.
Response mode
Asynchronous. Use the get_status interface to check the status.
Request parameters
| 参数名称 | 参数类型 | 是否必填 | 备注 |
|---|---|---|---|
| job_type | string | 1 | create_machine |
| hostaddr | string | 1 | 服务器 IP |
| machine_type | string | 1 | 设备类型。 设备类型包括两种 storage 和 computer。 分别代表了存储机型和计算机型。存储机型表示，该设备可以被用来部署 KlustronDB_storage。相同的，计算机型代表着该设备可以被用来部署 KlustronDB_server。同一台物理设备，其类型可以是存储设备或者计算设备，也可以既是存储设备，也是计算设备，这种情况下，端口范围需要明确的区分开，确保在同一台设备上安装的计算节点和存储节点不会端口冲突。 |
| port_range | string | 1 | 端口分配范围。即新创建的实力端口在该参数指定的范围内分配 |
| rack_id | string | 1 | 机架信息 |
| datadir | string | 1 | 实例数据目录存放位置 |
| logdir | string | 1 | 日志目录存放位置 |
| wal_log_dir | string | 1 | wal 日志存放位置 |
| comp_datadir | string | 1 | 如果当前设备为计算机型，则该字段标识计算节点的数据目录 |
| total_mem | string | 1 | 该机型可用总内存 单位为 MB |
| total_cpu_cores | string | 1 | 该机型可用总的逻辑核心数 |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"create_machine",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"hostaddr":"192.168.0.135",
"machine_type":"computer",
"port_range":"45000-48000",
"rack_id":"1",
"datadir":"/home/play_ground/run_data",
"logdir":"/home/play_ground/run_data",
"wal_log_dir":"/home/play_ground/run_data",
"comp_datadir":"/home/play_ground/run_data",
"total_mem":"8192",
"total_cpu_cores":"8"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"create machine successfully",
"status":"done",
"attachment":{
}
}
1.2 Remove Device
Interface Description
Perform the delete operation on servers that have already been registered in the system.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_machine |
| hostaddr | string | 1 | Machine IP |
| machine_type | string | 1 | The device model information: storage or computer |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"delete_machine",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"hostaddr":"127.0.0.1",
"machine_type":"storage"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"delete machine successfully",
"status":"done",
"attachment":{
}
}
1.3 Set Server Status
Interface Description
Set the status of a specified server, such as dead or running, which indicate that the current device is offline or online, respectively. Only online devices can be assigned to the resource pool by the system.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_machine |
| hostaddr | string | 1 | Machine IP |
| node_stats | string | 1 | running, dead |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"set_machine",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"hostaddr":"127.0.0.1",
"node_stats":"running"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"set machine success",
"status":"done",
"attachment":{
}
}
2. Cluster Purchase and Deletion
2.1 Purchase a New Cluster
Interface Description
Create a new kunlunBase cluster according to the specified parameter configuration.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | Yes | create_cluster |
| nick_name | string | Yes | Business name, e.g., 'Reconciliation and Interest Settlement Database' |
| ha_mode | string | Yes | rbr, mgr. Indicates high-availability mode. The former is in strong synchronization mode, while the latter is in mgr mode. |
| shards | string | 1 | Number of shards |
| nodes | string | 1 | Number of nodes in each shard. At least three nodes, high-availability clusters that form 1 master and slave |
| comps | string | 1 | Number of compute nodes |
| max_storage_size | string | 1 | Maximum storage space |
| max_connections | string | 1 | Maximum number of connections |
| cpu_cores | string | 1 | Maximum number of cores that can be used |
| cpu_limit \isolation_node | soft string | shot | 1 |
| innodb_size | string | 1 | KlustronDB the innodb_buffer size used by _storage (this field innodb_size unit: M) |
| dbcfg | string | 0 | sets the small memory mode, 0 and without this field for normal mode, 1 for small memory mode takes the value 0 or 1. |
| fullsync_level | string | 1 | The required number of strongly synchronized standby nodes in this shard. This value must be less than or equal to the number of standby nodes. |
| storage_iplists | Json array | 0 | Specifies the hosts on which to install the storage nodes. If not specified, existing hosts will be used by default. |
| computer_iplists | Json array | 0 | Specifies the hosts on which to install the compute nodes. If not specified, existing hosts will be used by default. |
| node_distribution | Json array | 0 | Configures the distribution information for storage nodes. |
| comp_distribution | Json array | 0 | Configures the distribution information for compute nodes. |
| install_proxysql | string | 0 | Whether to install ProxySQL during cluster deployment. If ProxySQL is installed, the cluster will be in single-shard mode. The default value is 0. |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"create_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"nick_name":"my_nick_name",
"ha_mode":"mgr",
"shards":"1",
"nodes":"3",
"comps":"1",
"max_storage_size":"20",
"max_connections":"6",
"cpu_cores":"8",
"innodb_size":"1024",
"rocksdb_block_cache_size_M":"1024",
"dbcfg":"1",
"fullsync_level":"1",
"install_proxysql":"0",
"storage_iplists":[
"192.168.0.2",
"192.168.0.3"
],
"computer_iplists":[
"192.168.0.1"
]
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"create cluster successfully",
"status":"done",
"attachment":{
"cluster_id":"3",
"cluster_name":"cluster_1651903646_000001",
"shards":"1",
"comps":"1"
}
}
2.2 Adding New Shards Within the Cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, new shards can be added, and multiple shards can be added.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | 1 | add_shards |
| cluster_id | string | 1 | Cluster ID |
| shards | string | 1 | Number of shards to add |
| nodes | string | 1 | Number of nodes per shard |
| storage_iplists | Json array | 1 | Machines specified for installing storage nodes |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"add_shards",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"shards":"2",
"nodes":"3",
"storage_iplists":[
"192.168.0.2",
"192.168.0.3"
]
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"add shards successfully",
"status":"done",
"attachment":{
"cluster_id":"2",
"cluster_name":"cluster_1651836141_000001",
"shards":"2",
"comps":"1",
"list_shard":[
{
"shard_id":"3",
"shard_name":"shard2"
}
]
}
}
2.3 Delete existing shards within the cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, shards can only be deleted one by one.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_shards |
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 1 | ID of the shard to be deleted |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"delete_shard",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"shard_id":"1"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"delete shard successfully",
"status":"done",
"attachment":{
}
}
2.4 Adding New Compute Nodes within the Cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, new shards can be added, and multiple shards can be added.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | add_comps |
| cluster_id | string | 1 | Cluster ID |
| comps | string | 1 | Number of new compute nodes |
| computer_iplists | Json array | 1 | Machines for installing specified compute nodes |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"add_comps",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"comps":"2",
"computer_iplists":[
"192.168.0.2",
"192.168.0.3"
]
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"add comps successfully",
"status":"done",
"attachment":{
"cluster_id":"3",
"cluster_name":"cluster_1651903646_000001",
"shards":"1",
"comps":"2",
"list_comp":[
{
"comp_id":"3",
"comp_name":"comp2"
}
]
}
}
2.5 Deleting compute nodes within the cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, deleting compute nodes can only be done one by one.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_shards |
| cluster_id | string | 1 | Cluster id |
| comp_id | string | 1 | ID of the computing node to be deleted |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"delete_comp",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"comp_id":"1"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"delete comp successfully",
"status":"done",
"attachment":{
}
}
2.6 Specify cluster to add node
Interface Description
Add nodes to all shards of the cluster according to the specified parameter configuration, asynchronously, requires querying.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | 1 | add_nodes |
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 0 | If this field is specified, nodes will be added only to this shard |
| nodes | string | 1 | Number of nodes to add |
| storage_iplists | JSON array | 1 | Machines designated for installing storage nodes |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"add_nodes",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"shard_id":"1",
"nodes":"2",
"storage_iplists":[
"192.168.0.2",
"192.168.0.3"
]
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"add new nodes successfully",
"status":"done",
"attachment":{
"cluster_id":"1",
"cluster_name":"cluster_1651907758_000001",
"shards":"1",
"comps":"1",
"list_shard":[
{
"shard_id":"1",
"shard_name":"shard1",
"nodes":"5",
"list_node":[
{
"hostaddr":"192.168.0.127",
"node_id":"4",
"port":"58509"
},
{
"hostaddr":"192.168.0.127",
"node_id":"5",
"port":"58512"
}
]
}
]
}
}
2.7 Specify cluster to delete node
Interface Description
According to the specified parameter configuration, delete the specified node from the cluster's designated shard asynchronously, query needed.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_nodes |
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 0 | If this field is specified, nodes will only be added to this shard |
| hostaddr | string | 1 | IP of the machine where the node to be deleted is located |
| port | string | 1 | Port number |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"delete_node",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1",
"shard_id":"1",
"hostaddr":"127.0.0.1",
"port":"57338"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"delete node successfully",
"status":"done",
"attachment":{
}
}
2.8 Redo the backup machine
Interface Description
Designate a backup machine to redo a certain shard.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | Yes | delete_nodes |
| cluster_id | string | Yes | Cluster ID |
| shard_id | string | No | If specified, nodes will be added only to this shard |
| hostaddr | string | Yes | IP address of the machine where the node to be deleted resides |
| port | string | Yes | Port number |
| need_backup | string | Yes | Whether a backup is required before redoing |
| hdfs_host | string | Yes | Cold backup location |
| pv_limit | string | Yes | Cold backup transfer rate limit, in MB |
| allow_pull_from_master | string | Yes | Whether to use data from the primary node to redo the standby node if the standby is unavailable |
| allow_replica_delay | string | Yes | Maximum allowable delay for the standby node; if the delay exceeds this value, the standby node will be considered unsuitable as a data source |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"rebuild_node",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"shard_id":"1",
"cluster_id":"1",
"rb_nodes":[
{
"hostaddr":"192.168.0.125",
"port":"57001",
"need_backup":"0",
"hdfs_host":"hdfs",
"pv_limit":"10"
}
],
"allow_pull_from_master":"1",
"allow_replica_delay":"15"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"version":"1.0",
"error_code":"0",
"error_info":"delete node successfully",
"status":"done",
"attachment":{
}
}
2.9 Remove the specified cluster
Interface Description
According to the specified parameter configuration, delete the specified node from the cluster's designated shard, asynchronously, requires querying.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_cluster |
| cluster_id | string | 1 | Cluster ID |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"delete_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
#查询接口get_status返回
{
"attachment": {
"cluster_id": "43" ,
"cluster_name": "cluster_1665971623_000043" ,
"computer_id": "776" ,
"computer_state": "done" ,
"computer_step": [
{
"cluster_id": "43" ,
"computer_hosts": "192.168.0.132_53704;" ,
"computer_id": "778" ,
"computer_state": "done" ,
"error_code": 0 ,
"error_info": "OK" ,
"rollback_flag": "1"
}
],
"job_steps": "shard,computer" ,
"shard_id": "775" ,
"shard_step": [
{
"cluster_id": "43" ,
"error_code": 0 ,
"error_info": "OK" ,
"rollback_flag": "1" ,
"shard_ids": "83," ,
"storage_hosts": "192.168.0.132_53407;192.168.0.132_53411;" ,
"storage_id": "777" ,
"storage_state": "done"
}
],
"storage_state": "done"
},
"error_code": "0" ,
"error_info": "OK" ,
"job_id": "" ,
"job_type": "" ,
"status": "done" ,
"version": "1.0"
}
2.10 Modify instance resource isolation parameters
Interface Description
According to the specified parameter configuration, modify the resource isolation parameters of the MySQL or PostgreSQL instance. Currently, only CPU isolation is supported.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | Yes | update_instance_cgroup |
| ip | string | Yes | Instance IP address |
| Port | string | Yes | Instance listening port |
| type | string | Yes | “mysql” indicates a storage node instance; “pg” indicates a compute node instance |
| cpu_cores | string | Yes | Number of CPU cores allocated |
| cgroup_mode | string | Yes | CPU resource limitation mode; the default is “quota”. It can be set to either “quota” or “share”, where “share” represents soft isolation and “quota” represents hard isolation. |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"update_instance_cgroup",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"ip":"192.168.0.135",
"port":"57001",
"type":"mysql",
"cpu_cores":"5",
"cgroup_mode":"quota"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
3. Cluster Backup
3.1 Manually Initiate Cluster Backup
Interface Description
Call this interface to manually initiate a full backup. When cluster_id is -1, it indicates backing up the metashard.
Response mode
Synchronous interface.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | Cluster ID |
Example
#请求
curl -d '
{
"version":"1.0",
"job_id":"",
"job_type":"manual_backup_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"cluster_id":"1"
}
}
' -X POST http://192.168.0.135:35001/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
4. Cluster Recovery
4.1 Cluster Rollback (Restore Cluster by Time Point)
Interface Description
Perform a rollback on the specified cluster according to the point in time.
Response mode
Asynchronous. Obtain task progress through the get_status interface.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| src_cluster_id | string | Yes | ID of the source cluster for the rollback operation |
| dst_cluster_id | string | Yes | ID of the target cluster for the rollback operation. This cluster must be a newly purchased empty cluster |
| restore_time | string | Yes | Rollback time point |
Example
Request
{ "version":"1.0", "job_id":"", "job_type":"cluster_restore", "timestamp":"1435749309", "paras":{ "src_cluster_id":"1", "dst_cluster_id":"2", "restore_time":"restore_time" } } -X POST http://192.168.0.135:35001/HttpService/Emit
Response
5. Cluster Expansion
5.1 Automatically Obtain Table Migration List
Interface Description
Get the list of tables automatically selected by the system for migration.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 1 | Source shard ID of the table to be migrated within the current cluster |
| policy | string | 1 | top_hit: select the table with the highest access frequency. top_size: select the table with the largest data volume |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| attachment->table_list | string | List of tables to be migrated |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_expand_table_list", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"3", "shard_id":"2", "policy":"top_size" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "table_list":"kunlundb_super_dba.kunlundb_super_dba_$$_public.test1" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
5.2 Initiate Table Migration Task
Interface Description
Initiate table migration task.
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | expand_cluster |
| cluster_id | string | 1 | Cluster ID |
| src_shard_id | string | 1 | Source shard ID of the table to be migrated in the current cluster |
| dst_shard_id | string | 1 | Target shard ID of the table to be migrated in the current cluster |
| table_list | json_array | 1 | List of tables to be migrated |
| drop_old_table | string | 0 | Whether to retain the source table |
Example
Request
curl -d ' { "version": "1.0", "job_id":"", "job_type": "expand_cluster", "timestamp" : "1435749309", "user_name":"kunlun_test", "paras": { "cluster_id": "1", "dst_shard_id": "5", "src_shard_id": "6", "table_list": [ "postgres.public.test2" ] } } ' -X POST http://192.168.0.135:8010/HttpService/Emit
Response
6. Metadata Management
6.1 Obtaining metashard Synchronization Mode
Interface Description
Query the metashard synchronization mode.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_meta_summary |
Response Parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| attachment->mode | string | ||
| metashard synchronization mode. Three types of results: no_rep, mgr, rbr |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_meta_mode", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "mode":"mgr" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.2 Obtaining Cold Backup Storage Service Information
Interface Description
Inquire about the detailed information of cold backup storage services.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_backup_storage |
Response Parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| hostaddr | string | ||
| Backup service IP | |||
| name | string | ||
| Backup service name | |||
| port | string | ||
| Backup service port | |||
| stype | string | ||
| Backup service type |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_backup_storage", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "list_backup_storage":[ { "hostaddr":"192.168.0.135", "name":"hdfs_backup1", "port":"9000", "stype":"HDFS" } ] }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.3 Obtaining Machine Status Information
Interface Description
Query detailed information of all machine statuses.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_machine_summary |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| hostaddr | string | ||
| Server Address | |||
| status | string | ||
| Server Status |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_machine_summary", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "list_machine":[ { "hostaddr":"192.168.0.135", "status":"online" }, { "hostaddr":"192.168.0.125", "status":"online" } ] }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.4 Get Cluster List
Interface Description
Query the list of all kunlunbase clusters.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_cluster_summary |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| name | string | Cluster Name | |
| nick_name | string | Business Name | |
| shards | string | Number of Shards | |
| comps | string | Number of Compute Nodes |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_machine_summary", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "list_machine":[ { "hostaddr":"192.168.0.135", "status":"online" }, { "hostaddr":"192.168.0.125", "status":"online" } ] }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.5 Getting Cluster Details
Interface Description
Get detailed information of the specified cluster.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_cluster_detail |
| cluster_name | string | 1 | Cluster Name |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| name | string | Cluster name | |
| nick_name | string | Business name | |
| shards | string | Number of shards | |
| comps | string | Number of compute nodes | |
| comp_name | string | Compute node name | |
| hostaddr | string | IP of compute or storage node | |
| port | string | Port of compute or storage node | |
| status | string | Current node status | |
| master | string | Whether the current storage node is the master node |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_cluster_detail", "timestamp":"1435749309", "paras":{ "cluster_name":"cluster_1659333382_000001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "cluster_name":"cluster_1659333382_000001", "comps":"1", "list_comp":[ { "comp_name":"comp_1", "hostaddr":"192.168.0.135", "port":"45001", "status":"online" } ], "list_shard":[ { "list_node":[ { "hostaddr":"192.168.0.135", "master":"true", "port":"55001", "status":"online" }, { "hostaddr":"192.168.0.135", "master":"false", "port":"55002", "status":"online" }, { "hostaddr":"192.168.0.135", "master":"false", "port":"55003", "status":"online" } ], "nodes":"3", "shard_name":"shard_1" } ], "nick_name":"my_nick_name", "shards":"1" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.6 Obtaining Storage Node System Variable Information
Interface Description
Query the value of a certain variable in KlustronDB_storage (such as innodb_buffer_pool_size).
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_variable |
| hostaddr | string | 1 | KlustronDB_storage IP |
| port | string | 1 | KlustronDB_storage port |
| variable | string | 1 | Variable name |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| result | string | ||
| Query success or not | |||
| value | string | ||
| Query result |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_variable", "timestamp":"1435749309", "paras":{ "hostaddr":"192.168.0.135", "port":"55001", "variable":"innodb_buffer_pool_size" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "result":"true", "value":"5242880" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
7. Table Redistribution
Interface Description
Based on the user input table, dump the data from the table and import it into a new table
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| src_cluster_id | string | 1 | The cluster_id where the original table is located |
| dst_shard_id | string | 1 | The cluster_id to which the target table is redistributed |
| repartition_tables | string | 1 | The target tables are redistributed to the target cluster_id |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"table_repartition", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "src_cluster_id":"3", "dst_cluster_id":"2", "repartition_tables":"test_
$$ public.t=>test1 $$
private.t2,test
$$ priv.ta=>test1 $$
_priv1.tb" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special Note:
1.It is necessary to pre-create the database, schema, and table in the target cluster, and the structure of the original and target tables must be consistent. This cluster_mgr will automatically perform the verification. 2.For the redistribution table task, the tables that need to be done at one time must be under the same database, but the source and target databases can be different.
8. Table Logical Backup
Interface Description
The user initiates a logical backup of a specific table and sets up scheduled logical backups.
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| cluster_id | string | 1 | The cluster_id where the logical backup table is located |
| backup_type | string | 1 | Logical backup type, supports db |
| db_table | string | 1 | Table to be logically backed up |
| backup_time | string | 1 | Scheduled backup time period |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"logical_backup", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "backup_type":"schema", "backup":[ { "db_table":"postgres_
$$ public", "backup_time":"01:00:00-02:00:00" }, { "db_table":"postgres $$
_trtest", "backup_time":"01:00:00-02:00:00" } ]
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special emphasis
If backup_type is db, it means that all tables in all schemas under that db need to be backed up. Enter the db name in db_table under backup.
If backup_type is schema, it means that all tables under that schema need to be backed up. In the backup input, enter db_$$_schema under db_table.
9. Table Logical Recovery
Interface Description
The user initiates the restoration of a certain logical backup table to a specified historical point in time
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| src_cluster_id | string | 1 | The cluster_id where the backup table is located |
| dst_cluster_id | string | 1 | The cluster_id to which the table will be logically restored |
| restore_type | string | 1 | Logical restore type, supports db |
| db_table | string | 1 | The table to be logically restored |
| restore_time | string | 1 | The historical point in time to restore to |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"logical_restore", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "src_cluster_id":"1", "dst_cluster_id":"2", "restore_type":"table", "restore":[{ "db_table":"postgres_
$$ public.t1", "restore_time":"2022-10-20 19:20:15" },{ "db_table":"postgres $$
_puingblic.t2", "restore_time":"2022-10-20 19:20:15" }]
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
10. Manually trigger cluster_mgr primary-standby switch
Interface Description
The user initiates a master-slave switch operation on the current cluster_mgr cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| task_type | string | 1 | Task type, here it is transfer_leader |
| target_leader | string | 1 | The target new leader to switch to |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"raft_mission", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "task_type":"transfer_leader", "target_leader":"127.0.0.1:25001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
11. Add nodes to cluster_mgr
Interface Description
User initiates the operation to add nodes to the current cluster_mgr cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| task_type | string | 1 | Task type, here it is add_peer |
| peer | string | 1 | The cluster_mgr node to add |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"raft_mission", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "task_type":"add_peer", "peer":"127.0.0.1:25001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
12. Delete nodes for cluster_mgr
Interface Description
The user initiates the operation to delete a node from the current cluster_mgr cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| task_type | string | 1 | Task type, here it is remove_peer |
| peer | string | 1 | The cluster_mgr node to be deleted |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"raft_mission", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "task_type":"remove_peer", "peer":"127.0.0.1:25001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special emphasis
To add/delete nodes to the cluster_mgr cluster, you need to manually update the raft_group_member_init_config configuration item in the cluster_mgr configuration file and update the configuration for adding/deleting nodes =============================================================================================================================
KlustronDB ClusterManager API
0. Introduction to Use
KlustronDB ClusterManager accepts requests via the HTTP protocol and performs corresponding tasks based on the type and parameters of the request. The interface response modes are divided into synchronous and asynchronous modes. In synchronous mode, the client receives a response that contains the execution result of the current interface after making a request (for example, when obtaining cluster information). Asynchronous mode is usually used for tasks that take a long time to execute. After issuing a request, the client needs to use the get_status interface to obtain the task's execution status until the task is completed successfully or fails.
0.1 Assign Tasks
Interface Description
Assign cluster management tasks.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| job_id | string | 0 | For issuing tasks, no need to fill in job_id. This job_id will be automatically generated by clustermanager and returned in the response. |
| job_type | string | 1 | Task type, related to the specific task content |
| timestamp | string | 1 | Timestamp when the task is initiated |
| paras | json | 0 | Custom parameter section, related to the task type |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| error_code | string | 1 | Error code |
| error_info | string | 1 | Error information |
| status | string | 1 | Task status, possible values are accepted, started, ongoing, done, failed |
| job_id | string | 0 | job_id automatically generated by clustermanager. If the task type is an asynchronous task, the task progress and status can be queried using this job_id. |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"expand_cluster", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "dst_shard_id":"2", "src_shard_id":"1", "table_list":[ "sbtest.public.test1" ] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
0.2 Get Task Execution Status
Interface Description
Get the execution status of the specified job_id task.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| job_id | string | 1 | ID of the task to be queried |
| job_type | string | 1 | Task type, current interface 'get_status' |
| timestamp | string | 1 | Timestamp when the task was initiated |
| paras | json | 0 | Custom parameter section, not required for get_status type interface, may be required for other task types. |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| version | string | 1 | Custom protocol version, default '1.0' |
| error_code | string | 1 | Error code |
| error_info | string | 1 | Error information |
| status | string | 1 | Task status, possible values are started, ongoing, done, failed |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"expand_cluster", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "dst_shard_id":"2", "src_shard_id":"1", "table_list":[ "sbtest.public.test1" ] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
1. Equipment Management
1.1 Reporting Machinery and Equipment
Interface Description
Register the initialized server device into the system.
Response mode
Asynchronous. Use the get_status interface to check the status.
Request parameters
| 参数名称 | 参数类型 | 是否必填 | 备注 |
|---|---|---|---|
| job_type | string | 1 | create_machine |
| hostaddr | string | 1 | 服务器 IP |
| machine_type | string | 1 | 设备类型。 设备类型包括两种 storage 和 computer。 分别代表了存储机型和计算机型。存储机型表示，该设备可以被用来部署 KlustronDB_storage。相同的，计算机型代表着该设备可以被用来部署 KlustronDB_server。同一台物理设备，其类型可以是存储设备或者计算设备，也可以既是存储设备，也是计算设备，这种情况下，端口范围需要明确的区分开，确保在同一台设备上安装的计算节点和存储节点不会端口冲突。 |
| port_range | string | 1 | 端口分配范围。即新创建的实力端口在该参数指定的范围内分配 |
| rack_id | string | 1 | 机架信息 |
| datadir | string | 1 | 实例数据目录存放位置 |
| logdir | string | 1 | 日志目录存放位置 |
| wal_log_dir | string | 1 | wal 日志存放位置 |
| comp_datadir | string | 1 | 如果当前设备为计算机型，则该字段标识计算节点的数据目录 |
| total_mem | string | 1 | 该机型可用总内存 单位为 MB |
| total_cpu_cores | string | 1 | 该机型可用总的逻辑核心数 |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"create_machine", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "hostaddr":"192.168.0.135", "machine_type":"computer", "port_range":"45000-48000", "rack_id":"1", "datadir":"/home/play_ground/run_data", "logdir":"/home/play_ground/run_data", "wal_log_dir":"/home/play_ground/run_data", "comp_datadir":"/home/play_ground/run_data", "total_mem":"8192", "total_cpu_cores":"8" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"create machine successfully", "status":"done", "attachment":{
} }
1.2 Remove Device
Interface Description
Perform the delete operation on servers that have already been registered in the system.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_machine |
| hostaddr | string | 1 | Machine IP |
| machine_type | string | 1 | The device model information: storage or computer |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"delete_machine", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "hostaddr":"127.0.0.1", "machine_type":"storage" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"delete machine successfully", "status":"done", "attachment":{
} }
1.3 Set Server Status
Interface Description
Set the status of a specified server, such as dead or running, which indicate that the current device is offline or online, respectively. Only online devices can be assigned to the resource pool by the system.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_machine |
| hostaddr | string | 1 | Machine IP |
| node_stats | string | 1 | running, dead |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"set_machine", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "hostaddr":"127.0.0.1", "node_stats":"running" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"set machine success", "status":"done", "attachment":{
} }
2. Cluster Purchase and Deletion
2.1 Purchase a New Cluster
Interface Description
Create a new KlustronDB cluster according to the specified parameter configuration.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | Yes | create_cluster |
| nick_name | string | Yes | Business name, e.g., 'Reconciliation and Interest Settlement Database' |
| ha_mode | string | Yes | rbr, mgr. Indicates high-availability mode. The former is in strong synchronization mode, while the latter is in mgr mode. |
| shards | string | 1 | Number of shards |
| nodes | string | 1 | Number of nodes per shard, with at least 3 nodes forming a highly available cluster consisting of one primary node and two secondary nodes |
| comps | string | 1 | Number of compute nodes |
| max_storage_size | string | 1 | Maximum storage space |
| data_storage_MB | string | 1 | Data storage space, in MB |
| log_storage_MB | string | 1 | Log storage space, in MB |
| max_connections | string | 1 | Maximum number of connections |
| cpu_cores | string | 1 | Maximum number of cores that can be used |
| cpu_limit_node | string | 1 | share indicates soft isolation mode; quota indicates hard isolation mode; |
| enable_backup_encrypt | string | 0 | 1 indicates enabling backup encryption; 0 indicates disabling backup encryption |
| backup_encrypt_key | string | 0 | Backup encryption key. If backup encryption is enabled, this option is required; |
| innodb_size | string | 1 | KlustronDB the innodb_buffer size used by storage (this field innodb_size unit: M) |
| innodb_page_size | string | 0 | KlustronDB the innodb_page_size used by storage, the default is 16384 (supported values are 4096 |
| dbcfg | string | 0 | Set the small memory mode, 0 and without this field for normal mode, 1 for small memory mode is 0 or 1. |
| fullsync_level | string | 1 | The desired number of strongly synchronized standby nodes in this shard. This value must be less than or equal to the total number of standby nodes. |
| storage_iplists | JSON array | 1 | Specifies the hosts on which to install the storage nodes. If not specified, the system will default to using existing hosts. |
| compute_iplists | JSON array | 1 | Specifies the hosts on which to install the compute nodes. If not specified, the system will default to using existing hosts. |
| install_proxysql | string | 0 | Whether to install ProxySQL when setting up the cluster. If ProxySQL is installed, the cluster will operate in single-shard mode. The default value is 0. |
Add input SN and CN node distribution parameters. In cross-city situations, specify master_idc/slave_idc for the main city.
{
"version":"1.0",
"job_id":"",
"job_type":"create_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"nick_name":"my_nick_name",
"ha_mode":"rbr",
"shards":"1",
"nodes":"3",
"node_distribution":{
"distribution_type":"cross_city",
"master_city":{
"city_name":"cname_11",
"master_idc":{
"idc":"name_11",
"node_num":"2"
},
"slave_idc":[
{
"idc":"name_22",
"node_num":"2"
},
{
"idc":"name_33",
"node_num":"2"
}
]
},
"slave_city":{
"city_name":"cname_22",
"idcs":[
{
"idc":"name_44",
"node_num":"2"
},
{
"idc":"name_55",
"node_num":"2"
}
]
}
},
"comps":"1",
"comp_distribution":[
{
"idc":"name_44",
"node_num":"2"
},
{
"idc":"name_55",
"node_num":"2"
}
],
"max_storage_size":"20",
"max_connections":"6",
"cpu_cores":"8",
"innodb_size":"128",
"dbcfg":"1"
}
}
In cross-city situations, the main city does not specify master_idc/slave_idc
{
"version":"1.0",
"job_id":"",
"job_type":"create_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"nick_name":"my_nick_name",
"ha_mode":"rbr",
"shards":"1",
"nodes":"3",
"node_distribution":{
"distribution_type":"cross_city",
"master_city":{
"city_name":"cname_11",
"idcs":[
{
"idc":"name_22",
"node_num":"2"
},
{
"idc":"name_33",
"node_num":"2"
}
]
},
"slave_city":{
"city_name":"cname_22",
"idcs":[
{
"idc":"name_44",
"node_num":"2"
},
{
"idc":"name_55",
"node_num":"2"
}
]
}
},
"comps":"1",
"comp_distribution":[
{
"idc":"name_44",
"node_num":"2"
},
{
"idc":"name_55",
"node_num":"2"
}
],
"max_storage_size":"20",
"max_connections":"6",
"cpu_cores":"8",
"innodb_size":"128",
"dbcfg":"1"
}
}
Local situation, specified master_idc/slave_idc situation
{
"version":"1.0",
"job_id":"",
"job_type":"create_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"nick_name":"my_nick_name",
"ha_mode":"rbr",
"shards":"1",
"nodes":"3",
"node_distribution":{
"distribution_type":"same_city",
"master_idc":{
"idc":"name_11",
"node_num":"2"
},
"slave_idc":[
{
"idc":"name_22",
"node_num":"2"
},
{
"idc":"name_33",
"node_num":"2"
}
]
},
"comps":"1",
"comp_distribution":[
{
"idc":"name_44",
"node_num":"2"
},
{
"idc":"name_55",
"node_num":"2"
}
],
"max_storage_size":"20",
"max_connections":"6",
"cpu_cores":"8",
"innodb_size":"128",
"dbcfg":"1"
}
}
Same-city situation, without specifying master_idc/slave_idc situation
{
"version":"1.0",
"job_id":"",
"job_type":"create_cluster",
"timestamp":"1435749309",
"user_name":"kunlun_test",
"paras":{
"nick_name":"my_nick_name",
"ha_mode":"rbr",
"shards":"1",
"nodes":"3",
"node_distribution":{
"distribution_type":"same_city",
"idcs":[
{
"idc":"name_22",
"node_num":"2"
},
{
"idc":"name_33",
"node_num":"2"
}
]
},
"comps":"1",
"comp_distribution":[
{
"idc":"name_44",
"node_num":"2"
},
{
"idc":"name_55",
"node_num":"2"
}
],
"max_storage_size":"20",
"max_connections":"6",
"cpu_cores":"8",
"innodb_size":"128",
"dbcfg":"1"
}
}
If the SN and CN node distribution parameters are not provided, it will be consistent with the original interface.
Special Note: If the user inputs storage_iplists and computer_lists, passing node_distribution and comp_distribution is not allowed. The number of nodes must equal the total of node_num in node_distribution, and the number of comps must equal the total of node_num in comp_distribution. When purchasing across cities, the information of the primary city and backup city must be specified.
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"create_cluster", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "nick_name":"my_nick_name", "ha_mode":"mgr", "shards":"1", "nodes":"3", "comps":"1", "max_storage_size":"20", "max_connections":"6", "cpu_cores":"8", "innodb_size":"1024", "dbcfg":"1", "fullsync_level":"1", "install_proxysql":"0", "storage_iplists":[ "192.168.0.2", "192.168.0.3" ], "computer_iplists":[ "192.168.0.1" ] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"create cluster successfully", "status":"done", "attachment":{ "cluster_id":"3", "cluster_name":"cluster_1651903646_000001", "shards":"1", "comps":"1" } }
2.2 Adding New Shards Within the Cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, new shards can be added, and multiple shards can be added.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | 1 | add_shards |
| cluster_id | string | 1 | Cluster ID |
| shards | string | 1 | Number of shards to add |
| nodes | string | 1 | Number of nodes per shard |
| storage_iplists | Json array | 1 | Machines specified for installing storage nodes |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"add_shards", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "shards":"2", "nodes":"3",
"storage_iplists":[ "192.168.0.2", "192.168.0.3" ] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"add shards successfully", "status":"done", "attachment":{ "cluster_id":"2", "cluster_name":"cluster_1651836141_000001", "shards":"2", "comps":"1", "list_shard":[ { "shard_id":"3", "shard_name":"shard2" } ] } }
2.3 Delete existing shards within the cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, shards can only be deleted one by one.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_shards |
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 1 | ID of the shard to be deleted |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"delete_shard", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "shard_id":"1" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"delete shard successfully", "status":"done", "attachment":{
} }
2.4 Adding New Compute Nodes within the Cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, new shards can be added, and multiple shards can be added.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | add_comps |
| cluster_id | string | 1 | Cluster id |
| comps | string | 1 | Number of added compute nodes |
| computer_iplists | Json array | 1 | Machines where the compute nodes will be installed |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"add_comps", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "comps":"2", "computer_iplists":[ "192.168.0.2", "192.168.0.3" ] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"add comps successfully", "status":"done", "attachment":{ "cluster_id":"3", "cluster_name":"cluster_1651903646_000001", "shards":"1", "comps":"2", "list_comp":[ { "comp_id":"3", "comp_name":"comp2" } ] } }
2.5 Deleting Compute Nodes within the Cluster
Interface Description
According to the specified parameter configuration, on the existing cluster, deleting compute nodes can only be done one by one.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_shards |
| cluster_id | string | 1 | Cluster id |
| comp_id | string | 1 | ID of the computing node to be deleted |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"delete_comp", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "comp_id":"1" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"delete comp successfully", "status":"done", "attachment":{
} }
2.6 Specify cluster to add node
Interface Description
Add nodes to all shards of the cluster according to the specified parameter configuration, asynchronously, requires querying.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | add_nodes |
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 0 | If this field is specified, nodes are added only on this shard |
| nodes | string | 1 | Number of nodes to add |
| storage_iplists | JSON array | 1 | Machines designated for installing storage nodes |
| rebuild_sync_limit | string | 0 | Data synchronization speed pulled from the master node, default value is 10 (unit: MB) |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"add_nodes", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "shard_id":"1", "nodes":"2", "storage_iplists":[ "192.168.0.2", "192.168.0.3" ] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"add new nodes successfully", "status":"done", "attachment":{ "cluster_id":"1", "cluster_name":"cluster_1651907758_000001", "shards":"1", "comps":"1", "list_shard":[ { "shard_id":"1", "shard_name":"shard1", "nodes":"5", "list_node":[ { "hostaddr":"192.168.0.127", "node_id":"4", "port":"58509" }, { "hostaddr":"192.168.0.127", "node_id":"5", "port":"58512" } ] } ] } }
2.7 Specify cluster to delete node
Interface Description
According to the specified parameter configuration, delete the specified node from the cluster's designated shard asynchronously, query needed.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_nodes |
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 0 | If this field is specified, nodes will only be added to this shard |
| hostaddr | string | 1 | IP of the machine where the node to be deleted is located |
| port | string | 1 | Port number |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"delete_node", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "shard_id":"1", "hostaddr":"127.0.0.1", "port":"57338" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"delete node successfully", "status":"done", "attachment":{
} }
2.8 Redo the backup machine
Interface Description
Designate a backup machine to redo a certain shard.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | Yes | delete_nodes |
| cluster_id | string | Yes | Cluster ID |
| shard_id | string | No | If specified, nodes will be added only to this shard |
| hostaddr | string | Yes | IP address of the machine where the node to be deleted resides |
| port | string | Yes | Port number |
| need_backup | string | Yes | Whether a backup is required before redoing |
| hdfs_host | string | Yes | Cold backup location |
| pv_limit | string | Yes | Cold backup transfer rate limit, in MB |
| allow_pull_from_master | string | Yes | Whether to use data from the primary node to redo the standby node if the standby is unavailable |
| allow_replica_delay | string | Yes | Maximum allowable delay for the standby node; if the delay exceeds this value, the standby node will be considered unsuitable as a data source |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"rebuild_node", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "shard_id":"1", "cluster_id":"1", "rb_nodes":[ { "hostaddr":"192.168.0.125", "port":"57001", "need_backup":"0", "hdfs_host":"hdfs", "pv_limit":"10" } ], "allow_pull_from_master":"1", "allow_replica_delay":"15" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "version":"1.0", "error_code":"0", "error_info":"delete node successfully", "status":"done", "attachment":{
} }
2.9 Remove the specified cluster
Interface Description
According to the specified parameter configuration, delete the specified node from the cluster's specified shard asynchronously, query needed.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | delete_cluster |
| cluster_id | string | 1 | Cluster ID |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"delete_cluster", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
The query interface get_status returns
{ "attachment": { "cluster_id": "43" , "cluster_name": "cluster_1665971623_000043" , "computer_id": "776" , "computer_state": "done" , "computer_step": [ { "cluster_id": "43" , "computer_hosts": "192.168.0.132_53704;" , "computer_id": "778" , "computer_state": "done" , "error_code": 0 , "error_info": "OK" , "rollback_flag": "1" } ], "job_steps": "shard,computer" , "shard_id": "775" , "shard_step": [ { "cluster_id": "43" , "error_code": 0 , "error_info": "OK" , "rollback_flag": "1" , "shard_ids": "83," , "storage_hosts": "192.168.0.132_53407;192.168.0.132_53411;" , "storage_id": "777" , "storage_state": "done" } ], "storage_state": "done" }, "error_code": "0" , "error_info": "OK" , "job_id": "" , "job_type": "" , "status": "done" , "version": "1.0" }
2.10 Modify instance resource isolation parameters
Interface Description
According to the specified parameter configuration, modify the resource isolation parameters of the MySQL or PostgreSQL instance. Currently, only CPU isolation is supported.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| job_type | string | Yes | update_instance_cgroup |
| ip | string | Yes | Instance IP address |
| Port | string | Yes | Instance listening port |
| type | string | Yes | “mysql” indicates a storage node instance; “pg” indicates a compute node instance |
| cpu_cores | string | Yes | Number of CPU cores allocated |
| cgroup_mode | string | Yes | CPU resource limitation mode; the default is “quota”. It can be set to either “quota” or “share”, where “share” represents soft isolation and “quota” represents hard isolation. |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"update_instance_cgroup", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "ip":"192.168.0.135", "port":"57001", "type":"mysql", "cpu_cores":"5", "cgroup_mode":"quota" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
2.11 Redo Cluster Computing Nodes
Interface Description
According to the specified parameters, redo all computing nodes in the cluster, that is, clear the data in the computing nodes.
Response mode
Asynchronous. Use the get_status interface to obtain the status.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | rebuild_comps |
| cluster_id | string | 1 | Need to redo all compute nodes of this cluster |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"rebuild_comps", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special Note:
This interface clears data on all computing nodes in the cluster. Generally, it is only needed when re-establishing the RCR relationship after the RCR relationship is disconnected.
3. Cluster Backup
3.1 Manually Initiate Cluster Backup
Interface Description
Call this interface to manually initiate a full backup. When cluster_id is -1, it indicates backing up the metashard.
Response mode
Synchronous interface.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | Cluster ID |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"manual_backup_cluster", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
4. Cluster Recovery
4.1 Cluster Rollback (Restore Cluster by Time Point)
Interface Description
Perform a rollback on the specified cluster according to the point in time.
Response mode
Asynchronous. Obtain task progress through the get_status interface.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| src_cluster_id | string | Yes | Source cluster ID for the rollback operation |
| dst_cluster_id | string | Yes | Target cluster ID for the rollback operation. This cluster must be a newly purchased empty cluster |
| restore_time | string | Yes | Rollback point in time |
Example
Request
{ "version":"1.0", "job_id":"", "job_type":"cluster_restore", "timestamp":"1435749309", "paras":{ "src_cluster_id":"1", "dst_cluster_id":"2", "restore_time":"restore_time" } } -X POST http://192.168.0.135:35001/HttpService/Emit
Response
5. Cluster Expansion
5.1 Automatically Obtain Table Migration List
Interface Description
Get the list of tables automatically selected by the system for migration.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | Cluster ID |
| shard_id | string | 1 | Source shard ID of the table to be migrated within the current cluster |
| policy | string | 1 | top_hit: select the table with the highest access frequency. top_size: select the table with the largest data volume |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| attachment->table_list | string | ||
| List of tables to be migrated |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_expand_table_list", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"3", "shard_id":"2", "policy":"top_size" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "table_list":"kunlundb_super_dba.kunlundb_super_dba_$$_public.test1" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
5.2 Initiate Table Migration Task
Interface Description
Initiate table migration task.
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | expand_cluster |
| cluster_id | string | 1 | Cluster ID |
| src_shard_id | string | 1 | Source shard ID of the table to be migrated in the current cluster |
| dst_shard_id | string | 1 | Target shard ID of the table to be migrated in the current cluster |
| table_list | json_array | 1 | List of tables to be migrated |
| drop_old_table | string | 0 | Whether to retain the source table |
Example
#请求
curl -d '
{
"version": "1.0",
"job_id":"",
"job_type": "expand_cluster",
"timestamp" : "1435749309",
"user_name":"kunlun_test",
"paras": {
"cluster_id": "1",
"dst_shard_id": "5",
"src_shard_id": "6",
"table_list": [
"postgres.public.test2"
]
}
} ' -X POST http://192.168.0.135:8010/HttpService/Emit
#应答
{
"attachment":null,
"error_code":"0",
"error_info":"",
"job_id":"35",
"status":"accept",
"version":"1.0"
}
6. Metadata Management
6.1 Obtaining metashard Synchronization Mode
Interface Description
Query the metashard synchronization mode.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_meta_summary |
Response Parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| attachment->mode | string | ||
| metashard synchronization mode. Three types of results: no_rep, mgr, rbr |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_meta_mode", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "mode":"mgr" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.2 Obtaining Cold Backup Storage Service Information
Interface Description
Inquire about the detailed information of cold backup storage services.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_backup_storage |
Response Parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| hostaddr | string | Backup service IP | |
| name | string | Backup service name | |
| port | string | Backup service port | |
| stype | string | Backup service type |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_backup_storage", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "list_backup_storage":[ { "hostaddr":"192.168.0.135", "name":"hdfs_backup1", "port":"9000", "stype":"HDFS" } ] }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.3 Obtaining Machine Status Information
Interface Description
Query detailed information of all machine statuses.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_machine_summary |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| hostaddr | string | ||
| Server Address | |||
| status | string | ||
| Server Status |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_machine_summary", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "list_machine":[ { "hostaddr":"192.168.0.135", "status":"online" }, { "hostaddr":"192.168.0.125", "status":"online" } ] }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.4 Get Cluster List
Interface Description
Query the list of all KlustronDB clusters.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_cluster_summary |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| name | string | Cluster Name | |
| nick_name | string | Business Name | |
| shards | string | Number of Shards | |
| comps | string | Number of Compute Nodes |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_machine_summary", "timestamp":"1435749309", "paras":{
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "list_machine":[ { "hostaddr":"192.168.0.135", "status":"online" }, { "hostaddr":"192.168.0.125", "status":"online" } ] }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.5 Get Cluster Details
Interface Description
Get detailed information of the specified cluster.
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_cluster_detail |
| cluster_name | string | 1 | Cluster Name |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| name | string | Cluster name | |
| nick_name | string | Business name | |
| shards | string | Number of shards | |
| comps | string | Number of compute nodes | |
| comp_name | string | Compute node name | |
| hostaddr | string | IP of compute or storage node | |
| port | string | Port of compute or storage node | |
| status | string | Current node status | |
| master | string | Whether the current storage node is the master node |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_cluster_detail", "timestamp":"1435749309", "paras":{ "cluster_name":"cluster_1659333382_000001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "cluster_name":"cluster_1659333382_000001", "comps":"1", "list_comp":[ { "comp_name":"comp_1", "hostaddr":"192.168.0.135", "port":"45001", "status":"online" } ], "list_shard":[ { "list_node":[ { "hostaddr":"192.168.0.135", "master":"true", "port":"55001", "status":"online" }, { "hostaddr":"192.168.0.135", "master":"false", "port":"55002", "status":"online" }, { "hostaddr":"192.168.0.135", "master":"false", "port":"55003", "status":"online" } ], "nodes":"3", "shard_name":"shard_1" } ], "nick_name":"my_nick_name", "shards":"1" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
6.6 Obtaining Storage Node System Variable Information
Interface Description
Query the value of a certain variable in KlustronDB_storage (such as innodb_buffer_pool_size).
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| job_type | string | 1 | get_variable |
| hostaddr | string | 1 | KlustronDB_storage IP |
| port | string | 1 | KlustronDB_storage port |
| variable | string | 1 | Variable name |
Response Parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| result | string | ||
| Query success | |||
| value | string | ||
| Query result |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_variable", "timestamp":"1435749309", "paras":{ "hostaddr":"192.168.0.135", "port":"55001", "variable":"innodb_buffer_pool_size" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
{ "attachment":{ "result":"true", "value":"5242880" }, "error_code":"0", "error_info":"", "job_id":"", "job_type":"", "status":"done", "version":"1.0" }
7. Table Redistribution
Interface Description
Based on the user input table, dump the data from the table and import it into a new table
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| src_cluster_id | string | 1 | The cluster_id where the source table is located |
| dst_shard_id | string | 1 | The cluster_id to which the target table is redistributed |
| repartition_tables | string | 1 | The target table to be redistributed to the target cluster_id |
| rename_tables | string | 0 | Whether to rename the target table to the source table name after redistribution |
| del_src_table | string | 0 | Whether to automatically delete the source table after successful redistribution |
| del_resv_day | string | 0 | If del_src_table=0, specify how many days to retain, default is 7 days |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"repartition_tables", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "src_cluster_id":"3", "dst_cluster_id":"2", "repartition_tables":"test_
$$ public.t=>test1 $$
private.t2,test
$$ priv.ta=>test1 $$
_priv1.tb" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special Note:
1.It is necessary to pre-create the database, schema, and table in the target cluster, and the structure of the source and target tables must be consistent. This cluster_mgr will automatically perform the verification. 2.For the redistribution table task, the tables that need to be done at one time must be under the same database, but the source and target databases can be different.
8. Table Logical Backup
Interface Description
The user initiates a logical backup of a specific table and sets up scheduled logical backups.
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| cluster_id | string | 1 | The cluster_id where the logical backup tables are located |
| backup_type | string | 0 | Type of logical backup, currently supports db, schema, table. If db is entered, all tables under that db will be backed up |
| db_table | string | 1 | The table to be backed up logically |
| backup_time | string | 1 | Scheduled backup time period |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"logical_backup", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "backup_type":"table", "backup":[ { "db_table":"postgres_
$$ public.t1", "backup_time":"01:00:00-02:00:00" }, { "db_table":"postgres $$
_puinblic.t2", "backup_time":"01:00:00-02:00:00" } ]
} } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
9. Table Logical Recovery
Interface Description
The user initiates the restoration of a certain logical backup table to a specified historical point in time
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Notes |
|---|---|---|---|
| src_cluster_id | string | 1 | The cluster_id where the backup table is located |
| dst_cluster_id | string | 1 | The cluster_id to which the table will be logically restored |
| restore_type | string | 0 | Type of logical restoration, currently supports db, schema, table. If 'db' is entered, all tables under that db in the source cluster will be restored to the target cluster |
| db_table | string | 1 | The table to be logically restored |
| restore_time | string | 1 | The historical time point to restore to |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"logical_restore", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "src_cluster_id":"1", "dst_cluster_id":"2", "restore_type":"table", "restore":[{ "db_table":"postgres_
$$ public.t1", "restore_time":"2022-10-20 19:20:15" },{ "db_table":"postgres $$
_publgic.t2", "restore_time":"2022-10-20 19:20:15" }] } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
10. Manually trigger cluster_mgr primary-standby switch
Interface Description
The user initiates a master-slave switch operation on the current cluster_mgr cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| task_type | string | 1 | Task type, here it is transfer_leader |
| target_leader | string | 1 | The target new leader to switch to |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"raft_mission", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "task_type":"transfer_leader", "target_leader":"127.0.0.1:25001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
11. Add nodes to cluster_mgr
Interface Description
The user initiates the operation to add nodes to the current cluster_mgr cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| task_type | string | 1 | Task type, here it is add_peer |
| peer | string | 1 | The cluster_mgr node to add |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"raft_mission", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "task_type":"add_peer", "peer":"127.0.0.1:25001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
12. Delete nodes for cluster_mgr
Interface Description
The user initiates the operation to remove a node from the current cluster_mgr cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| task_type | string | 1 | Task type, here it is remove_peer |
| peer | string | 1 | The cluster_mgr node to be deleted |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"raft_mission", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "task_type":"remove_peer", "peer":"127.0.0.1:25001" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special emphasis
To add/remove nodes to/from the cluster_mgr cluster, you need to manually update the raft_group_member_init_config configuration item in the cluster_mgr configuration file and update the configuration for adding/removing nodes.
13. Manual cross-IDC primary-standby switch
Interface Description
The user initiates an operation to switch a certain cluster across IDCs
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | The cluster ID to switch to |
| target_idc | string | 1 | The target IDC to switch to |
| need_rollback | string | 0 | Whether to roll back the successfully switched shard if the switch fails |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"manual_idcsw", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "target_idc":"abc" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
14. Forced Cross-City Switching
Interface Description
The user initiates a cross-city switch operation for a certain cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | The cluster id to switch |
| target_city | string | 1 | The target city to switch to |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"force_citysw", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"1", "target_city":"abc" } } ' -X POST http://192.168.0.135:35001/HttpService/Emit
Response
Special emphasis
This operation is a forced switch and cannot guarantee data consistency.
15. Establish RCR data synchronization
Interface Description
The user initiated RCR data synchronization between two clusters
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| master_info | Json_Array | 1 | Main RCR parameter, meta_db and cluster_id |
| cluster_id | string | 1 | ID of the backup RCR cluster |
| ignore_db | string | 0 | Default is 0. If 1 is passed, data in the backup cluster will be cleared and re-pulled from the main cluster; both storage and compute node data will be cleared |
| rebuild_pv_limit | string | 0 | Maximum sync speed when pulling data from the main cluster. Default is 10 MB |
| parallel_rebuild | string | 0 | Default is 0. If 1 is passed, redo backup cluster shard nodes in parallel |
| skip_sync_roles | string | 0 | When establishing sync relationships, whether to skip certain roles not present in the backup cluster. Format: ba,cd,ef |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"create_rcr", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "master_info":{ "meta_db":"127.0.0.1:57001,127.0.0.2:57001,127.0.0.3:57001", "cluster_id":"1" }, "cluster_id":"2", "ignore_db":"0" } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
16. Delete rcr data synchronization
Interface Description
The user initiated the cancellation of RCR data synchronization for two clusters
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| master_info | Json_Array | 1 | Main rcr parameter, meta_db and cluster_id |
| cluster_id | string | 1 | Backup rcr cluster id |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"delete_rcr", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "master_info":{ "meta_db":127.0.0.1:57001,127.0.0.2:57002", "cluster_id":"1" }, "cluster_id":"2" } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
17. Manual RCR Switching
Interface Description
User initiates manual switching of RCR synchronization
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| master_info | Json_Array | 1 | Main rcr parameter, meta_db and cluster_id |
| cluster_id | string | 1 | Backup rcr cluster id |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"manualsw_rcr", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "master_info":{ "meta_db":127.0.0.1:57001,127.0.0.2:57002", "cluster_id":"1" }, "cluster_id":"2" } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
17. Configure RCR data synchronization parameters
Interface Description
The user initiates an adjustment to the RCR data synchronization parameters
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remark |
|---|---|---|---|
| master_info | Json_Array | 1 | Main rcr parameter, meta_db and cluster_id |
| work_mode | string | 1 | Modify parameter type, currently supports modify_sync_delay |
| sync_delay | string | 0 | When work_mode=modify_sync_delay, set delay time in seconds |
| cluster_id | string | 1 | Backup rcr cluster id |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"modify_rcr", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "master_info":{ "meta_db":127.0.0.1:57001,127.0.0.2:57002", "cluster_id":"1" }, "work_mode":"start_rcr", "cluster_id":"2" } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
18. Set the automatic physical backup time period
Interface Description
User initiates adjustment of the physical automatic backup time period
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| cluster_id | string | 1 | ID of the cluster |
| time_period_str | string | 1 | Backup time period, format is “08:00:00-24:00:00” |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"update_cluster_coldback_time_period", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "cluster_id":"2", "time_period_str":"08:00:00-24:00:00" } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
19. Operations on storage nodes or compute nodes in the cluster
Interface Description
The user initiates operations on storage nodes or compute nodes in the cluster
Response mode
Asynchronous.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|---|---|---|
| control | string | 1 | Operation type, start |
| hostaddr | string | 1 | IP of the machine of the instance to operate |
| port | string | 1 | Port of the machine of the instance to operate |
| machine_type | string | 1 | Type of instance to operate, storage |
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"control_instance", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ "control":"start", "hostaddr":"127.0.0.1", "port":"27001", "machine_type":"storage" } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
20. Get the current metashard master node
Interface Description
User initiates a task and obtains the current metashard master node
Response mode
Synchronize.
Request parameters
| Parameter Name | Parameter Type | Required | Remarks |
|---|
Example
Request
curl -d ' { "version":"1.0", "job_id":"", "job_type":"get_meta_master", "timestamp":"1435749309", "user_name":"kunlun_test", "paras":{ } } ' -X POST http://127.0.0.1:35001/HttpService/Emit
Response
{ "attachment":{"master_host":"127.0.0.1:23301"}, "error_code":"0", "error_info":"", "job_id":"35", "status":"accept", "version":"1.0" }
END
33542df8d527d8698510020e4f1eed55588fb013:cluster_mgr_api.md