Introduction to KlustronDB Serverless

Overview of Serverless

Serverless is a new form of cloud services, and it has a series of technological advantages, such as freeing users from installing and upgrading database clusters, as well as eliminating most operational and maintenance work. Therefore, Serverless is rapidly becoming the basic form of fundamental software services, including database systems.

KlustronDB Serverless Overview

KlustronDB is already running in the Serverless mode on AWS's DBaaS service, which we call KlustronDB Serverless. Users can already purchase it on AWS Marketplace. In 2022, when we discussed cooperation with Amazon Web Services (AWS), they actively suggested and encouraged us to provide cloud services in the Serverless form. Considering that Serverless is indeed a new model of cloud computing and is already widely applied in scenarios such as PaaS, its user value is also suitable for DBaaS. For example, it can significantly reduce users' management and maintenance costs, improve user efficiency. Users only need to purchase KlustronDB from the AWS KlustronDB Serverless service page using the GUI, and they can use KlustronDB for data storage and transaction processing without installing or upgrading KlustronDB clusters or performing daily database operations. Therefore, we decided to implement KlustronDB Serverless on AWS, and during this process, we also received active cooperation from the AWS technical team, making the collaboration very efficient.

KlustronDB Serverless is based on KlustronDB, adding tenant management, data isolation, and usage statistics for billing on the basis of the KlustronDB distributed database, and restricting some cluster management functions in multi-tenant scenarios to ensure that these functions are not exposed to tenants and can only be used by us as the service provider.

We deployed a KlustronDB distributed database cluster using AWS EC2 nodes and EBS storage services, and used it to provide KlustronDB Serverless services to multiple tenants. At Klustron, we are responsible for operating and maintaining the KlustronDB cluster deployed on AWS, so users do not need to install or manage the KlustronDB cluster at all. During operation, we only need to add more EC2 nodes and EBS storage space as needed to provide more storage and computing capacity to current tenants and more new tenants.

Each tenant uses their private account and password to connect to KlustronDB Serverless and read and write their data. No tenant can access the data of other tenants, nor do they know which tenants are currently using the cluster.

We design billing rules based on the amount of data stored by each tenant and the number of computing resources used. AWS implements the KlustronDB Serverless billing module according to the billing rules we designed and installs it into its billing system to operate. AWS's billing system bills regularly and transfers the charges from the user accounts to our account.

Based on the existing capabilities of the KlustronDB distributed database, building a Serverless model DBaaS has a relatively controllable level of complexity, mainly involving work in several aspects. This is also the main content of this article.

Scenarios suitable for KlustronDB Serverless version

A. SaaS scenario: the same business logic, different user scales, and the business growth rate of different users also varies.

B. Unified Data Platform: The data platform department within a large company provides DBaaS similar to a private cloud for use by various departments or different product and service lines of the company

C. Public Cloud DBaaS Provider

Implementation of KlustronDB Serverless

Data Isolation

Data isolation is crucial for multi-tenant DBaaS. The system must ensure that no tenant can access other tenants' data, and cannot even see which databases, schemas, tables, or other database objects belong to other tenants — the existence and names of these objects must be unknown to other tenants.

We use the isolation capability of KlustronDB's databases to achieve data isolation for different tenants. Each tenant in KlustronDB Serverless can connect to its dedicated database to execute DDL and DML statements. Tenants can create schemas within their databases to achieve logical separation of data. However, tenants are not allowed to use DML statements to read or write metadata tables in the system catalog.

Unlike MySQL, once a client connects to a database, it cannot switch to another database using the USE command or mysql_select_db(). At the same time, a tenant cannot connect to another tenant's database. KlustronDB ensures this through permission settings. The business logic of KlustronDB Serverless is to create a dedicated account for each tenant in the KlustronDB cluster and configure appropriate permissions, as detailed below.

User account

Each tenant needs to use a dedicated user account to use KlustronDB Serverless, in order to achieve access control and other advanced management functions.

After a user purchases a KlustronDB Serverless through the DBaaS purchase interface on the AWS Marketplace, the built-in logic in the purchase process will use the database connection username and password provided by the user to create an account for that user in the KlustronDB cluster. The permissions configured for each account prohibit it from connecting to or accessing other tenants' databases, creating accounts or databases, being a superuser, or inheriting or modifying permissions. This account is the main account for the tenant, and the tenant can use this account to create more sub-accounts for internal permission control. Multiple schemas can also be created within their database for different business purposes and assigned to different sub-accounts for use by each business. All these sub-accounts can only connect to this tenant's database, and the KlustronDB management module will aggregate their usage of computing resources so that the AWS billing system can uniformly bill this tenant.

Tenant Cluster Management

We have expanded KlustronDB's XPanel cluster management system into XPanel Serverless, allowing it to provide each KlustronDB Serverless tenant with independent and limited management functions. Compared with on-premise deployed KlustronDB clusters, many cluster management functions are not applicable to KlustronDB Serverless tenants, including scaling, adding/removing cluster nodes and storage shards, physical cluster backup and recovery, full cluster logical backup and recovery, multi-availability zone (multi-datacenter) high availability, and dual-active clusters across local/remote regions. Only database, schema, and table-level logical backup and restore, online DDL & repartition, CDC, and similar functions remain valid. Additionally, tenants using the CDC function can only export data update event streams for their own databases. Each tenant logs into XPanel Serverless with their username and password and can only access and operate the databases, schemas, tables, and stored procedures they own.

Backend Cluster Management

Our company, as the technical service provider of KlustronDB Serverless, is responsible for the management and control of KlustronDB clusters, including scaling up and down, adding/removing cluster nodes and storage shards, physical backup and recovery of the cluster, logical backup and recovery of the entire cluster, high availability across multiple availability zones (multiple data centers), and active-active clusters in the same city or across different locations. We perform these functions by logging into the cluster with an administrator account through XPanel.

Log access control

KlustronDB supports using ElasticSearch to collect logs from all nodes in the cluster. For data security reasons, these logs can only be fully accessed by our technical support staff through the backend cluster management interface. Tenants cannot access logs generated by operations of other tenants. Tenants can only access the interface SQL logs corresponding to their databases (i.e., SQL statements sent from compute nodes to storage nodes), slow query logs on storage nodes, and slow query logs and SQL logs on compute nodes.

Resource Isolation

Currently, KlustronDB Serverless provides services to users on a best-effort basis, meaning it uses all available computing resources in the cluster to execute every SQL statement from each connected client, without implementing resource isolation. From the user's perspective, KlustronDB Serverless without resource isolation is very cost-effective. The only restriction we place on users is the number of connections, which is provided when the user purchases the KlustronDB Serverless service, and this parameter is also used as one of the basic parameters in the billing rules.

In the Serverless mode, the traditional approach of using cgroups for resource isolation is no longer suitable, because any process/thread of a KlustronDB storage node may serve any tenant, and there is no one-to-one correspondence with the tenant. Therefore, if resource isolation is to be implemented for tenants, it would be necessary to track the resource consumption of each tenant and perform resource scheduling, which in itself consumes considerable CPU and memory resources. So far, we have not done work in this area, but it will be completed at an appropriate time in the future.

END