Introduction to KunlunBase Serverless

Overview of Serverless

Serverless represents a new paradigm in cloud services, boasting a range of technological advantages such as freeing users from the responsibility of installing and upgrading database clusters, and eliminating most operational tasks. As a result, Serverless is rapidly becoming the default mode for basic software services, including database systems.

Overview of KunlunBase Serverless

KunlunBase operates in a Serverless mode on AWS's DBaaS service, which we refer to as KunlunBase Serverless. It's available for purchase on the AWS Marketplace. In 2022, during our discussions with Amazon Web Services (AWS) regarding a potential collaboration, they strongly suggested and encouraged us to offer our services in a Serverless format. Recognizing Serverless as the emerging trend in cloud computing, with widespread applications in scenarios like PaaS, and its inherent user value especially for DBaaS - significantly reducing user management and operational costs, enhancing user efficiency - we made the decision. With AWS's KunlunBase Serverless service, users can effortlessly utilize KunlunBase for data storage and transaction processing post-purchase through a GUI, eliminating the need for installation, upgrades, and regular database maintenance. The execution of KunlunBase Serverless on AWS was seamless, thanks in part to the proactive collaboration of the AWS technical team.

KunlunBase Serverless is built upon KunlunBase. It adds tenant management, data isolation, usage statistics for billing purposes, and limits certain cluster management functions in multi-tenant scenarios to ensure these functions aren't exposed to tenants - they remain exclusive to us as the service provider.

Using AWS's EC2 nodes and EBS storage services, we've deployed a KunlunBase distributed database cluster to serve multiple tenants with the KunlunBase Serverless service. Zezhuo Technology is responsible for the maintenance of our KunlunBase cluster deployed on AWS, absolving users of any need for installation or operational concerns. As we move forward, we can easily increase our storage and computational capabilities for both current and new tenants by simply adding more EC2 nodes and EBS storage space.


Each tenant connects to KunlunBase Serverless using their private account and password to read and write their data. No tenant can access another's data, nor are they privy to information about which other tenants are currently using the cluster.

Our billing structure is based on the volume of data stored by each tenant and the computational resources they utilize. AWS has designed the KunlunBase Serverless billing module according to our specified rules, integrating it into their existing billing system. AWS's billing system routinely processes payments, transferring funds from user accounts to ours.

Given the existing capabilities of the KunlunBase distributed database, constructing a Serverless DBaaS was relatively straightforward, involving a few critical aspects. This constitutes the core content of this article.

Suitable Scenarios for KunlunBase Serverless

A. SaaS scenarios: Identical business logic, varying user scales, and different growth rates for each user's business size.

B. Unified Data Platform: Larger corporate data platform departments offering DBaaS similar to private clouds for various company departments or different products/services.

C. Public Cloud DBaaS Providers.

Implementation of KunlunBase Serverless

Data Isolation

Data isolation is crucial for multi-tenant DBaaS. The system must ensure that no tenant can access other tenants' data or even see which databases, schemas, tables, etc., exist for other tenants - the existence and names of these database objects must be unknown to other tenants.

We use KunlunBase's database isolation capability to achieve data separation among different tenants. Each tenant of KunlunBase Serverless can connect to its dedicated database and execute DDL and DML statements. Tenants can create schemas within their database for logical data segmentation. However, tenants cannot use DML to read/write system catalog metadata tables.

Unlike MySQL, once a client connects to a database, it can't switch to another using the USE command or mysql_select_db(). Moreover, tenants can't connect to other tenant databases. KunlunBase ensures this via permission settings. The business logic of KunlunBase Serverless creates a dedicated account on the KunlunBase cluster for each tenant and sets appropriate permissions, as discussed below.

User Accounts

Each tenant needs a unique user account to use KunlunBase Serverless, enabling access control and other advanced management functionalities.

When users purchase a KunlunBase Serverless through AWS marketplace's DBaaS purchasing interface, embedded logic in the purchase process uses the database connection username and password provided by the user to create an account on the KunlunBase cluster for them. The permissions set for each account prohibit access or connection to other tenant databases, account and database creation, superuser privileges, or changing permissions. This primary account allows the tenant to create more sub-accounts for internal permission management. These sub-accounts can only connect to the tenant's database. The management module of KunlunBase aggregates their computational resource usage for AWS billing.

Tenant Cluster Management

We've extended KunlunBase's XPanel cluster management system to XPanel Serverless, giving each KunlunBase Serverless tenant specific and limited management features. Compared to the on-premise deployed KunlunBase cluster, many cluster management functions aren't suitable for KunlunBase Serverless tenants, such as scalability, adding/removing cluster nodes and storage shards, full cluster logical backup and recovery, multi-availability zone availability, and bi-active cluster features in the same/different cities.

Backend Cluster Management

Our company, as the technical service provider for KunlunBase Serverless, manages the KunlunBase cluster, including scalability, adding/removing cluster nodes and storage shards, physical and logical cluster backup and recovery, multi-availability zone high availability, and bi-active clusters in the same/different cities. We use the XPanel with administrator accounts to perform these functions.

Log Access Control

KunlunBase supports ElasticSearch for collecting logs from all cluster nodes. For data security, only our technical support staff can access all logs from the backend cluster management interface. Tenants can only access SQL logs related to their database, slow query logs, and SQL and slow query logs from the computing node.

Resource Isolation

Currently, KunlunBase Serverless offers services to users on a best-effort basis. This means it utilizes all available computational resources in the cluster to execute every SQL statement from each connected client, without implementing resource isolation. From a user's perspective, KunlunBase Serverless without resource isolation is very cost-effective. Our only limitation for users is the number of connections, a parameter provided when users purchase KunlunBase Serverless services. This parameter is also used as one of the foundational billing factors in our billing rules.

In the Serverless mode, the traditional practice of using cgroups for resource isolation is no longer appropriate. This is because any process/thread of a KunlunBase storage node might serve any tenant, with no 1:1 correspondence to a tenant. Hence, if resource isolation targeting tenants is to be implemented, it would require tracking the resource consumption of the tenants and managing resource allocation, tasks that would, in turn, consume significant CPU and memory. As of now, we haven't embarked on this work, but plan to undertake it when the time is right.

The Role of Database System Performance and Efficiency in DBaaS Serverless

For DBaaS in a Serverless mode, the performance and efficiency of the database system are crucial for service providers. Higher efficiency and performance translate to lower costs for the DBaaS service providers.

The outstanding performance of the KunlunBase distributed database is particularly beneficial for KunlunBase Serverless, as it enables us to reduce our service costs—fewer machine resources can provide a higher QPS. We will continue to enhance the performance of KunlunBase to further reduce the service costs of KunlunBase Serverless.

The Advantages of Using Distributed Database for Serverless DBaaS

The following are the advantages of using a distributed database for a Serverless form of DBaaS:

Unified Management

With unified database management, monitoring, maintenance, and upgrades for a cluster, the workload of the DBA is far less than operating hundreds of independent small clusters. Moreover, DBaaS users don't need to perform maintenance or upgrades. These processes are transparent to tenants and they won't be aware of them.

Flattening Peaks with a Huge Resource Pool

With thousands of tenants sharing hundreds of servers in KunlunBase Serverless, this hardware resource pool can flatten peak demand, ensuring a high level of hardware resource utilization at all times, and adding more hardware resources as needed.

In contrast to the current practice by mainstream cloud manufacturers, which is to use several virtual machines to form a cluster dedicated to a single customer, with each virtual machine hosting an independent database instance. Compared to this traditional method, a large cluster doing Serverless DBaaS can achieve better performance and reduce the performance overhead brought by the virtual machines themselves, greatly improving the efficiency of hardware resource utilization.

Unlike current cloud databases that require several hours to several days in advance to reserve DBaaS storage and computing resource expansion, KunlunBase Serverless tenants don't need to make reservations in advance. If the business load of individual tenants temporarily increases, KunlunBase can handle it. This capability is due to the large amount of reserved resources in the KunlunBase Serverless resource pool (20% of the total resources required by all current tenants), as well as KunlunBase's flexible scalability and the ability to add or subtract computing or storage nodes as needed.

Resource Isolation or Not?

KunlunBase Serverless currently does not perform resource isolation, but charges based on usage, allowing high-resource-consuming users to pay more, and then uses these funds to add hardware resources to serve more other customers. Resource isolation itself also consumes computing resources, so we believe that instead of restricting users from using resources, we should let users fully utilize computing resources to serve their businesses and provide a smooth user experience for their end users.

Also, as DBaaS service providers, we are able to detect these situations of resource shortage promptly, and quickly provide more computing resources to serve more user requests. As billing is based on usage, this is beneficial for both parties.

The Impact of DBaaS and Serverless on DBA's Career Development

In the future, Serverless database services will be very attractive to small and medium-sized users, because the Serverless model greatly simplifies database operation and management work. The workload of users' DBA is significantly reduced, they can focus on query performance optimization, Serverless service status monitoring, designing and defining data security and access control mechanisms, as well as data validity verification mechanisms, etc. The cost of users' database use will also be significantly reduced accordingly.

Conclusion

KunlunBase Serverless is designed based on our own research and application experience in the field of databases and our understanding of DBaaS and Serverless concepts. After the launch of KunlunBase Serverless on AWS, we will continue to summarize experiences and lessons, continuously iterate and improve KunlunBase Serverless, and share timely with industry peers.

Please keep an eye on KunlunBase, visit www.kunlunbase.com for related resources and information.

END