Abstract:Preface

Abstract:Cloud-native databases, with advantages such as out-of-the-box functionality, elastic scalability, and pay-as-you-go, are currently a research hotspot in academia and industry. Currently, cloud-native databases only support “single writer and multiple readers”, that is, read-write transactions are concentrated on a single read-write node, and read-only transactions are distributed to multiple read-only nodes. This limitation restricts the system's ability to process read-write transactions, making it difficult to meet the demands of write-intensive businesses. To this end, we propose the D3C (deterministic concurrency control cloud-native database) architecture in this study. It breaks through the limitation of “single writer and multiple readers” and supports concurrency execution of read-write transactions on multiple read-write nodes by designing a cloud-native database transaction processing mechanism based on deterministic concurrency control. D3C splits transactions into sub-transactions and independently executes them on each node according to a predefined global order, ensuring serializability for transaction execution on multiple read-write nodes. Additionally, in this study, we introduce mechanisms like asynchronous batch data persistence mechanisms based on multi-version to ensure transaction processing performance and propose a consistency point-based failure recovery mechanism to achieve high availability. Experimental results show that D3C can achieve 5.1 times the performance of the “single writer and multiple readers” architecture in write-intensive scenarios while meeting the key requirements of cloud-native databases.

Abstract:Single-master is the mainstream architecture of cloud-native databases. In the cluster, the slave node can share the read-only requests of the master node, and the master node processes the write requests. To meet the demands of large-scale transaction processing, on this basis, some of the cloud databases attempt to further extend multi-write transactions. One way to implement a multi-write extension is to implement a shared cache among compute nodes and support cross-node data access. In database systems based on shared cache, the cost of cross-node remote access is much larger than that of local access, so the design of the cache protocol is the key factor affecting the system's performance and scalability. In this study, two innovative improvements to the cache protocol are proposed, and PG-RAC, a shared cache database supporting multi-write transaction processing, is implemented based on PostgreSQL. On the one hand, PG-RAC proposes a new distributed chained routing strategy to distribute routing information among compute nodes. The average transaction latency is reduced by about 20% compared to the routing strategy of single-point directory management. On the other hand, the replica invalidation mechanism is improved, and the invalidation operation is separated from the transaction path, reducing the latency of the critical path in transaction processing. On this basis, PG-RAC utilizes the characteristics of multi-version concurrency control (MVCC) to further propose to delay replica invalidation, effectively improving the cache utilization. The results of the TPC-C experiment show that in a cluster with four compute nodes, the throughput rate is nearly twice that of PostgreSQL and 1.5 times that of the distributed database Citus.

Abstract:One of the most important features of multi-tenant databases in cloud environments is scalability. However, most elastic scaling techniques struggle to make effective scaling decisions for dynamically changing loads. If load changes can be predicted in advance, resource supply can be accurately adjusted. Given this, in this paper we propose a load-prediction-based elastic scaling method for multi-tenant databases. It includes a combined load prediction model and an elastic scaling strategy. The load prediction model combines the advantages of convolutional neural networks, long short-term memory networks and gated recurrent units. It can accurately forecast memory requirements of database clusters. Based on the prediction results, the elastic scaling strategy adjusts the number of virtual machines to ensure that resource supply remains within a reasonable range. Compared to existing methods, the combined load prediction model can reduce prediction errors by 8.7% to 21.8% and improve prediction fitting degree by 4.6%. Furthermore, we improve the Bayesian optimization algorithm for hyperparameter tuning of the combined prediction model. The enhanced hyperparameter tuning model reduces errors by above 20% and improves fitting degree by 1.04%, which proves that it can well address the poor performance of Bayesian optimization in combined domains of discrete and continuous solutions. Compared to the most widely used scaling strategy in Kubernetes, the proposed elastic scaling method reduces response time by 8.12% and latency by 9.56%. It can avoid the latency and the waste of resources to a large extent.

Abstract:Time series data are widely used in industrial manufacturing, meteorology, electricity, automotive, and other fields, which has promoted the development of time series database management systems. More and more database systems are migrating to the cloud, and the architecture of terminal-edge-cloud collaboration is becoming more common, leading to increasingly large data scales to be processed. In scenarios such as terminal-edge-cloud collaboration and massive time series, a large number of short time series are generated due to short synchronization cycles, frequent data flushing, and other reasons, posing new challenges to database systems. Effective data management and compression methods can significantly improve storage performance, enabling database systems to handle the storage of massive time series. Apache TsFile is a columnar storage file format designed for time series scenarios, playing an important role in database management systems such as Apache IoTDB. In this paper, we propose the group compression and merging methods to be used in Apache TsFile to address the scenario of a large number of short time series, especially in applications with a large number of time series such as industrial Internet of Things. This group compression method fully considers the data characteristics in the short time series scenario, improves the utilization of metadata through device grouping, reduces file index size and short time series, and significantly improves compression efficiency. After validation with real-world datasets, our grouping method shows significant improvements in compression efficiency, reading, writing, file merging, and other aspects, enabling better management of TsFiles in scenarios with short time series.

Abstract:Databases are foundational components in computer services. However, performance anomalies can damage service quality. How to diagnose performance anomalies in databases has become a hot problem in industry and academia. Recently, a series of automated anomaly diagnosis methods have been proposed. They analyze the runtime status of the database and find the most likely anomalies. However, with the expansion of data scale, distributed databases are becoming increasingly popular in enterprises. In a distributed database, which is composed of multiple nodes, existing anomaly diagnosis methods struggle to effectively locate anomalies that can occur on nodes, and fail to identify compound anomalies across multiple nodes, resulting in insufficient diagnostic capabilities. To address these challenges, we propose an anomaly diagnosis method for compound anomalies in distributed databases, DistDiagnosis. It models the anomalous state of distributed databases by using a compound anomaly graph, which not only represents anomalies at each node but also captures the correlations between nodes. DistDiagnosis introduces a correlation-aware root cause ranking method, locating root cause anomalies based on the relation between nodes. In this work, we construct anomaly testing cases for different scenarios on the domestically developed distributed database OceanBase. The experimental results show that DistDiagnosis outperforms other SOTA baselines, achieving the AC@1, AC@3, and AC@5 values of 0.97, 0.98, and 0.98, respectively. Compared to the second-best method, DistDiagnosis improves accuracy by up to 5.20%, 5.45%, and 4.46%, respectively.