PayPal last month released the source code for JunoDB, a distributed key-value store it developed internally and which today powers a variety of backend services at the payment site, including 350 billion transaction requests per day, the company says. By Alex Woodie.
JunoDB was originally developed over a decade ago in C++ to address the specific needs of the company. Since no commercial or open-source solutions were available to handle the required scale out-of-the-box, we developed our own solution to adopt a horizontal scaling strategy for key-value stores.
The new database would addresses two primary scaling needs in distributed key-value stores. Handling the a large number of client connections, and handling growth in read and write throughput. PayPal database developers created JunoDB with a proxy-based architecture to enable horizontal scaling. The JunoDB client library, which resides in the application, was developed to enable simple data actions through the JunoDB proxy, which manage requests from the clients, coordinates with the data stored on the JunoDB storage server, and provides load balancing. All data is encrypted, either at the client or the proxy layer using TLS; all stored data is also encrypted using TLS.
JunoDB utilizes consistent hashing to partition data and minimize data movement. To support horizontal scale, it shards data among a number of database partitions located on server nodes. JunoDB has since been rewritten in Golang to provide multi-threading and multi-core capabilities. With JunoDB’s data replication methods, including within-data center and cross-data center replication, the key-value store delivers six 9’s of system availability for PayPal. Interesting read!
[Read More]