ZTO Express Improves Infrastructure Efficiency

Industry background: Digitalization ability becomes the key to the development of the logistics industry

The rapid development of China’s e-commerce industry has given birth to an enormous and highly developed logistics market. Statistics released by the State Post Bureau shows that in 2018, 50.71 billion express delivery articles were delivered, while during the 2018 “Singles’ Day” shopping festival alone, the number of express delivery parcels delivered reached 416 million. While delivery volumes experienced rapid growth, logistical and delivery processing has become the new battlefield in the e-commerce industry. In order to satisfy consumers’ express delivery needs, which grow on a daily basis, same-day, and next-day deliveries have already become the standard for many logistics companies, and “deliveries within minutes” has also become the norm.

In this ultra-large scale and constantly growing business environment, how can we guarantee such a fast parcel processing and delivery speed? One of the “secrets” lies in constant innovation and the implementation of digital technologies. WeChat public accounts, applets, apps, and other optimizations are used to create a “one-stop-shop” ordering service catered to the different needs of e-commerce sellers, micro businesses, individuals, companies, and other client groups to ensure that the user requirements can be quickly transmitted to the back-end processing center in a standardized manner. Transfer and distribution, electronic bills, automated sorting, smart parcel lockers, smart voice, smart storage, smart security inspection machines, facial recognition, and other digital technologies are used so that logistics companies can now accurately dispatch and collect parcels using the least man power possible, while also lowering the logistical costs for logistics companies while increasing efficiency.

Challenge: Handling the huge amount of orders in a timely manner and establishing a foundation for innovative technology applications

As China’s largest express delivery service provider, ZTO Express (ZTO) has experienced rapid growth in recent years. In May 2019, the company handled over 1 billion parcels for the first time in history, and it regularly handles tens of millions of parcels daily during “Single’s Day” and other peak shopping periods. However, at the same time, ZTO still retains its extremely fast parcel processing and delivery speeds. Behind this achievement is ZTO’s relentless pursuit for digital innovation and constant improvements in its data center infrastructure.

The real-time logistics order management system is an important factor that influences logistics processing efficiency. This system usually includes receiving orders, splitting and merging orders, transportation planning, inventory control strategies, logistics execution and collaboration, visualization of logistics dynamics, event and exception management, and other functions. If the system performance is inadequate, it may lead to unpredictable losses due to effects on the order processing speed and environment, and may even lead to backlogging. In particular, during “Singles’ Day” and other peak periods, a backlog in orders does not only means a congestion in the business flow, but means that consumers will not be able to receive their deliveries on time, which will cause serious negative effects on the company and brand’s reputation.

Traditional real-time logistics order management systems usually use databases such as MySQL, Microsoft SQL Server*, and Sybase. These databases can easily handle applications in small-scale services, but their performance is insufficient for massive data processing requirements required by ZTO. First of all, ZTO processes tens of millions of orders daily on average, and the data processing capacity of traditional databases fails to support the high-speed throughput required by such massive data. When there is a large amount of data, performance reduces significantly. Secondly, problems such as high cost of traditional databases and not supporting linear scaling are increasingly prominent, restricting the sustainable growth of the business.

To this end, ZTO uses Redis database for its real-time logistics order management. This database is a log-based key-value database that runs on memory and can also be stored into persistent memory. By reading and writing data into memory, Redis significantly improves read and write speeds. This is an essential part for implementing high concurrency networking. However, it must be mentioned that Redis is a typical memory-intensive application. If the amount of data exceeds the maximum capacity of the cluster memory, on one hand, this will lead to a reduction in the amount of data Redis can process and may lead to memory overflow; on the other hand, if there are not enough Redis instances on a single host, the computing and networking resources will not be sufficiently utilized, which will lead to a low utilization rate of the entire system.

Besides innovation in the real-time order management systems for logistics applications, ZTO also actively utilizes virtual machines, containers, micro-services, and other innovative technologies to implement its digital transformation. In order to quickly transform into a high performance and scalable micro-service architecture with quick delivery, ZTO developed a new generation development framework, Titans. Titans provides developers with services, asynchronous messages, configuration management, distributed scheduling, and a series of other “out of the box” components. This self-developed DevOps collaboration platform helps individual developers to complete tasks that usually require a large team under traditional architectures. Currently ZTE runs over 50,000 virtual machines on its private cloud. At the same time, its developers are migrating an increasing number of applications onto containers to increase development agility and efficiency.

Of these innovative technologies, virtual machines and containers are memory intensive applications and the higher the number of virtual machines and containers loaded onto a single server, the lower the corresponding TCO. This also enables better control of the scale of data centers. Memory capacity and performance are very important factors that cannot be neglected in improving virtual machine and container density. ZTO hopes to find a low-cost memory scaling solution to significantly increase the density of virtual machines and containers to reach a balance between cost and scale.

In order to satisfy data storage, processing and other requirements, ZTO has built its own data center with over 2,000 servers, and is also building another data center in Huzhou, Zhejiang. Hundreds of servers will be deployed at this data center, which has the goal of increasing overall density of computation, storage, and networking resources and their overall utilization, and reducing TCO to support the development of the business and to be fully prepared for this year’s “Single’s Day.”

Servers in ZTO data centers have used DRAM memory for a long time, which has much lower capacity than what is actually needed. Therefore, ZTO has scaled its memory pool due to the urgent need. The simplest way to reach this goal is to directly deploy more DRAM memory. However, enterprise-level high capacity DRAM memory is expensive and is also limited by the number of slots available in servers and on the capacity of a single stick of memory. A lot of money will be wasted in order to get the ideal memory pool capacity, which makes it difficult to control the total cost of ownership (TCO) of the data center.

Solution: Combination of innovative Intel® processors and memory accelerates reform

Due to limitations in the medium, technology and other reasons, the issue with expensive traditional DRAM memory is difficult to be settled in the short term. Something worth mentioning is that the relationship between DC DRAM memory capacity and price is not a simple linear one. A doubling of capacity on a single stick of memory corresponds to a 2-4 times increase in price. This represents very poor cost efficiency for large scale memory pools. Therefore, ZTO decided to find an innovative memory scaling solution, and chose to use Intel® Optane™ DC persistent memory.

The first reason is that Intel Optane DC persistent memory provides the industry’s leading throughput and latency performance, as well as the best service quality while being highly durable. It can provide latency performance comparable to RAM and support high-speed caching and super-fast storage, and can be accelerated for applications leading to outstanding performance.

Secondly, the largest capacity for a single Intel Optane DC persistent memory device is 512GB, much higher than 256GB for DRAM. Therefore a higher total memory capacity can be easily installed on a single server. Furthermore, the unit price for Intel Optane DC persistent memory is much lower than DRAM. These features enable ZTO to build a higher capacity memory pool at a lower cost.

For memory-intensive operation loads, Intel Optane DC persistent memory can provide higher memory capacity, while compared to systems only configured with DRAM, it can provide more software service instances under the same service-level agreement (SLA). Using virtual environments, more memory can be provided using the same hardware budget. And under the same SLA, significantly more virtual machines can be loaded on a single server.

Besides its extreme performance, Intel Optane DC persistence memory also has better security and availability. Under the App Direct mode, Intel Optane DC persistent memory can guarantee that the data written will not be lost and when compared to the DRAM solution, it can satisfy even more application requirements.

As for processors, ZTO chose to use the 2nd Generation Intel® Xeon® Scalable processor, which perfectly supports Intel Optane DC persistent memory. This processor is optimized for workloads and has the industry’s leading performance and has implemented platform-wide innovations across computing, networking, and storage, while providing enhanced hardware virtualization features.

Solution test: Improved performance and effective control of TCO

In order to verify the performance of the “Intel Optane DC persistent memory and 2nd Generation Intel Xeon Scalable processor” solution under real applications, ZTO ran a simulation production environment with Redis stress tests on two different configurations (test configurations are shown in table 1).

Table 1: Suggested Test Configurations

The simulation production environment test shows that under the same CPU utilization rate, compared to configuration 1, the total memory load capacity was increased by 3 times in configuration 2. This means that when using the Redis database, ZTO is able to easily increase the number of Redis instances that can be loaded on servers and can adequately unleash the high speed data processing advantages of Redis. At the same time, the purchasing cost of configuration 2 is only 70% more than configuration 1, which indicates that costs can be better controlled by ZTO during Redis database memory scaling.

Furthermore, ZTO also discovered that, under the same Redis instance parameter, configuration 2 showed certain improvements in Transactions Per Second (TPS) compared to configuration 1. This is mainly due to the multi-core and load performance optimization of Intel Optane DC persistent memory and the 2nd Generation Intel Xeon Scalable processor, which results in further improvement in performance.

Thanks to the “Intel Optane DC persistent memory + 2nd Generation Intel Xeon Scalable processor” solution, Intel has not only successfully helped ZTO effectively control its TCO, but significantly increased the amount of Redis instances, virtual machines, and containers that can be loaded on a single server in the test environment, thereby improving cost effectiveness. According to estimates, using the same budget, the Intel Optane DC persistent memory solution could handle close to 40% more virtual machines and containers when compared to the older configuration.

Implementation results: Successfully handled the business peak rush

During the large-scale promotion, ZTO received a huge amount of orders coming from Tmall, Taobao, and other platforms, hitting the order processing system, with volumes that were several times the average daily amount. Under this extreme load, ZTO successfully withstood the surge of order data requests, and all orders received before 17:00 were dispatched on the same day. There were no significant order backlog issues and many customers were able to receive their purchased items by the next day or on the same day even. Since deliveries were on time as promised, customer satisfaction also increased.

This outstanding performance was due to ZTO’s continued investment into digital innovation and has also benefited from the evolution of ZTO’s data center infrastructure. Equipped with Intel Optane DC persistent memory, the servers guaranteed to withstand the surge in order data during the business peak period and ensured that ten millions of orders were processed and distributed quickly. The system has adequately proved itself in real life application. In terms of costs, ZTO infrastructure technicians also made some rough estimates and found that if traditional DRAM memory were used to build the memory pool to handle the challenge posed during business peak, then it would have cost an extra 30%. By using Intel Optane DC persistent memory, the saved costs helped improve business profit margins, and provided more momentum for the continued expansion of the business.

Application prospects: Full link digitalization, moving towards “smart logistics”

As one of the pioneers of digital transformation in China’s logistics industry, ZTO highly values digital technology innovation, and has implemented informatization, digitalization, and smart tools in all areas of its express delivery business. ZTO’s technology team has worked closely with the company’s strategy and business to establish a complete internet product research and development system, which guarantees the core requirement of “fast express deliveries”. ZTO also provides learning resources, lifestyle consumption, finance services, and other company-run or third party resources to provide more to users.

Intel will continue to help ZTO to innovate in its infrastructure while it moves towards full link digitalization, and reinforce its digital transformation foundations. The performance and cost benefits of the 2nd Generation Intel Xeon Scalable processor and Intel Optane DC persistent memory solution have been outstanding. This will also help in the acceleration of the application of artificial intelligence for workloads in data centers, companies and smart edge computing environments, and play an important role in the development of smart logistics while promoting the deployment of innovative services.

Download the PDF ›