Netstore: leveraging network optimizations to improve distributed transaction processing performance

Xu Cui, Michael Mior, Bernard Wong, Khuzaima Daudjee, Sajjad Rizvi. Netstore: leveraging network optimizations to improve distributed transaction processing performance. In Mohammad Sadoghi, Ilia Petrov, editors, Proceedings of the Second International Workshop on Active Middleware on Modern Hardware, ACTIVE@Middleware 2017, Las Vegas, NV, USA, December 11-15, 2017. pages 1-10, ACM, 2017. [doi]


A tremendous amount of data is processed daily in the cloud, which inevitably introduces large amounts of data transfer within a datacenter network. Numerous studies have reported that the network is often the performance bottleneck for cloud applications. While flow scheduling techniques have been proposed to mitigate this performance problem, distributed network-aware applications have not received much attention. The advent of software-defined networking (SDN) enables leveraging flow scheduling to make cloud applications network-aware, opening up the potential to improve the performance of these applications.

In this paper, we propose co-designing network and cloud applications to optimize performance in the presence of data-center network congestion. In particular, we built a network-aware distributed database framework which uses network state information to improve performance. We present two techniques to reduce transaction completion time for cloud database applications. First, we apply a novel load balancing algorithm at the network layer to support intelligent network route selection. Second, we introduce a network-aware caching algorithm to retrieve fresh data replicas from links in the network with low congestion. Our experimental results show that our techniques can significantly reduce average transaction completion times compared to an ECMP-based baseline.