Indexing in Cloud File Systems

Zeyuan Tan
Shuangning Liu

Abstract

Most of the data flow generated on the Internet consists of tiny
objects. Those tiny data flows are generally intensive, thus the
overhead brought about by the frequent requests greatly limit the
performance of a cloud file system. The cloud storage service usually
imposes a restriction on the maximum requests to be processed per
second, and an intensive workload would not only delay the
interactions but also add a burden to the budget.  In this project, a
bin-packing solution is introduced to manage a large number of tiny
objects and pack them into blobs. The size of blobs and the interval
of packing are decided by the packing policy. Experiments are
conducted to inspect the effectiveness and applicability of different
policies on the optimization of tiny data flows. As a result, the
packing process turned out to be efficient in improving throughputs
compared to baselines, and the combination of packing by interval and
packing by blob size could achieve the most steady performance.