Search

In modern applications the demand for real-time processing of high-volume data streams is growing. Common application scenarios include market feed processing and electronic trading, maintenance of IoT devices and fraud detection. In some scenarios reliability is the utmost concern while in others speed and simplicity are the top priority. Apache Kafka is a high-throughput distributed messaging system and its reliable stream delivery capability makes it an ideal source of data for stream-processing systems. With various configurable parameters Kafka is very flexible in reliable data delivery thus allowing all kinds of reliability tradeoffs. In this paper we introduce a tool for Testing the Reliability of Apache Kafka (TRAK), to study different data delivery semantics in Kafka and compare their reliability under poor network quality. We build a Kafka testbed using Docker containers and use a network emulation tool to control the network delay and loss. Two metrics, message loss rate and duplicate rate, are used in our experiments to evaluate the reliability of data delivery in Kafka. The experimental results show that under high network delay the size of messages matters. The at-least-once semantics is more reliable than at-most-once in a network with high packet loss, but can lead to duplicated messages.

OpenFlow is an innovative approach to manage networks. It separates the control plane out of forwarding devices and uses a centralized controller to manipulate all the forwarding devices. The separation offers many benefits in terms of network flexibility and management. However, these benefits come with a performance penalty. Since the controller manipulates all the forwarding devices, the performance of the controller impacts the performance of the whole network. In addition, the controller is a single point failure of an OpenFlow network. I investigate the performance and dependability of OpenFlow controllers during my PhD. I implement a benchmark tool for OpenFlow controllers to fit the response time of an OpenFlow controller into a PH (PHase-type) distribution. Based on the distribution of response time, I can build queueing models for multiple controllers in an OpenFlow network. Based on the queueing models, the parameters that can minimize the response time of the controllers will be determined. I also use a stochastic model to evaluate the dependability of multiple controllers.