Search

This paper shares our experience in supporting and running the Open vSwitch (OVS) software switch, as part of the NSX product for enterprise data center virtualization used by thousands of VMware customers. Starting in 2009, the OVS design split its code between tightly coupled kernel and userspace components. This split was necessary at the time for performance, but it caused maintainability problems that persist today. In addition, in-kernel packet processing is now much slower than newer options. To solve the problems caused by the user/kernel split, OVS must adopt a new architecture. We describe two possibilities that we explored, but did not adopt, one because it gives up compatibility with drivers and tools that are important to virtual data center operators, the other because it performs poorly. Instead, we endorse a third approach, based on a new Linux socket type called AF\_XDP, which solves the maintainability problem in a compatible, performant way. The new code is already merged into the mainstream OVS repository. We include a thorough performance evaluation and a collection of lessons learned.

VMware NSX virtualizes network functionality in a manner anal- ogous to how hypervisors virtualize compute resources. To do this, NSX must faithfully recreate virtual versions of network compo- nents, such as switches, routers, and firewalls. As this functionality becomes commoditized, NSX must move "up the stack" to provide more advanced features, such as load-balancers, IDS/IPS (intrusion detection and prevention systems), and DPI (deep packet inspec- tion) for classification. NSX is designed to work in all types of deployments-even those without any other VMware software. It integrates with ESXi, Linux KVM, and Hyper-V hypervisors; it is even being made to work on systems without a hypervisor, such as containers and third- party clouds. Each of these platforms has its own native forwarding plane. For the best user experience, all of the forwarding planes should provide the same behavior, but the disparate implemen- tations make this difficult in practice. As network functions be- come more complex and as NSX supports more forwarding planes, both duplication of effort and undesirable diversity of behavior in- creases. We propose a new approach to building advanced network func- tions in NSX. Under this approach, identical code runs on all of NSX's supported platforms. Applications will run at or near native performance, but with better security and identical cross-platform behavior. We demonstrate this by writing a single application to provide DPI functionality that runs in the fast paths of each of NSX's primary platforms: ESXi, Linux, and Edge gateway appli- ance. We evaluate the performance and correctness of our imple- mentation on the three platforms.

Hypervisors use software switches to steer packets to and from virtual machines (VMs). These switches frequently need upgrading and customization—to support new protocol headers or encapsulations for tunneling and overlays, to improve measurement and debugging features, and even to add middlebox-like functions. Software switches are typically based on a large body of code, including kernel code, and changing the switch is a formidable undertaking requiring domain mastery of network protocol design and developing, testing, and maintaining a large, complex codebase. Changing how a software switch forwards packets should not require intimate knowledge of its implementation. Instead, it should be possible to specify how packets are processed and forwarded in a high-level domain-specific language (DSL) such as P4, and compiled to run on a software switch. We present PISCES, a software switch derived from Open vSwitch (OVS), a hard-wired hypervisor switch, whose behavior is customized using P4. PISCES is not hard-wired to specific protocols; this independence makes it easy to add new features. We also show how the compiler can analyze the high-level specification to optimize forwarding performance. Our evaluation shows that PISCES performs comparably to OVS and that PISCES programs are about 40 times shorter than equivalent changes to OVS source code.

As anyone who has operated a large network can attest, enterprise networks are difficult to manage. That they have remained so despite significant commercial and academic efforts suggests the need for a different network management paradigm. Here we turn to operating systems as an instructive example in taming management complexity. In the early days of computing, programs were written in machine languages that had no common abstractions for the underlying physical resources. This made programs hard to write, port, reason about, and debug. Modern operating systems facilitate program development by providing controlled access to high-level abstractions for resources (e.g., memory, storage, communication) and information (e.g., files, directories). These abstractions enable programs to carry out complicated tasks safely and efficiently on a wide variety of computing hardware. In contrast, networks are managed through low-level configuration of individual components. Moreover, these configurations often depend on the underlying network; for example, blocking a user’s access with an ACL entry requires knowing the user’s current IP address. More complicated tasks require more extensive network knowledge; forcing guest users’ port 80 traffic to traverse an HTTP proxy requires knowing the current network topology and the location of each guest. In this way, an enterprise network resembles a computer without an operating system, with network-dependent component configuration playing the role of hardware-dependent machine-language programming. What we clearly need is an “operating system” for networks, one that provides a uniform and centralized programmatic interface to the entire network. Analogous to the read and write access to various resources provided by computer operating systems, a network operating system provides the ability to observe and control a network. A network operating system does not manage the network itself; it merely provides a programmatic interface. Applications implemented on top of the network operating system perform the actual management tasks. The programmatic interface should be general enough to support a broad spectrum of network management applications. Such a network operating system represents two major conceptual departures from the status quo. First, the network operating system presents programs with a centralized programming model; programs are written as if the entire network were present on a single machine (i.e., one would use Dijkstra to compute shortest paths, not Bellman-Ford). This requires (as in [3, 8, 14] and elsewhere) centralizing network state. Second, programs are written in terms of high-level abstractions (e.g., user and host names), not low-level configuration parameters (e.g., IP and MAC addresses). This allows management directives to be enforced independent of the underlying network topology, but it requires that the network operating system carefully maintain the bindings (i.e., mappings) between these abstractions and the low-level configurations. Thus, a network operating system allows management applications to be written as centralized programs over highlevel names as opposed to the distributed algorithms over low-level addresses we are forced to use today. While clearly a desirable goal, achieving this transformation from distributed algorithms to centralized programming presents significant technical challenges, and the question we pose here is: Can one build a network operating system at significant scale?

This paper overviews a Web-based collaborative system called TeamSCOPE that has been designed to support awareness needs of globally distributed teams. Four types of awareness needs of virtual teams are defined and the awareness support features of TeamSCOPE are described. The usage patterns of eight globally distributed engineering design teams are outlined, and evaluation results are provided. Findings illustrate how group process interacts with technology to create design challenges in the support of virtual team awareness needs.

This paper overviews a Web-based collaborative system called TeamSCOPE that has been designed to support awareness needs of globally distributed teams. Four types of awareness needs of virtual teams are defined and the awareness support features of TeamSCOPE are described. The use of TeamSCOPE in a project involving a number of globally distributed engineering design teams is outlined, and evaluation results are provided. Findings illustrate how group process interacts with technology to create design challenges in the support of virtual team awareness needs.

Pintos is an instructional operating system, complete with documentation and ready-made, modular projects that introduce students to the principles of multi-programming, scheduling, virtual memory, and filesystems. By allowing students to run their work product on actual hardware, while simultaneously benefiting from debugging and dynamic analysis tools provided in simulated and emulated environments, Pintos increases student engagement. Unlike tailored versions of commercial or open source OS such as Linux, Pintos is designed from the ground up from an educational perspective. It has been used by multiple institutions for a number of years and is available for wider use.

This paper shows how to quickly move the state of a running computer across a network, including the state in its disks, memory, CPU registers, and I/O devices. We call this state a capsule . Capsule state is hardware state, so it includes the entire operating system as well as applications and running processes.We have chosen to move x 86 computer states because x 86 computers are common, cheap, run the software we use, and have tools for migration. Unfortunately, x 86 capsules can be large, containing hundreds of megabytes of memory and gigabytes of disk data. We have developed techniques to reduce the amount of data sent over the network: copy-on-write disks track just the updates to capsule disks, "ballooning" zeros unused memory, demand paging fetches only needed blocks, and hashing avoids sending blocks that already exist at the remote end. We demonstrate these optimizations in a prototype system that uses VMware GSX Server virtual, machine monitor to create and run x 86 capsules. The system targets networks as slow as 384 kbps.Our experimental results suggest that efficient capsule migration can improve user mobility and system management. Software updates or installations on a set of machines can be accomplished simply by distributing a capsule with the new changes. Assuming the presence of a prior capsule, the amount of traffic incurred is commensurate with the size of the update or installation package itself. Capsule migration makes it possible for machines to start running an application within 20 minutes on a 384 kbps link, without having to first install the application or even the underlying operating system. Furthermore, users' capsules can be migrated during a commute between home and work in even less time.

In this paper, we describe a collaborative system specifically designed to address problems faced by distributed (or virtual) teams. TeamSCOPE (Team Software for a Collaborative Project Environment) is a web-based work environment that has emerged from a research project studying the communication needs of internationally distributed engineering design teams. The paper begins by outlining some of the needs of virtual teams. An integrative framework that focuses on facilitation of group members' awareness of group activities, communications and resources is proposed. These needs and awareness requirements are then translated into a set of collaborative system design goals which have guided the implementation of TeamSCOPE. The features of TeamSCOPE are briefly reviewed, and some preliminary observations from early users are provided. We conclude by noting some of the new features planned for TeamSCOPE based on our early trials.