Search

This paper presents SnappView, an open-source software development kit that facilitates end-user review of graphical user interfaces for mobile applications and streamlines their input into a continuous design life cycle. SnappView structures this user interface review process into four cumulative stages: (1) a developer creates a mobile application project with user interface code instrumented by only a few instructions governing SnappView and deploys the resulting application on an application store; (2) any tester, such as an end-user, a designer, a reviewer, while interacting with the instrumented user interface, shakes the mobile device to freeze and capture its screen and to provide insightful multimodal feedback such as textual comments, critics, suggestions, drawings by stroke gestures, voice or video records, with a level of importance; (3) the screenshot is captured with the application, browser, and status data and sent with the feedback to SnappView server; and (4) a designer then reviews collected and aggregated feedback data and passes them to the developer to address raised usability problems. Another cycle then initiates an iterative design. This paper presents the motivations and process for performing mobile application review based on SnappView. Based on this process, we deployed on the AppStore "WeTwo", a real-world mobile application to find various personal activities over a one-month period with 420 active users. This application served for a user experience evaluation conducted with N1=14 developers to reveal the advantages and shortcomings of the toolkit from a development point of view. The same application was also used in a usability evaluation conducted with N2=22 participants to reveal the advantages and shortcomings from an end-user viewpoint.

A hardware Trojan is a malicious modification to an integrated circuit (IC) made by untrusted third-party vendors, fabrication facilities, or rogue designers. Although existing hardware Trojans are designed to be stealthy, they can, in theory, be detected by post-manufacturing and acceptance tests due to their physical connections to IC logic. Manufacturing tests can potentially trigger the Trojan and propagate its payload to an output. Even if the Trojan is not triggered, the physical connections to the IC can enable detection due to additional side-channel activity (e.g., power consumption). In this article, we propose a novel hardware Trojan design, called Soft-HaT , which only becomes physically connected to other IC logic after activation by a software program. Using an electrically programmable fuse (E-fuse), the hardware can be “re-programmed” remotely. We illustrate how Soft-HaT can be used for offensive applications in system-on-chips. Examples of Soft-HaT attacks are demonstrated on an open source system-on-chip (OrpSoC) and implemented in Virtex-7 FPGA to show their efficacy in terms of stealthiness.

We introduce AB4Web, a web-based engine that implements a balanced randomized version of the multivariate A/B testing, specifically designed for practitioners to readily compare end-users' preferences for user interface alternatives, such as menu layouts, widgets, controls, forms, or visual input commands. AB4Web automatically generates a balanced set of randomized pairs from a pool of user interface design alternatives, presents them to participants, collects their preferences, and reports results from the perspective of four quantitative measures: the number of presentations, the preference percentage, the latent score of preference, and the matrix of preferences. In this paper, we exemplify the AB4Web tester with a user study for which N=108 participants expressed their preferences regarding the visual design of 49 distinct graphical adaptive menus, with a total number of 5,400 preference votes. We compare the results obtained from our quantitative measures with four alternative methods: Condorcet, de Borda count starting at one and zero, and the Dowdall scoring system. We plan to release AB4Web as a public tool for practitioners to create their own A/B testing experiments.

The growth of online social networks has seen the utilisation of these network graphs for the purpose of providing recommendations. Automated recommendations, however, do not take into account inter-personal trust levels that exist in a social network. In this article, we propose a privacy-preserving trusted social feedback (TSF) scheme where users can obtain feedback on questions from their friends whom they trust. We show that the concept can be extended to the domain of crowdsourcing -- the trusted crowdsourcing (TCS) scheme. In crowdsourcing, instead of asking friends, one can solicit opinions from experts in the crowd through a privacy preserving trusted feedback mechanism. Our proposal supports categorical answers as well as single-valued numerical answers. We evaluate our proposals in a number of ways: based on a prototype implementation built atop the Google App Engine, we illustrate the performance of the trusted social feedback. In addition, we present a user study to measure the impact that our trusted social feedback proposal has on users' perception of privacy and on foreground trust. We also present another user study to capture a model for user acceptance testing of the trusted crowdsourcing.

In spite of its importance in software reliability, testing is labor intensive and expensive. It has been found that software testing without a good strategy may not be more effective than testing the system with random data. Obviously, the effectiveness of testing relies heavily on how well the test suite --- the set of test cases actually used --- is generated. This is because the comprehensiveness of the test suite will affect the scope of testing and, hence, the chance of revealing software faults. There are two main approaches to generating test suites: specification-based and code-based. The former generates a test suite from information derived from the specification, without requiring the knowledge of the internal structure of the program. The latter approach, on the other hand, generates a test suite based on the source code of the program. Neither of these approaches is sufficient; they are complementary to one another. In software development, the requirements have to be established before implementation, and the specification should exist prior to coding. In this respect, the specification-based approach to test suite generation is particularly useful because test cases can be generated before coding has been completed. This facilitates software development phases to be performed in parallel, thus allowing time for preparing more thorough test plans and yet shortening the length of the whole process.

University professors traditionally struggle to incorporate software testing into their course curriculum. Worries include double-grading for correctness of both source and test code and finding time to teach testing as a topic. Test-driven development (TDD) has been suggested as a possible solution to improve student software testing skills and to realize the benefits of testing. According to most existing studies, TDD improves software quality and student productivity. This paper surveys the current state of TDD experiments conducted exclusively at universities. Similar surveys compare experiments in both the classroom and industry, but none have focused strictly on academia.

Component-based software testing is an important capability that supports productivity and quality assurance in component-based software development. The increased size and complexity of software systems has led to the current focus on developing distributed applications that are constructed primarily using components. Thus, the component-based systems require efficient and effective ways to test these systems and need to develop effective techniques for testing various aspects of the components such as reusability, security, dependability and safety. Study on the subject by several researchers indicates that more than fifty percent of the cost of software development is devoted to testing and it results into very high cost for testing complex software. The present paper is aimed at improving component-based system testing while considering two factors: component-based system testing requirement and test case process documentation. Lastly, the paper also discusses the limitations of component-based software testing that hinders component-based development.

Empirical research on software testing that aims to be relevant to industry faces important challenges. In this position paper, we review the background on software testing research and discuss some of the most important challenges. We then give suggestions on how to respond to these challenges.

Unit testing is one of the core practices in the Extreme Programming lightweight software development method, and it is usually carried out with the help of software frameworks that ease the construction of test cases as an integral part of programming tasks. This work describes our first results in studying the integration of automated unit testing practices in conventional 'introduction to programming' laboratories. Since the work used a classical procedural language in the course's assignments, we had to design a specific testing framework called tpUnit. The results of the experiment points out that a straightforward approach for the integration of unit testing in first-semester courses do not result in the expected outcomes in terms of student's engagement in the practice.

Testing accounts for 50% of the cost of software, yet it receives little treatment in most curricula. This paper presents some approaches to giving all students multiple, incremental exposures to software testing throughout the curriculum. A unifying framework is presented which identifies a minimal set of test experiences, skills and concepts students should accumulate. The integrated approach combines common test experiences in core courses, an elective course in software testing, and volunteer participation in a test laboratory.

his article discusses these problemsand presents both management andtechnical techniques to address them.Some of these solutions are alreadybeing used by software developers,even though they may not be widelyknown. This article proposes a multilevel regressiontesting framework that can be easily adapted into asoftware development and maintenance process inwhich regression testing can play a key role inimproving and ensuring consistent software quality.Our observations are based on our accumulatedexperience in software development and mainte-nance in the U.S. and Japan, including Hitachi Soft-ware Engineering (HSK).Regression testing has been extensively studiedby researchers [1, 3, 7, 10, 11] from a theoreticalpoint of view. However, there has been little discus-sion on applying regression testing in industrialenvironments. While researchers are mostly con-cerned with reducing the number of test cases forretesting, there are other important issues in usingregression testing in an industrial environment.Some of our observations include:• Regression testing is used extensively. In fact

This position paper aims at discussing a number of issues that typically arise when performing empirical studies with software testing techniques. Though some problems are general to all empirical disciplines, software testing studies face a number of specific challenges. Some of the main ones are discussed in sequence below.

The Continuous Media File System, CMFS, supports real-time storage and retrieval of continuous media data (digital audio and video) on disk. CMFS clients read or write files in “sessions,” each with a guaranteed minimum data rate. Multiple sessions, perhaps with different rates, and non-real-time access can proceed concurrently. CMFS addresses several interrelated design issues; real-time semantics fo sessions, disk layout, an acceptance test for new sessions, and disk scheduling policy. We use simulation to compare different design choices.

The 5th Workshop on the Automation of Software Test co-located with ICSE 2010 provides a publication forum that bridges the gap between the theory and practice of automated testing. The workshop featured a twoday high quality programs with a multi-session charette on integration testing. Researchers shared with us their insights on software testing through 15 regular papers and 6 case-study papers were presented in six sessions at the workshop. This report summarizes the organization of the workshop as well as the sessions and papers presented, the results of the charette session, attendee feedback, and lessons learned.

The design of software acceptance tests is as important to meeting contract goals as is the design of algorithms. This statement is particularly significant on fixed price contracts with tight schedules. An extreme instance of the demand placed on acceptance testing can be found in software projects wherein the only rigorous testing that required the Computer Program Configuration Item (CPCI) to exercise its repertoire of load and store instructions was the Formal Qualification Test (FQT). This paper is about such a project, the lessons learned from it, and provides an effective test approach for fixed price contracts. A word or two about the project is appropriate to establish the context that underscores the impact of the above assertion. Initially 30K (core words), 16-bit program instructions were to be developed within one year using a Varian 73 computer with 32K words of memory for a Command and Control application under a fixed price contract. A set of a priori conditions existed that tended to convey the impression that the inherent risks of this endeavor were reasonable. They were the “facts” that: Of the 30K (core words) to be written, 30% of this code already existed and would be used. Contractor standards would be allowed for documentation with limited use of Military Specifications No formal Design Reviews or audits would accompany the deliverable CPCI. Existent executive software would suffice. A competent and enthusiastic team was committed to the effort.

Current software testing practices focus, almost exclusively, on the implementation, despite widely acknowledged benefits of testing based on software specifications. We propose approaches to specification-based testing by extending a wide variety of implementation-based testing techniques to be applicable to formal specification languages. We demonstrate these approaches for the Anna and Larch specification languages.

A programmer often writes and tests programs in a bottom-up manner, producing code fragment by fragment and testing each fragment on a few examples to convince himself that the program works so far. These intermediate tests are typically lost without full utilization. The objective of this project is to create a kind of information retrieval system for test cases to remedy this situation. The "program testing assistant" described herein is intended to aid BASIC-PLUS programmers during incremental program development. As in the production of any piece of software tool, issues of ease of use and user-friendliness are of main concern in our testing assistant, along with software engineering considerations such as maintainability and reliability.

Several methods are currently employed by software developers to improve software quality. This paper explores the application of three of these methods: quality assurance, acceptance testing, and independent verification and validation. At first glance these methods appear to overlap, but a closer evaluation reveals that each has a distinct objective and an established set of procedures. The purpose of this paper is to clarify the role of each of these methods by examining their scope, organization, and implementation in the software development process.

This paper describes the structure and expounds the underlying motivations of the 'Testing System' which is presently being designed and experienced in testing the Honeywell GCOS Level 62 operating system. TS is a hierarchical system in which two main levels are individuated: tests and 'testing structures'. The former are automatic 'system exercisers' - either 'testing programs' and 'testing units' at JCL level; the latter are hierarchically organized 'catalogues', allowing the management of testing activities through the use of automatic tools.

We can trace the evolution of software test engineering by examining changes in the testing process model and the level of professionalism over the years. The current definition of a good software testing practice involves some preventive methodology.

The IBM Magnetic Drum Calculator Type 650 is an electronic calculator intermediate in speed, capacity and cost. It takes a logical position between the IBM Card Programmed Electronic Calculator and the IBM Electronic Data Processing Machines Type 701. It is a more powerful computing tool as required by those who have “outgrown” the Card Programmed Electronic Calculator. It is also a machine which may be used economically by those who are not as yet ready for a large scale computer such as the 701. It will serve not only to perform their required computing tasks, but it will also result in gaining valuable experience for later use of large scale equipment. The Magnetic Drum Calculator, through its stored program control, comprehensive order list, punched card input-output, self-checking and moderate memory capacity, gains the flexibility required of a computer which is to serve in both the commercial and scientific computing fields. Approximately 2000 tubes are used in the machine. They are the 5965, 6211, 12AY7, 6AL5, 2D21 and 5687. Types 6211, and 5965 are similar to the 12AV7. These are used because they meet special IBM acceptance tests.

This is the first in a series of reports to acquaint the AI community with the first commercially available natural language processor for data base query. The system, called ROBOT, is being marketed by the Artificial Intelligence Corporation of Kensington, Maryland, and is currently installed in three commercial sites. In this report we trace the origins of the system, the installation and acceptance testing performed by the user sites, and the impact the system has had on the trade media's view of AI.