Search

TraceLab is designed to empower future traceability research, through facilitating innovation and creativity, increasing collaboration between researchers, decreasing the startup costs and effort of new traceability research projects, and fostering technology transfer. To this end, it provides an experimental environment in which researchers can design and execute experiments in TraceLab's visual modeling environment using a library of reusable and user-defined components. TraceLab fosters research competitions by allowing researchers or industrial sponsors to launch research contests intended to focus attention on compelling traceability challenges. Contests are centered around specific traceability tasks, performed on publicly available datasets, and are evaluated using standard metrics incorporated into reusable TraceLab components. TraceLab has been released in beta-test mode to researchers at seven universities, and will be publicly released via CoEST.org in the summer of 2012. Furthermore, by late 2012 TraceLab's source code will be released as open source software, licensed under GPL. TraceLab currently runs on Windows but is designed with cross platforming issues in mind to allow easy ports to Unix and Mac environments. [ABSTRACT FROM AUTHOR]

Human analysts working with results from automated traceability tools often make incorrect decisions that lead to lower quality final trace matrices. As the human must vet the results of trace tools for mission- and safety-critical systems, the hopes of developing expedient and accurate tracing procedures lies in understanding how analysts work with trace matrices. This paper describes a study to understand when and why humans make correct and incorrect decisions during tracing tasks through logs of analyst actions. In addition to the traditional measures of recall and precision to describe the accuracy of the results, we introduce and study new measures that focus on analyst work quality: potential recall, sensitivity, and effort distribution. We use these measures to visualize analyst progress towards the final trace matrix, identifying factors that may influence their performance and determining how actual tracing strategies, derived from analyst logs, affect results. [ABSTRACT FROM PUBLISHER]

This paper posits that a theoretical model of analyst effort in tracing tasks is necessary to assist with study of the analyst. Specifically, it is clear from prior work by numerous research groups that the important factors in such a model are: the amount of time it takes for an analyst to vet a given candidate link and the amount of time it takes an analyst to find a missing link. This paper introduces a theoretical model of analyst effort as well as a simplified model. A number of simulations were undertaken in order to build effort curves to assist in evaluating numerous tracing scenarios, such as determining at what point in time an analyst should switch from vetting candidate links to manually searching for links not in the candidate list. [ABSTRACT FROM AUTHOR]

The challenges of implementing successful and cost-effective traceability have created a compelling research agenda that has addressed a broad range of traceability related issues, ranging from qualitative studies of traceability users in industry to very technical and quantitative studies. Unfortunately, advances are hampered by the significant time and effort needed to establish a traceability research environment and to perform comparative evaluations of new results against existing baselines. In this panel we discuss ongoing efforts by members of the Center of Excellence for Software Traceability (CoEST) to define the Grand Challenges of Traceability, develop benchmarks, and to construct TraceLab, an extensible and scalable visual environment for designing and executing a broad range of traceability experiments. [ABSTRACT FROM AUTHOR]

Our research group recently discovered that human analysts, when asked to validate candidate traceability matrices, produce predictably imperfect results, in some cases less accurate than the starting candidate matrices. This discovery radically changes our understanding of how to design a fast, accurate and certifiable tracing process that can be implemented as part of software assurance activities. We present our vision for the new approach to achieving this goal. Further, we posit that human fallibility may impact other software engineering activities involving decision support tools. [ABSTRACT FROM AUTHOR]

Traceability recovery is a tedious, error-prone, person-power intensive task, even if aided by automated traceability tools. Human analysts must vet candidate traceability links retrieved by such tools and must often go looking for links that such tools fail to locate as they build a traceability matrix. This paper examines a research version of the traceability tool REquirements TRacing On target (RETRO) that logs analyst actions. We examine the user logs in order to understand how analysts work on traceability recovery tasks. Such information is a pre-requisite to understanding how to better design traceability tools to best utilize analyst time while developing a high quality final traceability matrix. [ABSTRACT FROM AUTHOR]