Search

Context: Software architectures should be evaluated during the early stages of software development in order to verify whether the non-functional requirements (NFRs) of the product can be fulfilled. This activity is even more crucial in software product line (SPL) development, since it is also necessary to identify whether the NFRs of a particular product can be achieved by exercising the variation mechanisms provided by the product line architecture or whether additional transformations are required. These issues have motivated us to propose QuaDAI, a method for the derivation, evaluation and improvement of software architectures in model-driven SPL development. Objective: We present in this paper the results of a family of four experiments carried out to empirically validate the evaluation and improvement strategy of QuaDAI. Method: The family of experiments was carried out by 92 participants: Computer Science Master s and undergraduate students from Spain and Italy. The goal was to compare the effectiveness, efficiency, perceived ease of use, perceived usefulness and intention to use with regard to participants using the evaluation and improvement strategy of QuaDAI as opposed to the Architecture Tradeoff Analysis Method (ATAM). Results: The main result was that the participants produced their best results when applying QuaDAI, signifying that the participants obtained architectures with better values for the NFRs faster, and that they found the method easier to use, more useful and more likely to be used. The results of the meta-analysis carried out to aggregate the results obtained in the individual experiments also confirmed these results. Conclusions: The results support the hypothesis that QuaDAI would achieve better results than ATAM in the experiments and that QuaDAI can be considered as a promising approach with which to perform architectural evaluations that occur after the product architecture derivation in model-driven SPL development processes when ca

Context: Software architectures should be evaluated during the early stages of software development in order to verify whether the non-functional requirements (NFRs) of the product can be fulfilled. This activity is even more crucial in software product line (SPL) development, since it is also necessary to identify whether the NFRs of a particular product can be achieved by exercising the variation mechanisms provided by the product line architecture or whether additional transformations are required. These issues have motivated us to propose QuaDAI, a method for the derivation, evaluation and improvement of software architectures in model-driven SPL development. Objective: We present in this paper the results of a family of four experiments carried out to empirically validate the evaluation and improvement strategy of QuaDAI. Method: The family of experiments was carried out by 92 participants: Computer Science Master s and undergraduate students from Spain and Italy. The goal was to compare the effectiveness, efficiency, perceived ease of use, perceived usefulness and intention to use with regard to participants using the evaluation and improvement strategy of QuaDAI as opposed to the Architecture Tradeoff Analysis Method (ATAM). Results: The main result was that the participants produced their best results when applying QuaDAI, signifying that the participants obtained architectures with better values for the NFRs faster, and that they found the method easier to use, more useful and more likely to be used. The results of the meta-analysis carried out to aggregate the results obtained in the individual experiments also confirmed these results. Conclusions: The results support the hypothesis that QuaDAI would achieve better results than ATAM in the experiments and that QuaDAI can be considered as a promising approach with which to perform architectural evaluations that occur after the product architecture derivation in model-driven SPL development processes when ca

Web applications should be usable in order to be accepted by users and to improve their success probability. Despite the fact that this requirement has promoted the emergence of several usability evaluation methods, there is a need for empirically validated methods that provide evidence about their effectiveness and that can be properly integrated into early stages of Web development processes. Model-driven Web development processes have grown in popularity over the last few years, and offer a suitable context in which to perform early usability evaluations due to their intrinsic traceability mechanisms. These issues have motivated us to propose a Web Usability Evaluation Process (WUEP) which can be integrated into model-driven Web development processes. This paper presents a family of experiments that we have carried out to empirically validate WUEP. The family of experiments was carried out by 64 participants, including PhD and Master¿s computer science students. The objective of the experiments was to evaluate the participants¿ effectiveness, efficiency, perceived ease of use and perceived satisfaction when using WUEP in comparison to an industrial widely used inspection method: Heuristic Evaluation (HE). The statistical analysis and meta-analysis of the data obtained separately from each experiment indicated that WUEP is more effective and efficient than HE in the detection of usability problems. The evaluators were also more satisfied when applying WUEP, and found it easier to use than HE. Although further experiments must be carried out to strengthen these results, WUEP has proved to be a promising usability inspection method for Web applications which have been developed by using model-driven development processes.

Web applications should be usable in order to be accepted by users and to improve their success probability. Despite the fact that this requirement has promoted the emergence of several usability evaluation methods, there is a need for empirically validated methods that provide evidence about their effectiveness and that can be properly integrated into early stages of Web development processes. Model-driven Web development processes have grown in popularity over the last few years, and offer a suitable context in which to perform early usability evaluations due to their intrinsic traceability mechanisms. These issues have motivated us to propose a Web Usability Evaluation Process (WUEP) which can be integrated into model-driven Web development processes. This paper presents a family of experiments that we have carried out to empirically validate WUEP. The family of experiments was carried out by 64 participants, including PhD and Master¿s computer science students. The objective of the experiments was to evaluate the participants¿ effectiveness, efficiency, perceived ease of use and perceived satisfaction when using WUEP in comparison to an industrial widely used inspection method: Heuristic Evaluation (HE). The statistical analysis and meta-analysis of the data obtained separately from each experiment indicated that WUEP is more effective and efficient than HE in the detection of usability problems. The evaluators were also more satisfied when applying WUEP, and found it easier to use than HE. Although further experiments must be carried out to strengthen these results, WUEP has proved to be a promising usability inspection method for Web applications which have been developed by using model-driven development processes.

[EN] Context: The conventional wisdom states that stereotypes are used to clarify or extend the meaning of model elements and consequently should be helpful in comprehending the diagram semantics. Objective: The main goal of this work is to present a family of experiments that we have carried out to investigate whether the use of stereotypes improves the comprehension of UML sequence diagrams. Method: The family of experiments consists of an experiment and two replications carried out with 78, 29 and 36 undergraduate Computer Science students, respectively. The comprehension of UML sequence diagrams with and without stereotypes was analyzed from three different perspectives borrowed from the Cognitive Theory of Multimedia Learning (CTML): semantic comprehension, retention and transfer. In addition, we carried out a meta-analysis study to integrate the different data samples. Results: The statistical analysis and meta-analysis of the data obtained from each experiment separately indicates that the use of the proposed stereotypes helps improving the comprehension of the diagrams, especially when the subjects are not familiar with the domain. Conclusions: The set of stereotypes presented in this work seem to be helpful for a better comprehension of UML sequence diagrams, especially with not well-known domains. Although further research is necessary for strengthening these results, introducing these stereotypes both in academia and industry could be an interesting practice for checking the validity of the results

[EN] Context: The conventional wisdom states that stereotypes are used to clarify or extend the meaning of model elements and consequently should be helpful in comprehending the diagram semantics. Objective: The main goal of this work is to present a family of experiments that we have carried out to investigate whether the use of stereotypes improves the comprehension of UML sequence diagrams. Method: The family of experiments consists of an experiment and two replications carried out with 78, 29 and 36 undergraduate Computer Science students, respectively. The comprehension of UML sequence diagrams with and without stereotypes was analyzed from three different perspectives borrowed from the Cognitive Theory of Multimedia Learning (CTML): semantic comprehension, retention and transfer. In addition, we carried out a meta-analysis study to integrate the different data samples. Results: The statistical analysis and meta-analysis of the data obtained from each experiment separately indicates that the use of the proposed stereotypes helps improving the comprehension of the diagrams, especially when the subjects are not familiar with the domain. Conclusions: The set of stereotypes presented in this work seem to be helpful for a better comprehension of UML sequence diagrams, especially with not well-known domains. Although further research is necessary for strengthening these results, introducing these stereotypes both in academia and industry could be an interesting practice for checking the validity of the results