Search

When it comes to the assessment of design behaviors and outcomes, direct observations by external viewers and subjective reflections by the participants themselves can all yield important information. External viewers, for example, may code video evidence or apply design metrics to a designer's solutions, both of which can lead to interesting statistical analyses and detailed insights. In collecting and analyzing designers' personal reflections and perceptions, researchers often utilize Likert-type scales, multiple choice questions, or short open-ended prompts. While these modes of data collection are useful and valid, they also constrain participants' responses to fixed options in the case of Likert-type scales and multiple-choice questions, and to verbal expressions in the case of open-ended prompts. Few examples of other types of reflection activities (e.g., graphing, sketching) have been presented or studied in the engineering education literature. In this project, which is part of a larger investigation into high performance design teams, we explored the use of graphing and other visual techniques for recording designers' perceptions of their design processes and products. Our primary aim was to introduce greater richness into the evaluation of designers' behaviors and outcomes as we posed research questions about their relationship to cognitive variables. In this paper, we will discuss two of these reflection activities-- an emotional state plot and a graphical assessment of a design solution's feasibility, usefulness, and novelty--in the context of a team design challenge. While the designers under study worked as teams, each individual designer provided her/his perceptions of the team process and design solutions using these reflection activities. Our analysis of individual perceptions from seven design teams of 4-5 designers each will be reported here. This investigation also includes the cognitive style of each participant (as measured by Kirton's Adaption-Innovation inventory--KAI®) as a potential mediating variable on the individual perceptions of emotions and outcomes. This paper will provide details of the two reflection activities, our associated methods of analysis, key findings related to the designers' perceptions and cognitive styles, benefits and limitations of these two reflection activities, and implications for design educators. [ABSTRACT FROM AUTHOR]

The design process can often introduce manufacturing challenges, and designers must be able to understand these challenges in order to minimize them. Frequently, the experience level of mechanical engineering students is insufficient for them to consider the limitations that manufacturing processes impose upon design, and they often design parts that are either difficult or impossible to manufacture. This work describes the development, implementation, and analysis of a system used to rapidly provide students with the knowledge they need to consider manufacturing challenges for machining processes. An experimental group of students was trained in the use of a software package (SculptPrint) that provides visualizations of the turning process and taught how to operate various computer numerical control (CNC) machine tools. A separate control group of students was trained on the operation of manual machine tools and did not receive access to the turning visualizations. Knowledge assessments were given to both groups to measure their understanding of a variety of topics in manufacturability. Analysis of the survey results indicates that student understanding of geometrical limitations in the turning process can be dramatically improved by employing visualizations of manufacturing processes. [ABSTRACT FROM AUTHOR]

Considering a wide range of ideas is crucial for engineers as they seek to solve complex problems. Paradigm-relatedness is one dimension on which ideas can range from more incremental--ideas that refine and improve existing solutions--to more radical--ideas that approach a problem from a new perspective or seemingly unrelated angle. We developed a tool that consists of a set of framing strategies an engineer can apply to a design task to assist in generating ideas that differ from their initial ideas. We created two versions of the tool--one with an incremental set of framing strategies and another with a radical set. We explored whether the framing tool impacted the paradigm-relatedness of ideas generated in conceptual design sessions. Forty-five students attending a summer engineering outreach program participated in the study. Participants were given a problem statement and generated ideas initially without the tool and then with the version of the tool of their choosing. Post hoc, we coded each idea as more incremental or more radical on a four-point scale and examined the change in ideas between the first ideation session and the second. It was found that 73% of the participants chose the framing tool version we would have assigned to them based on our coding of their initial ideas. All of these participants exhibited a shift in the predicted paradigm-relatedness direction, providing support for the overall effectiveness of the tool. However, some participants selected a version of the tool that was not consistent with our intended use of the tool, and this happened more often with the incremental version of the tool. Those participants were inconsistent in shifting their ideas. Case examples were explored to gain insight into the ways in which the tool was effective as well as how it could be improved. [ABSTRACT FROM AUTHOR]

Many sound methods exist for creating the items or questions that make up educational surveys and assessments. These methods include the use of content experts, reviews of existing instruments, and lists of behaviors and descriptors commonly associated with the construct(s) we wish to assess. Unfortunately, however, item creation sometimes becomes overly dependent upon a researcher's personal attitudes about the construct(s) being tested, or on "borrowing" items from other instruments that may or may not be sound measures of the construct(s) of interest. These risks are particularly likely for new researchers in engineering education, who may have little experience with best practices in social science research. One way to support best practices in the development of new surveys and assessments is to use an instrument blueprint to guide the creation of items, as well as the collection of validity evidence. This paper outlines a process for instrument blueprint creation and content validation to help support best practices in educational assessment. Based on Messick's unified theory of validity1, the instrument blueprint includes a process for item construction that incorporates multiple resources, including: (1) the views of content experts; (2) research from the relevant domain of interest; (3) reviews of existing instruments; and (4) the expertise of the research team. This paper uses the development of a new instrument to measure engineering innovativeness as an illustrative example of the blueprinting process. Our new instrument will assess 20 characteristics of innovative engineers as identified by in-depth studies of expert engineering innovators in previous research2. This work highlights the positive impact of using a systematic process for item construction to transform current methods of assessment in engineering education. [ABSTRACT FROM AUTHOR]

Few empirical studies describe the individual behavioral characteristics that lead to discovery, development and implementation of innovations. Even fewer studies focus on innovative behavior by engineers. After interviewing 53 engineering innovators about their experiences as an innovator and qualitatively analyzing the interview data, we identified twenty unique characteristics of engineers who had demonstrated extraordinary innovative behavior (Ferguson D., 2013). This finding was corroborated by a separate focus group study (Ferguson D. et al., 2014). We then initiated a modified Delphi study with 150 engineering innovators drawn from academic, corporate, and entrepreneurial organizations to examine the complex constructs associated with engineering (Ferguson D, Purzer S, Ohland M, Jablokow K, & Menold J, 2014). Delphi study participants were nominated as extraordinary engineering innovators from large, medium and small firms; from many different types of engineering activities or professional classifications; and as individuals who had experienced many of the different phases of the innovation process. This paper discusses our modified Delphi study methodology and the analysis process executed for the first Delphi survey round. In the first round of the Delphi Study, 150 participants ranked the importance of the innovative characteristics of engineers and indicated an 87% agreement with the initial characteristic definitions. Delphi study participants also provided over 600 comments to help us clarify and improve those definitions. After integrating the Delphi participant comments, we received a 96% consensus agreement on the revised definitions. Delphi participants also confirmed that the engineering innovativeness characteristics developed from our interviews covered all of the behaviors they had personally exhibited or observed as engineering innovators. The implications for engineering education and for professional engineering training and management are that these innovative characteristics should be developed, supported and enhanced in engineers if innovative behavior by engineers is desired. [ABSTRACT FROM AUTHOR]

Ideation is a key component of the design process, which often takes place in team settings. The team approach to design requires the involvement of all parties demonstrating the use of their expertise and knowledge in the area. Teaming has been shown to have both positive and negative impacts on ideation activities1-5. Cognitive style - or the stable, preferred way that people manage and seek to bring about change - helps us understand the different ways in which people solve problems individually and as part of a team. When team members' cognitive styles are diverse, creating an effect known as cognitive gap, the team may experience the advantages of approaching problems in diverse ways, but the likelihood of conflicts and misunderstandings increases6. This study investigated the relationship between cognitive style and the perceptions of students working in teams about their own ideation. Through the analysis of reflection surveys from 202 pre-engineering, engineering, and design students participating in an ideation study, we explored the following questions: (1) how does working in teams impact students' perceptions of their own ideation?; and (2) how do team members' cognitive styles impact their perceptions of their ideation and team contributions? In this paper, we report on the significant correlations between cognitive style and changes in students' perceptions about their ideation creativity, diversity, elaboration, and difficulty, as well as our exploration of the impact of teaming on these perceptions. [ABSTRACT FROM AUTHOR]

The article defines the key characteristics of engineering innovativeness in student and practicing engineers. It places those key factors in the phases of the innovation process and collects initial validity evidence for tools to measure the characteristics. It includes research collaborators to assist in instrument testing and validation.

Design thinking is a popular design methodology that encourages iterative problem solving and fosters the creation of human-centered products. The design thinking method, born at the famous product development firm, IDEO, is intentionally loosely structured and fast paced, forcing designers to rapidly prototype and test potential solutions. With its less structured format, design thinking may be more appealing and more readily accepted and adopted by some individuals than others -- as anecdotal evidence collected from design classrooms and design thinking workshops seems to indicate. The aim of this study is to determine whether student receptivity to design thinking might be linked to individual cognitive characteristics that reflect innate structural preferences. This research could help educators determine the most appropriate design methodology based on the cognitive preferences of their students, as well as the need to teach coping strategies when students are required to engage in design activities that do not align with their natural cognitive preferences. Our work presents the results of data gathered during a design thinking workshop in which students engaged in the design thinking process by working through a real world challenge. Student receptivity was evaluated using an ecological mood assessment,1 and cognitive characteristics were assessed using the KAI, which evaluates the cognitive styles of individuals2. A statistically significant relationship was observed between cognitive style and the happiness and excitement scores (two subscales of the ecological momentary mood assessment); we feel that with future experiments, even more correlations might be discovered. [ABSTRACT FROM AUTHOR]

The role of ideation in design is to generate design solutions that have the potential for further development. Having many diverse ideas increases the potential for successful design outcomes by increasing the number of possibilities available during concept evaluation and selection phases. How do we define the problems that would allow for the most diverse solution space? The purpose of this study was to gain an understanding of how different contexts impacted the variety of solutions generated within the solution space, by a diverse group of students. In this exploratory case study, we report on (1) how we identified a set of design problems with diverse contexts appropriate for students with varied backgrounds, and (2) how we explored the impact of these problem contexts on the size of the solution space, aiming to select the contexts with the most diverse pool of ideas for our ongoing studies1 . Our results show that diversity judged by multiple raters was consistent and provided us with evidence to support the decision of which design problems to use in our further studies. [ABSTRACT FROM AUTHOR]

The way design problems are presented may influence an engineer's ideation process, and eventually, the design outcomes. We aimed to explore the ways in which pre-engineering students shift their design ideas based on different framings of design problems. We evaluated ideas with respect to the metric of paradigm-relatedness, which refers to the extent to which an idea works within the explicitly stated and commonly understood bounds of a problem, versus moves beyond those bounds. Thirteen prospective engineering students participated in the study. Students were first given a problem statement framed in a way that didn't encourage any particular type of solution. The students were asked to generate solutions to the problem using visual and verbal depictions. Subsequently, they were given a second problem framed either to encourage practical solutions based on pre-existing designs or framed to encourage radical solutions not based on pre-existing designs. Ideas were coded as either paradigm-preserving or paradigm-modifying. We identified students whose ideas shifted from more of one type to more of another from their first ideation session to their second, as well as students whose ideas remained consistent. We analyzed their generated idea sets and reflection questionnaires to describe the influence of the framed design problem statements on their ideation approaches. The findings illustrate that problem framing can influence the paradigm-relatedness of ideas generated in a design tasks, both in more adaptive directions and in more innovative directions. However, our findings also illustrate that problem framing is not always successful in causing an individual to shift in their ideation approach, and so additional factors such as individuals' cognitive styles should also be taken into account. [ABSTRACT FROM AUTHOR]