Research on Engineering Design

How Students Learn Engineering with Computer-Aided Design and Fabrication Tools

Computer-aided design (CAD) tools are fundamentally important in engineering. They also have great potential in transforming engineering classes. The objectives of this pilot study are: 1) understand the pros and cons of using CAD tools in student engineering projects, 2) identify different patterns in which students use the tools to solve design problems, 3) find effective instructional strategies for using CAD tools, and 4) investigate what it will take to make CAD tools work to their full potential in the classroom.

Time series analysis

We are developing a performance assessment tool based on time series analysis to shed light on how students solve design challenges and develop their design thinking. Time series analysis comprises methods for analyzing a time sequence of data in order to reveal characteristics of the data. A simple type of time series tool is an action logger that records the number of the learner's design actions at a given frequency (say, 2-4 times per minute) during a design session. These design actions are defined to be the "atomic" actions stored in the undo manager of the CAD tool (not random mouse clicks). The time series of actions approximately describes the user's activity level during a design period. At first glance, a time series may look stochastic. But there are complicated driving forces behind it (such as those economic forces that drive the Dow Jones Index). The goal of time series analysis for measuring engineering design learning gains is to detect if there is any consistent improvement of student design over time.

Detailed analysis of student design logs from an urban design project

Categories of design actions that are logged:

Build	Revise	Switch
b1: build a foundation b2: build a wall/walls b3: build a roof b4: build a window b5: build a door b6: build a floor b8: build a sidewalk	r1: revise a foundation r2: revise a wall (resize, delete) r3: revise a roof (reshape) r4: revise a window (resize, delete) r5: resize a door r6: revise a floor r7: revise a building (resize, move, or add) r8: revise a sidewalk	o1: Open another design or template

Female student F1: Screenshots of design solutions (click to enlarge, * final design)

a	b	c	d*

Design process of F1: compact timeline (click the image to enlarge)

Design activity of F1: accurate timeline (click the image to enlarge)

Female student F2: Screenshots of design solutions (click to enlarge, * final design)

a	b	c	d	e*	f

Design process of F2: compact timeline (click the image to enlarge)

Design activity of F2: accurate timeline (click the image to enlarge)

Female student F3: Screenshots of design solutions (click to enlarge, * final design)

a	b	c	d	e*

Design process of F3: compact timeline (click the image to enlarge)

Design activity of F3: accurate timeline (click the image to enlarge)

Female student F4: Screenshots of design solutions (click to enlarge, * final design)

a	b	c	d	e	f	g*

Design process of F4: compact timeline (click the image to enlarge)

Design activity of F4: accurate timeline (click the image to enlarge)

Male student M1: Screenshots of design solutions (click to enlarge, * final design)

a	b*	c

Design process of M1: compact timeline (click the image to enlarge)

Design activity of M1: accurate timeline (click the image to enlarge)

Male student M2: Screenshots of design solutions (click to enlarge, * final design)

a	b	c	d	e*

Design process of M2: compact timeline (click the image to enlarge)

Design activity of M2: accurate timeline (click the image to enlarge)

Male student M3: Screenshots of design solutions (click to enlarge, * final design)

a	b*	c

Design process of M3: compact timeline (click the image to enlarge)

Design activity of M3: accurate timeline (click the image to enlarge)

Male student M4: Screenshots of design solutions (click to enlarge, * final design)

a*	b	c

Design process of M4: compact timeline (click the image to enlarge)

Design activity of M4: accurate timeline (click the image to enlarge)

Male student M5: Screenshots of design solutions (click to enlarge)

a	b	c	d

Design process of M5: compact timeline (click the image to enlarge)

Design activity of M5: accurate timeline (click the image to enlarge)

Male student M6: Screenshots of design solutions (click to enlarge)

a*	b

Design process of M6: compact timeline (click the image to enlarge)

Design activity of M6: accurate timeline (click the image to enlarge)

Summary

Action intensity

Students	F1	F2	F3	F4	M1	M2	M3	M4	M5	M6	M7
Highest action intensity in a period	3.9	4.1	4.0	5.2	4.9	7.4	4.4	4.9	9.2	3.3	6.1

"Build vs. revise"

Assembled timelines of part counts

Comparing ten students (six males and four females, click the image to enlarge)

Students	F1	F2	F3	F4	M1	M2	M3	M4	M5	M6	M7
Maximum number of parts in a design	49	41	30	58	88	70	133	220	35	26	67

Limitations of time series analysis

Time series analysis has limitations in measuring learning. Time series data should be juxtaposed with other artifacts gathered from students to provide a more comprehensive picture of design learning. Despite the limitations, the time series analysis represents a promising step towards a more rigorous methodology for performance assessment of engineering design.

Product performance analysis

Product analysis is a method of assessing students' performances based on evaluating their products. It is based on the hypothesis that the performance of the product and the performance of the producer are highly correlated. The following tables shows the houses high school students designed and made with or without using our Energy3D, a simplified CAD/CAM tool that we have developed from scratch to support engineering education and research.

School A: 176 high school students in eight classes