Research on Engineering Design

How Students Learn Engineering with Computer-Aided Design and Fabrication Tools

Computer-aided design (CAD) tools are fundamentally important in engineering. They also have great potential in transforming engineering classes. The objectives of this pilot study are: 1) understand the pros and cons of using CAD tools in student engineering projects, 2) identify different patterns in which students use the tools to solve design problems, 3) find effective instructional strategies for using CAD tools, and 4) investigate what it will take to make CAD tools work to their full potential in the classroom.

Time series analysis

We are developing a performance assessment tool based on time series analysis to shed light on how students solve design challenges and develop their design thinking. Time series analysis comprises methods for analyzing a time sequence of data in order to reveal characteristics of the data. A simple type of time series tool is an action logger that records the number of the learner's design actions at a given frequency (say, 2-4 times per minute) during a design session. These design actions are defined to be the "atomic" actions stored in the undo manager of the CAD tool (not random mouse clicks). The time series of actions approximately describes the user's activity level during a design period. At first glance, a time series may look stochastic. But there are complicated driving forces behind it (such as those economic forces that drive the Dow Jones Index). The goal of time series analysis for measuring engineering design learning gains is to detect if there is any consistent improvement of student design over time.

Detailed analysis of student design logs from an urban design project

Categories of design actions that are logged:

Build Revise Switch
  • b1: build a foundation
  • b2: build a wall/walls
  • b3: build a roof
  • b4: build a window
  • b5: build a door
  • b6: build a floor
  • b8: build a sidewalk
  • r1: revise a foundation
  • r2: revise a wall (resize, delete)
  • r3: revise a roof (reshape)
  • r4: revise a window (resize, delete)
  • r5: resize a door
  • r6: revise a floor
  • r7: revise a building (resize, move, or add)
  • r8: revise a sidewalk
  • o1: Open another design or template

Female student F1: Screenshots of design solutions (click to enlarge, * final design)
a b c d*

Design process of F1: compact timeline (click the image to enlarge)


Design activity of F1: accurate timeline (click the image to enlarge)

Female student F2: Screenshots of design solutions (click to enlarge, * final design)
a b c d e* f

Design process of F2: compact timeline (click the image to enlarge)


Design activity of F2: accurate timeline (click the image to enlarge)

Female student F3: Screenshots of design solutions (click to enlarge, * final design)
a b c d e*

Design process of F3: compact timeline (click the image to enlarge)


Design activity of F3: accurate timeline (click the image to enlarge)

Female student F4: Screenshots of design solutions (click to enlarge, * final design)
a b c d e f g*

Design process of F4: compact timeline (click the image to enlarge)


Design activity of F4: accurate timeline (click the image to enlarge)

Male student M1: Screenshots of design solutions (click to enlarge, * final design)
a b* c

Design process of M1: compact timeline (click the image to enlarge)


Design activity of M1: accurate timeline (click the image to enlarge)

Male student M2: Screenshots of design solutions (click to enlarge, * final design)
a b c d e*

Design process of M2: compact timeline (click the image to enlarge)


Design activity of M2: accurate timeline (click the image to enlarge)

Male student M3: Screenshots of design solutions (click to enlarge, * final design)
a b* c

Design process of M3: compact timeline (click the image to enlarge)


Design activity of M3: accurate timeline (click the image to enlarge)

Male student M4: Screenshots of design solutions (click to enlarge, * final design)
a* b c

Design process of M4: compact timeline (click the image to enlarge)


Design activity of M4: accurate timeline (click the image to enlarge)

Male student M5: Screenshots of design solutions (click to enlarge)
a b c d

Design process of M5: compact timeline (click the image to enlarge)


Design activity of M5: accurate timeline (click the image to enlarge)

Male student M6: Screenshots of design solutions (click to enlarge)
a* b

Design process of M6: compact timeline (click the image to enlarge)


Design activity of M6: accurate timeline (click the image to enlarge)

Summary

Action intensity

Students F1 F2 F3 F4 M1 M2 M3 M4 M5 M6 M7
Highest action intensity in a period 3.9 4.1 4.0 5.2 4.9 7.4 4.4 4.9 9.2 3.3 6.1

"Build vs. revise"


Assembled timelines of part counts


Comparing ten students (six males and four females, click the image to enlarge)

Students F1 F2 F3 F4 M1 M2 M3 M4 M5 M6 M7
Maximum number of parts in a design 49 41 30 58 88 70 133 220 35 26 67

Limitations of time series analysis

Time series analysis has limitations in measuring learning. Time series data should be juxtaposed with other artifacts gathered from students to provide a more comprehensive picture of design learning. Despite the limitations, the time series analysis represents a promising step towards a more rigorous methodology for performance assessment of engineering design.

Product performance analysis

Product analysis is a method of assessing students' performances based on evaluating their products. It is based on the hypothesis that the performance of the product and the performance of the producer are highly correlated. The following tables shows the houses high school students designed and made with or without using our Energy3D, a simplified CAD/CAM tool that we have developed from scratch to support engineering education and research.

School A: 176 high school students in eight classes

























School B: 69 high school students in four classes













School C: 77 high school students in three classes










Click an image to view an enlarged version. Photos taken by Edmund Hazzard.