Research on Engineering Design

Process Analaytics will Reveal How Students Learn Science and Engineering through Creative Processes

How secondary students learn and apply science concepts in engineering design processes is one of the most fundamental research topics in learning sciences. Although previous research suggests that engineering design is an effective pedagogical approach to promoting science learning, there are also concerns about the so-called "design-science gap" that fails science learning in design projects. To expand the research foundations for understanding and solving this problem, we are conducting a large-scale study based on our Energy3D. The datasets to be collected will be large in two ways. First, over 3,000 students from diverse socioeconomic backgrounds will participate in this project. Second, each design challenge will require 5-7 hours of classroom time to complete. Throughout the entire design process, Energy3D will automatically log and sort all user actions, electronic notes, and design snapshots (collectively referred to as process data hereafter) of each student at an extremely fine-grained level.

The process data are critical to explaining student learning outcomes and effects of interventions. But the process data are so complex and large that they offer little clue at first glance. To probe deeply into student learning, this project will develop, refine, and apply a set of computational techniques to analyze these large, high-dimensional process datasets. These techniques (collectively referred to as process analytics hereafter) will quantify each individual student's learning progress and provide a holistic method to assess his/her performance. For example, these process analytics will be used to detect iterative cycles in a design process, measure the design space explored by a student, gauge the extent and effect of scientific inquiry throughout a design process, and diagnose the bottlenecks that prevent students from completing design tasks and reaching learning goals. Some of these analytics will be the computational counterparts of traditional performance assessment methods based on student articulation, classroom observation, or video analysis. Combining these process analytics with pre/post-test results and demographic data, this project will address research questions related to the "design-science gap" from the following three aspects:

1. Patterns and relationships in engineering design processes: What are the common patterns of student design behaviors and how are they associated with prior science and engineering knowledge, project duration, design performance, learning outcomes, and demographic factors?
2. The effect of engineering design processes on science learning outcomes: How do students deepen their understanding of science concepts involved in engineering design projects? For example, to what extent does design iteration contribute to science learning?
3. The effect of scientific inquiry processes on engineering design outcomes: How often and deeply do students use (simulated) scientific experimentation to make a design choice? To what extent does experimentation trigger students to revise a design or add a new feature?