EMAT 8990 – Seminar Spring 2006
Methodology for Assessment of Academic Content Review in a Game Format
Bill Daly
The purpose of this document is to outline a methodology to evaluate the beneficial or detrimental effect of using the EMAT7650 Applied Project, “AcademiaMania” game as a learning reinforcement tool. Briefly, the game is a jeopardy format, or 5x5 array, in which the rows are topic categories and the columns are point values. Ideally, the higher point values represent questions of greater depth of understanding or complexity. While this format of review has been met with student enthusiasm, its usefulness also needs to be gauged against other approaches to be certain that student gains are, as a minimum, break even. Furthermore, the methodology seeks to garner student feedback for the purpose of improving application of the review format.
The test setting for this report is introductory vectors at the beginning of the academic year for 11th grade honors Physics or AP-B Physics students.
At the outset, the sample size will be limited to 5 classes of students, just due to the practicality and scope of applying this study to one teacher’s students. Since it is impractical to attempt any type of blind study for the initial evaluation, to reduce influence of teacher disposition, this trial will be administered at the beginning of the year. Consequently, “Introductory Vectors” is an ideal topic area for the evaluation. It would be desirable the refine and expand the scope of the student to achieved a greater sample size and statistical confidence.
Although it is impossible to know the class makeup for next year, it is assumed am assuming it will be similar prior years. That is, in the morning, there will one AP class and one Honors class. In the afternoon there will be one AP class and two Honors classes. As a result of the limited sample size, “afternoon” and “morning” influences will be difficult to assess. The best that can be hoped for is to reverse the roles of the Honors and AP classes, versus morning and afternoon. Also, the fifth group, which is an Honors class may offer some insight into the time of day effect. Table 1 shows the preliminary make up of the control (conventional worksheet review) and the Test Group (AcamediaMania review). This will be adjusted as the class make up crystallizes.
|
Physics Class |
Estimated Number of Students |
Period |
Time |
Control Group (Conventional Review). |
Test Group (AcamediaMania Review). |
|
Honors |
24 |
1 |
7:30 – 8:25 |
|
X |
|
AP-B |
20 |
3 |
9:33 – 10:33 |
X |
|
|
Lunch |
20 |
|
|
|
|
|
Honors |
24 |
4 |
11:08 – 12:03 |
X |
|
|
Honors |
24 |
5 |
12:09 – 1:04 |
|
X |
|
AP-B |
20 |
6 |
1:10 – 2:12 |
|
X |
Table 1: Composition of Test and Control Groups.
A flow chart for assessment is shown in Figure 1.
|
|
|
|
|
|
Figure 1: Flow diagram for assessment methodology of game format review.
The vector pretest and lecture will be administered in a classroom setting. The control group will work on hardcopy worksheets for reinforcement of the lesson in a collaborative setting in groups of two. This will be done as an attempt to eliminate possible variability in the results. In other words, gains in one method over the other will not be attributable to collaborative versus individual study. The test group will be exposed to AcademiaMania in a computer lab setting in which student will compete with each other in pairs.
Following the reinforcement phase, effectiveness will be gauged on two fronts, student opinion and score gains between pre-test and post-test.. All students will be given the same post test. The results will be statistically analyzed to gauge score improvements. The questionnaire will be review more subjectively to assess student motivation to review the material for both methods. Although to control the variables and scope of this assessment, if results are break even to positive for the game format, future study should try to focus on the game format versus other review formats, such as Web Assign or individual review. Throughout the study, although not apparent on how to definitively determine this, the assessment should be sensitive to differentiation opportunities based on student demeanor or preference. .