Awareness of Uncertainty as a Prelude to Learning

ABSTRACT: Students commencing first year Physics subjects often have a poor understanding of the basic ideas of Newtonian mechanics despite having considerable experience in studying the topic. Because the topic is not new, students pay less attention to it than they should, and often emerge from the subject with their misunderstandings intact. If students can be jolted into re-examining their ideas, they are more likely to learn effectively.

One way of encouraging students to rethink their ideas is to require them to complete a multiple-choice test with a difference. Instead of being asked to select the best answer, students are asked to state their degree of certainty about whether each alternative is correct or incorrect. They become more highly motivated to re-learn Newtonian mechanics when they realize that some of their ideas are incorrect, and even their correct ideas are not held with confidence.

A sample test of this type is presented and student responses are compared to responses of a similar group of students to the same questions in standard multiple-choice format. The analysis indicates the extent to which an admission of uncertainty may affect the students' response.

Force Concept Inventory Item 24 The diagram below shows a rocket coasting in space in the direction indicated by the arrows. Between N and P no external force acts on the rocket. When it reaches point P, the rocket fires its engines as shown to provide a constant acceleration until it reaches a point Q in space. Which path best represents the path of the rocket between P and Q?
Figure 1: Force Concept Inventory Item 24.

The responses given by the students are shown in Figure 2. Just over half the class has chosen the correct response, (e); the most popular incorrect response, (c), chosen by more than a quarter of the class, is consistent with the belief that a constant force causes a constant component of velocity in the direction of the force.

Such a test is useful in identifying the areas where a large part of the class has a deficient understanding, and also in indicating to students the topics where their understanding must be improved. However, by its very nature, a multiple-choice test gives an illusion of certainty even where no such certainty exists. One student quizzed about giving the answer (d) to this question said:

"Well I knew it wasn't (a), (b) or (c), so that just left (d) and (e), and I picked the wrong one."

Figure 2: Response histogram for Physics I students.

This implies that the student thinks that:

• (d) and (e) are equally `likely' or `valid';
• he `nearly' knows the answer because his final choice included the correct response.

A proposal for encouraging this has been outlined by Acredolo and O'Connor (1991), and used by Rowell, Dawson and Madsen (1993) and by Rowell and Pollard (1995) with science students in different situations. Instead of choosing the best answer, students are asked to indicate their belief about each alternative by completing Table 1. They are also told that one answer does not necessarily exclude another, so they may regard more than one answer as correct. In addition, there is space for the student's own answer which is designated (f), if it is different from any of those given.

The responses of about 100 General Physics students to Item 24 are shown in Figure 3. The overall picture looks fairly similar to that for Physics  I students. About half the class identified the correct response as either `definitely' or `probably' a correct answer, and the most common alternative choice was again (c). Further analysis of the pattern of responses allows a comparison with the responses of the Physics I class.

The top row of Table 2 shows the responses of students who were confident that their chosen answer is correct, and that the others answers are incorrect. Less than 20% of the students had this degree of confidence, though the percentage of these students making the correct choice was higher than in the class as a whole.

The lower rows allow us to estimate the choice each student would have made if required to choose the best response. The most likely answer is recorded in Row 2 for students who chose one response as `definitely correct' and the remainder as either incorrect or `probably correct', and also for those who chose one response as `probably correct' and the remainder as either incorrect or `maybe correct'. Rows 3 and 4 show the number of cases where 2 or 3 responses respectively were accorded equal probability of correctness. The last row gives an estimate of the way the students would have responded if they had been asked to make a definite choice; it assumes that those with equally probable alternatives would have made a random choice for their single response.

Figure 3: Histogram for General Physics students (Item 24)

Figure 4: Comparison of responses from General Physics and Physics I students to Item 24.

In Figure 4, the deduced responses to this Item for General Physics students are compared with the responses given by Physics I students in the standard multiple-choice test. The comparison shows reasonable similarity between the two cohorts of students. The difference may be attributable to statistical fluctuations, and perhaps to some difference in the student groups, since the General Physics class comprises students who do not intend to continue with physics beyond first year, including some who have not studied Year 12 Physics.

A similar comparison can be made of the responses to Force Concept Inventory Item 13, where the choice of response seems to be much more clear-cut.

A large truck breaks down on the road and receives a push by a small car as shown in the diagram. While the car, pushing the truck, is speeding up to get to cruising speed: the magnitude of the force with which the car pushes the truck is equal to the magnitude of the force with which the truck pushes the car. the magnitude of the force with which the car pushes the truck is less than the magnitude of the force with which the truck pushes the car. the magnitude of the force with which the car pushes the truck is greater than the magnitude of the force with which the truck pushes the car. the car's engine is running so it applies a force as it pushes the truck, but the truck does not apply a force to the car. neither the car nor the truck exerts a force on each other; the truck is pushed forward because it is in the way of the car.

Figure 5: Force Concept Inventory Item 13.

The analysis of the responses of the General Physics class is shown in Table 3. About one third of the students are confident that a single answer is correct, with less than a third of the class confidently choosing the correct response. Two students identified as `definitely correct' both the statements (a) and (c), and several students identified both these statements as being `probably correct'.

The distribution of responses for General Physics students is shown in Figure 6.

In Figure 7, the estimate of the single responses these students would have chosen is compared with the responses actually given by the Physics I students; it shows a surprising result. More than 60% of the General Physics students identified the correct response as the answer most likely to be correct, whereas only 40% of the Physics I students actually chose the correct response.

Figure 6: General Physics students' responses to Item 13.

Figure 7: Comparison on responses from General Physics and Physics I students.

The degree of uncertainty is presented in more detail in Table 4. A single answer was chosen as `definitely correct' or `probably correct' by 70 students, with 63 of them choosing the answer which is actually correct. There were 16 students who identified 2 possibly correct responses (with varying degrees of certainty) and 12 who identified 3 possibly correct responses. Thus even in a situation which seems fairly clear-cut, almost a third of the students could not make a definitive choice.

A similar analysis on Item 24, presented in Table 5, shows far greater indecision, with only a quarter of the students choosing a single answer as correct.

This comparison of responses to the same test items in situations where different degrees of commitment were required shows that:

• students may be much less confident than their responses to a standard multiple-choice test indicate; and

• asking students to explicitly indicate their degree of certainty may change the way they think about the question.

To determine the change in students' concepts after the lecture series, students were given the same test, to be answered in the same way, during the final mechanics lecture. There was a small improvement in the responses, both in the number of students choosing correct answers and in the number being sure that incorrect answers were actually incorrect. When disappointment was expressed to my Tutorial group, about how small a change had occurred, the students said:

"You didn't give as a chance to revise."

To my suggestion that these questions tested understanding of mechanics at such a fundamental level that revision would be neither necessary nor helpful, they responded:

"We've been thinking that way for 17 years. You can't expect to change our whole way of thinking after only 17 lectures."

This is a reminder that non-Newtonian ideas are very firmly held by many students. The process of changing them is painfully slow, but it begins by persuading students that a change is needed. One way to do this is to give students an opportunity to see that their present understanding is inadequate since it does not allow them to give confident answers to questions about mechanics.

References

A B Arons (1990) A Guide to Introductory Physics Teaching, Wiley, New York.

D Hestenes, M Wells and G Swackhamer (1992) Force concept inventory, The Physics Teacher, 30, pp. 141-158

L C McDermott (1993) How we teach and how students learn, Australian and New Zealand Physicist, 30, pp. 151-163.

J A Rowell, C Dawson and P Madsen (1993) Probing students' non-scientific conceptions: a new tool for conventional and action-research in science teaching, Australian Science Teachers Journal, 39, pp. 62-68.

J A Rowell and J M Pollard (1995) Raising awareness of uncertainty: a useful addendum to courses in the history and philosophy of science for science teachers?, Science and Education, 4, pp. 87-97.