PHYSICS EDUCATION RESEARCH SECTION The Physics Education Research Section 共PERS兲 publishes articles describing important results from the field of physics education research. Manuscripts should be submitted using the web-based system that can be accessed via the American Journal of Physics home page, http://www.kzoo.edu/ajp/, and will be forwarded to the PERS editor for consideration.

Connecting symbolic difficulties with failure in physics Eugene T. Torigoea兲 Department of Physics, Allegheny College, Meadville, Pennsylvania 16335

Gary E. Gladding Department of Physics, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801

共Received 3 February 2010; accepted 19 August 2010兲 We find that symbolic physics questions are significantly more difficult than their analogous numerical versions. Very few of the errors are due to manipulation errors of the symbolic equations. Instead, most errors are due to confusions of symbolic meaning. We also find that performance on symbolic questions is more highly correlated with the overall performance than performance on numeric questions. We devised a coding scheme that distinguishes questions based only on the mathematical structure of the solutions. The coding scheme can be used to identify both difficult and discriminating physics questions. The questions identified by this coding scheme require an algebraic representation and discourage problem solving strategies that do not require an understanding of symbolic equations. Our results suggest that an inability to interpret physics equations may be a major contributor to student failure in introductory physics. © 2011 American Association of Physics Teachers.

关DOI: 10.1119/1.3487941兴

I. INTRODUCTION Our work is motivated by the assumption that poor mathematical preparation is a major reason why many students fail introductory physics. Physics instructors often encounter students with mathematical difficulties, which hinder their ability to be successful. In our earlier work, we quantified mathematical difficulties in introductory physics by comparing the differences in the score on numeric and symbolic versions of the same question.1,2 We gave a question in which the numeric version had an average of 95%, and the otherwise equivalent symbolic version had an average of 45%. Although some errors originated from incorrect algebraic manipulations, the great majority of the errors were related to the confusion of the meaning of the symbols. We calculated the ratio of the symbolic score to the numeric score for the two numeric-symbolic pairs of questions we studied 共see Fig. 1兲. A comparison of subgroups of the class based on the overall course performance showed that the top 1/4 of students showed little difference in performance between the versions, but there was a large difference in performance for the bottom 1/4 students. Of the latter group of students who could correctly solve the numeric version 共80% of the group兲, the ratio for these questions suggests that only 40% would also be able to solve the symbolic version. We believe that the mathematical difficulties underlying the differential performance on numeric and symbolic questions reflect students’ algebraic difficulties. As a result, education research concerning the teaching and learning of algebra is relevant to the teaching of physics. The emphasis of 133

Am. J. Phys. 79 共1兲, January 2011

http://aapt.org/ajp

much of this research is on the facilitation of the transition between the arithmetic and the algebraic stages of development. The difficulties that students have making the transition from the arithmetic stage to the algebraic stage foreshadow the symbolic difficulties that we have observed in college students. Although the arithmetic stage emphasizes the calculation of numeric results, the algebraic stage emphasizes the setup of symbolic equations. When students make the transition from arithmetic to algebra, they are often confused by algebraic expressions because the two subjects are similar enough that students can recognize the symbols used, but sufficiently different that students have difficulty understanding the meaning of the expressions that use those symbols. Kieran3,4 noted that in arithmetic, mathematical expressions contain procedural information about how to proceed to the solution. For example, 3 + 5 = ?, contains the symbol + to denote that we must use the addition to find the solution, and the equal sign, which is used to assert the solution to the expression. In contrast, algebraic expressions express relations and do not necessarily yield information about the process to solve for an unknown in the equation. Expressions in arithmetic are thought of only as a process to a numerical solution. In algebra the expressions must be understood both in terms of the process to solve for a particular quantity and as an expression of a relation between quantities. For example, from the algebraic perspective, the expression a + b is thought of as a process of combining the values of a and b and also as an object that represents the sum of a and b. The transition is especially troublesome for students be© 2011 American Association of Physics Teachers

133

Write an equation using the variables S and P to represent the following statement: “There are six times as many students as professors at this university.” Use S for the number of students and P for the number of professors.

Fig. 1. Ratio of the symbolic version score to the numeric version score for different subgroups of the class averaged over two questions analyzed in Ref. 1. We interpret this ratio to represent the likelihood that the students who could solve the numeric version correctly would also solve the symbolic version correctly.

cause procedures that successfully solve questions in arithmetic often fail for questions in algebra. Filloy and Rojano5 showed that although solutions of the form Ax + B = C can be solved using arithmetic methods, they cannot be used to solve equations of the form Ax + B = Cx + D. The following question can be solved with a series of simple calculations, which makes symbolic representation unnecessary. Daniel went to visit his grandmother, who gave him $1.50. Then he bought a book costing $3.20. If he has $2.30 left, then how much money did he have before visiting his grandmother?6 If the question is designed to use an equation Ax + B = Cx + D, then students must first represent the relation mathematically before calculating a number. In the following question, if the student is unable to reconceptualize the question, they must use an algebraic representation. The Westmont Video shop offers two rental plans. Then first plan costs $22.50 per year plus $2.00 per video rented. The second plan offers a free membership for one year but charges $3.25 per video rented. For what number of rental videos per year will these two plans cost exactly the same?6 Similar results have also been observed by Larkin et al.7 while studying novice and expert problem solving procedures in physics. They found that two equation and two unknown simultaneous-equation questions where both unknowns appear in both equations could not be solved by novice students. The common backward working strategy employed by novice students was not effective during the interviews. Similar to algebraic solutions with the form Ax + B = Cx + D, simultaneous-equation questions require students to setup the algebraic relations before they can calculate a number. Clement and colleagues8 found that many introductory physics students have difficulty mathematically representing relations between quantities. They studied variations of what is known as the “Students and Professors” problem. 134

Am. J. Phys., Vol. 79, No. 1, January 2011

Only 63% of first-year engineering students were able to correctly answer this question.8 A more recent study involving introductory physics students found a similar percentage.9 The most common error was a reversal of the correct quantitative relation 共6S = P instead of S = 6P兲. Although it might first appear that this error was due to carelessness, the authors found that for many students this error was due to an alternative conception of the meaning of algebraic equations.8 During interviews many students who made this reversal error knew which group was larger, but believed that the larger number should be placed next to the letter representing the larger group. In an equation such as 6S = P, the symbol S is used to signify a single student rather than the number of students. This use is similar to the use of labels in a unit conversion such as 100 cm= 1 m, in which we consider cm to mean a single centimeter rather than the number of centimeters. In a related study, Soloway et al.10 showed that when the question was asked in the context of a computer program, the likelihood of success was significantly higher 共87% correct兲 than when asked in the context of an algebraic expression. The authors demonstrated that although many students could successfully express the relation in the context of a numeric computation, fewer could correctly produce the relation in the context of an algebraic representation. These results suggest that there are many students in introductory physics with major difficulties with algebraic relations. In this paper, we explore the numeric and symbolic distinction with a greater number of question pairs than in our previous work1 and determine the correlations between performance on questions that require symbolic representation and overall success in introductory physics.

II. EXPANDED STUDY OF QUESTIONS IN MECHANICS In our previous study1 we investigated numeric and symbolic versions of two kinematics questions. These questions both had high numeric version scores and involved onedimensional kinematics. To understand different question properties, we expanded our study of numeric and symbolic questions to include ten pairs of questions that span many different introductory mechanics topics and a variety of difficulty levels. The 765 students in the study were enrolled in the calculus-based introductory mechanics course, Physics 211, at the University of Illinois at Urbana-Champaign in Spring 2007. Students in Physics 211 take three midterm exams and a multiple-choice cumulative final.11 The students completed one of the two randomly administered versions of the final exam. Each version of the final contained the numeric or the symbolic version of each of the ten questions. To ensure that any differences between the versions were not due to systematic differences between the groups, we compared the average midterm exam score of the students for each of the final exam versions. The average midterm E. T. Torigoe and G. E. Gladding

134

Table I. Average and standard error for numeric and symbolic versions of each question in the study. The p-values were calculated using a two-tailed t-test. The p-value represents the likelihood that such a difference could be observed under the assumption that the null hypothesis is true.

Numeric Symbolic Difference p-value

Question 1

Question 2

Question 3

Question 4

Question 5

Question 6

Question 7

Question 8

Question 9

Question 10

91.5⫾ 1.4 70.4⫾ 2.3 21.1⫾ 2.7 ⬍0.001

93.3⫾ 1.3 56.8⫾ 2.5 36.5⫾ 2.8 ⬍0.001

79.6⫾ 2.1 63.4⫾ 2.4 16.2⫾ 3.1 ⬍0.001

90.5⫾ 1.5 82.3⫾ 1.9 8.1⫾ 2.4 ⬍0.001

44.9⫾ 2.5 31.9⫾ 2.3 13.0⫾ 3.4 ⬍0.001

61.2⫾ 2.4 52.9⫾ 2.5 8.3⫾ 3.4 0.01⬍ p ⬍ 0.05

76.0⫾ 2.1 75.6⫾ 2.1 0.4⫾ 2.9 0.5⬍ p

33.2⫾ 2.3 29.8⫾ 2.2 3.4⫾ 3.1 0.2⬍ p ⬍ 0.4

78.6⫾ 2.0 54.5⫾ 2.5 24.1⫾ 3.2 ⬍0.001

48.8⫾ 2.3 52.8⫾ 2.4 −3.9⫾ 3.3 0.2⬍ p ⬍ 0.4

score and standard error for final 1 was 78.9⫾ 0.5 and for final 2 was 78.6⫾ 0.5, and we conclude that the two groups were equivalent. A sample of some question pairs used in the study is found in Appendix A.12 All but one of the ten paired questions contained analogous choices for each the numeric and symbolic versions of the question.13 To discourage cheating, many of the choices were in a different order in the two versions.

1. Physics difficulty The score for the top 1/4 on the numeric version was used as a measure of each question’s physics difficulty. Questions 5 and 8 were significantly more difficult for the top 1/4 students than the other questions, and the difficulty of the physics content overwhelmingly dominated the mathematical difficulty of the questions. Consequently, we removed them from further analysis in this study. This removal is supported by the fact that the average scores by the bottom 1/4 for the numeric versions of these questions was 23%, consistent with the random guessing rate of 20%.

A. Results Table I shows the score, the standard error, the difference, and the p-value of the difference for the numeric and symbolic versions of each question.14 Although there exist large differences between numeric and symbolic scores for some questions, which is consistent with our earlier findings, other questions show very little difference in score between versions. In the following, we describe properties of the questions that influenced the difference in score between versions. An analysis of each of the ten questions, including the popularity of the correct choices and incorrect choices, and an analysis of students’ written work were conducted to understand why some questions showed differences between the versions, while others did not. Table II shows the difference in scores between versions and the associated question properties. Many of the question properties we identified relate to the meaningful use of symbolic equations. The following is a list of question properties with explanations.

Table II. Questions ranked by the difference in the score between numeric and symbolic versions of each question with the associated question properties. The abbreviations are MultEq 共multiple equations in the solution兲, MoGE 共manipulation of general equations兲, CompExp 共use of a compound expression兲, SingEq 共single equation in solution兲, SimulEq 共simultaneous equations兲, ManipErr 共manipulation error兲, Diff 共difficult question兲. Difference

Question

Question properties

36.5⫾ 2.8 24.1⫾ 3.2 21.1⫾ 2.7 16.2⫾ 3.1 13.0⫾ 3.4 8.3⫾ 3.4 8.1⫾ 2.4 3.4⫾ 3.1 0.4⫾ 2.9 −3.9⫾ 3.3

2 9 1 3 5 6 4 8 7 10

MultEq, MoGE, CompExp MultEq, MoGE, CompExp MultEq, MoGE MultEq, MoGE Diff MultEq, CompExp SingEq, ManipErr Diff SingEq SimulEq

135

Am. J. Phys., Vol. 79, No. 1, January 2011

2. Multiple equations This property distinguishes whether the problem is commonly solved with one equation or with multiple equations. The symbolic questions tend to be more difficult because symbol confusion often occurs when equations are combined. The presence of multiple equations allows for the possibility of confusion of the same type of quantity 共for example, confusing two different velocities兲. Questions 1–3, 6, and 9 require multiple equations for their solution. Questions 4 and 7 are the only questions in which the solution can be found with a single equation. 3. Simultaneous equations The solution is simultaneous if, for example, there are two equations and two unknowns and both unknowns are present in both equations. In contrast, a sequential set of equations has an equation containing only a single unknown. Although numeric values cannot be obtained until the equations are combined in the first case, a numeric value can be obtained immediately from one of the sequential equations. Question 10 is the only question in our sample that requires simultaneous equations for the solution. The numeric and symbolic versions were equally difficult because students are forced to use the same procedure to solve both the numeric and the symbolic versions. 4. General equation manipulation This property signifies whether it is possible to obtain one of the incorrect choices by combining general equations or by manipulating a single general equation with minimal changes 共for example, replacing x by d兲. This property enhances the difference in score between the numeric and the symbolic versions because it penalizes students who do not meaningfully use the equations. Questions 1–3 and 9 contain incorrect choices corresponding to the blind manipulation of the general equations. This error was the most popular incorrect choice for Questions 1–3. E. T. Torigoe and G. E. Gladding

135

Fig. 2. Ratio of the symbolic version score to the numeric version score for different subgroups of the class for all ten questions.

5. Use of a compound expression This property signifies that to reach the correct symbolic solution students must replace a variable in a general equation with a compound expression 共for example, replacing the variable v by a more specific compound expression v / 2兲. Questions 2, 6, and 9 require that students specify a variable as a compound expression. For question 2, the second most popular incorrect choice corresponded to using v instead of v / 2 or t instead of t / 2. For question 6, the most popular incorrect choice corresponded to a failure to specify the variable m as 2M. Question 9 required that students specify both the variable m as 3m and the variable v as v / 3. 6. Manipulation error This property signifies that a common error on this question is related to an incorrect manipulation of a symbolic equation. The mean score for the symbolic version of question 4 was 8.2% higher than the numeric version because of this effect. An analysis of student work found that the difference in score was in large part due to students who setup the energy conservation equation correctly, but who choose an option corresponding to an incorrect manipulation of that equation. B. Connection to failure in physics To study the relation between the specific questions and the overall course performance, we divided the class into three subgroups based on the total course points. The groups are the bottom 1/4, the middle half, and the top 1/4. For each group and for each question in the study, we calculated the ratio of the average score on the symbolic version to the average score on the numeric version 共see Fig. 2兲. This ratio represents the fraction of the students who solved the numeric version correctly and who would also be able to solve the symbolic version. Even though no student was given both versions of a single question, this interpretation is justified because of the equivalence of the midterm exam averages of the students given each version of the final exam 共see Sec. II兲. In Sec. I we showed that an analysis of data from our previous study1 revealed that the lowest ratio of symbolic to the numeric scores was observed for the bottom 1/4 of the class. Our present results are consistent with that result. For almost all questions, the smallest ratio was observed for the bottom 1/4 of the class. In contrast, the top 1/4 rarely showed 136

Am. J. Phys., Vol. 79, No. 1, January 2011

any distinction between the numeric and symbolic versions. For the top 1/4 only, one question had a ratio lower than 0.93. A two-tailed t-test was performed to compare the score for the bottom 1/4 on the numeric and symbolic versions of each question. Questions 4, 7, 8, and 10 were the only questions for which p ⬎ 0.05, and thus the hypothesis that the performance by the bottom 1/4 on the numeric and symbolic versions of these questions were equivalent remains tenable.15 The high ratio for questions 8 and 10 is due to an equally poor performance on both versions of those questions. Question 4 and 7 were the only questions that could be solved with single equations, and the error for question 4 was related to a manipulation error. The lack of a correlation indicates that those types of question properties do not correlate well with the overall success in the course. The low ratios for the bottom 1/4 for the remaining questions, which contain properties that stress meaningful representation, suggests that the bottom 1/4’s algebraic difficulties may be related to their poor performance in physics. III. CODING QUESTIONS BY MATHEMATICAL STRUCTURE Many of the question properties described in Sec. II A were discovered after the research was performed and were not independently validated. In this study, we attempt to validate those question properties by analyzing an entire semester of exam questions. Specifically, we are interested in determining if question properties that stress meaningful algebraic representation can be used to identify discriminating exam questions. We examined the set of exam questions administered in Spring 2006 in Physics 211 and categorized them by the mathematical properties of their solutions. The exam questions were made by faculty members with no knowledge of the question properties we intended to study. There were 169 unique16 multiple-choice questions administered 共three midterms and two versions of the final exam兲 and 870 students who completed the course without any missing exam grades. We analyzed the mathematical structure of the solutions to all of the questions in our sample and coded for the following types of questions: 共1兲 Multiple-equation symbolic questions; 共2兲 simultaneous-equation numeric questions; and 共3兲 single-equation numeric questions where the target unknown appears on opposite sides of the equal sign. This last property is equivalent to the form Ax + B = Cx + D, which is a structure that requires students to use algebraic methods. An example of a question with a solution of this form can be found in Appendix B. We referred to questions with any of these three properties as equation priority questions. As the name suggests, the solutions to these questions emphasize the meaningful representation of symbolic equations. The three question properties require students to formally represent the equations and prevent simple one equation at a time numeric sequential solutions. Specifically omitted were single-equation and sequential-equation numeric questions, both of which were shown in Sec. II B to be poorly correlated with the overall performance in the course. Two additional properties identified in Sec. II A were not included in this coding scheme: General equation manipulation and the use of compound expressions. Even though we have some evidence that these properties tend to make symbolic problems more difficult, such a distinction is not imE. T. Torigoe and G. E. Gladding

136

Table III. Comparison of equation priority and other questions. The error on the mean difference was calculated using the distribution of question score differences between groups 共Ref. 18兲.

Number Bot. 1/4 score Top 3/4 score Mean difference Pearson r

Equation priority questions

Other questions

40 33.9% ⫾ 2.5% 62.0% ⫾ 2.9% 28.1% ⫾ 1.7% 0.38

129 56.9% ⫾ 1.8% 77.5% ⫾ 1.5% 20.6% ⫾ 0.9% 0.29

portant for this analysis. In this coding we were not interested in distinguishing one type of symbolic question from another, but questions whose mathematical structure require the formal setup of equations from those that do not. A. Results After analyzing the solutions to all of the exam questions, 40 out of 169 共24%兲 were coded as equation priority questions. One author and another experienced physics instructor agreed on 92% of the question categorizations. Disagreements were due to differences in judgment of what constituted a single-equation solution. The two raters agreed on the coding of all of the questions after a discussion of what the average student would consider to be a single equation. Although most of the solutions of the other, non-equation priority questions required equations, they did not satisfy our coding scheme. To study the ability of these questions to differentiate students who failed or were on the verge of failing from the rest of the class, we compared the performance of the bottom 1/4 and the rest of the class for both equation priority and the other questions. Table III shows that for both the bottom 1/4 and the rest of the class, the equation priority questions are significantly more difficult than other questions. This finding is consistent with our results in Sec. II A in which we found that questions with these mathematical properties were more difficult. Also consistent with our earlier findings is that symbolic difficulties are the most pronounced for the bottom 1/4 of students; we find that the difference in score between the bottom 1/4 and the rest of the class is larger for equation priority questions than the other questions. The mean difference for equation priority questions was 28.1% ⫾ 1.7% compared to 20.6% ⫾ 0.9% for other questions. A common measure of discrimination is the Pearson coefficient r.17 In this context r is a measure of the relation between the performance on individual questions and the overall performance in the course. We find an average value of r = 0.38 for equation priority coded questions and r = 0.29 for the other questions. Figure 3 shows a histogram of the values for r for both sets of questions. Although 50% of equation priority questions were highly correlated with course score 共r ⬎ 0.4兲, only 17.8% of the other questions were.

Fig. 3. Histogram of values for the Pearson correlation coefficient r for equation priority questions and the other questions.

discriminating. To address this concern, we performed an analysis using three methods for controlling for the effect of the question difficulty on the discrimination. First we combined the results for the class to determine the relation between difficulty and discrimination for the entire class. The size of the bins for the question score was 15%, except the highest bin that spanned the range from 90% to 100%. The size of the bins was chosen so that there were an adequate number of questions from each category. We calculated the fraction of questions that were highly discriminating 共r ⬎ 0.4兲 in each bin for both types of questions. Figure 4 shows that the fraction in each bin of highly discriminating equation priority questions is equal to or greater than that of the other questions. A chi-squared test of association was performed in each bin and it was found that the fraction of questions that were highly discriminating was significantly higher for equation priority questions in three bins: 30%–45%, 60%–75%, and 75%–90% 共p ⬍ 0.05兲. The discrimination is also reflected in the fact that the mean difference between the bottom 1/4 and the rest of the class was greater for equation priority questions than for the other questions. It is possible that this effect was due to the fact that the equation priority questions were also more difficult, on average, than the other questions. To control the

B. Analysis of systematic errors: The relation between difficulty and discrimination The higher discrimination of equation priority questions might be due to a positive correlation between difficulty and discrimination, that is, difficult questions tend to be more 137

Am. J. Phys., Vol. 79, No. 1, January 2011

Fig. 4. Equation priority and other questions were binned by their difficulty for the entire class. A histogram was created by calculating the fraction of highly discriminating questions 共r ⬎ 0.4兲 in each bin. E. T. Torigoe and G. E. Gladding

137

Table IV. Comparison between the bottom 1/4 and the rest of the class on groupings of questions based on fixing the score for the top 3/4 of the class. Other difficult non-equation priority questions were found by omitting easy questions until the other question score was the same as the equation priority question score for the top 3/4 of the class. Other matched questions were created by matching each equation priority question to other non-equation priority questions by difficulty for the top 3/4 of the class.

Number Bot. 1/4 score Top 3/4 score Mean difference Pearson r

Equation priority

Other difficult

Other matched

40 33.9% ⫾ 2.5% 62.0% ⫾ 2.9% 28.1% ⫾ 1.7% 0.38

57 40.7% ⫾ 1.9% 61.9% ⫾ 1.8% 21.2% ⫾ 1.3% 0.29

40 41.7% ⫾ 2.9% 62.3% ⫾ 2.8% 20.6% ⫾ 1.6% 0.29

effect of question difficulty, we used two methods of omitting the other questions so that the difficulty for the top 3/4 of the class on other questions was the same. This procedure allowed us to compare the performance of the bottom 1/4 on two sets of questions that were equally difficult for the top 3/4. The first method equalized the difficulty of both questions for the top 3/4 of the class by omitting the easiest other non-equation priority questions. The second method achieved the same result by matching the difficulty of each equation priority question to another non-equation priority question for the top 3/4 of the class. Table IV shows that a gap in the mean difference remains almost unchanged after these two corrections for question difficulty. The discriminatory ability between equation priority and other questions is unchanged even when different methods are employed to correct for question difficulty. Hence, the discriminatory ability of equation priority questions is not an effect of the difficulty of the questions. IV. DISCUSSION Our studies show the importance of mathematical structures in the difficulty and discrimination of questions in introductory physics. Questions whose structure force algebraic representation tend to be more difficult and more discriminating than questions that can be solved by a series of calculations. Even though these properties are not necessarily related to the physics content of the questions, performance on questions with these properties is correlated with the overall performance in the course. This relation may exist for a variety of reasons. One reason might be the teaching methods employed in introductory physics. Because symbolic algebra is vital to the understanding of the expert physicists who teach introductory physics, it is often a prominent component of instruction. At the University of Illinois symbolic derivations are often used to introduce new concepts, and symbolic example questions are often shown to demonstrate general problem solving procedures. However, students with algebraic difficulties may be easily confused by such methods. The correlations might also reflect that the ability of students to solve questions symbolically may increase the effectiveness of practice problems by allowing for a more general understanding of the solution. The ability to see the symbolic structure of a problem might improve the probability that students with this ability will be able to solve similarly structured questions on later exams. 138

Am. J. Phys., Vol. 79, No. 1, January 2011

Some research suggests that the structure of equations can assist the acquisition of physical concepts. Schwartz et al.19 found that equations help children develop an understanding of balance problems. They hypothesize that the structure of the equations supports the precision of conceptual ideas, alleviates working memory load, and allows for the organization of multiple parameters. Sloutsky et al.20 found that learning and transfer can be facilitated when knowledge is expressed in an abstract generic form. In introductory physics the ability to meaningfully construct and interpret symbolic equations may be beneficial to learning in all aspects of the course. Conversely, students who are unable to understand symbolic equations may learn less than those that do. The question properties we have identified penalize students who attempt to solve problems without understanding the meaning of the symbolic equations they use. Tuminaro21 observed an activity that he called “recursive plug-and-chug” while studying students working on homework questions in groups. In this activity students try to match variables to a list of equations. If they find an equation where the target quantity is the only unknown, they solve for the target quantity; if it is not, they replay the game until an appropriate equation is found. Although this procedure might result in the correct answer for some questions, it does not require an understanding of the underlying physics, and it is not very effective for simultaneous-equation and symbolic questions. Symbolic questions inhibit strategies like recursive plugand-chug by allowing for a greater amount of confusion of meaning. Questions that contain incorrect options corresponding to the manipulation of general equations and the failure to specify variables as compound expressions penalize students who use strategies that do not understand the meaning of the symbols and symbolic expressions. These confusions of meaning might be alleviated by the use of subscripts on symbols, but few students use them. Although seemingly trivial, subscripts allow experts to distinguish variables from specific quantities belonging to specific objects or individuals. A common error on symbolic questions was the confusion of two quantities of the same type. For example, combining two equations by inappropriately equating object A’s velocity with object B’s velocity. If students were in the practice of immediately writing subscripts after choosing a general equation, they would be forced to consider the meaning of the symbols that they were using. Conceptual exams like the force concept inventory 共FCI兲 have demonstrated that the ability to solve quantitative problems does not always indicate conceptual understanding. Mazur22 demonstrated that high average scores for complex quantitative questions can often be associated with low average scores on analogous conceptual questions with the same population of students. We suggest that the question properties we have identified can bridge the gap between quantitative and conceptual questions. The question properties we have described can be used by instructors to produce quantitative questions that emphasize meaningful symbolic representation. The use of these question properties may have the effect of reorganizing the reward structure in our physics courses. Students who were rewarded on exams and homework assignments for using strategies that did not require an understanding of the equations they used would not be rewarded with the types of questions we have described. Improving students’ ability to symbolically represent relaE. T. Torigoe and G. E. Gladding

138

Fig. 5. Diagram for question 2.

tions may be an important factor in lowering student failure in and retention through the introductory physics sequence. ACKNOWLEDGMENTS The authors would like to thank the reviewers for their helpful comments. The authors would also like to thank the members of the physics education research group at the University of Illinois at Urbana-Champaign. This material is based upon work supported by NSF DUE 0088734 and NSF DUE 0341261. APPENDIX A: QUESTIONS IN THE PHYSICS 211 SPRING 2007 FINAL EXAM STUDY The following are a sample of the questions in the Physics 211 Spring 2007 final exam study. The symbolic questions used the same wording but replaced the numbers with symbols. An asterisk has been placed next to the correct option for each question. Each option is accompanied in brackets by the percentages of all students, the bottom 1/4, the middle half, and the top 1/4 that choose that option. The sum of the percentages do not always add up to 100% due to rounding errors and students who left the question blank. Question 2 (numeric). A car can go from 0 to 60 m/s in 8 s. At what distance d from the start 共at rest兲 is the car traveling 30 m/s? 关Assume a constant acceleration 共see Fig. 5兲.兴 共a兲 30 m 关1%, 4%, 0%, 1%兴 共b*兲 60 m 关93%, 80%, 96%, 99%兴 共c兲 120 m 关3%, 10%, 2%, 0%兴 共d兲 240 m 关2%, 4%, 2%, 0%兴 共e兲 480 m 关0%, 1%, 0%, 0%兴 Question 2 (symbolic). A car can go from 0 to v1 in t1 seconds. At what distance d from the start 共at rest兲 is the car traveling 共v1 / 2兲? 关Assume a constant acceleration 共see Fig. 5兲.兴 共a兲 d = v1t1 关1%, 1%, 1%, 1%兴 共b兲 d = v1t1 / 2 关20%, 37%, 16%, 11%兴 共c兲 d = v1t1 / 4 关22%, 28%, 27%, 8%兴 共d*兲 d = v1t1 / 8 关57%, 35%, 56%, 81%兴 共e兲 d = v1t1 / 16 关0%, 0%, 0%, 0%兴 Question 9 (numeric). A block of mass M = 1.5 kg slides on a frictionless surface with a velocity V = 4 m / s. It strikes a second block of mass 2M = 3.0 kg that is at rest and is attached to a long, relaxed ideal 共massless兲 spring of spring constant k = 80 N / m. Assume that the blocks stick together after colliding and the collision takes place very quickly. Provided that the spring is stiff enough to stop the blocks before striking the wall, determine ␦x, the maximum amount the spring is compressed from its relaxed length 共see Fig. 6兲.

Fig. 6. Diagram for question 9. 139

Am. J. Phys., Vol. 79, No. 1, January 2011

Fig. 7. Diagram for question 10.

共a兲 ␦x = 0.18 m 关4%, 8%, 4%, 1%兴 共b*兲 ␦x = 0.32 m 关79%, 63%, 78%, 94%兴 共c兲 ␦x = 0.45 m 关6%, 11%, 5%, 2%兴 共d兲 ␦x = 0.55 m 关11%, 14%, 13%, 4%兴 共e兲 ␦x = 0.95 m 关1%, 4%, 0%, 0%兴 Question 9 (symbolic). A block of mass M slides on a frictionless surface with a velocity V. It strikes a second block of mass 2M that is at rest and is attached to a long, relaxed ideal 共massless兲 spring of spring constant k. Assume that the blocks stick together after colliding and the collision takes place very quickly. Provided that the spring is stiff enough to stop the blocks before striking the wall, determine ␦x, the maximum amount the spring is compressed from its relaxed length 共see Fig. 6兲. 共a兲 ␦x = 共3MV2 / k兲1/2 关14%, 33%, 12%, 0%兴 共b兲 ␦x = 共MV2 / k兲1/2 关13%, 17%, 16%, 2%兴 共c兲 ␦x = 共2MV2 / 共3k兲兲1/2 关5%, 9%, 6%, 0%兴 共d*兲 ␦x = 共MV2 / 共3k兲兲1/2 关55%, 29%, 50%, 90%兴 共e兲 ␦x = 共MV2 / 共9k兲兲1/2 关13%, 12%, 15%, 8%兴 Question 10 (numeric). A uniform disk of mass M = 8 kg and radius R = 0.5 m has a string wound around its rim. The disk is free to spin about a pin through the center of the disk. A mass M = 8 kg 共same mass as the disk兲 is connected to the string and is dropped from rest. What is the acceleration a of the block? 共See Fig. 7.兲 共a兲 a = 2.45 m / s2 关5%, 8%, 6%, 1%兴 共b兲 a = 3.27 m / s2 关8%, 16%, 7%, 3%兴 共c兲 a = 4.91 m / s2 关30%, 37%, 36%, 11%兴 共d*兲 a = 6.54 m / s2 关49%, 28%, 43%, 82%兴 共e兲 a = 7.36 m / s2 关7%, 11%, 7%, 4%兴 Question 10 (symbolic). A uniform disk of mass M and radius R has a string wound around its rim. The disk is free to spin about a pin through the center of the disk. A mass M 共same mass as the disk兲 is connected to the string and is dropped from rest. What is the acceleration a of the block? 共See Fig. 7.兲 共a兲 共b*兲 共c兲 共d兲 共e兲

a = 共3 / 4兲 ⴱ g 关8%, 10%, 10%, 2%兴 a = 共2 / 3兲 ⴱ g 关53%, 26%, 50%, 84%兴 a = 共1 / 2兲 ⴱ g 关32%, 45%, 35%, 15%兴 a = 共1 / 3兲 ⴱ g 关3%, 5%, 3%, 0%兴 a = 共1 / 4兲 ⴱ g 关4%, 11%, 3%, 0%兴 E. T. Torigoe and G. E. Gladding

139

7

Fig. 8. An example of a question requiring a single numeric equation where the target unknown appears on opposite sides of the equal sign.

APPENDIX B: SAMPLE EQUATION PRIORITY QUESTION We give an example of a question that requires a single numeric equation, where the unknown target appears on opposite sides of the equal sign. 共Physics 211 Spring 2006 exam 2, question 10.兲 Three boxes are arranged as shown. The middle box has a mass of 2 kg and accelerates to the right at 5 m / s2 on a horizontal frictionless table. The boxes to the left and right hang freely, suspended by strings over massless, frictionless pulleys. The tension in the left string is T1 = 10 N. What is the mass of the box M 1 on the left? 共See Fig. 8.兲 共a兲 共b兲 共c兲 共d兲 共e兲

M 1 = 0.675 kg M 1 = 1.91 kg M 1 = 2.13 kg M 1 = 3.75 kg M 1 = 4.16 kg

a兲

Electronic mail: [email protected] E. Torigoe and G. Gladding, “Same to us, different to them: Numeric computation versus symbolic representation,” in 2006 Physics Education Research Conference, edited by L. McCullough et al. 共AIP, New York, 2007兲, pp. 153–156. 2 E. Torigoe, “What kind of math matters? A study of the relationship between mathematical ability and success in physics,” Ph.D. dissertation, University of Illinois at Urbana-Champaign, 2008. 3 C. Kieran, “Cognitive processes involved in learning school algebra,” in Mathematics and Cognition: A Research Synthesis by the International Group for the Psychology of Mathematics Education, edited by P. Nesher and K. Kilpatrick 共Cambridge U. P., Cambridge, 1990兲, pp. 96–112. 4 C. Kieran, “The learning and teaching of school algebra,” in Handbook of Research on Mathematics Learning and Teaching, edited by D. Grouws 共Macmillan, New York, 1992兲, pp. 390–419. 5 E. Filloy and T. Rojano, “Solving equations: The transition from arithmetic to algebra,” For the Learning of Mathematics 9 共2兲, 19–25 共1989兲. 6 The examples shown are from Ref. 4. 1

140

Am. J. Phys., Vol. 79, No. 1, January 2011

J. H. Larkin, J. McDermott, D. P. Simon, and H. A. Simon, “Models of competence in solving physics problems,” Cogn. Sci. 4 共4兲, 317–345 共1980兲. 8 J. Clement, “Algebra word problem solutions: Thought processes underlying a common misconception,” J. Res. Math. Educ. 13 共1兲, 16–30 共1982兲. 9 E. Cohen and S. E. Kanim, “Factors influencing the algebra ‘reversal error’,” Am. J. Phys. 73 共11兲, 1072–1078 共2005兲. 10 E. Soloway, J. Lochhead, and J. Clement, “Does computer programming enhance problem solving ability? Some positive evidence on algebra word problems,” in Computer Literacy, edited by R. J. Seidel, R. E. Anderson, and B. Hunter 共Academic, Burlington, 1982兲, pp. 171–201. 11 M. Scott, T. Stelzer, and G. Gladding, “Evaluating multiple-choice exams in large introductory physics courses,” Phys. Rev. ST Phys. Educ. Res. 2 共2兲, 020102 共2006兲. 12 See supplementary material at http://dx.doi.org/10.1119/1.3487941 for all ten numeric and symbolic pairs of questions used in this study. 13 Question 4 was created by modifying an existing symbolic question. When numbers were introduced to create the numeric version, one of the symbolic options corresponded to an imaginary quantity. To ensure the similarity of all of the options, only the magnitude of this quantity was displayed in the numeric version. Two of the other five options for this question do not agree between the versions, but each of these options was chosen by 2% or less of the students. 14 The p-value represents the likelihood that such a difference can be observed under the assumption that the null hypothesis is true 共see Ref. 15兲. 15 G. V. Glass and K. D. Hopkins, Statistical Methods in Education and Psychology, 2nd ed. 共Prentice-Hall, Englewood Cliffs, NJ, 1984兲, pp. 229–235. 16 Some questions were common between the two versions of the final exam. 17 The discrimination of multiple-choice questions is most commonly measured using the point biserial coefficient of correlation because the result of a multiple-choice question is most commonly dichotomous. The multiple-choice questions in this study were analyzed using the Pearson correlation coefficient r, because students were given partial credit for multiple selections. As a result a student could receive a score of 0, 0.33, 0.5, or 1 on each question. 18 The error of the mean difference shown in Table III is less than what one would calculate if the errors for the top and bottom groups were combined in quadrature. To calculate the error shown, we took advantage of the fact that the difference in score between the top and bottom groups could be determined for each question. The error in the mean difference for the equation priority questions was determined by calculating the variance of the distribution of differences for the 40 equation priority questions. This process of pairing data is analogous to how one would calculate gains on the FCI by pairing the each precourse and postcourse score by student, rather than finding the mean difference between the average precourse score and the postcourse score for the class as a whole. 19 D. L. Schwartz, T. Martin, and J. Pfaffman, “How mathematics propels the development of physical knowledge,” Cognit. Dev. 6 共1兲, 65–88 共2005兲. 20 V. M. Sloutsky, J. A. Kaminski, and A. F. Heckler, “The advantage of simple symbols for learning and transfer,” Psychon. Bull. Rev. 12 共3兲, 508–513 共2005兲. 21 J. Tuminaro and E. F. Redish, “Elements of a cognitive model of physics problem solving: Epistemic games,” Phys. Rev. ST Phys. Educ. Res. 3 共2兲, 020101 共2007兲. 22 E. Mazur, Peer Instruction: A User’s Manual 共Prentice-Hall, Upper Saddle River, NJ, 1997兲, pp. 5–7.

E. T. Torigoe and G. E. Gladding

140