Measurement, 6: 1–6, 2008 Copyright © Taylor & Francis Group, LLC ISSN 1536-6367 print / 1536-6359 online DOI: 10.1080/15366360802035471

FOCUS ARTICLE SET

On the Conceptual Foundations of Psychological Measurement Brian D. Haig University of Canterbury

Denny Borsboom University of Amsterdam This special double issue of Measurement addresses a number of foundational issues in psychological measurement. It assembles three focus articles, with associated peer commentary, and replies by the authors. The first article challenges the routine assumption that psychometrics is a quantitative science; it argues that there may be no such thing as psychological measurement at all. The second article lays out a systematic framework for latent variable modeling that is intended to capture the notion of psychological measurement in its terms. The third article seeks to provide a conceptual distinction between theoretical variables that allow for an extension in possible worlds, and theoretical variables that do not. These focus articles are followed by a number of associated peer commentaries and replies by the authors. Key words: pathological science, quantitative attributes, latent variable theory, epistemological status of latent variables, constructs and concepts, nonactualized possibilities

Correspondence should be addressed either to Brian D. Haig, Department of Psychology, University of Canterbury, Private Bag 4800, Christchurch, New Zealand. E-mail: [email protected], or Denny Borsboom, Department of Psychology, Faculty of Social and Behavioral Sciences, University of Amsterdam, Roetersstraat 15, 1018 WB, Amsterdam, The Netherlands. E-mail: [email protected]

2

HAIG AND BORSBOOM

Scientific research requires the connection of theoretical terms (e.g., force in physics, fitness in biology, or intelligence in psychology) to observations. This necessitates the use of observational procedures that are set up to work in such a way that, if an object of study is subjected to them, they generate output that carries information about the state, value, position, or constitution of the system with respect to the concept of interest. In psychological research, such procedures range from subjecting persons to fMRI to gauge brain activity, to recording reaction times to study semantic associations, to administering IQ items to estimate a person’s level of intelligence. In ordinary language, such observational procedures are commonly called measurement procedures. In fact, in everyday conversation, it seems that the term measurement is used indiscriminately to refer to any procedure that leads to observational data. However, for a procedure to count as a measurement procedure, it must yield measurements of something; that is, it requires that there be a certain connection between the observations and some theoretical attribute (e.g., the use of a tape measure does not constitute measurement simpliciter; it constitutes the measurement of length, where length functions as a theoretical attribute). Most scientists, laypeople, and philosophers are inclined to view this relation between measures and a measured attribute as unproblematic in the case of physical measurements of length and the like. However, when it comes to psychology, many do view this connection as deeply problematic. What does it mean to say that one measures intelligence? How does one know that the measurement procedure offered (say, administering an IQ test) indeed measures this particular attribute and not something else or nothing at all? Can one measure a theoretical attribute such as intelligence in the absence of a scientific theory that characterizes the attribute? Are the proposed measurement procedures instances of measurement at all? Such questions concern the foundations of psychological measurement. They can be raised for most, if not all, proposed procedures for psychological measurement; and, even after more than a century of scientific psychology, it is rare to find good answers to any of them. The ubiquity and uniformity of this situation across many of the subdisciplines in psychology may point to the existence of problems in the foundations of psychological measurement; that is, it is not implausible that such problems arise from the lack of a strong conceptual foundation that specifies what psychological measurement is or when it is adequately performed. This hypothesis is strengthened by the fact that recent years have seen a flurry of conceptually oriented attacks on common practices in psychological measurement (e.g., Barrett, 2005; Borsboom, 2006; Borsboom, Mellenbergh, & Van Heerden, 2004; McGrath, 2005; Maraun & Peters, 2005; Michell, 1997, 1999, 2000). This signals a more than superficial dissatisfaction with existing frameworks for the conceptualization of measurement in psychology.

FOUNDATIONS OF MEASUREMENT

3

This special double issue of Measurement represents an attempt to put the conceptual foundations of psychological measurement squarely on psychology’s methodological agenda. It assembles three focus articles that are concerned with topics that are central to these conceptual foundations. Each of these papers is concerned with problematic aspects of the common treatment of measurement procedures in psychology. The first article challenges the routine assumption that psychometrics is a quantitative science and in fact argues that there may be no such thing as psychological measurement at all. The second article lays out a systematic framework for latent variable modeling, which is intended to capture the notion of psychological measurement in these terms. The third article seeks to provide a conceptual distinction between theoretical variables that allow for an extension in possible worlds and theoretical variables that do not; the paper argues that psychology stands in need of an extension of its vocabulary in order to accommodate this distinction. These focus articles are followed by a number of associated peer commentaries. The issue concludes with a reply to the relevant commentaries by each of the authors. Although the focus of these articles is on measurement in psychological science, their content has clear relevance for the human sciences more generally. As befits the nature of Measurement, all three focus articles are interdisciplinary in nature; while all three authors draw from the indigenous theoretical literatures in the field of psychometrics, particularly those of a foundational nature, they also make constructive use of the work of philosophers, including philosophers of science. Indeed, given their general, abstract, and critical nature, the focus articles themselves can be regarded as contributions to the philosophy of measurement. This emphasis on the conceptual foundations of psychological measurement is not common, and it is a part of measurement theory that is sometimes thought to be largely irrelevant to the more pragmatic concerns of measurement specialists. However, we think that resolving philosophical-cum-theoretical issues plays an important role in furthering our understanding of measurement. We think this is most likely to happen when philosophy is viewed as an abstract critical part of science, wherein it is well positioned both to learn from science and to inform science. Without appropriate philosophical study, the further development of measurement theory and practice will almost certainly be hindered. It is no accident that the development of measurement theory since the time of antiquity has received regular seminal input from philosophers (cf. Michell, 1999). For example, operationism, as a philosophical theory of measurement, retains a discernable influence within psychology, for example, through S. S. Stevens’s theory of measurement, and through its links to true score theory (Borsboom, 2005). Yet, operationism is deeply problematic and is widely considered to be philosophically unacceptable. The philosophical literature on operationism has been a major impetus to the adoption of less restrictive, and more defensible, views of measurement.

4

HAIG AND BORSBOOM

The three articles in the present issue are characterized by their focus on a number of fundamental general and specific issues and concepts in the field of psychometrics. These include the following: • The question of whether psychometrics uncritically accepts the hypothesis

that psychological attributes are quantitative • The issue of whether psychometrics is a pathological science, and if so,

what conditions make it pathological • The general problem of how one relates observed test scores to psycho-

logical attributes • The need to move validity theory from a concern with epistemology,

meaning, and correlation to a concern with ontology, reference, and causality • The importance of differentiating constructs and concepts in test validation and psychometrics more generally • The methodological importance of incorporating the set-theoretic apparatus of modal worlds into future efforts to understand and advance measurement theory and research methods For the last decade, Joel Michell has argued for, and defended, the arresting conclusion that quantitative psychology suffers from the methodological thought disorder of unwittingly accepting the untested empirical assumption that the attributes it measures are quantitative in nature. In the first focus paper, “Is Psychometrics Pathological Science?” Michell runs the same line of argument for the science of psychometrics, concluding that it is in a pathological state. For him, a pathology of science occurs “when the normal processes of scientific investigation break down and a hypothesis is accepted as true within the mainstream of the discipline without a serious attempt being made to test it and without any recognition that this process has occurred.” Michell argues that psychometrics proceeds on the assumption that psychological attributes are quantitative, but its mainstream scarcely acknowledges, let alone attends, to this hypothesis. Michell suggests that two sets of social interests to be found in the history of modern psychology are responsible for this situation, namely the ideological and the economic advantages that occur as a result of presenting psychology as a quantitative science. In the last part of his article, Michell reasons for the conclusion that modern item response models suffer from this pathological state because its advocates have neither recognized this problem nor seriously tested the relevant hypothesis. In his recent book, Measuring the Mind (2005), the author of the second focus article, Denny Borsboom, undertook an extensive examination of the conceptual foundation of psychological measurement. An important part of this examination centered on the theoretical status of latent variables. In his

FOUNDATIONS OF MEASUREMENT

5

focus article, “Latent Variable Theory,” Borsboom continues his work on the conceptual foundations of latent variable modeling under the broad heading of latent variable theory. Borsboom argues that all variables are ontologically on a par and that the distinction between latent and observed variables should not, therefore, be cast in ontological terms. Rather, he maintains that the distinction should be framed in epistemological terms with observed variables being judged more accessible to investigation than latent variables. For Borsboom, observed variables are cast as fully accessible, involving error-free inferences from data to variable structure, whereas latent variables are not. He maintains that when the conditions for full accessibility are not met, the to-be-measured variables should be regarded as latent. Because current substantive theory gives no guidance on how to formulate the variable structure when setting up a model, Borsboom draws the arresting conclusion that a variable is latent until proven observed. Science is commonly thought to aim at producing knowledge about states of affairs that exist in the world. However, students of science are increasingly coming to appreciate that thinking about nonactual possibilities plays an important role in scientific research. In the third focus article, titled “Concepts, Constructs, and the Worlds of Possibility: Connecting the Measurement, Manipulation, and Meaning of Variables,” Keith Markus shows how thinking about nonactualized possibilities in terms of modal set theory can help us understand how variables can be applied to hypothetical populations. Markus notes that the literature on methodology describes theoretical variables interchangeably as constructs and concepts. He proceeds to redeploy these two terms to mark a neglected, but important, distinction: Constructs are taken to correspond to variables that range over actual cases, whereas concepts are taken to range over both actual and possible cases. As such, constructs apply to a specified population and are therefore important in test validation and in psychometric contexts. In virtue of extending over possible populations, concepts can become goals for systemic change, as well as allowing for a comparison of constructs across populations. Markus notes a number of changes in practice that follow from the adoption of the distinction between constructs and concepts. We believe that this special issue of Measurement will be of value to reflective empirical researchers in the behavioral and social sciences who use statistical and psychometric methods and concepts and who are interested in furthering their understanding of the nature of measurement. We think the three focus articles should be of particular interest to research methodologists and psychometricians, including those who teach and research in the areas of psychological measurement. We hope philosophers of science with an interest in scientific measurement in the human sciences might also judge these articles to be worthy objects of future philosophical study.

6

HAIG AND BORSBOOM

REFERENCES Barrett, P. (2005). What if there were no psychometrics? Constructs, complexity, and measurement. Journal of Personality Assessment, 85, 134–140. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge: Cambridge University Press. Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71, 425–440. Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071. Maraun, M. D., & Peters, J. (2005). What does it mean that an issue is conceptual in nature? Journal of Personality Assessment, 85, 128–133. McGrath, R. E. (2005). Conceptual complexity and construct validity. Journal of Personality Assessment, 85, 112–124. Michell, J. (1997). Quantitative science and the definition of measurement in psychology. British Journal of Psychology, 88, 355–383. Michell, J. (1999). Measurement in psychology: A critical history of a methodological concept. New York: Cambridge University Press. Michell, J. (2000). Normal science, pathological science, and psychometrics. Theory and Psychology, 10, 639–667.

On the Conceptual Foundations of Psychological Measurement

On the Conceptual Foundations of Psychological Measurement. Brian D. Haig. University of Canterbury. Denny Borsboom. University of Amsterdam.

70KB Sizes 0 Downloads 238 Views

Recommend Documents

On the Conceptual Foundations of Psychological ...
Psychology, University of Canterbury, Private Bag 4800, Christchurch, New .... be largely irrelevant to the more pragmatic concerns of measurement specialists.

Applied Psychological Measurement
Mar 9, 2011 - http://apm.sagepub.com/content/early/2011/03/09/0146621610392911 ..... Compared to rXX0 (Equation 3), in Guttman's lambda-2 (Equation 10) the ..... Mokken scale analysis in R. Journal of Statistical Software, 20(11), 1-19.

Philosophical and Psychological Foundations of Education.pdf ...
Philosophical and Psychological Foundations of Education.pdf. Philosophical and Psychological Foundations of Education.pdf. Open. Extract. Open with. Sign In.

Report of the commission on the measurement of economic ...
coming from government, and interest payments on household loans going to financial corporations. Properly .... But while the precise list of the features affecting quality of life inevitably rests on value ... of-life domains for each person, and th

Measurement of vibrations induced on the surface of crystalline eye ...
line scan camera (Basler L104K-2k, having 2048 pixel resolution, line rate of 29.2 ..... speaker diaphragm surface at without vibration. Frequency (Hz). D s ip la.

Measurement of vibrations induced on the surface of crystalline eye ...
aDepartment of Electrical and Computer Engineering, University of Houston, ... cDepartment of Biomedical Engineering, University of Texas at Austin, Austin, TX, ...

unit 2 conceptual foundations information systems - eGyanKosh
Objectives. After going through this unit, you should lie able to: • appreciate the significance of information systems in an organisation. • understand the information subsystems which could be defined within a typical organisation. • differen

unit 2 conceptual foundations information systems - eGyanKosh
software. men, procedures as well as supplies." As the above given definition ..... application software, it becomes a complete conceptual structure of an information system. The Physical Structure. It is quite ... by recruitment of manpower (compute

REMARKS ON CINEMA: THE PSYCHOLOGICAL THEORY.pdf ...
Page 3 of 120. REMARKS ON CINEMA: THE PSYCHOLOGICAL THEORY.pdf. REMARKS ON CINEMA: THE PSYCHOLOGICAL THEORY.pdf. Open. Extract.

Temporal stability and psychological foundations of ...
Oct 20, 2011 - We conducted our experiments in the computer lab of a European university, using the software z-Tree (Fischbacher,. 2007). We observed the decisions of ..... of Conditional Cooperators (56–60 percent) and Free Riders (19–25 percent

Probabilistic models of cognition: Conceptual foundations
logical roots, and become an autonomous, and highly formal, discipline. ... predicate logic. ..... mind map naturally onto highly distributed, autonomous,.

PdF Conceptual Foundations of Occupational Therapy ...
... Conceptual Foundations of Occupational Therapy Practice read PDF free, ... a Web page URL associated with that DOI name Send questions or comments to ...

The Psychological Benefits of Being Authentic on Facebook.pdf ...
The Psychological Benefits of Being Authentic on Facebook.pdf. The Psychological Benefits of Being Authentic on Facebook.pdf. Open. Extract. Open with.

On the psychological reality of verb typologies
Our specific processing predictions take into account previous findings from studies .... experiment was run on Apple Macintosh computers running PsyScope (Cohen et al. 1993). 6. Results ..... The developer intentionally destroyed the city. 6.