When we assess psychological attributes, like depression, intelligence, or attitudes, we commonly use sets of items that “measure” the construct in question. A sizable literature in psychometrics turns on the question of what that means. It’s easy to say you measure attitudes with questionnaires, but it’s clear that this refers to a process that’s very different from, say, measuring length with a meter stick or measuring weight with a pan balance.
One answer to the question of what measurement means in psychology is to say that answers to questionnaire items are “indirect” measures that “reflect” constructs. This view is bolstered by an elegant set of mathematical models generically known as Item Response Theory. In these models, the probability of item responses (e.g., your answer to the question “are you in favor of building new nuclear power plants?”) are related to a construct (e.g., a mathematical representation of your attitude towards nuclear power) through a mathematical function. That function says how responses to the item depend on a person’s position on the construct, which is typically thought of as a point on a line (representing the degree to which one is pro or con; however, there are many other ways to represent constructs, including multidimensional spaces, circles, fuzzy sets, and networks).
Applied mathematical models, however, cannot contain their own meaning: and after all the mathematical functions have been explicated and fitted to the data, we are typically still left with the question of what these functions represent. Thinking of measurement in terms of responses that “reflect a construct” does not solve the measurement problem but replaces it; for now we are left with the question of what the word “reflect” means.
One way of understanding reflective measurement is through a causal relation, in which one’s position on a construct causes one’s responses to the items. This way of thinking commonly leads to a reification of the construct, which is considered to play a causal role in the generation of the item response. However, it requires considerable mental gymnastics to think of attitudes, intelligence, or personality as “causing” item responses. After all, as our former colleague Paul de Boeck used to say, the cause of an answer is not a construct, but that we asked a question.
An alternative metaphor is to think of items as “probes”. We send them into the depth of a person’s psychological world, and then they come back carrying an information signal – the person’s answer. Just like submarines judge the depth of the sea by sending a ping and recording how long it takes to get back as it is reflected off the bottom. So interpreted, a person’s position on a construct is causally relevant to the signal we get back, but it’s not an efficient cause in the way striking a match is an efficient cause of a forest fire. It’s more like the pre-existing drought that doesn’t directly cause the fire, but does determine whether it will spread.
Unfortunately metaphors aren’t real theories. Just like Shakespeare’s “Julia is the sun” would not lead physicists to investigate whether there’s nuclear fusion going on in her belly, we don’t really believe that psychological constructs are landscapes that reflect probes. So in the end, we’re still stuck with the question of what psychological measurement really is.
When we assess psychological attributes, like depression, intelligence, or attitudes, we commonly use sets of items that “measure” the construct in question. A sizable literature in psychometrics turns on the question of what that means. It’s easy to say you measure attitudes with questionnaires, but it’s clear that this refers to a process that’s very different from, say, measuring length with a meter stick or measuring weight with a pan balance.
One answer to the question of what measurement means in psychology is to say that answers to questionnaire items are “indirect” measures that “reflect” constructs. This view is bolstered by an elegant set of mathematical models generically known as Item Response Theory. In these models, the probability of item responses (e.g., your answer to the question “are you in favor of building new nuclear power plants?”) are related to a construct (e.g., a mathematical representation of your attitude towards nuclear power) through a mathematical function. That function says how responses to the item depend on a person’s position on the construct, which is typically thought of as a point on a line (representing the degree to which one is pro or con; however, there are many other ways to represent constructs, including multidimensional spaces, circles, fuzzy sets, and networks).
Applied mathematical models, however, cannot contain their own meaning: and after all the mathematical functions have been explicated and fitted to the data, we are typically still left with the question of what these functions represent. Thinking of measurement in terms of responses that “reflect a construct” does not solve the measurement problem but replaces it; for now we are left with the question of what the word “reflect” means.
One way of understanding reflective measurement is through a causal relation, in which one’s position on a construct causes one’s responses to the items. This way of thinking commonly leads to a reification of the construct, which is considered to play a causal role in the generation of the item response. However, it requires considerable mental gymnastics to think of attitudes, intelligence, or personality as “causing” item responses. After all, as our former colleague Paul de Boeck used to say, the cause of an answer is not a construct, but that we asked a question.
An alternative metaphor is to think of items as “probes”. We send them into the depth of a person’s psychological world, and then they come back carrying an information signal – the person’s answer. Just like submarines judge the depth of the sea by sending a ping and recording how long it takes to get back as it is reflected off the bottom. So interpreted, a person’s position on a construct is causally relevant to the signal we get back, but it’s not an efficient cause in the way striking a match is an efficient cause of a forest fire. It’s more like the pre-existing drought that doesn’t directly cause the fire, but does determine whether it will spread.
Unfortunately metaphors aren’t real theories. Just like Shakespeare’s “Julia is the sun” would not lead physicists to investigate whether there’s nuclear fusion going on in her belly, we don’t really believe that psychological constructs are landscapes that reflect probes. So in the end, we’re still stuck with the question of what psychological measurement really is.
