Running Head: COGNITIVE COUPLING DURING READING

1

Cognitive Coupling during Reading Word Count: 4267

Caitlin Mills a, Art Graesser b, Evan F. Riskoc, Sidney K. D’Mello d, e Department of Psychology a, University of British Columbia, Vancouver, BC V6T 1Z4 Canada [email protected] Department of Psychology b, University of Memphis, Memphis, TN 38152, USA [email protected] Department of Psychology c, University of Waterloo, Waterloo, ON N2L3G1, Canada [email protected] Departments of Psychology d and Computer Science e University of Notre Dame, Notre Dame, IN 46556, USA [email protected]

Corresponding Author Caitlin Mills 2136 West Mall University of British Columbia Vancouver, BC V6T 1Z4 Canada [email protected]

COGNITIVE COUPLING DURING READING

2

Abstract We hypothesize that cognitively engaged readers dynamically adjust their reading times with respect to text complexity (i.e., reading times should increase for difficult sections and decrease for easier ones) and failure to do so should impair comprehension. This hypothesis is consistent with theories of text comprehension but has surprisingly been untested. We tested this hypothesis by analyzing four data sets in which participants (N = 484) read expository texts using a selfpaced reading paradigm. Participants self-reported mind wandering in response to pseudorandom thought-probes during reading and completed comprehension assessments after reading. We computed two measures of cognitive coupling by regressing each participant’s paragraph-level reading times on two measures of text complexity: Flesch Kincaid Grade Level and Word Concreteness scores. The two coupling measures yielded convergent findings: coupling was a negative predictor of mind wandering and a positive predictor of both text- and inference-level comprehension. Goodness of fit, measured with Akaike information criterion (AIC), also improved after adding coupling to the reading-time only models. Furthermore, cognitive coupling mediated the relationship between mind wandering and comprehension, supporting the hypothesis that mind wandering engenders a decoupling of attention from external stimuli.

Keywords: mind wandering, comprehension, reading, cognitive coupling

COGNITIVE COUPLING DURING READING

3

Cognitive Coupling during Reading It is widely accepted that reading time varies as a function of text complexity (Graesser, Hoffman, & Clark, 1980; Graesser & McNamara, 2011; Haberlandt & Graesser, 1985; McNamara & Magliano, 2009). For example, longer reading times are associated with texts that use unfamiliar words, are less cohesive, or contain contradictory information (Cirilo & Foss, 1980; Gernsbacher, 1991; Kintsch, 1998a; McNerney, Goodwin, & Radvansky, 2011; Zwaan, Magliano, & Graesser, 1995). Because readers need more time to process and integrate information as text complexity increases, comprehension should be closely related to the extent to which reading time and text complexity are coupled, a concept we refer to as cognitive coupling. Although there is theoretical support that comprehension should be facilitated by a coupling between reading time and text complexity (Graesser & McNamara, 2011; McNamara & Magliano, 2009), this relationship has not yet been empirically tested. Accordingly, we test the hypothesis that the alignment of reading time and text complexity (referred to as cognitive coupling) should predict comprehension. We consider one type of text complexity, namely the readability of a text. Here, complex texts contain longer, more complex sentences, and less familiar, abstract words. Our measures of cognitive coupling capture the extent to which an individual’s reading times align with text complexity. We expect that comprehension will be facilitated when longer reading times are associated with more difficult parts of a text and vice versa for easier parts (high in cognitive coupling). Conversely, comprehension will suffer if reading time does not align with the current complexity of the text (low in cognitive coupling). For example, skimming, as manifested via faster reading times irrespective of the complexity of the current text, should result in poor

COGNITIVE COUPLING DURING READING

4

comprehension because reading time is not calibrated to text complexity (Carver, 1992) due to a shallower level of text processing. Consistent with contemporary models of comprehension, allocation of reading time is thought to predominantly occur below the threshold of conscious deliberation (Kintsch, 1998b; Myers & O’Brien, 1998; O’Brien & Myers, 1999). In fact, readers may not even be aware of when they become “decoupled” from the text, such as when they experience a lapse in attention (or begin to mind wander). In this event, the continual subconscious evaluation of text complexity may become disrupted, subsequently influencing how readers allocate their reading time, and eventually stifling comprehension. Therefore, we also test the relationship between cognitive coupling and attention using the propensity to mind wander as a measure of an attentional breakdown. Mind wandering is defined as a shift in attention away from the external task-related thoughts towards internal, task-unrelated thoughts. Mind wandering is reported as much as 20-40% of the time during reading (Smallwood, McSpadden, & Schooler, 2007) and is negatively related to reading comprehension (Feng, D'Mello, Graesser, 2013; Randall, Oswald, & Beier, 2014; Smallwood, Fishman, & Schooler, 2007). We expect that a breakdown in cognitive coupling might be one mechanism to explain the negative influence of mind wandering on reading comprehension. This is based on the perceptual decoupling hypothesis, which posits that attention becomes decoupled from the external environment when the mind wanders (Schooler et al., 2011). Specifically, when mind wandering occurs, attentional focus is diverted from the text content towards internal, unrelated thoughts, causing a break in the alignment between reading times and complexity because the text is only being superficially processed. In support of this idea, research shows that readers’ eyes continue to move through the text even when they mind wander, albeit with different gaze

COGNITIVE COUPLING DURING READING

5

patterns compared to when they are paying attention to the text (e.g., fewer and longer fixations during mind wandering; Bixler & D’Mello, 2015; Faber, Bixler, D'Mello, 2017; Reichle, Reineberg, & Schooler, 2010; Smilek, Carriere, & Cheyne, 2010). We suggest that when readers progress through the text during mind wandering, their reading times become decoupled from text complexity, which should impair comprehension because sufficient time is not being allocated to process the content. We use four datasets to investigate relationships between cognitive coupling, mind wandering, and comprehension. Cognitive coupling was computed based on the strength of the relationship between self-paced reading times and text complexity. We hypothesize that a positive relationship between paragraph-level reading time and text complexity scores would be indicative of cognitive coupling (i.e. slowing down when text becomes more difficult). Our cognitive coupling measures are based on two different text complexity metrics – one content-free and another content-based. Using dissimilar metrics provides a more comprehensive test of the cognitive coupling hypothesis because the hypothesized relationships should be consistent across metrics. The content-free measure is the Flesch-Kincaid Grade Level (FKGL), a widely-used measure of text complexity that indicates a text’s level of difficulty expressed in school grades (e.g., 5th grade reading level; Klare, 1974). Although FKGL is strongly correlated with more sophisticated models of text complexity (e.g. r = .716 with a multifaceted measure of complexity in Graesser et al., 2014), it is considered to be content-free, and somewhat shallow, because it is computed from the length of words and sentences alone. Our second cognitive coupling measure is based on a deeper, content-based metric called word concreteness (WC) that assesses how concrete versus abstract a text is based on the words it contains. WC indexes text complexity since texts containing more concrete words, which

COGNITIVE COUPLING DURING READING

6

evoke mental images, are easier to understand compared to texts with more abstract words (Graesser et al., 2014; Medimorec, Pavlik, Olney, Graesser, & Risko, 2015). We derive WC from Coh-Metrix, a computational text analysis tool that analyzes texts at various levels of language and discourse (Graesser et al., 2014; Graesser, McNamara, Louwerse, & Cai, 2004). There are alternative metrics that capture complex properties of the text, such as text cohesion, but their reliability is dependent on having texts of sufficient length. In contrast, WC is not as sensitive to text length (McNamara, Graesser, McCarthy, Cai, 2014). It is also not highly correlated with FKGL (r = -.21), with higher values representing more concrete and thereby “easier” texts (Graesser et al., 2014). We note that although our measures of coupling are derived from different complexity metrics, our goal is not to contrast the coupling measures, but to assess whether there are convergent patterns between the two. We predict that both measures of cognitive coupling should negatively relate to mind wandering measured with thought probes during reading (Schooler et al., 2011) and positively relate to comprehension assessments administered after reading. Importantly, cognitive coupling should explain unique variance after accounting for the well-known relationships between reading time, mind wandering, and comprehension (Feng, D’Mello, & Graesser, 2013; Reichle et al., 2010). Moreover, we predict that the influence of mind wandering on comprehension will be mediated through cognitive coupling, because mind wandering engenders perceptual decoupling from the text (Schooler et al., 2011) which results in poor comprehension. Datasets We analyzed four existing datasets, where participants (N = 484) learned about scientific research methods by reading computerized texts. The datasets were originally collected to investigate mind wandering under various online reading conditions (described below and

COGNITIVE COUPLING DURING READING

7

summarized in Table 1). The advantages of these four datasets include variability in reading conditions and diverse populations. Two of the datasets were collected from a private Midwestern university’s subject pool and two were collected via Amazon Mechanical Turk. Mechanical Turk has been found to be a valid source to collect reliable data (Buhrmester, Kwang, & Gosling, 2011; Mason & Suri, 2012; Rand, 2012). Only dataset 4 has been published as part of a study that explored mind wandering during rereading (Phillips, Mills, D’Mello, & Risko, 2016).

Table 1 Overview of design and experimental manipulations for the four datasets Dataset 1

Dataset 2

Dataset 3

Dataset 4

Source of Data Collection

Mechanical Turk

Univ. subject pool

Univ. subject pool

Mechanical Turk

Number of Texts Read

1

1

2

2

Number of Participants

191

136

71

86

Compensation

$2.50

course credit

$2.50

$5.00

Experimental Design

Between

Between

Within

Within

Text Difficulty x Text Presentation

Text Presentation

Text Difficulty

Re-reading vs. Read Once

Manipulations

Notes. Univ. = University; Prop. Between = between-subjects manipulation; Within = within-subjects manipulation.

Texts and Text Presentation. We used the same two texts on scientific research methods in all four datasets. One text was on the topic of dependent variables while the other focused on causal claims. We used research methods topics because the content is educational, relatively unfamiliar to the average person, and useful to a diverse range of citizens.

COGNITIVE COUPLING DURING READING

8

We created easy and difficult versions of the texts corresponding to each topic. Both versions were equated on conceptual content and were approximately 1,500 words in length (see (Fulmer, D’Mello, Strain, & Graesser, 2014; Mills & D’Mello, 2012 for details). The easy and difficult versions were created with respect to key dimensions that contribute to text complexity (Graesser & McNamara, 2011): narrativity, sentence length, word frequency, and syntactic simplicity. For example, easy versions were more narrative, had shorter sentences, and had fewer low frequency words (average FKGL = 9). Difficult texts had longer, more complex sentences, and more low frequency words (average FKGL = 13). There were two between-subjects manipulations in Dataset 1. The text difficulty manipulation involved participants reading texts that were either easy or difficult, whereas the text presentation manipulation involved texts being presented with either one sentence or one paragraph on the screen at a time. Dataset 2 only used difficult texts, but text presentation was manipulated (sentence vs. paragraph). In contrast, in Dataset 3, both texts were presented at the paragraph level, but text difficulty shifted at the midpoint of each text. Dataset 4 used a sentencelevel presentation and included a within-subjects manipulation of reading a text once versus rereading the same text twice. Only data from the read-once condition and the first read of the reread condition were considered here because the second-read was confounded by familiarity effects from the first read (see Phillips et al., 2016). Thought Probes. Mind wandering was tracked via auditory thought probes in all four datasets. A standard description of mind wandering (Smallwood & Schooler, 2006) was provided to participants before reading: “At some point during reading the texts, you may realize that you have no idea what you just read. Not only were you not thinking about the text, you were thinking about something else altogether.” The probe consisted of a beep that would sound on

COGNITIVE COUPLING DURING READING

9

pseudo-random screens of text. Probes were triggered when participants pressed the space bar to advance to the next screen of text, so reading times were unaffected by probing. Participants were instructed to press the “Y” key if they were mind wandering or the “N” key if they were not. Comprehension measures. We used four-alternative multiple-choice questions that tapped comprehension at the text- and inference- level. Text-level questions were based on factual or text-level characteristics. Inference questions were designed to elicit patterns of reasoning and required participants to make inferences or apply abstract concepts to novel examples (Graesser, Ozuru, & Sullins, 2010). Each participant completed a comprehension posttest on the text(s) they read. It could contain up to six inference questions and up to 12 textlevel questions, with the exact number varying across datasets. Reliability statistics for comprehension and mind wandering measures are listed in Appendix A. Procedure. A standard procedure was used for all datasets. Participants completed an electronic consent form. They were subsequently given instructions on the self-paced reading task and on how to respond to the mind wandering probes. Participants pressed the space bar to move through each screen of the text, but could not return to a previously read screen. There were six to nine auditory thought probes for each text. Participants completed the comprehension assessment immediately after reading each text (datasets 1, 3, 4) or at the end of the session (dataset 2). Computing Cognitive Coupling. We computed cognitive coupling from self-paced reading times collected during reading and from the text complexity measures. There were two measures of cognitive coupling, one based on Flesch–Kincaid Grade Level (FKGL) and the other based on Word Concreteness (WC). Cognitive coupling was computed in three steps: (1)

COGNITIVE COUPLING DURING READING

10

Reading times for sentence-level screen presentations were summed in order to align with the content of the paragraph-level screen presentations, yielding a reading time measure for each paragraph. This was done because the text complexity measures are more reliable for longer stretches of texts compared to single sentences (Klare, 1974; Graesser, et al. 2011). (2) Paragraph level reading times were z-score standardized by participant. Reading times were also z-score standardized by text in datasets 2 and 3, and by reading condition in dataset 4 (re-read vs. read once). (3) Paragraph-level reading time was regressed on raw FKGL/WC for each participant and the regression coefficient was taken as a measure of cognitive coupling. Since participants read two texts in datasets 3 and 4, coupling was computed separately for the two texts, and then averaged. Higher values indicate more cognitive coupling between reading time and FKGL. Because, higher WC scores from Coh-Metrix correspond to more concrete words and easier texts, WC-coupling coefficients were reverse coded so that positive coefficients reflect more coupling, similar to FKGL. Figure 1 shows an example of FKGL coupling. The left panel illustrates how reading times can be aligned (top) or misaligned (bottom) with text complexity by plotting standardized reading time and FKGL for each paragraph of the text. The resulting regressions and coupling scores are shown in the right column.

COGNITIVE COUPLING DURING READING

11

Figure 1. Line graphs (left) and scatter plots (right) of two participants’ z-score standardized reading times and Flesch Kincaid Grade Level. Regression coefficients were computed by regressing standardized (by participant) paragraph level reading time on unstandardized FKGL scores. The top graph demonstrates low cognitive coupling (negative coefficient) whereas the bottom graph demonstrates high cognitive coupling (positive coefficient).

Results On average, participants spent a total of 4.95 minutes (SD = 5.46) reading each text across the four datasets. We removed eleven participants (less than 3% of total participants; 473 included in analyses) who did not spend at least 30 seconds reading. To control for the amount of information on a given screen of text, we divided reading times per screen with the number of characters on that screen at the paragraph level (McNerney et al., 2011). These average adjusted reading times were used in all subsequent analyses moving forward, but note that the coupling measures were computed from unadjusted reading times. Correlations between adjusted reading

COGNITIVE COUPLING DURING READING

12

time with cognitive coupling scores suggest no collinearity concerns as the correlations ranged from -.158 to .190 (all ps > .05). The analyses consisted of linear regression models to predict mind wandering, text-level, and inference-level comprehension from the experimental variables (covariates), adjusted reading time, and cognitive coupling (separate models for FKGL and WC coupling). The goal was to ascertain if cognitive coupling explains additional variance after controlling for adjusted reading time, which is critical since cognitive coupling is derived from reading time itself. We built models for the individual data sets as well as for a combined dataset (here, dataset identity was used as a dummy-coded control variable). Table 2 presents a summary of the findings (See Appendices B-F for full models).

Running Head: COGNITIVE COUPLING DURING READING

13

Table 2 Summary of standardized regression coefficients (β) for predicting mind wandering, text-level, and inference-level comprehension from adjusted reading time and coupling. Mind Wandering D1 Datasets Word Concreteness Adj. RT WC coupling

D2

-.277** -.169+

Text-level

D3

D4

DC

-.102

-.127

-.147*

-.062

-.182*

-.220

-.236*

-.146

FKGL coupling -.237*

-.118

+

D1

D2

.371** .108

Inference-level

D3

D4

.199+

.180+

-.250*

-.136*

.053

.349** .237* .293*

-.134

-.140

-.131*

.324** .039

-.275*

-.216*

-.225**

.285** .407** .326* .383*

DC .154**

D1 .070

D2 .127

D3 .195

D4 .087

+

DC .098*

.222**

-.058

.285*

.220

.444** .147*

.132*

.063

.076

.224+

.112

.084+

.322**

.090

.292*

.352*

.367*

.212**

FKGL Adj. RT

.232* .183+

Notes. * = p < .05; ** = p < .001; + = p < .10; Adj RT = adjusted reading time; D1, D2, D3, D4 – datasets 1, 2, 3, and 4. DC = combined dataset; Models contain experimental covariates, adjusted reading time, and cognitive coupling measures.

Running Head: COGNITIVE COUPLING DURING READING 14 Main effects. Focusing on the individual datasets, adjusted reading time only significantly predicted mind wandering in dataset D1 and was marginal when combined with WC coupling in D2 (p = .054). WC coupling predicted mind wandering in two out of four datasets (D2 and D4) and was marginally significant in D3 (p = .078), whereas FKGL coupling was a significant predictor in three out of four datasets (D1, D3, and D4). And while all three measures predicted mind wandering for the combined dataset DC, FKGL coupling was the strongest predictor. Thus, while the results were somewhat inconsistent with respect to predicting mind wandering across the datasets, cognitive coupling was a better predictor, and more consistent predictor, compared to adjusted reading time. With respect to text level comprehension, both FKGL (4/4 datasets) and WC coupling (3/4 datasets) consistently predicted text-level comprehension in the individual datasets. On the other hand, when combined with WC coupling, adjusted reading time was only significant in dataset D1 and marginal in D3 (p = .097) and D4 (p = .090). When combined with FKGL coupling, it was significant in D1 and D3 and marginal for D4 (p = .074). All three measures were significant predictors of text-level comprehension in the combined dataset, but FKGL coupling was again the strongest predictor. For inference level comprehension, WC coupling was a significant predictor in two datasets (D2 and D4) and marginal in D3 (p = .074), whereas FKGL coupling was a significant predictor in 3/4 datasets (D2, D3, and D4). In contrast, adjusted reading time was not a significant predictor in any individual datasets when combined with WC coupling, and was only a marginally significant predictor in one dataset when combined with FKGL coupling (D3, p = .053). In the combined dataset, both FKGL and WC were significant predictors of inferencelevel comprehension. Adjusted reading time was only a significant predictor of comprehension

COGNITIVE COUPLING DURING READING

15

when combined with the WC coupling model and was marginal when combined with FKGL coupling (p = .053). FKGL coupling was also the strongest predictor in the combined data. In sum, FKGL coupling emerged as the strongest predictor of all three dependent variables, while WC coupling showed similar, albeit slightly weaker patterns. Compared to adjusted reading time, both coupling measures were more consistent predictors across datasets. Goodness of fit. We also computed the change in Akaike information criterion (ΔAIC) after adding each cognitive coupling measure to a model with adjusted reading time and the experimental covariates. AIC provides an estimate for goodness of fit while penalizing for model complexity (Burnham & Anderson, 2004). AIC values are not meaningful in isolation because they are influenced by arbitrary constraints, but evaluating ΔAIC is a common procedure for model comparison. Here, we computed ΔAIC (AIC adjusted reading time model  AIC coupling added model), with positive values demonstrating better fit for the coupling added models compared to the adjusted reading time only models. Burnham & Anderson (2004) provided some “rules of thumb” for model comparisons based on ΔAIC values: ≤ 2 provide evidence that two models are similar (e.g., that adding coupling to the model is not beneficial), values between 4 and 7 provide considerably less support that the models are similar, and values > 10 provide no support that the models are the same. Focusing on the combined dataset, adding WC coupling to an adjusted reading time model resulted in ΔAIC values of 6, 23, and 9 for mind wandering, text-level, and inferencelevel comprehension, respectively. Adding FKGL coupling resulted in even larger ΔAIC values: mind wandering changed by 23, text-level comprehension by 58, and inference-level comprehension by 22. These ΔAIC values provide considerable support for selecting the models

COGNITIVE COUPLING DURING READING

16

that include cognitive coupling over the adjusted reading time only models (ΔAIC > 5), with even stronger evidence for the FKGL coupling models (ΔAIC > 15). Mediation. Finally, we tested if FKGL and WC coupling mediated the relationship between mind wandering and comprehension. Our hypothesis is that participants no longer attend to the current task demands during mind wandering, leading to “decoupling” of attention from reading. According to Shrout and Bolger’s (2002), the path between wandering and coupling is consistent with small to medium sized effects in all datasets (see Table 2) and a sample size of 400 or greater would be required to approach power levels of .80 in detecting small effects for direct or indirect paths in the model (Fritz & MacKinnon, 2007). Thus, we use the combined dataset (N = 473) in order to maximize our power for the mediation analyses. Because adjusted reading time, FKGL, and WC coupling each explained unique variance in predicting mind wandering and comprehension, we tested all three as mediators in separate models (Figure 2). This approach was used in lieu of a multiple mediation model in order to remove shared variance between cognitive coupling and adjusted reading time for each path in the model. Accordingly, reading time was included as a covariate for the coupling mediation models, while both coupling measures were included as covariates in the reading time models. Study was dummy-coded and also included as a covariate in all models.

COGNITIVE COUPLING DURING READING

17

Figure 2. Mediation models constructed using Preacher and Hayes PROCESS macro in SPSS (Darlington & Hayes, 2016). Panel A depicts the indirect effect of mind wandering through WC coupling on comprehension while controlling for adjusted reading time. Panel B depicts the indirect effect of mind wandering on comprehension through FKGL coupling, while controlling for adjusted reading time. Panel C depicts the indirect effect of mind wandering on comprehension through adjusted reading time while controlling for WC and FKGL coupling. Note that there are separate models for text- and inference-level comprehension. Values represent unstandardized regression coefficients with standard error in parentheses. * indicates the 95% confidence interval of the indirect effects does overlap with 0 and is taken as a significant effect.

COGNITIVE COUPLING DURING READING

18

Following recommendations by Preacher and Hayes (2008), evidence for mediation can be demonstrated by a significant indirect effect estimated from bootstrapping procedures. Accordingly, we computed unstandardized indirect effects across 10,000 percentile bootstrapped samples and deemed effects to be significant if the 95% confidence intervals did not overlap with zero (Preacher and Hays, 2008). We found that the direct effect of mind wandering on text and inference-level comprehension was mediated by both FKGL and WC coupling. There was also evidence of mediation through adjusted reading time after controlling for the coupling measures. Discussion The idea that the alignment between reading times and text complexity should positively influence text comprehension is widely acknowledged (Graesser et al., 1980; Graesser & McNamara, 2011; Kintsch, 1998), but has not been empirically tested in the reading comprehension literature. We tested this fundamental claim by introducing two measures of cognitive coupling that are based on different aspects of text complexity: Flesch-Kincaid Grade Level (FKGL) and Word Concreteness (WC). FKGL is a content-free measure derived from counts and lengths of words and sentences, whereas WC is based on the content of the text. This was done to show that our results are not exclusive to a single metric of text complexity. Our results support the hypotheses that alignment between reading times and text complexity is positively related to comprehension. Whereas adjusted reading time and cognitive coupling significantly predicted text-level comprehension, the cognitive coupling measures were more consistent across datasets. The biggest differences between adjusted reading time and the cognitive coupling measures was seen in the inference-level comprehension models: the cognitive coupling measures were much better predictors of inference-level comprehension in

COGNITIVE COUPLING DURING READING

19

comparison to adjusted reading time (based on regression coefficients and ΔAIC values). This suggests that although longer reading times may be beneficial for “shallow” comprehension, deeper levels of understanding require more fine-grained allocation of resources to meet the demands of the unfolding text. Indeed, adding cognitive coupling to models that previously only contained adjusted reading time as a predictor improved model fit for both text-level comprehension (ΔAIC = 23 for WC and 58 for FKGL), as well as inference level comprehension (ΔAIC = 9 for WC and 22 for FKGL). Our results also provide insights into key processes during reading. Specifically, cognitive coupling was negatively related to mind wandering, albeit somewhat inconsistently (yet more consistently than adjusted reading time). Notable ΔAIC values (6 for WC and 23 for FKGL) were observed after adding cognitive coupling to a model that previously contained adjusted reading time as the only predictor of mind wandering, suggesting it can offer additional insights into attentional focus during learning. Further, our mediation analyses suggest that cognitive coupling provides one mechanism through which attentional focus (measured via mind wandering) influences comprehension. Our interpretation is that readers are perceptually decoupled (Schooler et al., 2011) from the environment when mind wandering, likely underallocating resources to difficult parts of the text. We note that reading time also mediated the relationship between mind wandering on comprehension. This suggests that cognitive coupling and adjusted reading time might provide insights into different processes, both of which are necessary for comprehension but in different ways. Cognitive coupling represents a local adaptation of reading time to text complexity, whereas adjusted reading time represents a global measure of the reader’s effort allocation.

COGNITIVE COUPLING DURING READING

20

It is important to mention three limitations with our research. First, all datasets were collected online, which necessitates a relinquishment of control over the setting. However, in our view, a lack of control makes the findings more compelling and makes the context more authentic with respect to how participants read in their day-to-day lives. Second, we only used instructional reading texts on research methods. Different types of texts may promote different patterns with respect to cognitive coupling, so replication with more diverse texts is warranted. Direct replication would also be prudent as this is the first demonstration of using cognitive coupling as a measure of mind wandering and text comprehension. Third, mind wandering probes were triggered when participants attempted to advance to the next screen of text, which inherently required some action immediately before answering the probe. The action itself may have somewhat limited the effectiveness of the probe-caught method (Smallwood & Schooler, 2006), so future work should eliminate this methodological artifact as well as explore alternative methods to measure mind wandering (e.g., self-caught mind wandering). For example, we can differentiate between different types of off-task thoughts, such as stimulus-independent, taskrelated interferences, external distractors, and stimulus independent thoughts (Stawarczyk, Majerus, Maj, Van der Linden, & D’Argembeau, 2011). We only excluded participants who read for less than 30 seconds to preserve as much data as possible. However, it is possible that effects might be driven by participants who simply did not follow directions or did not spend sufficient time reading (resulting in off-task thoughts and poor comprehension). We addressed this concern by repeating the analyses with a stricter data inclusion criterion, where participants who read for less than 2.5 minutes per text (roughly 600 words per minute; half of the time it took participants to read a 1500-word text on average) were removed. As shown in Appendix H, significance values changed for the individual data sets

COGNITIVE COUPLING DURING READING

21

(compared to Table 2), but this is unsurprising given the loss in power. Importantly, WC and FKGL coupling were remained significant predictors of mind wandering, text level, and inference level comprehension in the combined dataset despite reducing the sample to 419 participants. We also acknowledge that there are other ways to conceptualize text complexity beyond the readability metrics used here (see Graesser et al., 2014 for a discussion ). Other types of cognitive complexity might be more explicitly apparent to readers, such as when major cohesion breaks occur. Future work might extend the idea of cognitive coupling to different types of text complexity. Future work should also explore how metacognition influences the allocation of reading time in situations when the reader is aware of shifts in difficulty. In conclusion, we introduce cognitive coupling as the dynamic alignment between reading time and text complexity. We derived two measures of cognitive coupling and show that they predict multiple levels of comprehension after accounting for reading time. We also show that cognitive coupling is one viable mechanism to explain how attentional lapses that arise during reading impair comprehension, essentially finding that mind wandering engenders a form of attentional decoupling from the text.

COGNITIVE COUPLING DURING READING

22

Author Note This research was supported by the National Science Foundation (NSF) (DRL 1235958 and IIS 1523091). Any opinions, findings and conclusions, or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of NSF. This research was also supported by a Natural Sciences and Engineering Research Council of Canada (NSERC) discovery grant, funding from the Canada Research Chairs program, and an Early Researcher Award from the Province of Ontario to EFR. Early findings from this research (dataset 1) were presented at the 2015 meeting for Society for Text and Discourse (2015) and the 2015 meeting of the Psychonomic Society (2015). No results on this work were been published prior to this journal article.

COGNITIVE COUPLING DURING READING

23

References Bixler, R., & D’Mello, S. (2015). Automatic Gaze-Based Detection of Mind Wandering with Metacognitive Awareness. In User Modeling, Adaptation and Personalization (pp. 31– 43). Switzerland: Springer International. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science, 6(1), 3–5. Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference understanding AIC and BIC in model selection. Sociological methods & research, 33(2), 261-304. Carver, R. P. (1992). Reading rate: Theory, research, and practical implications. Journal of Reading, 36(2), 84–95. Cirilo, R. K., & Foss, D. J. (1980). Text structure and reading time for sentences. Journal of Verbal Learning and Verbal Behavior, 19(1), 96–109. Darlington, R. B., & Hayes, A. F. (2016). Regression analysis and linear models: Concepts, applications, and implementation. Guilford Publications. Faber, M., Bixler, R., & D’Mello, S. K. (2017). An automated behavioral measure of mind wandering during computerized reading. Behavior Research Methods, 102(3), 1-17. Feng, S., D’Mello, S., & Graesser, A. C. (2013). Mind wandering while reading easy and difficult texts. Psychonomic Bulletin & Review, 20(3), 586-592. Fritz, M. S., & MacKinnon, D. P. (2007). Required sample size to detect the mediated effect. Psychological Science, 18(3), 233–239.

COGNITIVE COUPLING DURING READING

24

Fulmer, S. M., D’Mello, S. K., Strain, A., & Graesser, A. C. (2015). Interest-based text preference moderates the effect of text difficulty on. Contemporary Educational Psychology, 41, 98-110. Gernsbacher, M. A. (1991). Cognitive processes and mechanisms in language comprehension: The structure building framework. Psychology of Learning and Motivation, 27, 217–263. Graesser, A. C., Hoffman, N. L., & Clark, L. F. (1980). Structural components of reading time. Journal of Verbal Learning and Verbal Behavior, 19(2), 135–151. Graesser, A. C., & McNamara, D. S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371–398. Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H., & Pennebaker, J. (2014). CohMetrix measures text characteristics at multiple levels of language and discourse. The Elementary School Journal, 115(2), 210–229. Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36(2), 193–202. Graesser, A. C., Ozuru, Y., & Sullins, J. (2010). What is a good question? In Bringing reading research to life (pp. 112–141). New York, NY, US: Guilford Press. Haberlandt, K. F., & Graesser, A. C. (1985). Component processes in text comprehension and some of their interactions. Journal of Experimental Psychology: General, 114(3), 357– 374. Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge: Cambridge University Press. Klare, G. R. (1974). Assessing Readability. Reading Research Quarterly, 10(1), 62–102.

COGNITIVE COUPLING DURING READING

25

Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44(1), 1–23. McNamara, D. S., & Magliano, J. (2009). Toward a Comprehensive Model of Comprehension. In B. Ross (Ed.) Psychology of Learning and Motivation (Vol. 51, pp. 297–384). Burlington: Academic Press. McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press. McNerney, M. W., Goodwin, K. A., & Radvansky, G. A. (2011). A novel study: A situation model analysis of reading times. Discourse Processes, 48(7), 453–474. Mills, C., & D’Mello, S. (2012). Emotions during writing on topics that align or misalign with personal beliefs. In Intelligent Tutoring Systems (pp. 638–639). Berlin Heidelberg: Springer. Myers, J. L., & O’Brien, E. J. (1998). Accessing the discourse representation during reading. Discourse Processes, 26(2–3), 131–157. O’Brien, E. J., & Myers, J. L. (1999). Text comprehension: A view from the bottom up. In S. Goldman, A.C. Graesser, & P. van den Broek (Eds.) Narrative Comprehension, Causality, and Coherence: Essays in Honor of Tom Trabasso, 35–53. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. Rand, D. G. (2012). The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments. Journal of Theoretical Biology, 299, 172–179.

COGNITIVE COUPLING DURING READING

26

Randall, J. G., Oswald, F. L., & Beier, M. E. (2014). Mind-wandering, cognition, and performance: A theory-driven meta-analysis of attention regulation. Psychological Bulletin, 140(6), 1411–1431. Reichle, E. D., Reineberg, A. E., & Schooler, J. W. (2010). Eye Movements During Mindless Reading. Psychological Science, 21(9), 1300–1310. Schooler, J. W., Smallwood, J., Christoff, K., Handy, T. C., Reichle, E. D., & Sayette, M. A. (2011). Meta-awareness, perceptual decoupling and the wandering mind. Trends in Cognitive Sciences, 15(7), 319–326. Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychological Methods, 7(4), 422. Smallwood, J., Fishman, D. J., & Schooler, J. W. (2007). Counting the cost of an absent mind: Mind wandering as an underrecognized influence on educational performance. Psychonomic Bulletin & Review, 14(2), 230–236. Smallwood, J., McSpadden, M., & Schooler, J. W. (2007). The lights are on but no one’s home: Meta-awareness and the decoupling of attention when the mind wanders. Psychonomic Bulletin & Review, 14(3), 527–533. Smallwood, J., & Schooler, J. W. (2006). The restless mind. Psychological Bulletin, 132(6), 130149. Smilek, D., Carriere, J. S. A., & Cheyne, J. A. (2010). Out of Mind, Out of Sight Eye Blinking as Indicator and Embodiment of Mind Wandering. Psychological Science, 21(6), 786–789. Stawarczyk, D., Majerus, S., Maj, M., Van der Linden, M., & D’Argembeau, A. (2011). Mindwandering: Phenomenology and function as assessed with a novel experience sampling method. Acta Psychologica, 136(3), 370–381.

COGNITIVE COUPLING DURING READING

27

Zwaan, R. A., Magliano, J. P., & Graesser, A. C. (1995). Dimensions of situation model construction in narrative comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(2), 386-397.

COGNITIVE COUPLING DURING READING

28

Appendix A

Reliability statistics for all four datasets. Datasets Cronbach's Alpha Mind Wandering Text-Level Comprehension Inference-level Comprehension

1

2

3

4

.750 .624 .430

.743 .808 .705

.737 .859 .719

.769 .701 .461

Running Head: COGNITIVE COUPLING DURING READING

29

Appendix B Full model statistics for Dataset 1 for Word Concreteness (WC) and Flesch Kincaid (FKGL) cognitive coupling Mind Wandering

Text-level

Inference-level

β

B

SE

p

β

B

SE

p

β

B

SE

p

-.030

-.017

.041

.679

.027

.009

.024

.698

.093

.050

.039

.207

Text Difficulty

.139

.080

.041

.053

-.144

-.049

.023

.037

-.191

-.103

.039

.009

Adjusted Reading Time

-.277

-.006

.002

.000

.371

.005

.001

.000

.070

.001

.002

.334

Coupling

-.062

-.239

.279

.393

.053

.122

.159

.446

-.058

-.207

.266

.438

.043

.025

.041

.549

-.056

-.019

.023

.404

.079

.043

.040

.287

Text Difficulty

.096

.055

.040

.168

-.096

-.033

.022

.146

-.187

-.101

.039

.011

Adjusted Reading Time

-.236

-.005

.002

.001

.324

.004

.001

.000

.063

.001

.002

.389

FKGL Coupling

-.237

-1.18

.359

.001

.285

.838

.201

.000

.090

.416

.351

.237

Dataset 1 Word Concreteness Model Text Presentation

FKGL Model Text Presentation

Notes. Text Presentation reference group = sentence by sentence; Text Difficulty reference group = easy

COGNITIVE COUPLING DURING READING

30

Appendix C Full model statistics for Dataset 2 for Word Concreteness (WC) and Flesch Kincaid (FKGL) cognitive coupling Mind Wandering

Text-level Comprehension

Inference-level Comprehension

β

B

SE

p

β

B

SE

p

β

B

SE

P

.052

.040

.193

.053

.019

.030

.532

.083

.033

.034

.332

Text Presentation

.115 -.035

-.016

.040

.691

-.022

-.008

.030

.791

-.104

-.042

.034

.221

Adjusted Reading Time

-.169

-.001

.000

.054

.108

.000

.000

.199

.127

.000

.000

.135

WC Coupling

-.182

-.335

.161

.039

.349

.506

.122

.000

.285

.463

.138

.001

.097

.044

.040

.269

.070

.025

.029

.390

.1011

.040

.034

.234

Text Presentation

-.027

-.012

.040

.762

-.045

-.016

.029

.582

-.120

-.049

.034

.154

Adjusted Reading Time

-.146

-.001

.000

.101

.039

.000

.000

.636

.076

.000

.000

.370

FKGL Coupling

-.118

-.760

.569

.184

.407

2.06

.416

.000

.292

1.66

.482

.001

Dataset 2 Word Concreteness Model Text Order

FKGL Model Text Order

Notes. Text Order reference group = dependent variable text first; Text Presentation reference group = paragraph by paragraph.

COGNITIVE COUPLING DURING READING

31

Appendix D Full model statistics for Dataset 3 for Word Concreteness (WC) and Flesch Kincaid (FKGL) cognitive coupling Mind Wandering

Text-level

Inference-level

Dataset 3 Word Concreteness Model Text Order

β

B

SE

p

β

B

SE

p

β

B

SE

p

.102

.047

.056

.400

.086

.030

.040

.458

-.101

-.039

.045

.395

Text Presentation

-.057

-.026

.057

.643

.115

.040

.041

.331

-.042

-.016

.046

.725

Adjusted Reading Time

-.102

-.001

.001

.407

.199

.001

.001

.097

.195

.001

.001

.112

WC Coupling

-.220

-.371

.207

.078

.237

.298

.148

.049

.220

.306

.168

.074

.058

.027

.056

.630

.138

.048

.039

.228

-.047

-.018

-.047

.680

Text Presentation Order

-.069

-.032

.056

.564

.127

.044

.039

.266

-.033

-.013

-.033

.774

Adjusted Reading Time

-.134

-.001

.001

.264

.232

.001

.001

.043

.224

.001

.224

.053

FKGL Coupling

-.275

-1.27

.550

.024

.326

.388

.288

.005

.352

1.34

.352

.003

FKGL Model Text Difficulty Order

Notes. Text Order reference group = dependent variable text first; Text Difficulty Order reference group = difficult half first and easy half second

COGNITIVE COUPLING DURING READING

32

Appendix E Full model statistics for Dataset 4 for Word Concreteness (WC) and Flesch Kincaid (FKGL) cognitive coupling Mind Wandering Dataset 4 WC Concreteness Model Rereading Condition Order

β

Text-level Comprehension

Inference-level Comprehension

B

SE

p

β

B

SE

p

β

B

SE

p

.057

.027

.049

.585

.021

.008

.039

.837

-.063

-.022

.034

.520

Topic Order

-.097

-.046

.050

.367

.096

.037

.040

.360

.077

.027

.035

.442

Adjusted Reading Time

-.127

-.001

.001

.241

.180

.001

.001

.090

.087

.000

.001

.383

WC Coupling

-.250

-.634

.270

.021

.293

.603

.214

.006

.444

.839

.186

.000

FKGL Model Rereading Condition Order

.112

.053

.051

.301

-.068

-.026

.038

.501

-.157

-.055

.036

.131

Topic Order

-.114

-.054

.050

.292

.110

.042

.038

.275

.107

.038

.036

.298

Adjusted Reading Time

-.140

-.001

.001

.197

.183

.001

.001

.074

.112

.001

.001

.278

FKGL Coupling

-.216

-.898

.448

.049

.383

1.29

.340

.000

.367

1.14

.319

.001

Notes. Text Order reference group = causal claims text first; Rereading Condition Text reference group = dependent variable text reread.

COGNITIVE COUPLING DURING READING

33

Appendix F Full model statistics for the combined datasets for Word Concreteness (WC) and Flesch Kincaid (FKGL) cognitive coupling Mind Wandering Combined Datasets Word Concreteness Model Study1

Text-level

Inference-level

β

B

SE

p

β

B

SE

p

β

B

SE

p

-.041

-.022

.033

.517

.508

.195

.023

.000

.114

.055

.030

.065

Study 2

.155

.090

.035

.011

.387

.163

.024

.000

.252

.133

.031

.000

Study 3

.090

.066

.041

.108

.198

.106

.028

.000

.287

.192

.036

.000

Adjusted Reading Time

-.147

-.001

.000

.001

.154

.001

.000

.000

.098

.001

.000

.026

WC Coupling

-.136

-.321

.112

.004

.222

.382

.076

.000

.147

.318

.099

.001

-.003

-.001

.032

.966

.445

.171

.022

.000

.073

.035

.029

.223

Study 2

.137

.079

.034

.022

.415

.175

.023

.000

.271

.143

.031

.000

Study 3

.064

.047

.040

.245

.236

.126

.027

.000

.312

.208

.036

.000

Adjusted Reading Time

-.131

-.001

.000

.003

.132

.001

.000

.001

.084

.001

.000

.053

FKGL Coupling

-.225

-1.13

.222

.000

.322

1.17

.148

.000

.212

.971

.198

.000

FKGL Model Study1

Notes. Study was coded as a dummy variable (3 binary variables) and used as covariates in the combined model instead of including all six experimental manipulations that were included in the individual dataset models.

COGNITIVE COUPLING DURING READING

34

Appendix G

Indirect effects for each dataset separately; Estimate (95% Confidence Intervals) Datasets D1

D2

D3

D4

Text-level

-.038 (-.081; -.013)

-.002 (-.069; .025)

-.031 (-.123; .026)

-.025 (-.070; .005)

Inference-level

-.005 (-.043, .031)

-.005 (-.057; .015)

-.031 (-.127; .-.017)

-.015 (-.047; .005)

Text-level

-.030 (-.062; -.011)

-.037 (-.106; .002)

-.048 (-.153;-.004)

-.048 (-.135; -.010)

Inference-level

-.011 (-.053; .019)

-.027 (-.096; .002)

-.053 (-.157;-.003)

-.220 (-.675; -.022)

Text-level

-.001 (-.017, .004)

-.047 (-.132; -013)

-.028 (-.122; .004)

-.051 (-.140, -.009)

Inference-level

.004 (-.009, .033)

-.039 (-.113; -.008)

-.024 (-.120; .008)

-.080 (-.171, -.027)

Adj. Reading Time

FKGL Coupling

WC Coupling

Notes. Models represent X = Mind wandering, M = coupling/adj. reading time, Y = comprehension; Bold indicates 95% CI does not contain zero. 10000 bias-corrected bootstrap samples used.

COGNITIVE COUPLING DURING READING

35

Appendix H Analyses were repeated with an alternate outlier removal (participants had to read at least 2.5 minutes of each text). Standardized regression coefficients (β) for predicting mind wandering, text-level and inference-level comprehension from adjusted reading time and coupling individually and combined in each dataset. Mind Wandering D1 Datasets Word Concreteness Adj. RT WC coupling

D2

-.128+

-.177+

-.095

-.177

+

-.123

-.142

Text-level

Inference-level

D3

D4

DC

D1

D2

D3

D4

.028

-.110

-.107*

.182*

-.040

.056

.120 +

-.163

-.278*

-.155*

.110

.280*

.060

.214

.002

-.135

-.096+

.174*

-.104

.068

.134

DC

D1

D2

D3

D4

DC

.037

.031

.063

.119

.001

.037

.167**

-.043

.271*

.049

.421**

.108*

.024

.029

.013

.136

.043

.029

.319*

.286*

.163**

FKGL Adj. RT FKGL coupling

-.215*

-.134

-.325*

-.258*

-.216**

.324**

.306*

.196

.321*

.271**

.097

.163

+

Notes. * = p < .05; ** = p < .001; + = p < .10; Adj RT = adjusted reading time; D1, D2, D3, D4 – datasets 1, 2, 3, and 4. DC = combined dataset; Models contain experimental covariates, adjusted reading time, and cognitive coupling measure.

COGNITIVE COUPLING DURING READING

36

Running Head: COGNITIVE COUPLING DURING READING 1 ...

Departments of Psychology d and Computer Science e ... University of British Columbia. Vancouver, BC V6T 1Z4. Canada .... investigate mind wandering under various online reading conditions (described .... Computing Cognitive Coupling.

632KB Sizes 0 Downloads 43 Views

Recommend Documents

Running head: REINTERPRETING ANCHORING 1 ...
should estimate probabilistic quantities, assuming they have access to an algorithm that is initially ...... (Abbott & Griffiths, 2011), and the dynamics of memory retrieval (Abbott et al., 2012; ..... parameter is the standard uniform distribution:.

Temporal Relations 1 Running head: EVENT ...
events do not overlap in any way, whereas OVERLAP means that two events share part of the time course but have .... sessions of a conference may not start at the same point in time, but people tend to perceive them as beginning .... MediaLab to displ

1 Running Head: RASCH MEASURES OF IMPLICIT ...
This paper provides a Many-Facet Rasch Measurement (MFRM) analysis of Go/No Go. Association .... correlation between the original variables, see Nunnally & Bernstein, 1994 for an analytical .... Data analysis was completed using Facets v.

Module 2 Summary 1 Running Head: MODULE 2 ...
change. By resisting today's ICT methods such as cell phones, Myspace, and Wikipedia, schools ... The only way to move forward effectively is to ... economic and business literacy; civic literacy; learning and thinking skills; creating the passion.

1 Running Head: ELEVATION AT WORK Elevation at ...
behavior. Study 1 used scenarios manipulated experimentally; study 2 examined employees' ..... represent a class of affective events that can evoke both positive and negative emotions in .... years, and 55% of them worked in production; the remaining

Social cognitive development during adolescence - Oxford Journals
However, the development of social cognition during adolescence and its neural underpinnings remains .... context of the Social Information Processing Network.

Structural Invariance SASH-Y 1 Running head ...
based on the factor analysis conducted by Barona and Miller .... applications and programming for confirmatory factor analytic ... Chicago: Scientific Software.

Cartography of Psychology 1 Running Head ...
Nanyang Technological University, Singapore. (in press). ... applied category included journals of applied, educational, measurement, and clinical ... collected 40 years of citation data from 1979 to 2009 and analyzed them with advanced MDS.