Journal of Experimental Psychology: Learning, Memory, and Cognition 1996, Vol. 22, No. 6, 1443-1462

Copyright 1996by the American Psychological~ i a t i o n , Inc. 0278-7393/96/$3.00

Sudden Insight: All-or-None Processing Revealed by Speed-Accuracy Decomposition Roderick W. Smith

John Kounios University of Pennsylvania

University of Colorado at Boulder

Issues surrounding the discreteness or continuity of cognitive processes have played a major role in experimental psychology, although there has been relatively little work that directly addresses these topics. Nevertheless, there has been a shift away from discrete models and toward continuous ones. The research reported in this article demonstrates discrete processing of information in an anagram task selected because of its similarity to insight problems, which seem subjectively to produce discrete "illumination" during processing. The authors used speed-accuracy decomposition (SAD), a relatively new technique for investigating the time course of information processing. The results of 2 experiments indicate little or no partial information in the anagram tasks, in contrast to previous research with SAD, all of which has revealed partial information. General models of human information processing must therefore be able to account for both patterns.

Many cognitive theories can be classified according to the basic assumptions they make about how information is processed. For instance, Miller (1988) has provided a taxonomy of ways in which information can be processed in a discrete or continuous fashion. This distinction, outlined in greater detail shortly, is an important one in cognitive psychology, and yet there is relatively little in the way of empirical research that directly demonstrates continuous processing and even less that shows discrete processing. Our goal with this research is to do just this, by examining tasks that we hoped would show evidence of discrete processing, in contrast to the continuous processing evidenced in previous research (e.g., Kounios, Montgomery, & Smith, 1994; Kounios, Osman, & Meyer, 1987; Meyer, Irwin, Osman, & Kounios, 1988; Ratcliff, 1988). Strong evidence for both discrete and continuous processing, particularly if obtained using the same experimental technique and, ideally, similar experimental tasks, would have strong implications for the nature of cognitive architectures, as defined by Anderson (1983) or Newell (1990a, 1990b). Because the phenomenon of insight seems, subjectively, to be one of a discrete change in cognitive states, it is insight-like tasks that have been selected for this study.

Insight The area of insight has a long history in psychology. Gestalt psychologists used the concept heavily, although not always in the way it has been used more recently. K6hler (1947), for instance, wrote extensively about insight, but he used the term to refer to the means by which a person correctly associates two things as being causally related. Schooler, Ohlsson, and Brooks (1993) laid out a fairly specific definition of insight as it relates to problem solving. They defined an insight problem as one that (a) is well within the competence of the average subject; (b) has a high probability of leading to an impasse, that is, a state in which the subject does not know what to do next; and (c) has a high probability of rewarding sustained effort with an 'Aha' experience in which the impasse is suddenly broken and insight into the solution is rapidly attained. (p. 168)

Roderick W. Smith, Department of Psychology, University of Colorado at Boulder; John Kounios, Institute for Research in Cognitive Science, University of Pennsylvania. This article is based on a doctoral dissertation submitted by Roderick W. Smith at Tufts University and was supported by National Institute of Mental Health Grant CEP 1 R29 MH45447-01AJ. We thank Alice Healy, Phillip Holcomb, Janet Metcalfe, Cynthia Thomsen, and Michael Wertheimer for helpful comments on drafts of this article. Correspondence concerning this article should be addressed to Roderick W. Smith, who is now at Department of Psychology,Indiana University, Bloomington, Indiana 47405. Electronic mail may be sent via Internet to [email protected].

Typically, research on insight has involved the use of some small set of presumed insight problems, which participants are asked to solve while articulating their thought processes (e.g., Duncker, 1945; Durldn, 1937) or manipulations of the task or environment with subsequent observations of frequency of or time to solution (e.g., Gick & Holyoak, 1980). Although interesting, such studies suffer from the intrusiveness and inexactness inherent in talk-aloud procedures, or from the limited nature of solution time and accuracy measures. For example, talk-aloud procedures have been found to interfere with some cognitive processes, including the solving of insight problems (Schooler et al., 1993). In addition, conventional solution time and accuracy measures cannot be used to directly study the time course of problem solving. Research has been done using measures of feeling of knowing and feeling of warmth in an effort to investigate the time course of insight and noninsight problem solution. Metcalfe (1986a, 198613), for instance, found that individuals' feelings of knowing accurately predicted which trivia questions they were able to answer but did not predict the correct solution of anagrams or other

1443

1444

SMITH AND KOUNIOS

insight problems. In fact, high ratings of feeling of warmth in Metcalfe's (1986b) research were actually predictive of incorrect solutions to insight problems. Further, Metcalfe and Wiebe (1987) found that algebra problems' correct solutions were predicted by feelings of warmth, unlike insight problems. This would tend to indicate that algebra problems are fundamentally different from insight problems, at least with respect to metacognitive processes having access to the workings of problem-solving mechanisms. Although the results obtained with feeling-of-knowing and feeling-of-warmth ratings of problem solving are intriguing, they are not conclusive. Feeling-of-knowing ratings can be criticized because they require repeated conscious responses at certain intervals, and these responses may interfere with normal processing (e.g., Schooler et al., 1993). Additionally, they tap conscious metacognitions, which may or may not accurately reflect underlying unconscious psychological processes, or which may reflect only a subset of the processes occurring. Finally, feeling-of-knowing judgments are metacognitive judgments that may or may not accurately reflect actual knowledge of the solution (e.g., Jameson, Narens, Goldfarb, & Nelson, 1990; Miner & Reder, 1994; Schwartz & Metcalfe, 1992). Still, metacognitive techniques represent one of the few ways used to date to study the time course of insight problem solving. In contrast, Bowers, Regehr, Balthazard, and Parker (1990) used a somewhat less direct approach. Bowers et al. used various tasks, none of them insight problems by a strict application of Schooler et al.'s (1993) definition, but all designed to produce similar subjective experiences. They measured the percentage of correct responses in forced-choice decisions between two items, one of which met some criterion and one of which did not. For instance, their Experiment 2 involved determining which of two degraded images was an image of a real object. Even when individuals could not name the object, their performance on the forced-choice task was above chance levels. Bowers et al. suggested that this was due to spreading activation, and that the subjective "aha" experience reported by many participants was the result of activation passing some threshold. When below this value, the activation could still influence the forced-choice task, but conscious recognition of the activation would be nonexistent. This represents an alternative explanation to Metcalfe's (1986a, 1986b; Metcalfe & Wiebe, 1987): Rather than a sudden shift in processing, Bowers et al. proposed an underlying continuity with a threshold for conscious recognition of the underlying activity. It should be noted, however, that Bowers et al. (1990) used different tasks from Metcalfe (1986a, 1986b; Metcalfe & Wiebe, 1987), so it is possible that task differences account for the obtained discrepancies in results, as Bowers et al. briefly discussed. Bowers et al.'s explanation would also require some reason for the fact that algebra problems (Metcalfe & Wiebe, 1987) produce accurate metacognitive judgments, whereas insight problems and anagrams do not (Metcalfe, 1986a, 1986b). If both problems were solved through similar types of spreading activation, why would one give metacognitive processes access to this and the other not? Thus, although Bowers et al. provided evidence for underlying continuity in at least

some seemingly discrete processes, it would be premature to conclude that all such processes are in fact completely continuous. In fact, the classification of a mental process as discrete or continuous is less straightforward than it might at first seem. Continuity and Discreteness o f Cognition The issue of insight is closely related to questions concerning the continuity and discreteness of cognition. Miller (1988) has laid out an analysis of these issues as dealt with implicitly and explicitly in information processing research. Such research typically does not use terms such as discrete and continuous in the same way they are used in mathematics. In information processing theories, these terms typically refer to two ends of a spectrum of possible processing strategies. For instance, a completely discrete process, as insight seems subjectively to be, might completely restructure a mental representation and pass it to the next processing stage instantaneously, whereas a completely continuous process would do so smoothly, with a literally infinite number of intermediate steps. Most cognitive theories, however, permit neither possibility, but instead posit some number of intermediate states. If the number of such states is small, the process may be considered discrete; if it is large, the process may be considered continuous. These definitions are admittedly imprecise compared with the mathematical definitions of the same terms, hut they are also more readily applicable to cognitive theories. According to Miller (1988), there are at least three senses in which information processing can be considered discrete or continuous: representation, transformation, and transmission. A representation is discrete or continuous to the extent that the symbols, links, or activations underlying the representation can take on a small or large number of values, respectively. Transformations are discrete or continuous insofar as they are built up of few or many distinct steps, respectively. Transmission is discrete or continuous depending on whether one stage passes information in a small number of chunks or in a larger number of "packages" or even as an ongoing "flow" of information. These senses of continuous and discrete are largely independent of one another, with one exception: Continuous transmission is an awkward concept if transformation is discrete, because in this case, there effectively is no partial information for the stage to pass on. Miller therefore considers this combination impossible. According to this analysis, an insight problem would seem to require, at the very least, discrete transmission of information to consciousness. The nature of the representation is relatively unimportant to the status of a problem as an insight or noninsight problem. An insight problem might at first seem to involve discrete transformation of information, but this is not necessarily so if the transmission of information is discrete, because it is the discrete transmission that allows for the "aha" experience. Of course, this does not preclude the possibility of both discrete transformation and discrete transmission of information in an insight task. Alternately, conscious awareness may be the result of activation in a single stage passing a threshold when continuously transforming, as Bowers et al. (1990) suggest. If consciousness operates parallel to the rest of the information processing

ALL-OR-NONE PROCESSING system after this threshold is reached, processing could proceed in a continuous fashion, but the perception would be of a discrete change--an "aha" experience. In some sense, however, even this represents a discrete transmission of information from the original nonconscious information processing system to the conscious system. It should be emphasized that it is primarily the suddenness of solution--hence the discrete nature of information transmiss i o n - t h a t seems to distinguish insight from noninsight problems, and that it is thus necessary to investigate the time course of information processing when studying insight. Metcalfe's (1986a, 1986b) research does this using metacognitive judgments, and Bowers et al. (1990) used accuracy on incomplete solution to get at (presumably) ongoing processing, but these are only two methods of acquiring information on the time course of information processing, and neither directly addresses the question of how information accumulates (or does not) over time. A fuller understanding of insight processes might be gained by using other techniques to investigate insight or insight-like problems. Although we speak of a "sudden" solution or an "aha" experience, these are inherently subjective terms in this context. It is possible that an objective measure of the time course of such an experience would reveal a process that occurs over some relatively wide periodusay, hundreds or even thousands of milliseconds---but the subjective experience of this would be distorted. This is why investigating the time course of such experiences is important. On the other end of the scale, a "sudden" solution or even discrete transformation or transmission cannot occur truly instantaneously, as neural impulses are limited in velocity. Newell (1990b, pp. 121-139) suggests splitting human activities into various time "bands" ranging from organelle activity within cells to the "social band," which can span months. By this analysis, a discrete change would occur on Newell's neural circuit system, which has a temporal extent measured roughly in the 10s of milliseconds up to perhaps 100 ms. Speed-Accuracy Decomposition One technique that has been used to investigate the time course of cognitive processes is speed-accuracy decomposition, or SAD, developed and most fully explained by Meyer et al. (1988; see also Kounios et al., 1987, 1994; Ratcliff, 1988; R. W. Smith, Kounios, & Osterhout, 1996). This technique is designed to measure the accrual of information over the course of time. That is, can a person's final response accuracy be explained by a continuous increase in knowledge over time or by a discrete jump in information immediately before a response occurs.'? SAD measures partial response information (or simply partial information), which can be defined as an individual's capacity to provide a correct response before processing is completed. Partial information could take many forms in various cognitive theories, and SAD does not discriminate among these. For instance, subthreshold activations in a parallel distributed processing (PDP) or semantic network could produce partial information; or a handful of insufficient "clues" (such as green color in a task involving the classification of photographs into images of plants and animals) might

1445

do so. Note that SAD measures partial response information. Although the word response is frequently omitted from the phrase for brevity's sake, it should not be entirely forgotten, as SAD utilizes a physical response by the participant, and so can only measure partial information to which response mechanisms have access. Given that every SAD study conducted to date (Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988) has produced evidence of partial information, it seems that a wide range of cognitive phenomena does provide partial information to response systems. As a means of measuring partial information, SAD has obvious applicability to insight phenomena; if they are what they subjectively seem to be, then insight problems will not produce partial information, whereas they likely will do so if they utilize some underlying continuous processing. In terms of Miller's (1988) taxonomy, SAD most directly addresses the question of information transmission. SAD generally utilizes probes at multiple points in time, and if these measures turn up increasing levels of partial information over time, it serves as evidence for continuous transmission of information, with increasing evidence as the number of probes increases. As continuous transmission is essentially impossible without continuous transformation, however, a finding of increasing levels of partial information in a SAD study implies similarly continuous transformation as well. A constant level of partial information, however, is consistent with relatively discrete transmission, whereas a constant level fixed at chance accuracy implies an extreme, all-or-none discrete transmission of information. In addition, because all previous SAD studies (Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988) have found evidence for partial information, it seems that much information is transmissible to response systems even before a continuous transformation is complete. This would therefore suggest the strong possibility, though not an absolute certainty, of discrete transformation should discrete transmission be found by SAD. SAD involves measuring the speed and accuracy of responses to stimuli, as in ordinary reaction time (RT) studies. In a SAD study, however, on some trials (regu/ar tr/a/s) participants have as much time as they need to make an accurate response, as in a conventional RT study. On signal trials (randomly intermixed with the regular trials), participants are presented with a visual or auditory response signal at some point after the onset of the test stimulus. This serves as a cue to the participant to respond immediately, effectively limiting the time available for processing. This differs from speed-aceuracy macro-trade-off experiments (e.g., Pachella, 1974; Wickelgren, 1977), in which all trials have response signals and participants are instructed to respond only after detecting a response signal) In SAD, participants are not forced to hold their

I Conventional speed-accuracy trade-off procedures contain an important limitation, in that they do not distinguish actual partial information accrual from an all-or-none jump in information with a variable time of that jump (Meyer et al., 1988;Wickelgren, 1977). For instance, if a discrete jump in information sometimes occurs at Time x and sometimes at Time y, a speed-accuracy macro-trade-off signal placed between these two times will reveal, given enough experimental trials, what seems to be an intermediate level of partial information,

1446

SMITH AND KOUNIOS

responses until a response signal; they may respond at any time before the response signal, or immediately after its presentation. Participants are instructed that accuracy is of primary importance and speed is of secondary importance on regular trials, whereas these priorities are reversed once a response signal occurs. Because regular trials and signal trials are randomly interspersed during the course of an experimental session, participants have no way of knowing whether a particular trial will contain a response signal and therefore cannot adopt different processing strategies for the different types of trials. The priority shift between the two types of trials must occur entirely after the response signal has appeared, if it does. SAD applies a parallel sophisticated guessing (PSG) model to the data from the regular and signal trials to derive an estimate of overall guessing accuracy. This model is described in Appendix A.

Advantages and Disadvantages of S A D Partial information detected by SAD may be conscious or unconscious. In either ease, it must directly contribute to an explicit response. Some types of partial information, such as information passed from one stage to another in a serial processing model, may not directly affect response systems, and so will not be detectable by SAD. Previous research using SAD (e.g., Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988), however, has always yielded evidence of partial information, suggesting that, in many experimental paradigms, much partial information is transmissible to response systems. One advantage of SAD is that it involves only one intrusion on processing per trial on signal trials (the response signal), and none on regular trials, so disruption of processing is virtually eliminated, compared to the repeated judgments required in feeling-of-knowing research or the continuous verbal responses required in verbal response protocol paradigms. On the other hand, a practical limitation of SAD is that such studies require a large number of stimuli to be effective, because the technique relies on accurate estimates of the distributions of response times for each signal lag and for the regular trials for each class of stimuli. This limitation is exacerbated by the need for a very large number of practice trials to train participants in the task. There must also be some way to measure RT, and the probability of producing a correct response at random must be known.

Experiments 1A and 1B In this research we used the SAD technique to investigate the presence of partial information in anagram problem solving, a task that has previously been shown (e.g., Metcalfe, 1986b) to produce a relatively rapid shift in people's metacognitive awareness of the solution. Metcalfe's research led us to

because some responses will be based on the state of no accuracy and others will be based on completed processing. The SAD technique avoids this problem by the use of regular-trial response information to factor out responses based on completed processing.

expect that an anagram-solving task would yield no partial information .2 Anagrams were selected as a task for a variety of reasons. In addition to there being a substantial body of research on anagram problem solving to use as a guide (e.g., Beilin & Horn, 1962; Mayzner & Tresselt, 1958, 1959; Mendelsohn & O'Brien, 1974; Metcalfe, 1986b; Richardson & Johnson, 1980), anagrams seem to be insight, or at least insight-like, problems, for which the subjective experience is one of sudden solution. Although not precisely an insight task by Schooler et al.'s (1993) definition, anagram problems have been studied alongside insight problems and have produced results that are in many ways similar to those obtained from insight problems (e.g., Metcalfe, 1986b). Equally important is the fact that a sufficient number of anagrams can be generated from English words for use in a SAD study--a criterion not met by the usual set of insight problems. The specific task used for anagram problem solving is an adaptation of one used by White (1988). In this task, participants are shown strings of letters, some of which are anagrams of real words and some of which are not. For instance, ROUYis an anagram of the word YOUR, whereas ROUV cannot be rearranged to form any English word. The participants' task is to determine whether the string can be rearranged to form a word. These experiments are not intended to test any specific model of anagram solution or even of problem solving per se; rather, they represent an investigation of broader issues of discreteness and continuity in human thought. Because previous studies using SAD have always shown partial information, it would be theoretically interesting and important to find a task that does not produce partial information. The existence of such a task would be strong evidence that human cognition is made up of both discrete and (with the evidence from previous SAD studies) continuous information transmission, and probably information transformation.

Experiment 1A The primary goals of this experiment were to collect normative RT and accuracy data in a conventional RT task and to confirm that the anagram task was workable and would produce RTs amenable to use in a SAD experiment. The normative data are used in determining placement of response signals in a subsequent SAD study using the same task.

Method Partic~vants. Thirty-four Tufts University undergraduates participated in partial fulfillment of course requirements. Materials. The stimuli were derived from the Ku~era and Francis (1967) word frequency norms. Four-letter words were selected that (a) e As discussed earlier, however, metacognitive judgments are limited in a number of ways, including their temporal resolution, so a SAD study serves to extend such work considerably. The mathematics of SAD allows it to produce accurate estimates of chance guessing levels, and this has been demonstrated in simulations reported by R. W. Smith et al. (1996) and summarized in this article (in Appendix B), so there is little doubt about SAD's suitability to this task.

ALL-OR-NONE PROCESSING had no other word as an anagram, (b) had no repeated letters, (c) were high in frequency in English (25-1,000 occurrences in the Ku~era & Francis sample), and (d) were high in bigram frequency (greater than 900, on the basis of Underwood and Schulz's, 1960, "total count" of bigram frequency). A total of 47 words were selected using these criteria. From these words, a computer program generated anagrams by swapping the positions of each possible combination of pairs of letters (for a total of six anagrams for each word) and selecting the anagrams of each word with both the highest and lowest bigram frequency. A letter string was then derived from these anagrams that was not an anagram of an English word and that was as close as possible in bigram frequency to the anagram by substituting individual vowels for vowels and consonants for consonants. The lettersa; y, and z were not used in this substitution. Thus, two word anagrams and two nonword anagrams were constructed for each word. These were split into two blocks of 94 stimuli each, with one anagram of each word appearing once in each list, and each anagram's matched nonword appearing in the other list. In addition, 10 practice stimuli were generated in a similar manner, but using relaxed criteria for frequency in English. Five word anagrams were used for the practice stimuli, each of which had one associated nonword anagram. Stimuli were presented in uppercase letters on IBM PC-compatible computers with amber (monochrome) monitors. Participants responded to the experimental task on four-button response boxes using their thumbs on the outermost two buttons. Procedure. The participants' experimental task was to decide whether each stimulus was an anagram of a real word. Each session consisted of three blocks: A short practice block (10 trials) followed by two experimental blocks (94 trials each). Stimuli were presented individually, preceded for 1,000 ms by a row of four asterisks as a fixation point and warning signal. The computer provided error feedback after each response. Participants could take up to 30 s to respond to each stimulus, at which point the computer automatically presented the next one, coding the previous trial as a late response.

Results and Discussion The results are summarized in Table 1. As SAD analysis relies on both correct and incorrect responses, both types are included in the reported RT values. In this and all subsequent experiments, an alpha level of .05 was used, with .10 for marginal significance. Separate repeated measures analyses of variance (ANOVAs) were performed on accuracies and RTs. Accuracies improved and RTs decreased from Block 1 to Block 2, F(1, 33) = 31.358,p < .001 for accuracy; F(1, 33) = 44.635,p < .001 for RTs. Accuracies and RTs were lower for word than for nonword stimuli, F(1, 33) = 177.796, p < .001 for accuracy; F(1, 33) = 87.972, p < .001 for RTs. Word Status x Block interaction effects were significant for accu-

1447

racy, F(1, 33) --- 5.081,p = .031, but not for RTs, F(1, 33) = 2.550, p = .120, reflecting a greater increase in accuracy from Block 1 to Block 2 for word than for nonword stimuli. The above results suggest that the task is suitable for use in a SAD experiment. Ideally, RTs and accuracies should be comparable to those used in previous SAD studies. Failing that, overall accuracies that at least approach 80% would be wanted, to ensure that sufficient regular response accuracy exists to be able to detect partial information; RTs that are not dramatically more than those observed in previous SAD experiments would also be wanted, to avoid the possibility of some unforeseen problem with SAD analysis with long RTs. Although the RTs from Experiment 1A are longer than those used in previous SAD experiments (1,821-3,472 ms vs., for instance, 720-1,115 ms in Kounios et al., 1994), the overall averages are not unreasonable. In addition, the fact that RTs decreased from the first to the second block suggests that highly practiced individuals, as are used in SAD, might have even shorter RTs. Similarly, accuracies are acceptable, at 82% overall. These results indicate that repetition or practice may have a strong effect on RTs in anagram solution, as the Block 2 RTs are considerably shorter than those of Block 1 (2,321 vs. 2,909 ms overall). Thus, subsequent experiments reported here provided practice stimuli at the beginning of a session to minimize effects of rapid learning and did not repeat words, even as different anagrams, for any critical stimuli.

Experiment 1B This experiment was an extension of Experiment 1A using the SAD technique. The goal of the experiment was to ascertain whether an anagram task would produce partial information. We anticipated, based on Metcalfe's (1986a, 1986b) results, that it would not. A finding contrary to this expectation would indicate a theoretically interesting dissociation between metacognition, as revealed by Metcalfe's work, and partial response information; people would know, but would not know that they know. To the extent that very low levels of partial information (e.g., chance responding) are revealed, it would be evidence that some form of discrete information processing, most likely taking the form of a discrete transmission, and possibly discrete transformation, is occurring. This would contrast with the partial response information found in all previous SAD studies (Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988), which indicates

Table 1

Mean Reaction Times ([RTs], in Milliseconds) and Accuracies (Proportion Correct) for Word-Nonword Anagram Judgments in Experiment 1A Block 1

Block 2

Stimulus type

RT

Accuracy

RT

Accuracy

Word Nonword

2,346 (130) 3,472 (248)

0.681 (0.016) 0.902 (0.013)

1,821 (98) 2,821 (176)

0.754 (0.019) 0.936 (0.011)

2,909

0.792

2,321

0.845

M

Note. Standard errors are in parentheses.

1448

SMITH AND KOUNIOS

continuous transmission, and almost certainly continuous transformation, of information.

Method Partic~oants. Twenty-seven participants were recruited, primarily through sign-up sheets on the Tufts University campus. Participants were paid $4.50 per session plus a bonus of $0.02 per correct answer in the task. Materials. Stimuli were selected in the same way as in Experiment 1A, with the following changes: (a) A total of 78 four-letter words and 78 four-letter nonwords were used as experimental stimuli, with an additional 54 of each used for practice and generated using relaxed criteria for frequency in English; (b) an equal number of anagrams were generated for each of the six two-letter swaps possible for four-letter words, to deter systematic search strategies; (c) anagrams were selected to bring the average bigram frequency as close as possible to 600, while still meeting Requirement b; and (d) stimuli were randomly assigned (in equal numbers) to one of three conditions-regular trial (no response signal), early-signai lag (1,000 ms), and late-signal lag (1,500 ms). These lags were selected so that the guessing responses on these trials would fall within the distribution of response times on regular trials, as obtained in Experiment 1A. The assignment of stimuli to these conditions was counterbalanced across subjects for the final test session. An additional 486 flve-letter anagrams and 486 nonwords were selected for practice sessions. These were generated in a manner similar to the selection of four-letter anagrams, except that the target bigram frequency was 750, and there were 10 possible two-letter swaps with five-letter words. The practice stimuli were assigned equally to regular trial, early lag, and late lag conditions. Stimuli were presented on the same apparatus as in Experiment 1A. Procedure. The procedure involved multiple practice sessions and one final data collection session. Except for the first practice session (described below), both practice and final sessions followed the same procedure. Each practice session used a unique random selection of stimuli from the five-letter stimulus pool for each block. The final session used all of the four-letter stimuli, presented in blocks, with order randomized for each participant within each block. Participants were told that they were to make word-nonword judgments about the stimuli. The warning signal was an arrow ( " ~ " ) to the left of the location the stimulus would ultimately occupy. Two-thirds of the trials were signal trials, in which the stimulus was masked out by pound signs ( " # # # # " ) as a signal to respond immediately. Whenever a participant took more than 325 ms to respond to the signal, the computer began a 75-ms, 880-Hz tone. If a participant took more than 200 ms after this tone (600 ms total), the computer coded the response as late and went on to the next stimulus. On regular trials, participants had up to 30 s to respond, after which point the computer would present a response signal. This was due to a timing limitation of the program. No participant took this long to respond on any trial in the final session. Error feedback was given after each trial. For the purposes of this feedback, late responses were reported as errors, though they were coded differently in the data file. The intertrial interval was 2,000 ms. Each session consisted of three blocks, each of which contained 10 practice stimuli and 78 stimuli identified as nonpractiee (i.e., experimental). In reality, the entire first block was practice in the final session. This was done to minimize the potentially disruptive effect of the switch from five-letter to four-letter anagrams on the experimental data, as well as to allow participants to reach full involvement in the task. The first session was intended to give participants some familiarity with the anagram judgment task, without the stress of the response signals. It was identical to the practice sessions, except that no response signals were presented; participants had up to 30 s to make

each response. It was virtually identical to the task used in Experiment 1A. The practice sessions gave participants sufficient practice with the anagram task and with the SAD procedure so that their data would be useful. Participants knew that these were practice sessions and were given feedback at the end of each session on their performance in terms of RTs, percentage of correct answers, and number of late responses to response signals. Before being tested in the final session, participants were required to exceed, for three or more consecutive sessions, a criterion of greater than 80% accuracy on regular trials with no more than one or two late responses on signal trials, these being defined as responses occurring more than 600 ms after onset of the response signal. Participants engaged in an average of 9.2 days of practice (range: 7-12) before the final session.

Results Criteria for inclusion of a participant's data in the final data analysis were as follows: (a) Fewer than 12 (11%) late responses on signal trials, defined here as responses more than 425 ms after onset of the response signal, and (b) an estimated percentage of guessing responses of greater than 20% for each signal-lag condition. Criterion a exists to eliminate individuals who had not, despite previous indications, learned the S A D task adequately. A large number of late responses might indicate that participants were attempting to steal a few extra milliseconds of thought before responding, possibly altering their cognitive processes and violating S A D ' s assumptions. Late responses might also indicate inattention, fatigue, or the development of some heuristic for quick responses in the practice sessions that was inapplicable in the final session. A threshold value of 425 ms was adopted because it is only 100 ms past the point at which the beep began, and thus should not be contaminated by processing alterations from the beep itself, given that fast simple RTs tend to be on the order of 100-200 ms (e.g., Ollman & Billington, 1972). Eleven responses were selected as the second part of the cutoff because this was a convenient cutpoint separating a group of participants showing relatively few late responses from a much smaller group showing many (e.g., as many as 29). Criterion b exists to remove participants for whom the response signal was too late relative to their regular trial responses. This would result in estimated guessing accuracies based on very few or no responses, because fast normal-process RTs would predominate even on signal trials, leaving few or no guessing process completions on which to base estimates of partial information. That is, because S A D uses, essentially, the accuracy of those trials identified as guessing trials for its estimate of guessing accuracy, if few trials are identified as guessing trials, estimates of guessing accuracy are likely to be inaccurate because of insufficient data. The 20% criterion amounts to 5.2 guessing trials and was selected because experience shows that estimates based on fewer than this percentage of guessing trials tend to be wildly and obviously erratic (e.g., 600% estimated guessing accuracy), whereas estimates based on more than 20% are not. Twenty-seven people were recruited. F o u r failed to complete the experiment, and 9 others were eliminated from the analysis because of failure to m e e t the above criteria, leaving 14 participants, 5 in each of two counterbalance conditions and

1449

ALL-OR-NONE PROCESSING Table 2

Mean Reaction Times ([RTs], in Milliseconds) and Accuracies for Word-Nonword Anagram Judgments in Experiment 1B Regular trials Stimulus type Word Nonword

RT

Accuracy

Lag 1a RT

Accuracy

Lag 2a RT

2,099 (164) 0.71 (0.031) 1,149 (21) 0.56 (0.034) 1,415(27) 3,835 (257) 0.96 (0.013) 1,290 (9.4) 0.82 (0.039) 1,699(28)

M 2,967 Estimated PI proportion correct Estimated PI d ' Estimated PI log 13

0.83

1,220

0.69 0.550 (0.025) 0.460 (0.222) 0.397 (0.215)

1,557

Accuracy 0.58(0.030) 0.88(0.028) 0.73 0.466 (0.048) -0.193 (0.413) -0.331 (0.426)

Note. Standard errors are in parentheses. PI -- partial information. PI cannot be estimated for regular trials in isolation. aThese columns represent all responses for each signal lag condition, including both normal and guessing processes.

4 in the third condition. One participant from each of the two conditions with 5 individuals was randomly selected for removal, to make the counterbalancing even at 12 participants total, 4 in each of the three conditions. RTs and accuracies (as well as their associated standard errors) are presented in Table 2. As in Experiment 1A, RTs are based on both correct and incorrect responses. Separate 2 x 3 (Stimulus Type x Lag) repeated measures ANOVAs were performed on the RTs and accuracies. For this and all subsequent ANOVAS, the Geisser-Greenhouse correction to probability levels was applied wherever appropriate. RTs and accuracies increased with increasing signal lag time (considering regular trials to have the longest lags), F(2, 22) = 72.428, p < .001, and F(2, 22) = 18.287,p < .001, respectively. Words produced faster and more accurate responses than nonwords, F(1, 11) = 172.236,p < .001 and F(1, 11) = 84.335,p < .001, respectively. For RTs, the interaction was also significant, F(2, 22) = 99.958,p < .001, apparently reflecting a greater regularversus signal-trial effect for nonwords than for words. For accuracy, the interaction was not significant, F(2, 22) = 0.579, p --- .526. Note that overall RTs and accuracies on regular trials are similar to those obtained in Block 1 of Experiment 1A; mean RT was 2,967 ms in the current experiment, compared with 2,909 ms in Experiment 1A, and aceuraeies averaged .83 in Experiment 1B, compared with.79 in Experiment 1A. There is therefore little evidence that the nature of processing in Experiment 1B was different from that in Experiment 1A, though of course this simple comparison is not conclusive. Table 2 also presents the estimated levels of partial information for each signal lag, as computed by an implementation of Equation A2 (see Appendix A) in a computer program, in terms of both proportion correct (PC) and d'. 3 T tests reveal that the Lag 1 result was significantly greater than chance, t ( l l ) --- 1.982,p = .036 for PC; t ( l l ) -- 2.069,p = .032 for d', when one-tailed tests were used, but that the estimated guessing accuracy did not differ significantly from chance for Lag 2, t(11 ) = -0.707, p = .494 for PC; t(11) = -0.467, p = .649 for d', both t w o - t a i l e d : Additionally, a within-subjects t test shows that the difference between Lag 1 and Lag 2 was not significant, t ( l l ) = 1.557,p = ,148 for PC; t ( l l ) = 1.665,p = .124 for d', both two-tailed, showing that the numerical shift from

positive to negative guessing accuracy over time does not necessarily indicate a shift in processing to which the test of absolute level of partial information was not sensitive. Regulartrial d' values averaged 1.913 across subjects: The log 13line in Table 2 shows a measure of response bias (J. E. K. Smith, 1974). Positive values indicate a bias to respond "word," whereas negative values indicate a bias to respond "nonword." The Lag 1 results are marginally significantly greater than 0, indicating a possible bias to respond "word," t ( l l ) = 1.847,p = .092, but the Lag 2 results did not differ significantly from 0, t ( l l ) = - 0 . 7 7 8 , p = .453. There was a marginally significant change from Lag 1 to Lag 2, t ( l l ) = 1.967, p = .075 (all tests two-tailed). SAD's mathematics makes no assumptions concerning response bias, and the accuracy of SAD's results has been shown (R. W. Smith et al., 1996) to not be affected by it, so the possibility of a shift in response bias is not a major concern here; however, such a shift can make interpretation of percent correct measures difficult, and performing a signal detection analysis on the results of a SAD analysis will of course bring in signal detection theory's

3 A logistic d' measure was used, as reported by J. E. IC Smith (1974). This measure is simple to compute and closely approximates the more common Gaussian d' function. This same measure was used by Kounios et al. (1987) and Meyer et al. (1988), and so is wellestablished in the SAD literature. SAD itself produces a PC measure, from which d' can then be computed. 4 We would normally expect either no partial information or positive partial information (estimated guessing accuracies greater than 50%), and would therefore use one-tailed t tests; however, the less-thanchance estimated partial information values for Lag 2 suggested that a two-tailed test would be more appropriate for this condition. A two-tailed test is appropriate for the Lag 1 to Lag 2 comparison because some models (e.g., Ratcliff, 1988) allow for partial information levels to rise and then fall again. 5 Although d' measures are frequently favored over proportion correct, in the current case, d' suffers from a disadvantage in that the SAD analysis software occasionally produces estimated guessing accuracies in excess of 1.0 or below 0.0, requiring rounding these values to 0.99 or 0.01, respectively, possibly introducing some distortions. We thus favor the PC measures in this study, though in most cases PC and d' yield identical results.

1450

SMITH AND KOUNIOS

¢ncn ~Q'°rc' n0.7105 sk Lag2 nonword n" " :~ 0.25 g

D.

o

-0.250

500 1000 1500 2000 Time (ms)

Figure 1. Averaged cumulative distribution functions for estimated guessing-time distributions from Experiment lB.

assumptions. The fact that PC and d' measures produced the same results is therefore encouraging, as the two measures, each with different strengths, converge on the same conclusions. In evaluating SAD results, the estimated proportion of guessing processes finishing before normal processes on signal trials is important. The lower this proportion, the less data on which the estimate of guessing accuracy is based, resulting in a less reliable estimate. Averaged across subjects, the estimated proportions of guesses finishing first (i.e., determining the response) on signal trials in the current experiment were .556 and .382 for word anagrams for Lags 1 and 2, respectively, and .931 and .867 for nonword anagrams for Lags 1 and 2, respectively. These values are all acceptably high; partial information estimates for each participant are based on an average of from 10 to 24 estimated guessing responses and 26 regular-trial responses per condition. Thus, considered across all 12 participants, the partial information estimates are based on an estimated 854 guessing responses (analyzed separately for each participant). With so many guessing responses, it is unlikely that the results are distorted by too small a sample size. These numbers also represent adequate placement of the signals relative to the regular-trial response-time distributions. If the signals had been placed too early, it could be that no partial information was detected simply because it had not begun to accumulate by the last signal. Considered across stimulus types, however, normal processes are estimated to account for 37.6% of the Lag 2 responses. With over a third of the signal-trial responses accounted for by normal-process completions, it seems doubtful that these same signals were placed too early to detect partial information. Tests of the PSG model. The primary assumptions of SAD can be tested by examining the cumulative distribution functions (CDFs) of the estimated guessing responses. These CDFs represent the accumulation of estimated guessing responses over the course of a trial for a specific condition. They are obtained from an implementation of Equation A1 in Appendix A. If the shapes of the guessing-time CDFs for the different stimulus types vary systematically, it is evidence that

different processes may be producing the different guessing responses, or that the guessing processes may be interacting with the normal processes in violation of the assumptions of SAD. Such a pattern of results in the PSG model tests should serve as a warning to examine the data with extreme care, for this indicates a possibility, but not a certainty, that the partial information estimates obtained from the SAD analysis may be spurious. Simulations summarized in Appendix B indicate that SAD is robust to such violations under certain circumstances, however. To study these CDFs systematically, it is necessary to first adjust the CDFs for the differences caused by the signal lag time itself. After this correction, six points on each CDF curve were selected for analysis by multivariate A N O V A (MANOVA), with the time, lag, and stimulus type as the independent variables and the proportion of responses at the various points in time as the dependent variables.6 For display purposes, these proportions of responses were averaged across subjects, and lag time was not subtracted. Figure 1 presents the averaged estimated guessing time CDFs. These data were analyzed by a 2 x 2 x 6 (Stimulus Type x Lag x Time) M A N O V A using the cumulative frequency of responses at specified times after the response signal as the dependent variables. Times used ranged from 200 ms to 400 ms at intervals of 40 ms. The results show a marginally significant main effect of stimulus type, F(1, 11) = 3.806, p = .077, and significant effects of both lag, F(1, 11) = 9.268, p = .011, and time, F(5, 7) --- 162.626,p < .001. The effect of time reflects the growth of responses over the course of a trial, and was expected. The lag and stimulus type effects were not expected, however, and represent differences in the mean estimated guessing times between the two lags and stimulus types. The Lag x Time interaction effect was also significant, F(5, 7) = 7.741,p = .009, indicating a difference in CDF shape between the two lags. No other interaction approached significance (allps > .230). Overall, these results are somewhat ambiguous. The Lag x Time interaction effect indicates that the shape of the guessing distribution may be different from Lag 1 to Lag 2, though the effect size seems small by an "eyeball" analysis. The marginally significant effect of stimulus type may be of some concern, because the overall guessing accuracy estimates rely on estimates for both stimulus types, and if the guessing processes for 6 It should be noted that combining RT distributions in this way has been criticized on the grounds that it can distort the shapes of the distributions (Ratcliff, 1979; Thomas & Ross, 1980). The suggested alternative is a procedure known as Vincentizing, in which the times corresponding to specific proportions of responses are compared and averaged. Vincentizing, however, requires monotonic functions. Because of the fact that, by chance, more regular-trial than signal-trial responses can occur in any given time period in the SAD procedure, nonmonotonic estimated guessing CDFs are common. Vincentizing therefore cannot be applied to the SAD estimated guessing data without first applying some sort of data smoothing algorithm, which may itself distort the curves. R. W. Smith et al. (1996) have shown that the distortions introduced by such a smoothing technique tend to produce spurious results in PSG model tests, whereas averaging does not. The averaging approach is therefore used exclusively in these experiments.

ALL-OR-NONE PROCESSING these types are different from one another, the overall guessing accuracy estimate may be distorted. Still, the magnitude of this effect is small in terms of the change in RTs---only 8 ms at the median--and the effect is only marginally significant. It seems likely that this effect represents a statistical artifact or some amount of predictability in stimulus orderings or assignment. Such predictability, however, would be expected to increase the estimates of partial information; because there is little or no estimated partial information, this would not affect the conclusions of this experiment. The effect of lag is also of some concern, but again, the effect is small in absolute magnit u d e - 2 4 ms faster responses in the late lag than in the early lag. Also, in the simulations reported by R. W. Smith et al. (1996) and summarized in Appendix B, although statistically significant stimulus type and/or lag effects do seem to point to possible violations of SAD's assumptions, the magnitude of the resulting error in estimated guessing accuracy was small (a .022 increase in estimated guessing accuracy, in the worst-case situation, on average). A n a l y s i s by n u m b e r o f moves. Because we used all possible two-letter swaps as anagrams (for different words), this experiment included both "one-move" and "two-move" anagrams, and this variable has been found to have an effect on anagram solution times (e.g., Hunter, 1959; Mendeisohn & O'Brien, 1974). It is therefore possible that processing, and therefore the accumulation of partial information, would vary for these different stimulus types. The regular-trial RTs on the two types of anagrams did vary systematically, with one-move anagrams being solved more quickly (M = 1,823 ms, S E = 196) than the two-move anagrams (M = 2,416 ms, S E = 206), t(ll) = -2.472,p = 0.031, two-tailed. These data have therefore been submitted to separate SAD analyses for the one-move and two-move anagrams. Unfortunately, this reduced the number of stimuli for each participant to half of the original number for each condition and also reduced the number of individuals in each analysis, to 6 for the one-move and 10 for the two-move anagrams, as a result of failure to meet the 20% cut criterion for individual anagram types for specific participants. This loss has also seriously disrupted the counterbalancing established for the main analysis; all of the participants for one condition (which also showed the lowest levels of partial information in the overall analysis) were removed. With these caveats, then, the one-move analysis showed the Lag 1 partial information level to be significantly greater than chance (PC = .671, S E = .052; d ' = 1.265, S E = 0.330), t(5) = 3.298,p = .011 for PC; t(5) = 3.827, p = .006 for d ' , both one-tailed. The one-move Lag 2 results were not significantly different from chance (PC = .562, S E = .058; d' = 0.728, S E = 0.491), t(5) = 1.066, p = .168, and t(5) = 1.482, p = .100, respectively, both one-tailed, though the d' measure hovered on the edge of marginal significance. The one-move Lag 1 values were marginally significantly different from the Lag 2 values for PC, t(5) = 2.146, p = .085, but not for d', t(5) = 1.767, p = .137, both two-tailed. The two-move anagrams were not significantly different from chance for the Lag 1 condition (PC = .522, S E = .048; d' = 0.192, S E = 0.334; bothps > .289, one-tailed), but the two-move Lag 2 anagrams showed marginally significant results with one-tailed tests (PC = .614, S E = .064; d' = .858, S E = 0.491), t(9) = 1.794,p = .053, and t(9) = 1.746,p = .058,

1451

respectively. The two-move partial information levels were not significantly different from one another (both ps > .256, two-tailed). The one-move PSG model tests showed no significant results aside from the necessary effect of time, but the two-move results showed significant effects of lag, F(1, 9) -15.089,p -- .004, and marginally significant Lag × Time effects, F(5, 5) = 4.284, p -- .068, suggesting the possibility of some PSG model violations. These results should all be taken with extreme caution, however, given the reduced sample sizes used. This is especially true in light of the total elimination of one counterbalance condition from the one-move results. Discussion

As expected, we found little or no partial information in this task. Although the estimates of partial information are significantly greater than 50% for Lag 1, this difference is small in an absolute sense. The Lag 2 estimate is below chance, but the difference from 50% is not statistically significant. Additionally, the estimates are not statistically different from each other, and the overall level of guessing accuracy (across lags) is only .508. Even if the differences from chance levels are real, however, it should be noted that the amount of partial information present is small compared with final accuracies (83%), and does not increase over the course of the 500-ms difference between the first lag and the second. Indeed, the nonsignificant trend is for a slight decrease in partial information. It is possible that a small amount of partial information is available for certain easy-to-solve anagrams, or participants may be utilizing some unknown heuristic on some trials, thus producing the small level of partial information observed in the early lag. As the split one- vs. two-move analysis hints, for instance, some one-move anagrams may produce some partial information. More difficult anagrams, by this view, may be solved by other means that do not produce partial information. The early lag might therefore show a small amount of partial information due to its presence in the solution of a few easy anagrams, whereas the later lag shows no or less partial information because the anagrams that produce partial information have already been solved. Thus, some process must produce an all-or-none jump in information available to response systems for a large number of stimuli. Placing a signal later than we did in the current experiment in an effort to detect this information increase in the event it is nondiscrete would be difficult for this task, as a later signal placement would result in fewer guessing responses, and thus reduced accuracy. Further, because the present signal placement yields guessing responses within the regular-trial response distribution, it is unlikely that a later placement would tap processes not already sampled by this experiment. Even if partial information exists at a point other than those that have been sampled here, those anagrams that are solved in this range yield little or no partial information, and because a demonstration of no partial information was the goal of this experiment, that goal has been satisfied. Experiments 2 A and 2B To extend the results obtained in the previous experiment, we modified the anagram task to one requiring a high-low

1452

SMITH AND KOUNIOS five, or six letters. High-imageability words were defined as those with an MRC database composite imageability rating of greater than 500 (M = 605.93), and low-imageability words had ratings of less than 425 (M -- 308.07). Anagrams were generated from these words in much the same way as for Experiments 1A and lB. Unlike in Experiment 1A, no word was repeated. In addition to the primary stimuli, 12 practice stimuli were generated in the same manner. All stimuli could be unscrambled into

imageability judgment on anagrams, rather than a w o r d nonword judgment. We anticipated that partial information might appear when an imageability judgment was included as part of the anagram task. This involves a memory access arguably similar to those in which both partial information in semantic verification tasks (Kounios et al., 1987, 1994; Meyer et al., 1988) and accurate feelings of knowing in memory trivia tasks (Metcalfe, 1986a, 1986b) have been found. This partial information would presumably be response information accumulated only in the imageability rating portion of the task, as the anagram solving process itself has already been shown to produce little or no partial information. This portion of the task may occur entirely serially (as in Sternberg, 1969) after the anagram solution phase, or may overlap partially in a cascaded fashion (as in McClelland, 1979). Alternatively, if anagram solution is driven by tests of "candidate" words generated from some subset of the anagram's letters, as some have proposed (e.g., Dewing & Hetherington, 1974; Richardson & Johnson, 1980; White, 1988), and if people either generate words of known imageability or can test the imageability of generated words while simultaneously testing the anagram for a match to the generated words, this imageability rating of anagrams task should produce no partial information. The first of these two possible findings would serve as strong evidence for the serial linkage of two cognitive processes, the first of which does not produce partial information and the second of which doe,s.The second possible finding would strengthen the conclusion of Experiment 2B that discrete transmission, and probably transformation, of information is poss~le.

English words. Stimuli were presented using the same apparatus as

was used in the first two experiments. Procedure. Participants were tested individually, following a procedure that closely paralleled that of Experiment 1A. A practice block was followed by two experimental blocks. The practice block contained 12 items, and the experimental blocks each contained 75 items. Within a block, each participant received a unique randomization of stimulus orders.

Results and Discussion Mean RTs and accuracies are presented in Table 3. Separate 3 x 2 (Number of Letters x Imageability) repeated measures A N O V A s were performed on RTs and accuracies. There was a marginal increase in RT with number of letters, F(2, 66) = 3.328,p = .052, and a marginal effect of number of letters on accuracy, F(2, 66) = 3.014, p = .066, with five-letter words being less accurate than the others. High-imageability words produced faster RTs than low-imageability words, F(1, 33) = 149.027, p < .001, but imageability had no effect on accuracy, F(1, 33) -- 0.046,p = .831. The interaction effect was not significant for RTs, F(2, 66) = 1.366, p = .262, but was marginally significant for accuracy, F(2, 66) = 2.854, p = .066. The results suggest that a SAD study using this procedure is feasible. The RTs (4,559-7,563 ms) were longer than for either Experiment 1A (1,821-3,472 ms) or 1B (2,097-3,766 ms on regular trials), and this is cause for some concern; however, they were not so long as to be unreasonable, and we expected that these RTs would diminish with practice. Accuracies were slightly higher than in Experiment 1A, overall (.89 vs..82), so we expected no problems as a result of low accuracy.

Experiment 2A This experiment was similar to Experiment 1A, but used a high-low imageability judgment on anagrams instead of a word-nonword classification. The goals were the same as those of Experiment 1A: collection of normative data and confirmation that the task would be amenable to SAD.

Method

Experiment 2B

Panic~oants. Thirty-four Tufts University undergraduates partici-

This experiment was analogous to Experiment 1B, but used a high-low imageability judgment task similar to that of Experiment 2A.

pated in partial fulfillment of course requirements. Materials. Seventy-five high-imageability and 75 low-imageability words were selected from the MRC Psycholingnistic Database (Wilson, 1987). Selected words (a) had no other word as an anagram, (b) had no repeated letters, (c) were high in frequency in English (M = 492, based on the Ku6era & Francis, 1967, norms), (d) were high in word bigram frequency (M = 1,189, based on the Underwood & Schulz, 1960, "total count" of bigram frequency), and (e) had four,

Method Participants. Eighteen participantswere recruited by sign-up sheets around the Tufts University campus and by in-class announcements.

Table 3

Mean Reaction Times ([RTs], in MiUiseconds) and Accuracies (Proportion Correct) for High-Low Anagram ImageabilityJudgments in Ert~riment 2,4 Four letters Stimulus type

RT

Accuracy

Five letters RT

Accuracy

Six letters RT

Accuracy

Low imageability 6,612(343) 0.891 (0.023) 6,528 (344) 0.871 (0.029) 7,563 (530) 0.895 (0.035) High imageability 4,560 (251) 0.921 (0.015) 4,772 (308) 0.859 (0.015) 4,734 (279) 0.888 (0.025) M

5,586

0.906

Note. Standard errors are in parentheses.

5,650

0.865

6,148

0.892

ALL-OR-NONE PROCESSING Participants were paid $4.50 per session plus a bonus of $0.02 per correct response. Materials. Stimuli were generated from the MRC Psycholinguistic Database (Wilson, 1987), using largely the same criteria as in Experiment 2A. Four-letter words were selected for the test session, and fiveand six-letter words were used for the practice sessions. For the final-session four-letter words, word bigram frequencies ranged from 148 to 2,696, with a mean of 935.53. The target bigram frequency for the anagrams was 600; actual values ranged from 110 to 817, with a mean of 538.85. A total of 84 Iow-imageability and 84 high-imageability four-letter test stimuli were selected. An additional 82 four-letter stimuli were generated for use in the first (practice) block, and 6 four-letter stimuli were used in warm-up trials. Because of the limited number of suitable English words, some of these noncritical (warm-up and practice block) stimuli had more than one solution. In such cases, both solutions were classified in the same manner--that is, both were high-imageability or both were low-imageability words. Three of the 6 warm-up stimuli were repeated in the first block, which was, in fact, also practice, though participants were unaware of the practice nature of this block. Each participant received a unique random assignment of critical stimuli to regular trial and response-signal onsets of 1,100 and 1,300 ms, with the restriction that an equal number of regular trial, 1,100-ms, and 1,300-ms trials occurred in each critical block, as well as a unique pseudorandom ordering of stimuli. This ordering included a restriction that runs of more than four occurrences of the same imageability or signal lag would not be tolerated. 7 The additional stimuli for the first block were assigned the same lags as the critical stimuli. The signal lags for this experiment were more closely spaced than in Experiment 1B because of an observation that a number of participants in Experiment 1B produced either 100% estimated guessing responses on the first lag or fewer than 20% estimated guessing responses on the second lag, and a similar pattern was observed on pilot participants tested with the procedure from Experiment 2B. That is, the first lag was placed so early that it tapped nothing but guessing processes, whereas the second lag was late enough that few or no guessing processes went into those signal trials, for at least some participants. This is an indication that the spread between the two lags might have been causing us to drop data because of the 20% criterion for the second lag while simultaneously sampling from an extremely early point in processing for the first lag. Thus, the lags were moved closer together for this experiment. It was originally expected that longer lags would be used, but pilot data showed shorter regular-trial RTs than anticipated on the basis of Experiment 2A's results, so the lags were shortened to 1,100 and 1,300 ms. s All possible two-letter swaps of the five- and six-letter words were used as practice stimuli. These were randomly assigned responsesignal times: 10% were placed at 500 ms, and 30% each were placed at 1,000 ms, 1,500 ms, and regular trial (i.e., no response signal). The 500-ms signal lag was intended to keep participants alert, because the results of Experiment 2A led us to believe that lags would be relatively long. All participants sped up considerably during practice, so practice lags were shortened partway through the practice sessions to 500 ms and 1,000 ms instead of 1,000 ms and 1,500 ms, with the original short 500-ms lag moved to 350 ms. This speedup presumably occurred because participants saw the same stimuli multiple times over the course of practice and so began using memory, rather than normal anagram-solving strategies. As this would not help participants on the final session, this session utilized longer lags than those used in the latter stages of practice. The change from problem-solving to recognition strategies during the practice sessions is largely unimportant, as the main purpose of practice was to familiarize participants with the SAD response deadline procedure itself; the task requirements would remain the same despite any changes in processing strategy from

1453

practice to final session. Stimuli were presented on the same apparatus as was used in the first three experiments. Procedure. The procedure paralleled that of Experiment lB. Participants practiced using the five- and six-letter words and the SAD technique. Two response mappings were used, counterbalanced across subjects: They used either the leftmost or rightmost button on the response box to indicate a high-imageability word, and the opposite button was used to indicate a low-imageability word. The first session lacked response signals; participants could take up to 30 s to respond to stimuli in these trials, making this session equivalent to Experiment 2A, except for the lack of four-letter anagrams. Participants were given feedback on overall accuracies and RTs after each session and engaged in an average of 8.5 practice sessions (range: 7-12) before engaging in the final session. This session used the same procedure as the practice sessions, except that four-letter stimuli were used exclusively, and the signal lags were different (see above).

Results Criteria for inclusion in the results were similar to those for E x p e r i m e n t 1B a n d were selected for similar reasons: fewer t h a n 12 ( 1 1 % ) late r e s p o n s e s o n signal trials, defined as r e s p o n s e s longer t h a n 425 ms after t h e o n s e t of t h e r e s p o n s e signal, a n d t h e e s t i m a t e d p e r c e n t a g e of guessing r e s p o n s e s 7 This restriction was one step taken in an effort to reduce a suspected predictability in stimulus ordering, as mentioned in the discussion of the tests of the race model for Experiment 1B, and as observed in pilot work with this experiment. Although placing a restriction on legitimate random runs might seem unjustified, the reasoning behind this decision was that long runs would be particularly salient to participants, who would then bias their responses because of a run. The second measure used to reduce the suspected predictability was to change the random number generator used in creating random orderings. In Experiments 1A, 1B, and 2A, the randomization routine built into the Micro Experimental Laboratory software package (Schneider, 1990) was used; but for Experiment 2B, a random number generator described by Wichmann and Hill (1987) was used instead. Neither of these measures was applied to the practice sessions; only in the final data collection session was this different randomization procedure applied. s Although these lags were shorter than the longest lag from Experiment 1B, we would still expect partial information, if present, to be detectable by SAD at these lags, for two reasons. First, the lags for Experiment 1B were placed in an attempt to overlap both the word and the nonword responses, but the nonword response distribution was later than the word response distribution; in both Experiment 2A and pilot work for this experiment, the split between high- and iow-imageability words was not so great, thus allowing earlier signal placements. Second, if this task were done through a serial concatenation of anagram solution and imagery judgment, partial information would begin to accrue at roughly the same point in time relative to a normal-process response, which translates to different points in time relative to stimulus onset. As Experiments 1A and 1B both showed substantial variability in normal-process completion times, the beginnings of partial information growth, considered across all trials, would begin quite early and would continue, statistically speaking, across a broad range of time. Put another way, for all SAD studies the important point in evaluating signal placement is the point where estimated guessing-process responses intersect the regular-trial RT distribution; if this "outpoint" is within the body of the regular-trial distribution (as opposed to before it or very late in it), SAD should be able to detect any existing partial information. Our signal placements result in cuts well within the regular-trial distribution.

1454

SMITH AND KOUNIOS Table 4

Mean Reaction Times ([RTs], in Milliseconds) and Accuracies (Proportion Correct) for High-Low Anagram ImageabilityJudgments in ~ n t 2B Regular trials Stimulus type Low imageability High imageability

RT

Accuracy

Lag 1a RT

Accuracy

3,827 (639) 0.78 (0.053) 1,248 (18) 0.71 (0.038) 2,729 (316) 0.83 (0.021) 1,229 (14) 0.62 (0.043)

M 3,278 Estimated PI proportion correct Estimated PI d ' Estimated PI log 13

0.81

1,238

0.66 0.548 (0.071) 0.335 (1.750) 0.181 (0.820)

Lag 2a RT

Accuracy

1,363 (37) 0.71 (0.044) 1,306 (31) 0.64 (0.018) 1,335

0.67 0.496 (0.057) 0.044 (0.816) 0.101 (0.680)

Note. Standard errors are in parentheses. PI = partial information. PI cannot be estimated for regular trials in isolation. aThese columns represent all responses for each signal lag condition, including both normal and guessing processes.

among signal trials of at least 20%. Of the 18 participants, 7 failed to meet these criteria, leaving 11 for the final analysis. Because a unique matching of stimuli to signal lags was used for each individual rather than three counterbalanced sets as in Experiment 1B, no participants were dropped to make counterbalancing even. 9 RTs and accuracies (and associated standard errors) are presented in Table 4. Separate 3 x 2 (Lag x Imageability) repeated measures ANOVAs were performed for both RTs and accuracies. RTs and accuracies both increased with lag (considering regular trials to be a higher lag than either signal time), F(2, 20) = 20.135,p = .001, and F(2, 20) = 10.449,p = .003, respectively. RTs were higher for low- than for highimageability words, F(1, 10) = 7.531,p = .021, but accuracy did not vary significantly with imageability, F(1, 10) = 0.548, p = .476. The interaction effect was significant for RTs, F(2, 20) = 6.856,p = .026, apparently due to a greater increase in RT for low- than for high-imageability words with the regular trials. The interaction effect for accuracy was also significant, F(2, 20) = 5.932,p = .012, reflecting greater accuracy for high-imageability words in the regular trials, but a reversal of this for both signal conditions. Even on regnlar-trial responses, participants in Experiment 2B took substantially less time to respond than did participants in Experiment 2A (an average of 3,278 ms in the regular trials of Experiment 2B vs. 5,585 ms in the four-letter trials of Experiment 2A). There are quite a few variables that may account for this difference, including more practice by participants in Experiment 2B, different stimuli used in the two experiments, and any time stress perceived by participants as a result of the response-signal trials in Experiment 2B. Estimated guessing accuracies, expressed as both proportion correct and logistic d', are also presented in Table 4. As in Experiment 1B, there is no significant change in estimated guessing accuracy from the early (1,100 ms, PC = .548, d' = 0.335) to the late (1,300 ms, PC ffi .496, d' ffi 0.044) response signal, t(10) = 0.547, p = .596 and tOO ) = 0.478, p = .643 for PC and d', respectively, both two-tailed. Unlike in Experiment 1B, the absolute levels of partial information were not significantly different from 50% for either signal lag, Lag 1: t(10) = 0.676, p = .257 for PC, t(10) - 0.636, p -- .270 for d', both

one-tailed; Lag 2: t(10) = - 0 . 0 6 8 , p = .947 for PC, two-tailed, tOO ) = 0.177,p = .432 for d', one-tailed. 1° The log 13estimates did not vary significantly from one lag to the next, t(10) = 0.246, p = .811, two-tailed, indicating no change in guessing bias between lags. A positive log 13 value indicates a bias to respond "low-imageability" when guessing, but neither the Lag 1 nor the Lag 2 log 15value was significantly different from 0, t(10) = 0.734,p = .480, and t(10) = 0.492,p = .634, respectively, two-tailed. Average estimated probabilities of guesses finishing before regular processes were .620 and .543 for low-imagery words in Lags 1 and 2, respectively, and .525 and .473 for high-imagery words in Lags 1 and 2, respectively. These represent placement of the response signals so that guesses fell within the regulartn'ai RT distributions. Overall, these numbers indicate that the estimates of guessing accuracy are based on approximately 666 guesses, and the fact that approximately 49% of the late signal-trial responses were based on normal-process completions indicates that this signal was late enough with respect to the regular-trial response-time distribution that it should have been possible to detect any partial information had it been present. Tests of the PSG model. Figure 2 shows the averaged estimated guessing CDFs. The estimated guessing times for the two stimulus types cluster tightly around each other, with medians approximately 300 ms after the response signals (1,100 and 1,300 ms; medians range from 275 ms to 300 ms after these response signals). In absolute terms, low-imageability guessing-time medians are 12 ms faster than highimageability medians, most of this difference coming in the late lag. The late lag's medians are 12 ms faster than the early lag's medians, most of this difference coming from low-imageability stimuli. Except for some minor deviations at the early tail of 9 Preliminary analyses showed no effects of counterbalanced response mapping, and so this was left unbalanced at five and four. 10As in Experiment 1B, a one-tailed test was used for theoretical reasons for Lag 1, but the numerically below-chance accuracy for Lag 2 when using the PC measure made a two-tailed test a practical necessity. The comparison of the two conditions required a two-tailed test for theoretical reasons.

ALL-OR-NONE PROCESSING 1 u~ ¢n t-- 0.75

Lag 2 low imageabiliW._

8. ~

0.5

Lag 2 high imageability

w.-

.g 0.25 Lag 1 low i m a g e ~ oQ.

2

o

o.

•

-

0

,

.

500

2

5

1000

•

~

. .

1500

2000

Time (ms) Figure 2. Averaged cumulative distribution functions for estimated

1455

0.449). None of these values was significantly different from chance (allps > .135 using one-tailed t tests on both PC and d' measures); the Lag 1 and Lag 2 values were not different from each other (both ps > .674 using two-tailed t tests). For the two-move anagrams, the Lag 1 mean estimated guessing accuracy was .496, SE = .069 (d' = -0.014, SE - 0.554) and the Lag 2 accuracy was .497, SE -- .058 (d' -- -0.291, SE = 0.455). Two-tailed t tests on both PC and d' measures showed no significant differences from chance accuracy (alips > .535) or from each other (bothps > .706). PSG model tests showed no significant results for the one-move condition, aside from the necessary effect of time (all ps > .422). For the two-move condition, the effect of stimulus type was significant, F( 1, 10) = 9,662,p = .011, but no other effect aside from the main effect of time approached significance (all ps > .484). There is thus no suggestion of differences between one- and two-move anagrams in this experiment.

guessing-time distributions from Experiment 2B. Discussion

the CDFs, these curves appear to be quite similar to one another. A 2 x 2 x 6 (Stimulus Type x Signal Lag x Time) M A N O V A was performed on these data using proportion of responses by specified times as the dependent variables, first subtracting the sigual-lag time from each distribution. Times used ranged from 200 ms to 400 ms, every 40 ms. The results showed significant main effects of stimulus type, F(1, 10) = 5.518,p = .041, and of time, F(5, 6) = 52.918,p < .001. The main effect of lag was not significant, F(1, 10) = 0.092, p = .768. No interaction effects reached significance (.759 > p > .104). The PSG model test for Experiment 213 is somewhat cleaner than the test in Experiment lB. The main effect of time is expected and necessary. The effect of stimulus type confirms the observation that low-imageability guessing times were slightly shorter than were high-imageability guessing times. The magnitude of this effect, however, when measured in terms of median guessing times, is small (12 ms). Additionally, as mentioned in reference to Experiment 1B, simulations reported by R. W. Smith et al. (1996; and summarized in Appendix B) suggest that main effects of stimulus type produce only very small changes in estimated guessing accuracy. Thus, it seems unlikely that this effect would produce a substantial bias in estimated guessing accuracies. Analysis by number o f moves. As in Experiment 1B, it is conceivable that one-move and two-move anagrams could be solved by different mechanisms. The regular-trial RTs in this experiment, however, show no main effects or interactions involving number of moves when submitted to a 2 x 2 (Number of Moves x Imageability) A N O V A (bothps > .491). The main effect of imageability was significant, F(1, 10) = 8.994, M S E = 730,687, p = .013. Nonetheless, the data were analyzed separately for one-move and two-move anagrams. This resulted in the loss of 4 participants in the one-move condition as a result of the 20% cut criterion. No data were lost in the two-move condition. For the one-move condition, Lag 1 estimated guessing accuracy was .651, SE = .143 (d' = 0.710, SE = 1.033) and Lag 2 was .579, SE = .075 (d' = 0.543, SE =

Experiment 2B provided no evidence for partial information in an anagram plus imageability judgment task. Although the estimated guessing accuracy of Lag 1 was slightly (but not significantly) above chance, Lag 2's estimated guessing accuracy was minutely (and also not significantly) below 50%. Even Lag l's levels of guessing accuracy are fairly uninspiring, representing, at most, an extremely low level of partial information. In a split one- versus two-move analysis, the one-move anagrams produced partial information levels that were nonsignificantly above chance, whereas the two-move anagrams produced minutely (and also nonsignificantly) negative guessing accuracies. These results, of course, do not prove that partial information does not exist in this task, but it seems likely that any partial information is minuscule or results from just a handful of stimuli. It is interesting that there was no increase in partial information from Lag 1 to Lag 2. As in Experiment 1B, this suggests that some later process produces a massive increase in available information at a later point in time in a discrete jump. On a methodological note, the large number of participants whose data were not included in final analysis for both Experiments 1B and 2B may seem unsettling at first. This does not seem to be a serious problem, however, for several reasons. The main reason for noninclusion is that responses did not occur within the expected time f r a m e k i n other words, the signals were placed incorrectly for specific individuals, given that some took more or less time to respond than others on regular trials. Specifically, in Experiment 1B, 5 participants' data were rejected for this reason, whereas in Experiment 2B, 3 participants' data were thus eliminated. The individuals whose data were not included therefore represent those with faster average RTs. In Experiment 2B, for instance, the 3 participants excluded because of a low proportion of estimated guessing process completions on signal trials had average regular-trial RTs of 1,368 ms, with accuracies of .79, compared with 3,278 ms and .81 accuracy for the included participants. Although it is conceivable that individuals who respond more quickly would use different methods of solution than those who respond more slowly, this would require an elaborate

1456

SMITH AND KOUNIOS

theory of individual differences in anagram solution. Even such a theory would have to encompass the results from these experiments for at least some participants, however, so the results from the present study remain valid: Partial information does not always exist. If an individual-differences explanation is not utilized, then the systematic removal of participants by their regular-trial RTs, as the 20% cut criterion effectively is, does not represent a threat to the validity of the results, as the faster individuals would be expected to show a pattern of results similar to the slower ones'. The broader goal of constraining theories of information processing based on the presence of partial information in some tasks (e.g., Kounios et al., 1987, 1994; Meyer et al., 1988; Rateliff, 1988) and its absence in others (the results reported here) is met whether or not the discarded data would show evidence for partial information. C o m b i n e d Analyses Because drawing a conclusion from a null result is difficult (see Frick, 1995, for a discussion of this point), it is important that we have as much statistical power as possible. It should be emphasized that the current SAD studies, although using relatively few participants by usual RT research standards, utilized more participants than in any previous SAD study, which have used as few as 3 participants (e.g., Kounios et al., 1987, Experiment 3; Ratcliff, 1988). In some ways, SAD studies resemble psyehophysical experiments in their collection of large amounts of data from few individuals. All previous SAD studies have found evidence of partial information, even with relatively small absolute levels (e.g., Ratcliff, 1988). The present experiments also utilized massive amounts of data collection from individual participants, but have increased the number of participants. Nonetheless, it is important to bring as much statistical power to bear as possible on the present data. As the preceding results showed little evidence for either partial information or differences in guessing accuracy levels across lags, we have combined the results from both studies to increase the statistical power of the tests. Although combining data across experiments is not always justified, the almost total lack of effects in within-experiment results is encouraging in this respect. In addition, if the hypothesis that the imageryrating task involved the use of high- or low-imagery words as cues to anagram solution is correct, as the results suggest, then the task differences between the two experiments would in fact be very minor, further justifying a combined analysis. The estimated guessing accuracies from Experiments 1B and 2B (see Tables 2 and 4) were submitted to a 2 x 2 (Experiment x Lag) ANOVA. The results showed no significant effects for either experiment, F(1, 21) = 0.073,p = .790, or lag, F(1, 21) = 1.623, p -- .217. The interaction effect was also nonsignificant, F(1, 21) = 0.091, p = .765. A similar A_NOVA using d' rather than percent correct revealed an identical pattern of results. This analysis suggests that there are no differences in any of the four conditions (two experiments by two lags), and so a further collapsing of data is justified. The data from each participant's two lags were averaged together and a confidence interval was computed around the

resulting grand mean. The overall level of guessing accuracy was .515 (d' = 0.160), with a 95% confidence interval of .464-.566 (-0.230-4).550 in d'). This compares with cross-lag confidence intervals of .449-.567 (-0.454--0.772 in d') for Experiment 1B and .425-.619 (-0.426--0.806 in d') for Experiment 2B. We can therefore be quite certain that there is very little or no partial information in the present tasks. 11 G e n e r a l Discussion Considered broadly, the purpose of the preceding experiments was to demonstrate discrete processing (very probably taking the form of discrete transmission of information), a phenomenon that has only been hinted at in the past. This would provide important constraints on higher level cognitive theories--namely, that such theories must be able to account for both discrete (because of these findings) and continuous (because of previous research) processing. Ideally, findings of no partial information in Experiment 1B and its presence in Experiment 2B would have provided an even more specific constraint in that both types of processing would have occurred in superficially very similar tasks. The fact that we did not find evidence for partial information in Experiment 2B is somewhat disappointing in this sense, but reassuring in another, as this serves as a replication of the no partial information finding using SAD, which would otherwise be entirely unique to Experiment lB. Thus, we can be more certain than we had been before that we have found a task that produces, very little or no partial information. Previous SAD studies (e.g., Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988) can still serve the function of providing evidence for the presence of partial information. Thus, there is evidence for both the presence and absence of partial response information using the same method of measurement, though in disparate tasks. It seems that the imagery task involved a word-driven anagram solution process (e.g., Dewing & Hetherington, 1974; Richardson & Johnson, 1980; White, 1988) using words of known imageability, or with parallel ongoing imageability judgment and anagram solution. Thus, with the anagram solution itself producing no partial information, neither does the combined solution-plus-judgment task. Another way of looking at this possibility is that people go through a generateand-test cycle for solving anagrams, and the generate portion of this cycle is driven by an imagery-based process, so that when the solution is found, the imagery classification is already known.

11Further, a power analysis on the combined tests shows that the combined analyses have substantial power: For PC, the power was .85, using a one-tailed test. Computing the same thing using d', the power was .99. Power for the individual experiments was .28 for PC and .46 for d' in Experiment 1B and .44 for PC and .62 for d' in Experiment 2B. As Frick (1995) argued at length, however, power analyses are of limited value in situations such as this, as it is always possible to argue that an experiment with more participants would detect a smaller effect, and computed power increases as the p value decreases. We therefore favor the confidence interval as being more straightforward and easier to interpret.

ALL-OR-NONE PROCESSING

H _

=

• Meyer et al. (1988), Experiment 3 Meyer et al. (1988), Experiment 5 -- Kounios et al. (1994), Experiment 1

= Kounios et al. (1994), Experiment 2

D ~ Q Experiment l b o ~ o Experiment 2b

2.5 2.25

¢1.75 ==

\

\ " 0

..........

10

on the word RTs in Experiment 1B; see Table 2) can be used to give an estimate of the imageability judgment RT, when combined with the mean regular trial RTs in Experiment 2B (3,278 ms; see Table 4), and the result is an estimated time of 1,179 ms. Although this comparison is far from perfect (different stimuli were used, with different participants, and so on), it does at least suggest that partial information in a serial model should accumulate over a relatively broad time range, and thus represent a very large moving target, which SAD should be able to detect, if it were present.

A Comparison With Previous SAD Results

1.25 /,,2.

0

1457

20

30

, ; i

40

50

........

60

% Regular Trials < Guessing Median Figure 3. Growth of partial information in Experiments 1B and 2B and in previous speed-accuracy decomposition studies. The x axis shows a measure of estimated guessing responses relative to regulartrial responses, and so is independent of the temporal differences between experiments.

Alternative explanations do exist, however. One possibility is simply that imageability judgment does not produce partial information, unlike other phenomena studied with SAD (Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988). This seems unlikely, however, as judging imageability would seem to involve a memory access similar to that used in these previous experiments. A second alternative is that the time scales of the two processes are too dissimilar. That is, if anagram solution takes a long time, with a relatively wide variance (as it seems to, given the results of Experiments 1A and 1B), and if imageability judgment occurs relatively rapidly, but still not in an all-or-none fashion, it will nonetheless be difficult to detect partial information from the imagery judgment task if it occurs after the anagram solution. In principle, such partial information should be detectable, because there will be response signals placed in such a way as to tap the imagery task's (hypothetical) growth in partial information; but if the point at which the anagram task ends and the imageability task begins (and, thus, the point at which partial information begins to grow) is relatively variable, and if growth in partial information is fairly rapid, response signals that tap this growth will be relatively rare, and estimates of partial information will be deflated. Loosely speaking, by this view, the use of the anagram task may have made any partial information in the postanagram portion of the task a "moving target" for SAD, and so it was largely missed. This seems unlikely, however, as by this model, the time for anagram solution (2,099 ms, based

To obtain some sense of the unusual nature of these results, it is necessary to compare them with results from previous SAD studies. To date, four such studies have been published, and a brief summary of the tasks and results is provided below. Meyer et al. (1988) applied SAD to two word recognition tasks in five experiments. One task was word-nonword discrimination. In the second task, participants compared two letter strings to determine whether they were the same or different in lexicality (i.e., both words or both nonwords). Estimated guessing accuracies ranged from approximately 50% for the earliest lags to over 80% for some late conditions, indicating considerable sensitivity of SAD to partial information when it is present. Ratcliff (1988) used a study-test recognition memory paradigm, applying SAD to the recognition phase of the study-test paradigm. The resulting d' values ranged from 0.358 to 0.771 (equivalent percents correct ranged from about 55% to 65%). Regular-trial accuracies ranged from 68% to 78%. Note that these regular-trial accuracies are very low compared with those obtained in other SAD experiments, including those reported here, thus accounting for some of the relatively low partial information levels obtained; but Ratcliff's results were, nonetheless, statistically significant for some lags. Kounios et al. (1987) performed three semantic memory experiments involving sentence verification (e.g., All dogs are animals). Estimated guessing accuracies were presented in d ' units, and ranged from 0.26 to 2.69 across lags and experiments. Finally, Kounios et al. (1994) used ownership relations (e.g., verification of sentences such as many people own galaxies) in two SAD experiments. Derived guessing accuracies (expressed as d ' ) ranged from 0.54 to 2.88 (approximately 60-86%). Figure 3 presents a comparison of the present results with those of Meyer et al. (1988, Experiments 3 and 5) and Kounios et al. (1994). 12This figure presents the obtained d ' values over a measure of time. The time measure used is the percentage of regular-trial responses occurring prior to the guessing median. This is effectively a measure of signal placement relative to the normal-process response distribution, and therefore automatically corrects for the different temporal scales of the various experiments. Increasing values indicate increasing normalprocess completions independent of absolute time. 12Data for the cutpoints used as the x axis in Figure 3 were not available for Rateliff (1988) or Kounios et al. (1987), making inclusion of these data in the figure impossible.

1458

SMITH AND KOUNIOS

The figure dearly shows that the levels of partial information present in the current experiments are far below the levels obtained in previous experiments, although occurring later in processing than in some other experiments. This figure also shows three different overall patterns of partial information growth: (a) continuous growth from the beginning of the regular-trial RT distribution or before (Kounios et al., 1994, both experiments [complex semantic relations]; Meyer et al., 1988, Experiment 5 [lexical decision]); (b) initial growth in partial information followed by a plateau at an intermediate level (Meyer et al., 1988, Experiment 3 [dual-string lexical decision]), presumably followed by a late discrete jump to final accuracy levels; and (c) constant low levels of partial information, indicating a probable single discrete jump from zero to final accuracy (Experiments 1B and 2B). Considered together, these three patterns indicate considerable variability in the information processing strategies used in human cognition and challenge information processing theorists to develop models that can account for all three patterns.

Views of the Mind as Both Discrete and Continuous Given that partial information has been found in other experiments, the question remains as to how the mind can produce partial information in some cases and not produce it in others. There are a number of possible explanations. One is simply that memory access is fundamentally different from problem solving. By this view, memory access may involve processes that build information from pieces, each of which contributes partial information. In contrast, problem solving, or at least anagram problem solving, may involve a search through a problem space, and this search does not yield a growth in actual knowledge of the solution until it is complete. This hypothesis predicts the possibility of a dissociation between partial information revealed by SAD and metacognitive judgments of the likelihood of reaching a solution in some tasks, as metacognitive knowledge of progress may not correlate with usable partial information (see below). This explanation of the findings of partial information and no partial information in different tasks, however, has the disadvantage of lacking guiding principles. That is, the mechanisms that account for the levels of partial information in different tasks are dependent on the specific task and would presumably vary, possibly in unpredictable ways, in different tasks. Thus, the predictive value of this view is limited. Another possibility is that the time involved is a critical issue. The anagram problems studied here took longer to solve than did the problems in previous SAD studies, and so may involve fundamentally different mechanisms. As Rumelhart, Smolensky, McClelland, and Hinton (1986) suggested, for instance, individual PDP networks can be linked together in a serial fashion, and it is conceivable that such linkages might produce continuous transformations of information within a network and discrete transmission of information between networks. Thus, microlevel cognitions--those processes that occur relatively rapidly--might be expected to produce partial information in SAD studies. Cognitive processes that take longer would be more likely to produce a lack of partial

information in a SAD study. In some sense, however, this explanation is little more than a rehashing of box model theories, which use (frequently poorly explained) information processing "boxes" linked together in various ways. This linked PDP model does, however, at least give a more coherent description of both the micro- and macrolevels of cognition, in that the workings of individual PDP networks are typically described in detail, and the linkages between networks could be, at least in principle, explicated in detail. Whatever the explanation, broader accounts of cognition, such as Anderson's ( 1983) ACT* (Adaptive Control of Thought ), or the Soar model de.sen'bed by Newell (1990a, 1990b), require some means of accounting for both types of processing.

SAD, Insight, and Metacognition Previous research using metacognitive judgments of insight and other problem-solving processes (e.g., Metcalfe, 1986a, 1986b) has typically found that insight and insight-like problems, including anagrams, are not readily accessible to metacognitive processes. Judgments of feeling of warmth in such studies tend to remain low up until the moment of solution, or as near to that as could be measured with the 10- or 15-s judgment intervals used. Similar judgments in other tasks, however, tend to produce slowly increasing warmth ratings. Although such ratings may not always indicate direct access to actual cognitive processes, as Miner and Reder (1994) argued at length, they do seem to represent, at the very least, some form of access to cognitive processing, even if it is, as Miner and Reder suggested, merely an estimate of the likelihood of future retrieval or solution based on cue familiarity or some other heuristic. The present research ties into this area in that metacognitive judgments and partial information estimates seem to be in agreement with one another. Specifically, in memory retrieval tasks, metacognitive judgments (e.g., Metcalfe, 1986a) and SAD (e.g., Kounios et al., 1987, 1994; Meyer et al., 1988; Ratcliff, 1988) produce evidence that individuals are aware of the retrievability of a solution (in metacognition) and have partial information about that solution (in SAD). In anagram problem solving, however, metacognitive judgments (Metcalfe, 1986b) draw a large blank, and there is little or no evidence for partial information (Experiments 1B and 2B). Although it is far too early to draw any strong conclusions from this, it does at least appear to be possible that positive metacognitive knowledge and partial information may be related in some way. This relationship need not be very direct; it could, for instance, be that metacognitive judgments and partial response information are both drawn from similar evaluative information. In other words, both might result from what may be called a very deep structure of cognition--a sort of guidance system for directing many cognitive activities by the use of incomplete information. This very deep structure might be able to guide cognitive processes in a relatively directed manner in some cases, producing both partial information and accurate rectacognitions, whereas in other cases it would essentially wander aimlessly, producing neither partial information nor accurate metacognitions. These ideas are admittedly very speculative,

ALL-OR-NONE PROCESSING and it is unclear at this point how they might be tested, but they are intriguing.

Conclusion The present research has found evidence for discrete transmission of information in an insight-like task. This may very well be the strongest evidence for all-or-none processing to date and, as such, has considerable importance to theories of information processing. More research is necessary, however, to extend these findings to cognitive domains other than anagram problem solving. Ideally, it should eventually be possible to formulate a theory of information processing that accurately predicts which tasks will produce partial information and which tasks will not.

References Anderson, J. R. (1983). The architectureof cognition. Cambridge, MA: Harvard University Press. Beilin, H., & Horn, R. (1962). Transition probability effects in anagram problem solving. Journal of Experimental Psychology, 63, 514-518. Bowers, K. S., Regehr, G., Balthazard, C., & Parker, K. (1990). Intuition in the context of discovery. Cognitive Psychology, 22, 72-110. DeJong, R. (1991). Partial information or facilitation? Different interpretations of results from-speed-accuracy decomposition. Perception & Psychophysics, 50, 333-350. Dewing, IC, & Hetbering,ton, P. (1974). Anagram solving as a function of word imagery. Journal of E,werimentai Psychology, 102, 764-767. Duncker, IC (1945). On problem-solving. Westport, CT: Greenwood Press. Durkin, H. E. (1937). Trial-and-error, gradual analysis, and sudden reorganization: An experimental study of problem solving. Archives ofPsychology,210, 2-85. Frick, R. W. (1995). Accepting the null hypothesis. Memory & Cognition,23, 132-138. Gick, M. L, & Holyoak, K. J. (1980). Analogical problem solving. CognitivePsychology,12, 306-355. Hunter, I. M. L. (1959).The solvingof five-letteranagram problems. British Journal of Psychology, 50, 193-206. Irwin, D. E., & Meyer, D. E. (1995). Sophisticated guessing vs. pure guessing in speed-accuracy decomposition." Comment on De Jong (1991). Unpublished manuscript. Jameson, IC A,, Narens, L., Goldfarb, IC, & Nelson, T. O. (1990). The influence of near-threshold priming on metamemory and recall. Acta Psychologica, 73, 55--68. K6hler, W. (1947). Gestaltpsychology: An introduction to new concepts in modernpsychology. New York: Liveright. Kaunios, J., Montgomery, E. C., & Smith, R. W. (1994). Semantic memory and the granularity of semantic relations: Evidence from speed-accuracy decomposition. Memory & Cognition, 22, 729-740. Kounios, J., Osman, A. M., & Meyer, D. E. (1987). Structure and process in semantic memory. New evidence based on speed-accuracy decomposition. Journal of Experimental Psychology: Genera~ 116, 3-25. Ku~ra, H., & Francis, W. (1967). Computational analysis ofpresentdayAmehcan English. Providence, RI: Brown University Press.

1459

Mayzner, M. S., & Tresselt, M, E. (1958). Anagram solution times: A function of letter order and word frequency. Journal of Experimental Psychology, 56, 376-379. Mayzner, M. S., & Tresselt, M. E. (1959). Anagram solution times: A function of transition probabilities. The Journal of PsychologF, 47, 117-125. McClelland, J. L. (1979). On the time relations of mental processes: An examination of systems of processes in cascade. Psychological Review, 86, 287-330. Mendelsohn, G. A., & O'Brien, A. T. (1974). The solution of anagrams: A reexamination of the effects of transition letter probabilities, letter moves, and word frequency on anagram difficulty. Memory & Cognition, 2, 566-574. Metcalfe, J. (1986a). Feeling of knowing in memory and problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12, 288-294. Metcalfe, J. (1986b). Premonitions of insight predict impending error. Journal of ExperimentalPsychology:Learning, Memory, and Cognition, 12, 623-634. Metcalfe, J., & Wiebe, D. (1987). Intuition in insight and noninsight problem solving. Memory & Cognition,, 15, 238-246. Meyer, D. E., Irwin, D. E., Osman, A. M., & Kounios, J. (1988). The dynamics of cognition and action: Mental processes inferred from speed-accuracy decomposition. PsychologicalReview, 95, 183-237. Miller, J. (1988). Discrete and continuous models of human information processing: Theoretical distinctions and empirical results. Acta Psychologica, 67, 191-257. Miner, A. C., & Reder, L. M. (1994). A new look at feeling of knowing: Its metacognitive role in regulating question answering. In J. Metcalfe & A. P. Shimamura (Eds.), Metacognition (pp. 47-70). Cambridge, MA: MIT Press. Neweli, A. (1990a). Metaphors for mind, theories of mind: Should the humanities mind7 In-J. Sheehan & M. Sosna (Eds.), Boundaries of humanity: Humans, animals, and machines (pp. 158-197). Berkeley, CA: University of California Press. Newell, A. (1990b). Unified theories of cognition. Cambridge, MA: Harvard University Press. OIIman, R. T., & Billington, M. J. (1972). The deadline model for simple reaction times. CognitivePsychology,, 3, 311-336. Pachella, R. G. (1974). The interpretation of RT in information processing research. In B. Kantowitz (Ed.), Human information processing: Tutorials in performance and cognition (pp. 41-82). New York: Halstead Press. Ratcliff, R. (1979). Group RT distributions and an analysis of distribution statistics. PsychologiculBulletin, 86, 446--461. Ratcliff, R. (1988). Continuous versus discrete information processing: Modeling accumulation of partial information. PsychologicalReview, 95, 238--255. Richardson, J. T. E., & Johnson, P. B. (1980). Models of anagram solution. Bulletin of the Psychonomic Society, 16, 247-250. Rumelhart, D. E., Smolensky, P., MeClelland, J. L., & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Volume 2: Psychological and biological models (pp. 7-57). Cambridge, MA: MIT Press. Schneider, W. (1990). Micro Experimental Laboratory [Computer program]. Pittsburgh, PA: Psychology Software Tools. Schooler, J. W., Ohlsson, S., & Brooks, IC (1993). Thoughts beyond words: When language overshadows insight. Journal of F2q~rimental Psychology: General, 122, 166-183. Schwartz, B. L, & Metcalfe, J. (1992). Cue familiarity but not target retrievability enhances feeling-of-knowing judgments. Journal of

1460

SMITH AND KOUNIOS

~ntal

Psychoiogy: Learning, Memory, and Cognition, 18, 1074-1083. Smith, J. E. K. (1974). Some siraplified indices of sensitivity and bias (Teeh. Rep. No. MMPP 74-5). Ann Arbor: University of Michigan, Mathematical Psychology Program. Smith, R. W., Kounios, J., & Osterhout, L. (1996). The robustness and of ~d--ata:umy ~ a technique for investigating the time-course of information processing. Manuscript submitted for publication. Sternberg, S. (1969). The discovery of processing stages: Extensions of Donders' method.Acta Psychologica, 30, 276-315. Thomas, E. A. C., & Ross, B. H. (1980). On appropriate procedures for combining probability distributions within the same family. Journal of Mathematical Psychology, 21, 136-152.

Underwood, B. J., & Schulz, R. W. (1960). Meaningfulness and verbal learning. Chicago: Lippincott. White, H. (1988). Semantic priming of anagram solutions. American Journal of Psychology, 101, 383-399. Wichmann, B., & Hill, D. (1987). Building a random-number generator: A Pascal routine for very-long-cycle random-number sequences. Byte, 12, 127-128. Wickelgren, W. A. (1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67-85. Wilson, M. (1987). MRC psycholingnistic database: Machine usable dictionary, Version 2.00 [Machine-readable data file]. Chilton, England: Rutherford Appleton Laboratory, Informatics Division (Producer). Oxford, England: Oxford Text Archive, Oxford University Computing Service, [email protected], (Distributor).

Appendix A The Mathematics of SAD

The following is a brief discussion of the mathematics underlying SAD. For more details, see Meyer et al. (1988) or R. W. Smith et al. (1996).

The CDF for guessing times can be given by

P(ts <_ C) = l - P(tg > C) = 1 - [e(ts > C)/P(tn > C)].

The PSG Model During signal trials, a "race" is presumed to occur between normal and guessing processes. The normal processes, assumed to be those that would occur in a corresponding conventional RT task, begin when a stimulus is detected and end when a response is made on the basis of essentially complete processing of the stimulus. Guessing processes begin with the detection of a response signal and end with a fast "best guess" response based on whatever partial information and response bias exist when the signal is detected. These processes will overlap temporally on at least some signal trials. Whichever process finishes first determines the response on a signal trial. They are assumed to be temporally independent in that neither process should interfere with the operation of the other. Because the guessing processes begin with the detection of a response signal, they arc absent on regular trials, which are handled entirely by normal processes. The normal processes, however, occur on both types of trials, although they may or may not produce the response on any given signal trial, depending on whether the normal process or the guessing process wins the "race." The fact that participants may respond on signal trials on the basis of either completed normal processing or a guess means that the distribution of signal-trial RTs is a mixture of normal and guessing processes. SAD is a technique for determining the RT distributions and accuracies of a participant's guesses based on responses to the randomly interleaved regular and signal trials.

Estimating the Distribution of Guessing Times Guessing time is defined to be the time from the onset of the test stimulus until the participant's guessing response. The PSG model allows guessing times to be estimated by using the distribution of RTs on regular trials to remove normal processing contributions to the signal-trial RT distribution, leaving an estimate of the guessing-time distribution.

(A1)

In this and all subsequent equations, ts represents the time for a guessing-process response, tn represents the time for a normal-process response (and can be estimated by the time for a regular-trial response), and ts is the time for a signal-trial response. This equation represents the probability that a guessing response on a signal trial with a given lag will have a time less than or equal to some constant C. Each term on the right side of the equation can be estimated from either the observed distribution of RTs on signal trials (Pits > C]) or from the observed distribution of RTs on regular trials (P[tn > C]). The temporal-independence assumption of the PSG model can be checked by examining the CDFs produced by Equation A1. If the guessing-time CDFs obtained for different signal lags take on different shapes or have different latencies from their respective response signals, then the temporal-independence assumption is suspect. (Fuller derivations of Equation A1 can be found in Meyer et al., 1988, and R. W. Smith et al., 1996.) Estimating the Accuracy of the Guessing Processes The guessing accuracy is usually of more theoretical interest than the guessing-time distribution. According to the PSG model, it can be titrated out of the signal-trial accuracy for any given response signal lag (P[correct]) because this is a mixture of the accuracies derived from the normal (P[correctltn -~ tg]) and guessing (P[correctlts < tn]) processes. This value can be obtained by solving, for each response signal lag,

P(correctlts < tn) =

e(correct) - P(tn < t~)P(correctltn ~ tg). P(tg < tn)

(A2)

All of the terms on the right side of the equation can be estimated from empirical data, either directly (e.g., P [correct]) or by utilizing the

1461

ALL-OR-NONE PROCESSING estimated guessing CDFs from Equation A1 in combination with observed regular-trial CDFs as estimates of normal-process completions, along with associated regular- and signal-trial accuracies. This equation may therefore be used for determining the amount of partial information available to the guessing processes at any point in time after the presentation of the test stimulus, if a response signal is placed

at that point. A1 If this value exceeds chance, it is an indication that some form of useful partial information exists.

A1 Actually, the response signal must be placed slightly before the point of interest, to allow time for the participant to detect the signal.

Appendix B Simulations Simulations of SAD data were reported by R. W. Smith et al. (1996). These simulations were aimed at verifying the validity and robustness of the SAD technique itself, as well as conditions under which SAD's assumptions may be violated. As such, they are relevant to the interpretation of the PSG model tests presented for Experiments 1B and 2B, and are briefly summarized here. Further details can be found in R. Wo Smith et al. (1996). The simulations generated RTs and associated accuracies for both normal and guessing processes, following a procedure that closely mirrored the process the SAD analysis assumes to occur during a SAD experiment (see R. W. Smith et al., 1996, for details). The parameters used in these simulations were loosely based on the data obtained from Experiment 2B, though highly idealized values were used. A summary of the results of these simulations is presented in Table B1. The Magnitude column represents the magnitude of the error resulting from the manipulation for a given line and should be interpreted as the amount by which SAD overestimates partial information. For instance, a value of 0.002 here means that an actual level of partial information of 0.700 would yield an estimate of 0.702. The other numeric columns represent the statistical significance of ANOVA tests performed on the error magnitude or the PSG model tests. The p values listed here show the significance level of the two tests, one with and one without partial information. In all columns, results for simulations with both partial information absent and partial information present are shown. The Core series test represents the results of a simulation with no violations of SAD's assumptions. SAD performed quite well here, as expected. Two different types of assumption violations were tested, as described by DeJong (1991): correlation and facilitation. In the first

scenario, normal- and guessing-process RTs are negatively correlated, as might occur if they shared some limited cognitive resource. In the facilitation test, normal-process RTs were reduced on simulation trials on which a response signal was presented. This was done in the way DeJong suggested, to model the intersensory facilitation effect he proposed, in which the presentation of a response signal might speed up normal processing, thus inflating estimates of partial information. These two forms of violation were tested individually and combined. For various reasons discussed by R. W. Smith et al. (1996) and Irwin and Meyer (1995), these simulations represent a worst-ease scenario. Although the violations of assumptions tests yielded statistically significant results in at least some conditions, the magnitude of the error in the SAD analysis was small when a correlation alone was present--the SAD procedure produced an average overestimate of .002 in that case. Even the largest error--.022 on average for the case in which both violations of assumptions were simultaneously pres e n t - i s not enough to substantively affect the conclusions of a SAD study. In addition to the violations of assumptions, tests were performed with unequal normal-process and unequal gnessing-process means, because regular-trial RTs were clearly unequal in both Experiments 1B and 2B, but especially in 1B, and because the PSG model tests suggested that guessing-process means were different in Experiment 213 and possibly in 1B, as well. These tests showed unreliable or no effects, as determined by the statistical tests on the accuracy of the estimates of partial information. The final relevant test from these simulations was of the effect of response bias on the SAD computations. Normal-process response accuracies were increased by 10% for one stimulus type and decreased by 10% for the other, whereas guessing-process accuracies were

T a b l e B1 Summary o f Simulation Results Guessing accuracy Test

Significance

Core series Violations of assumptions Correlation Facilitation Both Unequal normal means Unequal guessing means Response bias

--/---/.038 --/.002 .001/.001 --/---/---/.068

Magnitude .002/.008 -.005/.009 .012/.013 .025/.020 .000/.004 - .007/.008 - .007/.011

PSG model tests Stimulus type

Lag condition

Lag x Time

--/--

--/--

--/--

.094/---/---/---/.092 .001 /.001

.003/.010 --/-.001/.006 --/---/--

--/-.035/-.085/.031 --/---/--

Three-way --/--/.005 m/.096 .064/---/--/-

Note. Values indicate the significance level of the tests for partial information absent/present conditions, except for the values under the magnitude column, which represent the magnitude of the effect. Dashes indicate a result that was not statistically significant. An empty cell indicates the test was not performed. PSG = parallel sophisticated guessing.

{Appenatr commuea, on next page:

1462

SMITH AND KOUNIOS

increased or decreased by 20% in the same direction as the normalprocess accuracies. This roughly parallels the pattern of results obtained in Experiment 213. As shown in Table B1, the response bias had, at most, a marginally significant effect. It should be noted that this effect was restricted to the partial information present condition and appeared to be the result of an inflation of estimated guessing accuracy in cases where the true level of partial information approached 100%# 1 Thus, this effect is not of concern in the present experiments, which had guessing accuracies of approximately 50%. The columns of Table B1 under the PSG model tests heading show which specific tests gave statistically significant results in the simulations. These tests showed different patterns of results for the different violations or other manipulations. Specifically, the Lag x Time interaction effect was sensitive to facilitation violations under at least some conditions, and the main effect of lag was sensitive to correlation violations; the presence of both PSG test effects indicates both violations. As expected, the stimulus type effect was sensitive to a difference in guessing means. Three-way effects may also indicate violations of assumptions. These simulation results indicate that the results of Experiments 1B and 2B are accurate. The PSG model tests for Experiment 1B showed significant effects of both lag and of Lag x Time. This is consistent

with a normal-guessing process correlation, probably combined with a facilitation effect. The simulations suggest, however, that the magnitude of the error induced by this violation of assumptions is small. Experiment 2B's PSG model tests showed no significant effects that may be diagnostic of violations of assumptions, although the effect of stimulus type was significant, suggesting that the guessing means were significantly different--but again, this should have, at most, only a very small effect on the estimated levels of partial information.

m The SAD analysis software does not impose a limit of 100% on accuracy levels---it can and does occasionally produce estimates of guessing accuracy that exceed 100% (or that are lower than 0%). In general, these overestimates are balanced by underestimates that fall within the 0-100% range and that are therefore not conspicuous; but when both a response bias and a high level of partial information are present, this balance is imperfect. R e c e i v e d April 7, 1995 Revision received April 2, 1996 A c c e p t e d April 2, 1996 •