Alternative Approaches to the Study of Change: When Syndromal Assessments Conflict with Adults’ and Children’s Impressions of Improvement By Sophia C. Choukas-Bradley

HONORS THESIS April, 2008

Submitted in partial fulfillment of the requirements for the degree of Bachelor of Arts with Honors in the Department of Psychology at Brown University

Approaches to Studying Change

2

Acknowledgments It would be impossible to adequately thank Jack Wright for his exceptional dedication as an advisor. Thank you for never faltering in your support of this thesis that has taken an inordinate amount of time from other parts of your life over the past two years. Our work has been a joy, and it has been the most important part of my college experience. I feel prepared for and excited about a future in this field, and I cannot thank you enough for what you have taught me. I would also like to thank Audrey Zakriski, for her support over these past two years, and for the laughs we have shared at the end of long days in the lab. A big thank you to their kids, Emily and Ethan: You have always been so friendly and patient on all research-related expeditions, you were invaluable pretesters of my interview protocol, and I loved our movie-watching night at Wediko. Thank you to Steph, Annie, Kristen, Natalie, the members of the summer research teams and my clinical team, and all the wonderful people of Wediko who have helped make this project happen. A special thank you goes to Lindsay Metcalfe. You have literally been there every step of the way and I can’t imagine this project without your friendship, advice, and support. Finally, thank you to all my friends and family who have listened to me talk endlessly about Wediko and my thesis and have supported me through it all—and thank you especially to Mom, Dad, Jesse, and Emily.

Approaches to Studying Change

Table of Contents Acknowledgments

2

Table of Contents

3

Abstract

4

Introduction

5

Method

17

Results

31

Discussion

40

References

53

Tables

58

Figure Captions

63

Figures

65

Appendices

73

3

Approaches to Studying Change

4

Abstract This research examined how context-sensitive assessments of behavior can reveal patterns of change in both adaptive and problem behaviors that are obscured by syndromal assessments. In a field study of children at a short-term residential treatment program, counselors’ impressions of change suggested improvement in all areas of functioning. These findings conflicted with the results from standardized syndromal assessments, which suggested increases in some problem behaviors and no change in others. Structured child interviews designed to parallel adults’ field observations revealed that children perceived improvement at the end of the program. However, further analyses showed that their self-assessments did not change from the first to the second interview administration in their ratings of their overall behavior frequencies, their reactions to events, or the frequencies of encountering those events. Extensive finegrained field observations revealed robust increases in overall prosocial behavior, and complex patterns of change in children’s responses to specific events, which demonstrated how children can simultaneously “worsen” in some areas and “improve” in others. The results underscore the importance of developing treatment outcome measures that are contextually sensitive and that assess changes in positive as well as negative behaviors.

Approaches to Studying Change

5

Introduction At a time when 14 million of the country’s youth are estimated to have serious emotional or behavioral problems, the field of child clinical psychology is moving towards evidence-based treatments (Kazdin, 2003). There is surprisingly little agreement, however, about what constitutes “evidence” in the study of change and about how such evidence should be used. Questions remain unresolved about the types of changes we should examine, the types of informants we should involve in assessments, when such assessments should occur, and how we should interpret them. Traditionally, change has been defined “as the simple difference of two measurements, such as the difference between pre- and postintervention therapist ratings of client symptomatology” (Eddy, Dishion, & Stoolmiller, 1998, p. 53). More recently, reviews of psychosocial interventions have advocated for an expansion of the study of change “beyond the exclusive focus on symptom change” (Kazdin, 2003, p. 269), to include greater attention to the situational factors in children’s environments that may mediate change processes (Hoagwood, Jensen, Petti, & Burns, 1996). Nevertheless, evaluation studies that define treatment outcomes beyond ratings of overall “symptom reduction” continue to be surprisingly rare (Hoagwood et al., 1996, p. 1056), leading to a “stymied” study of change (Eddy et al., 1998). Controversies in Assessment The field’s overemphasis on symptomatology can be traced to the framing of personality in terms of the frequency of “acts” over some period of time. Buss and Craik (1983) defined the “act frequency approach” to personality as a method of summarizing past behavior and predicting future behavior based on “act trends”—that is, rates of

Approaches to Studying Change

6

specified, decontextualized actions, collapsed over situations. In this and related approaches to personality assessment, variations in behavior across situations are viewed as deviations from one’s “true” nature, which should be reduced through aggregation (Mischel and Shoda, 1995; Wright & Mischel, 1987). The “five factor approach” to personality (McCrae & Costa, 1987) claims that personality can be measured based on the “Big Five” traits (e.g., extraversion, agreeableness, neuroticism). Traits are measured through mostly acontextual statements (e.g., “she really enjoys talking to people”; “she is not a worrier”), and participants are provided with post-assessment summaries of their personalities that state: “Unless you experience major life changes or make deliberate efforts to change yourself, this summary should apply to you throughout your adult life” (Costa & McCrae, 1991). Such trait theories favor a view of personality as a set of relatively stable and enduring characteristics within the person that are relatively free from contextual influences. Although trait theories that emphasize aggregated act frequency statements have continued to receive support and empirical testing, researchers have also developed other theories of personality that emphasize the complexity of person-situation interactions. For example, although Roberts and Caspi (2001, p. 105) argue that there is continuity in traits from childhood to adulthood, they also acknowledge that personality involves “complex patterns of behavior,” based on “if-stimulus-then” relationships. This formulation of personality as a series of if…then… links between situations and behaviors has been discussed and tested over the past two decades (see Mischel & Shoda, 1995; Zakriski, Wright, & Underwood, 2005). This view suggests that personality is stable, but it also acknowledges the importance of the environment in shaping behavior, treating variations

Approaches to Studying Change

7

across situations as meaningful clues in our attempt to solve the puzzle of personality, rather than as “noise” that we must filter out. Behavioral observations collected at a summer program for emotionally and behaviorally disturbed youth revealed that children displayed stable reaction patterns in response to situations that demanded similar competencies (Shoda, Mischel, & Wright, 1993). Mischel and Shoda (1995, p. 257) claimed that the underlying theory, that personality can be understood as a person’s pattern of if…then… situation-behavior relations, helps to resolve the person-situation debate by recognizing the importance of both stable behavioral tendencies and situational factors. Implications for Assessment It has been difficult for the field to make the translation from theory into the practice of assessment. In spite of the theoretical questions raised about acontextual approaches to assessment, the field continues to rely on methods that are designed to provide highly aggregated statements about people’s behavioral tendencies, because such assessments are reported to show high reliability and validity (Achenbach & Rescorla, 2001). Cervone, Shadel, and Jencius (2001, p. 36) note that, despite repeated concerns about context-free approaches, “the central elements of personality assessment are measures of global, decontextualized psychological qualities, and individuals are characterized by their average tendency to exhibit each of the qualities.” This disconnect reflects the ambivalence of a field that wants to acknowledge the variability of people’s behaviors, but is reluctant to depart from widely tested assessment tools. The disconnect between theory and practice can often be seen within papers, and sometimes within sentences. For example, Roberts and Caspi (2001, p. 105) claim that the “complex

Approaches to Studying Change

8

patterns of behavior,” mentioned previously, “need to be examined, interpreted, and aggregated across numerous situations, places and times, to arrive at a reliable and valid index of a personality trait.” This statement calls for two contradictory actions: an interpretation of different responses to situations that are aggregated across time and place. We cannot understand these complex patterns of behavior if we are to filter out the very contextual information that makes an interpretation necessary. The need for highly context-sensitive measurements is arguably greatest in the field of child clinical psychology, because children may differ significantly in their responses to peers and adults, and in their home and school settings. And yet, the majority of assessment tools ask raters to retrospectively assess the children’s behaviors through act statements which are insensitive to situational variability. One widely used instrument is the Conners parent and teacher scales (Conners, Sitarenios, Parker, & Epstein, 1998), which consist of 38 items that reflect a range of behavior problems (e.g., oppositional; anxious/shy). For each item (e.g., “difficulty being quiet”), the rater is asked how often the behavior has occurred (i.e., “just a little,” “pretty much,” or “very much”). A similar instrument is the Behavioral Assessment System for Children (BASC, Reynolds & Kamphaus, 2002), which asks informants to rate a range of behaviors on a frequency scale of “never” to “almost always.” The behaviors probed in these assessments are typically acontextual statements, which are aggregated into scale scores or “syndromes” that attempt to summarize children’s dispositions. The most widely used standardized instruments in this field, with over 6,000 citations, are Achenbach’s (2001) Teacher Report Form (TRF) and the parallel Child Behavior Checklist (CBCL), which is used by parents. Each instrument consists of 118

Approaches to Studying Change

9

items, the majority of which are decontextualized act statements (e.g., “threatens people”; “teases a lot”). Raters are asked to assess these behaviors on a scale of 0-2 (“not true,” “somewhat or sometimes true,” and “very true or often true”). Some items take context into account (e.g., “argues when denied own way”), and such items could even be reframed in terms of if…then… statements (i.e., “if denied own way, then argues”). However, this contextual information is lost in the aggregation of all relevant items into syndromal scale scores. These scores are converted to T scores so that children can be compared to means for “normal” or “clinical” samples. Various studies have highlighted the importance of using context-sensitive measures in the assessment of behavior in children and adolescents. Wright, Lindgren, and Zakriski (2001) found that in both a field study and in a controlled laboratory study, the TRF failed to discriminate between children with similar behavioral base rates who were functionally different. Specifically, a target child who was highly aggressive when provoked, but encountered few aversive events, received the same aggression scores as a functionally opposite target child, who was less reactive but embedded in a hostile social environment. The researchers found that raters were capable of encoding information about the children’s social environments when forming global personality impressions. The raters recognized that the children differed in their event rates and reaction rates, and in forming their global judgments of the children’s personalities, they weighted reactions more heavily than event rates. It was only in completing the TRF that raters seemed to “filter out” this important information and only focus on their overall act frequencies. Syndromal assessments can fail to recognize important differences in individual children, and also in groups of children. Zakriski, Wright, and Underwood (2005) found

Approaches to Studying Change 10 that gender differences emerged in girls’ and boys’ responses to specific events and in their rates of encountering those events. Their results also elucidate how complex interactions can occur between the person and his/her environment. For example, older girls (age 12 and up) encountered more prosocial peer talk than did older boys, and they responded with a higher rate of prosocial reactions in the face of such events. Thus, older girls appeared to be involved in a positive reciprocal cycle, in which their positive reactions encouraged future positive events, and vice-versa. Older boys, on the other hand, seemed to be involved in a negative cycle; they more frequently encountered aversive peer events, and “appear[ed] to be enmeshed in a cycle in which their counterattacks to peer provocation [led] to escalated coercion from peers” (Zakriski et al., p. 852). Implications for the Study of Change Discrepancies between “evidence” and impressions. Assessment measures that do not examine the person-situation interactions that contribute to behavior may fail to uncover the change processes that influence treatment outcomes (Choukas-Bradley, Banducci, Metcalfe, Wright, & Zakriski, 2008). Indeed, if it is difficult to assess behaviors, it is likely to be even more challenging to assess behavioral change. A common approach to measuring children’s change in response to treatment uses difference scores from two administrations of a syndromal measure. A smaller number of studies use a more direct approach to measuring change, by asking observers, such as clinicians or parents, to reflect on children’s improvement over the full course of treatment, and to make summary judgments about this improvement. One such measure is the Clinical Global Impressions (CGI, Guy, 1976) instrument, which asks raters to

Approaches to Studying Change 11 provide impressions of change in six areas (global improvement, relationships with adults, relationships with peers, aggressive behavior, withdrawn behavior, and prosocial behavior). A study of residential treatment outcomes by Connor, Miller, Cunningham, and Melloni (2002) found that greater improvement was reported by the CGI than by a standardized syndromal instrument similar to the TRF (the Devereux Scales of Mental Disorder, or DSMD). The researchers attributed this discrepancy to the fact that teachers completed the DSMD forms and residential clinical staff completed the CGI forms, and point out that the low level of staff agreement from these two measures “highlight[ed] the difficulties in defining and reliability assessing outcome for youth in residential care” (Connor et al., 2002). In analyses of multiple measures of children’s behavioral changes during short-term residential treatment, Metcalfe (2007) and Choukas-Bradley et al. (2008) found results consistent with those of Connor et al. (2002). Counselors’ impressions of change appeared to contradict pre-post difference scores based on the TRF, even when the same raters completed both forms. Specifically, the CGI reported improvement in all areas, but the TRF did not show a linear change in aggression over time. Problem behaviors vs. adaptive functioning. In trying to understand the differences between raters’ global impressions of change and standardized syndromal measures, it is important to note that instruments such as the CGI ask raters to assess changes in both problem and prosocial behaviors, while the syndromal instruments focus on “symptoms,” that is, problem behaviors. Indeed, the trend in the study of children’s behavioral change—and the study of children’s problem behaviors in general—is to

Approaches to Studying Change 12 focus on the reduction of symptoms. As Kazdin (2003, p. 269) points out, “Although symptom change is important, it is difficult to find compelling evidence that symptom change, as opposed to reduced impairment or improvements in prosocial functioning, family interaction, or peer relations, is the best predictor of long-term adjustment and functioning.” Some evidence suggests that during treatment, children improve more in this realm than in their problem or antisocial behaviors. In the Metcalfe (2007) study, counselors’ judgments of improvement for prosocial behavior were higher than for problem behaviors such as aggression. Moreover, field observations revealed increases in prosocial behavior, whereas aggression showed relatively little change. To summarize, global impressions of change and field observations of behavior reveal improvements in children’s adaptive, prosocial behaviors, and such changes are not revealed in syndromal assessments that focus on problem behaviors. Contextual specificity of change. A second possible explanation for the discrepancy between counselors’ impressions of change and syndrome change scores is that the latter may obscure important context-specific changes in children’s reactions to social events. Wright and Zakriski (in press) found that complex change patterns occurred during residential treatment. For example, prosocial reactions to positive events increased, yet they decreased in response to aversive events (e.g., adult discipline and peer provocation). Similarly, the overall rate of aggressive behavior decreased, yet aggressive reactions to aversive events increased. Metcalfe’s (2007) study also revealed complicated patterns of change. Children’s “total adaptive functioning” (a measure that included increases in prosocial behavior and decreases in problem behaviors) improved in response to adult events, but changed little if at all in their reactions to peer events.

Approaches to Studying Change 13 Choukas-Bradley et al. (2008) found that children rated as “most improved” in counselors’ impressions of change showed significant decreases in aggression over time, but that their aggression increased in response to aversive events. Other research found that counselors were able to incorporate important contextual information about a target child in their impressions of his personality, whereas their TRF ratings did not discriminate between functionally different children (Wright et al., 2001). It seems that similar processes may mediate the discrepancies between people’s impressions of change and changes in TRF scores; counselors’ impressions of change reflect their internalization of contextual specificity, but their TRF assessments may not. Children’s Interviews Though previous research has provided evidence of the relation between extensive field observations and people’s impressions of change, such studies neglect a key group of raters: the children themselves. If we are to view inter-rater discrepancies as informative rather than problematic, and if we hope to incorporate the ratings of many different types of informants in order to understand differences in children’s behaviors across a wide range of situations, then it follows that we should incorporate children’s views of their own behaviors in our model of change. Achenbach, McConaughby, and Howell (1987, p. 213) argue: “Because we lack definitive criteria against which to validate measures of childhood problems, it is essential to preserve the contributions of different informants, even if they do not correlate well with each other.” Achenbach, who developed the TRF/CBCL, also developed the Youth Self Report (YSR), the child version of the behavioral checklist. He reasoned that the low mean correlation (.22) between children’s self-ratings and their ratings by parents, teachers, and mental health

Approaches to Studying Change 14 workers, demonstrates that children’s self-ratings should not stand alone as measures of their behavioral problems, but that neither should they be ignored, because “[l]ow correlations between informants may indicate that the target variables differ from one situation to another, rather than that the informants’ reports are invalid or unreliable” (Achenbach et al., 1987, p. 213). Giving voice to children in the assessment of their own behaviors is an important step towards understanding their behavioral changes, but important information could be lost in the aggregation that occurs in existing child versions of syndromal scales. A context-sensitive assessment that measures children’s perceptions of their behaviors would help clarify the processes that mediate change, and would help us understand whether children’s self-perceptions of change agree with adults’ judgments. In his critique of the act frequency approach’s (AFA) lack of regard to intentionality, Block (1989, p. 238) writes, “…if I affirm the context-unspecifying statement ‘I demand a backrub’ (a dominance ‘act’ statement), the AFA is not interested in and cannot know the meaning of my yes response.” Context-sensitive ratings by adults could help us to understand the situations in which a person is most likely to demand a backrub, but only in talking to the person directly could we learn more about the “meaning of his yes response.” In the context of a residential treatment program, an observer might recognize that a child responds aggressively to peer prosocial talk, but only the child herself could reveal that she believes other children are mocking her when, from the adults’ perspectives, they are simply trying to engage her in conversation. Children are valuable informants because they can provide us with insights into their behaviors to which they have better access than outside observers. Kazdin (2001, p.

Approaches to Studying Change 15 789) states that “aggression is not merely triggered by environmental effects, but rather through the way in which these events are perceived and processed.” His claims are complemented by considerable research on social information processing models of aggression, which suggest that aggressive children may interpret their social surroundings in distinctive ways. Social Information Processing (SIP) theory, as reformulated by Crick and Dodge (1994), suggests that, when children are confronted with a social situation, they (1) encode external and internal cures, (2) interpret and mentally represent those cues, (3) select a goal, (4) access or construct responses, (5) decide on a response, and (6) enact that response. The most common method of studying children’s social informational processing involves an interview or questionnaire in which children are presented with hypothetical situations and are asked questions that probe their processing patterns at various steps of the SIP model, such as how the situation might make them feel and what they might do next (Crick & Dodge, 1994). As support for the SIP model has increased, studies have attempted to link children’s vignette responses to actual behaviors, by using data from peer nominations about social preference and/or aggression (e.g., Dodge, Lansford, Burks, Bates, Pettit, Fontaine, & Price, 2003; Parker, Hubbard, Ramsden, Relyea, Dearing, Smithmyer, & Schimmel, 2001) or teacher and parent reports (e.g., Dodge et al., 2003). Few researchers have studied the SIP patterns of children with “clinically severe aggressive behavior problems” (Orobio de Castro, Merk, Koops, Veerman, & Bosch, 2005, p. 105). The extant studies have compared samples of clinically referred children with nonclinical samples (e.g., Coy, Speltz, DeKlyen, & Jones, 2001; Orobio de Castro et al., 2005) or have examined differences between different groups of aggressive children, such as

Approaches to Studying Change 16 children with conduct disorder versus oppositional defiant disorder (Dunn, Lochman, & Colder, 1997). Findings indicate that children with clinically aggressive behavior have trouble encoding social cues (Coy et al., 2001), and tend to attribute hostile intentions, generate more aggressive responses, and evaluate aggressive responses less negatively than their nonaggressive peers (Orobio de Castro et al., 2005). My child interview method resembled SIP studies in its use of vignettes to probe children’s reactions to hypothetical social events, but its main purpose differs from the goals of most related studies in several key ways. Firstly, most studies seek to explore the cognitive processes underlying children’s social perceptions and decisions (see Crick & Dodge, 1994). Studies that focus on distinguishing the processing patterns of clinical samples from non-clinical samples generally try to isolate the specific processing step at which children with maladaptive social patterns deviate from the norm in their thinking (e.g., Orobio de Castro et al., 2005). The current study focused on the less studied question of whether children’s verbal responses to hypothetical situations are related to the behaviors they display in response to real social events. Secondly, rather than comparing the responses of a clinical sample to a normative sample, my study focused on the relationship between children’s interview responses and extensive field observations recorded by adults in a short-term residential treatment program for children with emotional and behavioral disorders. The vignettes depicted social events commonly encountered by children in their social interactions with peers and adults (e.g., teasing from peers or instructions from adults). My review of the literature revealed no studies that directly compared children’s vignette responses to field observations of their reactions to similar events. The current study also differed from past work in the field of

Approaches to Studying Change 17 child vignette research by asking children straightforward questions about their overall behavior ratings, event ratings, and impressions of their own behavioral changes. Finally, the current study sought to embed the acquired information about children’s selfperceptions into an extensive study of behavioral change. Hypotheses of the Current Study The study examined counselors’ global impressions of change through the CGI, as in Metcalfe (2007) and Choukas-Bradley et al. (2008). Based on past research, I hypothesized that counselors would perceive children as having improved in all areas of functioning. Secondly, in contrast to people’s impressions of change, I hypothesized that standardized assessments (TRF) at three time points would not reveal changes in problem behaviors. Thirdly, I expected that extensive fine-grained field observations would help elucidate the discrepancies between the CGI and TRF judgments of change, by revealing complex patterns of change in children’s reactions to events. Fourthly, based on findings about low correlations between children’s self-reports and observer reports (Achenbach et al., 1987), and findings that aggressive children do not have accurate perceptions of their social environments (e.g., Coy et al., 2001), I hypothesized that children’s perceptions of their own behaviors, social events, and behavioral changes would differ from adults’ ratings at the individual difference level. Such discrepancies were not expected to act as hindrances to our understanding of children’s behavioral changes, but rather to help us untangle the complicated web of change.

Method This research was collected as part of the second year of a three-year study at Wediko Children’s Services’ residential summer treatment program for children with

Approaches to Studying Change 18 emotional and behavioral problems. The program admits approximately 125-150 children each summer. Many of the children are referred through the Boston public school system and the majority of the children come from the New England region. The children are 719 years old and live on the setting for 45 days during July and August in clinical groups of 8-12 same-sex, roughly same-aged peers. They participate in a range of daily activities including classroom learning, recreational activities (e.g., athletics, art), and group therapy. Approximately 140 counselors and teachers work on the setting and participate in data collection during the summer. During the first year of data collection, when these methods were developed, I worked as a counselor and as a member of the research team. During the current year of study, I developed the child interview protocol, oversaw all interviews, and helped with data management and staff training for the overarching study. Parental consent to use children’s data for research was obtained from parents/guardians during the pre-summer interview process. Child participants. For field observations and standardized assessment forms, data were obtained for a total of 148 children during the summer. Children who arrived late, departed early, were placed in a new clinical group after their arrival, or spent more than three days away from their group at any point during the summer were excluded. The final sample consisted of 139 children; 55.4% White, 30.2% African-American, 12.9% Hispanic, 0.7% Asian, and 0.7% other. For analysis purposes, children were placed in two age groups, using the age guidelines of past research (Achenbach & Rescorla, 2001). Specifically, children over age 11 were categorized as “older” and all others as “younger.” Each gender and age group was as follows: younger boys (N = 29, Myears = 9.14, SD = 1.55), younger girls (N = 9, M = 10.22, SD = .97), older boys (N =

Approaches to Studying Change 19 67, M = 14.42, SD = 1.93), older girls (N = 34, M = 14.26, SD = 1.96). For every analysis reported in this thesis, gender and age were tested as factors, but no significant results were found, and thus will not be further discussed. Adult participants. A total of 144 counselors and teachers (mainly college students or recent graduates) were employed at Wediko over the summer. Teachers and activity counselors ran classes and activities for six hours each day, and other residential counselors as well as supervisory staff spent the majority of the day with their clinical groups of children. End of summer assessments from counselors who had arrived late to or departed early from Wediko were excluded from analyses, as they were not in a position to provide impressions of children’s changes over the full summer. Hourly observations of behavior (see below) from all counselors were included in the behavior data analyses. Standardized Syndromal Assessment Teacher Report Form (TRF). The TRF (Achenbach & Rescorla, 2001) is a standardized measure that assesses behavior problems in children. It is one of the most widely used instruments of its type in research and clinical settings. It consists of 118 items (see Appendix D) grouped into eight narrowband measures and two broadband measures. The narrowband measures include Aggression, Delinquency, Withdrawal, Somatic Complaints, Anxiety, Thought Disorder, Attention Problems, Rule Breaking, and Social Problems; the broadband measures include Internalizing and Externalizing. “Internalizing” is the aggregate of Withdrawal, Somatic Complaints, and Anxiety; “Externalizing” is the aggregate of Aggression and Rule Breaking (called “Delinquency” in earlier versions of the TRF). The “Total Problem Behavior” (TPB) scale was also used

Approaches to Studying Change 20 in my analyses; in addition to the eight narrowband scales, it incorporates additional problem behaviors such as “bedwetting.” Raters are asked to assess these behaviors on a scale of 0-2 (“not true,” “somewhat or sometimes true,” and “very true or often true”). All results are reported in the form of T scores, using Achenbach and Rescorla’s (2001) procedure. Scores are calculated based on age and gender groups, with children over 11 categorized as “older,” as mentioned before. Reliability information is available from Achenbach and Rescorla (2001). Procedure. With the exception of supervisory staff, all staff—including teachers, residential counselors, and activity counselors—completed standardized assessments of behavior every two weeks and participated in daily behavioral observations. There were three TRF administrations during the summer: July 15 (10 days after the children arrived), July 29 (24 days after the children arrived), and August 12 (38 days after the children arrived). At each administration, counselors were assigned 1-2 children from their clinical team (or in the case of teachers, from their classroom) to rate using the TRF. For the first assessment, they were instructed to assess the children’s behavior since the beginning of the session (a 10 day period). For the second and third assessments, they were instructed to base their ratings on the most recent two weeks. The completion of each child’s TRF assessment took approximately 30 minutes. If a child was rated by more than one adult, the raters’ scores were averaged. Field Observations Daily behavior observations were recorded for up to six days per week during hourly activity periods. Coders recorded behaviors at the end of the period, using 79 hand-held computers (“Palm”© Z-22s) running SatForm software written for this project.

Approaches to Studying Change 21 Each day, the 79 Palms were updated with the appropriate schedule of activities, including the names of the children and staff who were scheduled to be present at that activity. The Palms were distributed each morning to activity sites, classrooms, and living groups and they were collected at the end of each day. Each night, they were docked, the data were uploaded to a desktop computer, the batteries were charged for use the next day, and the new schedule was uploaded. 18,649 observation sessions were obtained. A manual provided specific code definitions and explained Palm input procedures (see Appendix A). Training at the beginning of the summer (before the children had arrived) included group presentations and demonstrations, hands-on practice in coding hypothetical situations, and a quiz of the codes used in the Palms. Counselors were also provided with “Quick Guides” summarizing the codes, which they were encouraged to carry with them for reference. Each coder typically coded 2-6 children per period and each child was observed for 2-6 periods per day. A coder started each coding session by logging into the software with his/her assigned username and password. The coder was then presented with the list of activities he/she was scheduled to attend during the day. At this point, the coder could select the activity to be coded, and would then be presented with a list of children expected to be present at that activity. He/she would coordinate the coding of the children with the other counselors present. Overall behavior frequencies. The coder was then prompted to record an overall assessment of eight of the target child’s behaviors using a 0-3 scale (“not at all,” “somewhat,” “moderately,” and “a lot”). The eight behaviors were: “withdrew/isolated,” “looked sad/cried,” “whined/complained,” “argued/quarreled,” “teased/bossed,”

Approaches to Studying Change 22 “hit/pushed/attacked,” “talked prosocially,” and “complied/cooperated.” If the rater failed to enter a rating for any of the eight behaviors, he/she was prompted to do so, as the Palm program required a rating for each of the eight behaviors (for more details, see Appendix A). Event frequencies. The coder was then prompted to record overall frequencies of eight social events that the target child encountered, using the same 0-3 scale. The social events were: “peer teased/bossed,” “peer argued/quarreled,” “peer asked/told,” “peer talked prosocially,” “adult disciplined/punished,” “adult warned/reprimanded,” “adult gave instructions,” and “adult talked prosocially.” As with the overall behaviors, the coder was prompted to enter a rating for each of the eight social events if he/she had not done so (see Appendix A). Target’s reactions to events. At this point in the coding procedure, the software selected two of the social events that the coder reported had occurred (i.e., an event receiving a code other than “0”) so as to obtain details about the event. Due to the time constraints faced by the counselors and teachers coding the children’s behaviors, it was not feasible for the Palms to probe coders for details about all of the reported events. The following procedure occurred for each of the two events probed. Once an event was selected to be probed, the coder was instructed to select the name of the peer or adult who had most recently (or most memorably) engaged the target child in the “event” in question. For example, if the coder was describing an event in which the target child “Max” was “teased/bossed” by a peer, the program would prompt the coder for the name of the peer who had teased or bossed Max (e.g., “John”). The coder would then be asked to assess which of the eight possible responses Max had

Approaches to Studying Change 23 displayed when John teased/bossed him, and to what extent. For responses that did occur, the coder used a 1-3 scale (“somewhat,” “moderately,” or “a lot”). For responses that did not occur, the coder did not enter any information. The possible responses were the same eight behaviors that were used earlier in the “overall behavior” part of the coding procedure. Interactant’s responses to target’s behavior. The data from the final part of the coding procedure were not used in analyses for the present study, and thus are described briefly. In the final step, the coder was prompted to assess how John (the peer interactant) responded to Max’s reactions (those described above), using the same 1-3 scale. The possible responses of peers were as follows: “ignored/no response,” “talked prosocially,” “showed positive emotion,” “asked/told,” “argued/quarreled,” “teased/bossed,” “hit/pushed/attacked,” and “yielded/gave in”; the possible adult responses were: “ignored/no response,” “talked prosocially,” “showed positive emotion,” “gave instructions,” “warned/reprimanded,” “disciplined/punished,” “physically controlled,” and “yielded/gave in.” After the coder entered information about the first social event, he/she was prompted about the second one with the same procedure. Once two behavioral sequences were coded, the coder was brought back to the original prompt screen and could begin to code another child or terminate the session. Preliminary analyses of behavioral data. As in previous work (Zakriski et al., 2005), individual items from the overall behavior ratings were grouped into three behavioral categories: aggression (argued/quarreled, teased/bossed, whined/complained, and hit/pushed/attacked), withdrawal (withdrew/isolated, looked sad/cried), and prosocial

Approaches to Studying Change 24 behavior (talked prosocially, complied/cooperated). As in Zakriski et al. (2005), adult warned/reprimanded and adult disciplined/punished were combined to form a single category called “adult warned/disciplined,” and peer teased/bossed and peer argued/quarreled were combined to form “peer argued/teased.” A total of six event categories resulted: adult prosocial talk, adult instruct, adult warn/discipline, peer prosocial talk, peer argue/tease, and peer ask/tell. For purposes of this thesis, adult prosocial talk and peer prosocial talk will be referred to as “positive events,” adult instruct and peer ask/tell as “neutral events,” and adult warn/discipline and peer argue/tease as “negative events.” Aggressive, prosocial, and withdrawn reaction rates were calculated based on data acquired from the stage of the Palm coding process in which coders enter information about the target child’s reactions to a social event. These data were grouped according to the three behavioral categories described above (i.e., aggression, withdrawal, and prosocial behavior), and expressed in terms of reactions to specific social events (e.g., “aggression to adult instruct”). Impressions of Change At the end of the session, all teachers, residential counselors, and activity counselors rated children’s overall “improvement” from the summer. Materials. Each adult rated all children in his/her clinical group using six items of a revised version of the CGI (Guy, 1976) scale mentioned in the Introduction section (see Appendix B). Items 1 through 3 examined the children’s overall adjustment (“global”), relationships and interactions with adults (“adults”), and relationships and interactions with peers (“peers”). Items 4 through 6 examined changes in aggressive behavior

Approaches to Studying Change 25 (“aggression”), withdrawn behavior (“withdrawal”), and prosocial/friendly behavior (“prosocial”). Aggressive behavior was defined as “arguing, teasing, bossing, bullying, threatening, pushing, and/or hitting.” Withdrawn behavior was defined as “isolating self from others, looking sad, and/or crying.” Prosocial/friendly behavior was defined as “talking, attending, and/or showing positive emotion to others.” Staff made their evaluations based on a 7-point scale: “very much worse” (1), “much worse,” “minimally worse,” “no change,” “minimally improved,” “much improved,” and “very much improved” (7). Internal consistency was high for the six items of the adult CGI; Cronbach’s α was .92 and the mean inter-item r was .65. Therefore an overall index of change (“overall”) was calculated for each child by averaging the six other items. Procedure. All raters were instructed to use their full knowledge of the child over the entire summer when evaluating his/her changes. The forms were individualized so that each staff member received a form with the names of the children who had been in his/her clinical group. The rater circled a number for each of the six items (using the 1-7 scales) for each of the children in his/her clinical group. The completion of these six items took approximately ten minutes. Children’s Self Assessments A structured interview protocol was developed to probe children’s perceptions of their social environments and their own behavioral changes. Children were interviewed at two points during the summer using a protocol that was designed to parallel as closely as possible the procedures used to collect field observations of children’s behavior in context.

Approaches to Studying Change 26 Participants. A sample of 40 children was used for purposes of interviews and related analyses. Based on the lower number of girls relative to boys attending the Wediko summer program, children aged 9 to 13 were targeted. An equal number of boys and girls were selected based on the first week’s observational data for aggressive, withdrawn and prosocial behaviors. Girls and boys might differ in their reasons for referral and in the severity of their behavior problems, which could confound comparisons of their behavior change. Therefore matched samples of girls and boys were identified based on their overall behavioral frequencies (using field observations) during their first week in the program. Specifically, each girl was matched to the boy who was the same age (i.e., within one year) and whose overall rates of aggressive, withdrawn, and prosocial behavior were the most similar to hers. One child was ill during the second interview administration, and thus was excluded from all data analyses, yielding a final sample of 39 children. The gender and age breakdown was as follows: younger boys (N = 6, Myears = 10.67, SD = .52), younger girls (N = 8, M = 10.50, SD = .53), older boys (N = 13, M = 12.38, SD = .51), older girls (N = 12, M = 12.42, SD = .52). Qualitative Responses. Interviews were organized into four parts (see Appendix C). The first part of the interview included four vignettes, which I developed to represent a specific social event that paralleled one of the event categories used in the field observations (see above). Due both to the nature of the population (e.g., limits in the children’s attention spans), and to the nature of the setting (e.g., limits to the amount of time that children could spend away from their clinical groups and structured activities), it was not possible to probe all eight social events examined in the field observations. Four events were chosen based on their frequency and the variety of behavioral responses

Approaches to Studying Change 27 evoked by them, according to Palm data from Year 1 of the study: “peer argue/tease,” “adult warn/discipline,” “peer ask/tell,” and “adult instruct.” Each of the four vignettes was created to exemplify one of the events, based on definitions and examples provided in the coding manual (see Appendix A). Five free-form questions followed the presentation of each vignette (see Appendix C). Children were asked: (1) “If a kid/adult did something like this to you, how would it make you feel?” (2) “Why would a kid/adult do something like this to you?” (3) “If a kid/adult did something like this to you, what would you do next?” (4) “Let’s say you did the things you just said you would do. What would the kid/adult do next to you?” (5) “Let’s say the kid/adult did the things you just said the kid/adult would do. How would that make you feel?” Questions 3 and 4 were designed to reflect Palm pilot probes: Question 3 corresponded with “target’s reactions to event,” and Question 4 corresponded with “interactant’s response to target’s behavior.” Social Event Rates. The second part of the interview asked children for discrete responses. Interviewees were asked to rate how often they had encountered the eight social events probed by the Palm. The language used in describing the social events to the children was as simple as possible without compromising the meaning of the code (see Appendix C). Interviewees could respond using a five-point scale: “Almost Never,” “Sometimes,” “Often,” “Very Often,” or “Almost Always,” which was converted to a 1-5 scale for the purposes of analysis. Overall Behavior Rates. The third part of the interview asked children to use the same scale to rate how often they themselves had displayed the eight overall behaviors probed by the Palms. As was the case for the social event ratings, the questions about

Approaches to Studying Change 28 behaviors were asked in language that was both child-friendly and true to the nature of the code. The same behavioral scales were formed for the interview data as were formed for the Palm data: aggression (argued/quarreled, teased/bossed, whined/complained, and hit/pushed/attacked), withdrawal (withdrew/isolated, looked sad/cried), and prosocial behavior (talked prosocially, complied/cooperated). Perceptions of Change. The fourth and final part of the interview was only used during the second interview administration, as it assessed children’s perceptions of their behavioral changes over the course of the summer. Children were asked six questions based on the six items of the Clinical Global Impressions (CGI) assessment completed by adults at the end of the summer. The seven-point scale was adapted from the scale used for adults, reworded with child-friendly language: “very much worse” (1), “much worse,” “a little worse,” “no change,” “a little improved,” “much improved,” and “very much improved” (7). The items remained functionally equivalent to those asked of adults but were revised with child-friendly language. To parallel the scales of the adult CGI, an overall index of change (“overall”) was calculated for each child by averaging the six other items (see Results for a discussion of internal consistency and the additional subscale created). Interview Procedure Two interviewers conducted each interview: a primary interviewer to ask the questions and a recorder to transcribe the interview. Such an arrangement reduced the total time of the interview and was clinically important. Due to the time-sensitive nature of the data and the special demands of the population and setting, it was not feasible for the same pairing of interviewer and recorder to conduct all interviews. Interviewers were

Approaches to Studying Change 29 senior staff members, and recorders were members of the research team; I served as the recorder in over 75% of the interviews. I carefully trained every interviewer in the vocal delivery of the vignettes so as to minimize inconsistencies. The first set of interviews was conducted 10 to 12 days into the summer session, on or within two days of the administration of the first TRF. The second set of interviews was conducted four to six days before the end of the summer session, on or within two days of the third TRF, and within six to eight days of the administration of the adult CGI. Children were never asked to leave school or group therapy for the interviews; appropriate times included activities (e.g., swimming or art), mealtimes, and free time. At the beginning of each interview, the child was told that he/she would be asked about things that kids or adults might say or do to him/her. The interviewee was informed that his/her answers would remain confidential and that there were no right or wrong answers. During the vignette portion of the interview, the child responded verbally to all questions. The interviewer prompted him/her to provide details about his/her answers, and summarized his/her previous responses when asking the final two questions. For the social event frequency section and the overall behavior frequency sections, all instructions about the questions and the five-point response scale were carefully explained to the interviewee and he/she was given a printed version of the scale with the text in large print. An example question was provided to ensure that the child understood the instructions (see Appendix C). During the first interview administration, each social event and overall behavior question was preceded by the phrase “since Wediko started,” and for the second interview administration, each question was preceded by the phrase “over the last two weeks.” This wording was based on the

Approaches to Studying Change 30 instructions given to adults during the corresponding administrations of standardized forms. The child was allowed either to verbally state his/her responses or to point to them, in which case the interviewer would repeat each response aloud to ensure the intended choice had been made. For the first administration, the interview ended after the overall behavior ratings section. In the case of the second administration, the child completed one final section: Perceptions of Change. The new scale was presented and explained, the interviewee again was given a piece of paper with the words of the scale largely printed, and he/she was asked another example question followed by the six change questions. Interviews in the first administration were completed in approximately 15 to 20 minutes and interviews in the second administration were completed in approximately 17 to 25 minutes. Coding of Responses Vignette responses involving the interviewee’s own behaviors were coded as a target child’s reactions to events would have been coded in field observations, then grouped based on the behavioral scales used for Palm data (i.e., “aggression,” “withdrawal,” “prosocial”; see “Behavior Coding”). Questions regarding affect, intent, and the interactant’s reactions were not used in the analyses for this thesis. Children’s social event frequency ratings were grouped into three scales: positive (peer prosocial talk, adult prosocial talk), neutral (peer ask/tell, adult instruct), and negative (peer tease/boss, peer argue/quarrel, adult warn/reprimand, adult discipline/punish). Children’s overall behavior frequency ratings were grouped in the same way as the behavioral vignette responses.

Approaches to Studying Change 31 Results Overall Impressions of Change (CGI) Means and standard errors for adults’ and children’s assessments of change are provided in Figure 1. For adults’ ratings of all children (Panel A), mean ratings for each item were reliably higher than 4 (“no change”), ts(138) ranged from 10.94 to 23.97, all ps < .001. A repeated measures ANOVA revealed that scales varied, F(5, 725) = 93.93, p < .001. (All significance levels provided in this report for repeated-measures analyses were based on the Greenhouse-Geisser adjustment.) Mean improvement was highest for global behavior, relationships with adults, relationships with peers, and prosocial behavior, with means between 5 (“minimally improved”) and 6 (“much improved”). Ratings were lower for improvement in aggressive behavior and withdrawn behavior, with means between 4 (“no change”) and 5 (“minimally improved”). For adults’ ratings of the interviewed children (Figure 1, Panel B), mean ratings for each item were again reliably higher than 4, ts(38) ranged from 7.12 to 13.07, all ps < .001. A repeated-measures ANOVA again revealed that scales varied, F(5, 38) = 17.91, p < .001. As was the case for the ratings of all children, improvement was lowest for aggression and withdrawal, with means between 4 and 5, and all other means were between 5 and 6. To assess whether the sample of interviewed children was representative of the full sample of children, I tested whether adults’ ratings of the interviewed children differed from their mean ratings of all children. Two-sample t tests would not have been appropriate given that the interviewed children were part of the full sample. One-sample t tests were performed against nulls of the mean level over all children for each scale, and

Approaches to Studying Change 32 no comparison was significant, with ts(38) ranging from .261 to 1.82 and ps ranging from < .08 to >.70. For children’s ratings (Figure 1, Panel C), mean ratings for each item were again reliably higher than the midpoint, with ts (38) ranging from 3.52 to 9.88, ps < .01. A repeated-measures ANOVA revealed that scales also varied for children’s assessments, F(5,190) = 3.08, p < .02. An inspection of Figure 1 reveals the similarities in the pattern of variation over scales between adults’ and children’s assessments. Children’s selfassessments yielded patterns of greatest and least improvement that was similar to the adults’ ratings, with mean improvement scores between 4 and 5 for aggression and withdrawal and between 5 and 6 for all other scales. Pairwise comparisons were performed to test whether children’s ratings differed for each scale when compared with adults’ ratings of the same children. No comparison was significant, with the largest t(38) = 1.87, p < .07, and all other ts < 1. Note that the mean level similarities discussed thus far are different from the discrepancies hypothesized to occur at the level of individual children. Although the adults’ CGI ratings of the interviewed children agreed with the children’s assessments of their own behavioral changes at the mean level, correlation analyses revealed no reliable relationship at the individual level for any scale, rs ranged from -.28 to .19, all ps > .08, with a mean r of .01. An inspection of the bivariate analyses revealed no obvious outliers or nonlinearities. Because internal consistency was low for the children’s items (mean inter-item r = .29), to further explore correlations between children’s and adults’ ratings, I created an additional subscale consisting of the four most highly inter-correlated items (i.e.,

Approaches to Studying Change 33 “global,” “adults,” “aggression,” and “withdrawal”). For this subscale, mean inter-item r was higher (.43), but α showed little increase (.73), due to the reduced number of items. For adults’ ratings of the interviewed children, α was .84, and for adults’ ratings of all children α was .86. However, using this subscale to compare the interviewed children’s own ratings to those made by adults, the correlation remained non-significant (r = -.11). Thus, I found little evidence of agreement between adults’ and children’s ratings at the individual level. Summary. For both children and adults, improvement was reported for each scale, the degree of improvement varied over scales, and children and adults agreed as to the scales for which the greatest and least improvement was shown. Adults and children showed little if any agreement about where change occurred at the level of the individual. Standardized Syndromal Assessments Figure 2 shows results for the Teacher Report Form (TRF) for three scales of interest: the Aggression narrowband scale, and the broadband Externalizing, Internalizing, and Total Problem Behavior broadband scales. In contrast to children’s and adults’ judgments of improvement, Externalizing and Total Problem Behavior showed significant linear increases over time. The Aggression scale increased slightly, but nonsignificantly. Internalizing, like most of the TRF scales, did not show change over time. Table 1 provides scale means and results of repeated-measures ANOVAs of all the narrowband and broadband scales of the TRF over time. The TRF showed no evidence of “improvement” in children, which would be revealed through decreases in their problem behaviors.

Approaches to Studying Change 34 For the subgroup of interviewed children, the TRF did not show significant changes for any of the narrowband or broadband scales, Fs ranged from .074 to 2.59. To further check possible differences between interviewed and non-interviewed children, one-way ANOVAs were performed to determine whether the group of interviewed children differed from the non-interviewed children at Time 1, the time of the first interview and the first TRF administration. A reliable difference was found for one scale of 11, the Aggression subscale, F(1, 138) = 4.85, p < .03. Summary. The TRF suggested that little change occurred, and changes that were revealed showed increases in problem behaviors. These results contrasted with adults’ and children’s impressions of change, which suggested improvement in all areas, including those assessed by the TRF (i.e., aggression and withdrawal). Field Observations of Behavior One possible explanation for why the TRF did not reveal “improvement” is that it focused on children’s problem behaviors at isolated time points, rather than continuously assessing both problem behaviors and adaptive behaviors over time. Extensive finegrained field observations examined changes in children’s overall behaviors, reactions to events, and rates of encountering those events. Linear regression. Mean behavior rates were computed over all behaviors in a given category for each child for each day of the summer. Raw scores were converted to z-scores (Xi-M)/spooled where Xi was each child’s score for a given measure on a given day, M was the mean for that measure for all children over all coded days, and spooled was the standard deviation for that measure over days, pooled over children. This transformation was performed because it preserves information about changes over time

Approaches to Studying Change 35 and differences among children in their amount of variation, but it removes differences in the overall elevations of the scales. Similarly, event rates were computed for each child for each day of the summer, and reactions to each event were computed by pooling all behavioral reactions in that category in response to that event, over all days, over all children. Analyses using distinct time periods. Because various forms of assessment (e.g., the TRF, and the child interviews to be discussed) assess behavioral change by comparing data acquired at different time points, the Palm data were divided into time periods for the purposes of certain analyses. “Days” refers to days when coding occurred. The first time period was defined as Days 1 through 9, the second time period was Days 10 through 22, and the third period was Days 23 through 36. For each of these periods, means were calculated for children’s overall behavior rates, reaction rates, and event rates. The first and third time periods corresponded with the two interview administrations. Day 36 was not the final day of coding, but it was the first day of the final interview administration, so the remaining Days were excluded from time period analyses. Overall Rates of Behavior Figure 3 provides the results of linear regression analyses for children’s daily global behavior ratings over time, for both the full sample of children and the sample of interviewed children. Overall behavior rates showed a significant increase in prosocial behavior r2 = .46, b = .27, F(1, 36) = 29.27, p < .001, and a significant decrease in withdrawal, r2 = .09, b = -.11, F(1, 36) = 4.43, p < .05. Aggression did not show significant linear change, r2 = 0, b = .02, F(1, 36) < 1.

Approaches to Studying Change 36 Overall behavior rates of the interviewed children showed similar patterns to those of the full sample. As was the case for the group of all children, the interviewed children showed a significant increase in prosocial behavior, r2 = .31, b = .38, F(1, 36) = 14.97, p < .001, and a decrease in withdrawal, though it did not reach significance, r2 = .08, b = -.13, F(1, 36) = 2.81, p < .20. Although aggression for the interviewed children showed clear nonlinearity, with a peak at time 2, the measure of linear change was not significant, r2 = 0, b = -.04, F(1, 36) < 1. Reactions to Events Another possible explanation for why “improvement” was not revealed through the TRF involves the measurement’s focus on “act frequencies,” and its inability to assess contextually specific behaviors. Overall behavior rates aggregate over contexts and thus can mask significant changes in children’s behavioral responses to specific situations (e.g., Choukas-Bradley et al., 2008). Aggressive, withdrawn, and prosocial reaction rates in response to individual events can reveal narrow yet important behavioral changes. In the analyses that follow, for each type of reaction, figures showing the children’s changes in responses to two events are shown (i.e., reactions to peer prosocial talk and to adult warn/discipline). These events were chosen so as to show the diversity of reactions and forms of reaction change in response to a positive peer event and an aversive adult event. For each type of reaction, complete analyses for reactions to all six events are shown in table form. Aggressive reactions to events. Although children’s overall aggression did not change significantly over time (see Figure 3), an examination of their aggressive responses to events revealed a complex pattern of context-specific changes. Children’s

Approaches to Studying Change 37 aggressive reactions decreased in response to positive events, increased in response to aversive events, and did not change in response to neutral events. Figure 4 shows the decrease in children’s aggressive reactions to peer prosocial and the increase in their aggression to adult warn/discipline; results of reactions to all events, with summary statistics, are shown in Table 2. The subgroup of interviewed children showed patterns that resembled those for the full sample. The main exception was that the changes in children’s responses to positive events were not significant. Withdrawn reactions to events. Although the overall rate of withdrawn behavior showed modest but significant change (see Figure 3), children decreased significantly in their withdrawn reactions to all events except to peer argue/tease, to which their withdrawal did not change. Figure 5 shows their significance decreases in withdrawal in response to peer prosocial and adult warn/discipline; Table 3 shows the results of analyses of reactions to all events, with relevant statistical summaries. The interviewed children showed a similar pattern of reactions in terms of direction and significance of change, with the possible exception of their withdrawn reactions to peer ask/tell, which was not significant. Prosocial reactions to events. Although children’s overall prosocial behaviors increased significantly (see Figure 3), their changes in prosocial reactions to most events were not significant. Strong positive increases were found only in response to peer prosocial talk (see Figure 6) and to adult prosocial talk. Children showed no significant changes in their prosocial reactions to peer ask/tell, adult instruct, or adult warn/discipline (Figure 6). Moreover, children decreased significantly over time in their prosocial responses to peer argue/tease. Table 4 provides the results of all analyses of the children’s

Approaches to Studying Change 38 prosocial reactions. The interviewed children’s prosocial reactions also decreased in response to peer argue/tease, but not significantly, and all other reactions followed the same patterns of directionality and significance as those of the full sample of children. Event Rates Table 5 shows the results of linear regression analyses of children’s event rates. No event showed significant linear changes except for adult instruct, which decreased. Interestingly, the interviewed children differed from the full group in their changes in event rates. They experienced significantly positive changes in their peer positive and peer neutral events. Summary. In contrast to the TRF’s finding of no “improvement” over time, linear regressions of data from field observations revealed important changes in children’s overall behaviors and reactions to events. Overall behavior rates showed strong improvement in prosocial behaviors and less improvement in problem behaviors. Children’s reaction rates further clarify change processes by revealing increases in adaptive behaviors in response to positive events, and increases in problem behaviors in response to aversive events. Such context-specific changes may help us understand adults’ and children’s reports of improvement in their global impressions of change. Children’s Self-Assessments An analysis of children’s interview responses can help clarify their own impressions of their behavioral changes; how those impressions may or may not be linked to changes in their ratings of overall behaviors, event rates, and reactions to events; and how their self perceptions may or may not agree with adults’ assessments.

Approaches to Studying Change 39 Overall Behavior Ratings Repeated-measures analyses of the children’s ratings of their own overall aggression, withdrawal, and prosocial behaviors at the two interview administrations yielded a time x behavior interaction, F(2, 76) = 3.96, p < .04. Withdrawal decreased significantly from time 1 to time 2, F(1, 38) = 8.37, p < .007, aggression decreased but not significantly, F(1, 38) = 2.04, p > .1, and prosocial increased but not significantly, F (1, 38) < 1. Children’s responses were given at two discrete time points and were not collected continuously. To make a more direct comparison of children’s results to field observations, overall behavior ratings were calculated for time points that corresponded with the first and second interview administrations. Using data from these two time points, field observations from the Palms did not reveal a time x behavior interaction for the interviewed children, F(2, 76) = 2.42, p > .10. Thus, there was a lack of correspondence between children’s and adults’ ratings of overall behavior, and children reported more change than did adults’ field observations based on these two time periods. Reactions to Events Children’s judgments of their reactions to events were assessed using their freeform responses to vignettes (see Method). Figure 7 shows the children’s self-reported hypothetical aggressive, withdrawn, and prosocial reactions to the vignettes designed to mirror the aversive and neutral events in the field observations. Panel A shows the results at the first interview administration, and Panel B shows the results at the second

Approaches to Studying Change 40 interview administration. Analyses revealed only a vignette x reaction interaction effect, F(6, 228) = 12.25, p < .001. In order to again directly compare the children’s responses to field observations, Figure 8 shows the children’s behavioral reactions to the four events based on field observations from the Palms, for the two corresponding time periods. A time x reaction interaction was revealed, F(2, 70) = 4.51, p < .03, with aggressive reactions increasing, F(1, 35) = 14.28, p < .002. As with the interview data, the behavioral reactions yielded by field responses did not show a time x reaction x vignette interaction, F(6, 210) < 1. Figures 7 and 8 appear to reveal similarities in the patterns between the children’s verbally-generated responses to hypothetical situations and their recorded behavioral responses to actual events. To assess whether the patterns were in fact similar at the time of the first interview administration, the means depicted in Figure 7A were correlated with the means depicted in Figure 8A, yielding a high correlation, r = .84, p < .002. The analyses for the second administration (Figures 7B and 8B) revealed a stronger relationship, r = .906, p < .001. Event Rates The means of children’s event ratings reported in the interviews did not change over time, F (3, 114) < 1. Corresponding analyses of event data from adults’ field observations at the two corresponding time points also yielded results that were not significant, F (3, 114) < 1. Discussion This research supports past work in clarifying how complex behavioral changes emerge when multiple, context-sensitive measures are used (Choukas-Bradley et al.,

Approaches to Studying Change 41 2008; Metcalfe, 2007; Wright & Zakriski, in press), and reinforces questions about the field’s overemphasis on measures of treatment outcomes that focus on the reduction of symptoms (Mash & Hunsley, 2005) and that neglect to measure changes in adaptive functioning (Kazdin, 2003). Four main findings emerged. First, counselors’ and children’s overall impressions of change indicated improvement in all areas of functioning. These findings are consistent with Metcalfe’s (2007) and Choukas-Bradley et al.’s (2008) in showing that residential counselors believe children have changed during treatment, and my findings add to our understanding of change by demonstrating that the children themselves also believe such improvement has occurred. Second, adults’ and children’s impressions of change conflicted with multiple administrations of the TRF, which showed increases in the Externalizing and Total Problem Behavior broadband scales and no changes in other scales.. These findings agree with past work showing that overall impressions of change yielded higher ratings of improvement than pre-post difference scores from standardized syndromal assessments (e.g., Connor et al., 2002; Metcalfe, 2007). Such differences may be due in part to the syndromal assessments’ focus on problem behaviors. Recall that the CGI asks raters for assessments of change in adaptive behaviors as well as problem behaviors. Mean ratings were higher for prosocial behavior than for aggression and withdrawal in both the current study and in Metcalfe’s (2007) and Choukas-Bradley et al.’s (2008). Specific CGI mean ratings were not provided in the Connor et al. (2002) article. Third, extensive field observations of children’s overall behaviors, social environments, and changes in context-specific reactions, revealed complex patterns of behavioral changes. Overall behavior rates showed strong improvement in prosocial

Approaches to Studying Change 42 behaviors and less improvement in problem behaviors, with robust increases in prosocial behavior, more modest decreases in withdrawal, and no linear change in aggression. Children’s reaction rates further clarify change processes by revealing increases in adaptive behaviors in response to positive events, and increases in problem behaviors in response to aversive events. Such context-specific changes illuminate how pre-post difference scores from standardized syndromal assessments can obscure complicated change patterns, and may help explain adults’ and children’s reports of improvement in their global impressions of change. Fourth, although children reported improvement in their overall self-assessments of change, their other responses did not change from the first to the second interview administration. Little if any change was found for their overall behavior ratings (with the exception of their mean overall behavior rate of withdrawal, which decreased), or their reported reactions to events described in the vignettes. Thus, the children’s interviews further highlight the complicated nature of change and the difficulties in measuring it, and reinforce the importance of using multiple forms of measurement that are sensitive to contextual specificity, that consider changes in reactions to positive as well as negative events, and that do not rely solely on retrospective ratings. Implications of contextual specificity for “iatrogenesis” My findings demonstrate how different instruments can reveal either overall “improvement” or “worsening” in the same group of children. Though many different kinds of discrepancies were revealed, the current section focuses on the lack of agreement in counselors’ ratings of aggression change. We cannot attribute these conflicting findings to the assessments’ having been completed by different raters who interacted

Approaches to Studying Change 43 with children in different environments, which was the case in the Connor et al. (2002) study. In the current study, the TRF and CGI forms and the extensive Palm coding of children’s behaviors were all shared by residential counselors. The counselors’ impressions of change suggested that children improved in their aggression and in all other areas of functioning. In contrast, according to the TRF, the children worsened during treatment: They did not change for some problem behaviors and they increased in their Externalizing and Total Problem Behavior broadband scales. Furthermore, data from field observations revealed that the overall frequency of children’s aggression did not change. Children themselves claimed that they improved in their aggression in their global impressions of change, but no change in aggression was revealed in their other interview responses. Without further information about changes in children’s contextspecific reactions, I might have reviewed my data and concluded that children had showed no improvement in their overall aggression, in contrast to children’s and adults’ impressions of change. However, the fine-grained field observations of children’s reactions revealed that children’s aggression should not be measured only as one aggregated behavior, because it increased in response to some events and decreased in response to others. These intricate patterns of context-specific improvement and worsening may help answer questions raised in the field of child clinical psychology about “iatrogenesis”— worsening through treatment. Although the evidence is mixed, some studies focusing on problem behaviors have found that children appear to worsen in response to group treatment programs (e.g., Dishion, McCord, & Poulin, 1999). Such “iatrogenic effects” have been shown to occur when children increase in their aggression, presumably

Approaches to Studying Change 44 because of “peer deviancy training,” in which aggression and other deviant behaviors (such as tobacco use) are rewarded by peers (Dishion & Dodge, 2005). Viewed in isolation, these findings are troubling. However, it may be helpful to view aggression as one piece of a behavioral change mosaic, in which children can improve in some areas and worsen in others (Wright & Zakriski, in press). Such contextual “unpacking” may help to explain why some studies found worsening in aggressive behavior (e.g., Dishion et al., 1999) while others did not (see Weiss, Caron, Ball, Tapp, Johnson, & Weisz, 2005); changes in children’s aggression may be contextually narrow and thus undetectable by certain assessment methods. Studies that found worsening in overall rates of aggression may have failed to detect contextually specific improvements. Conversely, studies that did not find overall worsening may have failed to recognize that aggressive reactions to certain events increased over time. The findings of Wright and Zakriski (in press) further emphasize the need for assessments that focus on context, because the overall behavior ratings from field data showed that aggression decreased during residential treatment, but that aggressive reactions nevertheless increased to aversive events. Those contextually narrow but clinically important changes made by children are aggregated and filtered out by standardized syndromal measures, but counselors seem to internalize them and form global impressions accordingly (Wright et al., 2001), possibly weighting the improvements more heavily. The importance of prosocial behavior My findings highlight the need for assessment measures that consider adaptive functioning as an important indicator of treatment outcomes. Studies that emphasize the

Approaches to Studying Change 45 presence of iatrogenic effects may not have recognized context-specific improvements in aggression, and may also have neglected to measure improvements in prosociality. In support of Kazdin’s (2003) assertion that a decrease in problem behaviors is not necessarily a better predictor of long-term adjustment than improvements in prosocial functioning, the current study suggests that counselors may be especially sensitive to increases in prosocial behavior when forming global impressions of change. My analyses of field observations reveal robust changes in children’s overall prosocial functioning and in their prosocial reactions to positive events. Results of the current study are in agreement with Metcalfe (2007), who found robust changes in overall prosocial behavior, including in the group of children who were “least improved” according to the CGI. My results also converge with Wright and Zakriski (in press), who found increases in overall prosocial behavior and in prosocial reactions to adult instruct and adult prosocial talk. Overall prosocial behaviors, prosocial reactions to events, and various responses to positive events appear to be important areas in the study of change, and they are all areas that cannot be understood using syndromal methods that emphasize “symptoms” and ignore contextual influences. Implications of children’s interview responses Field observations revealed that children showed greater improvement in response to positive events than aversive ones. This may help explain the lack of change I found in children’s responses to the vignettes. It should be noted that adults’ field observations revealed that all forms of children’s reactions improved in response to positive events: prosocial behavior increased over the course of the summer, and withdrawal and aggression decreased. The opposite pattern was found for aversive events: prosocial

Approaches to Studying Change 46 reactions decreased, aggression increased, and withdrawal decreased to adult warn/discipline but did not change to peer argue/tease. Note that my interview protocol included four of the six event categories in the form of vignettes, and that the two events I excluded were the two positive events. I made this decision on the basis of previous research showing that children’s reactions to positive events yielded less variability as compared to neutral and aversive events (Wright & Zakriski, 2001, 2003), and I believed that such effects might be exaggerated in children’s responses to interview vignettes, due to possible social desirability biases. However, my analyses of field data revealed a rich variety in responses to positive events. Including these events in the interview vignettes might have led to interesting interview responses, and potentially to more hypothetical prosocial reactions and fewer aggressive and withdrawn reactions in the second interview administration. Another important point to note is that, although the linear regressions from Palm data revealed changes in reactions to events, no changes were revealed when we analyzed reactions to the aversive and neutral events based on average rates from the two time periods corresponding to the interviews. This finding suggests that children’s responses should not necessarily have been surprising. Children’s ratings of their overall behaviors and social event frequencies revealed further methodological limitations. The lack of change in children’s social event ratings between the first and second interview administrations was unsurprising given that adults’ field observations also did not reveal changes in event ratings over the course of the summer. More surprising was the absence of change in children’s ratings of overall prosocial behavior. However, two factors must be kept in mind. Firstly, when we examine adults’ ratings of children’s overall behaviors as a function of two time periods

Approaches to Studying Change 47 equal to those we asked the children to consider in their responses, no overall behavioral changes were revealed. Secondly, children were interviewed beginning the 36th day of treatment, six days prior to the end of the summer session. When I performed linear regression analyses of their behaviors for the first 36 days of treatment, no changes were significant. In sum, little if any change was revealed through children’s ratings of their overall behavior and social event frequencies or through their responses to vignettes, but when adults’ field observations were viewed as means over blocked periods of time, or in terms of linear change for the appropriate time period, the apparent discrepancies between adults’ and children’s perceptions were reduced. What remains unclear is why children’s global impressions of change showed improvement. The children formed these impressions at a time when the field observations, the TRF pre-post difference scores, and the children’s own ratings, all suggested that their behaviors had not improved over the course of the summer. The fact that the children still believed they had improved may suggest that they were not capable of accurately assessing their behavioral changes. The social information processing models suggest that aggressive children interpret their surroundings in skewed ways (Coy et al., 2001; Orobio de Castro et al., 2005), and it is possible that clinically aggressive children are unable to accurately assess their behaviors, let alone any changes in those behaviors. Indeed, even carefully trained clinicians have trouble agreeing about which children have changed (Connor et al., 2002), and research suggests that children’s reports generally have low correlations with adults’ reports (Achenbach, et al., 1987). What we should not do is conclude that children are poor judges who should not be used in future studies of behavioral change. As Kazdin (2003) points out, the way

Approaches to Studying Change 48 children interpret their surroundings can have a significant impact on their behaviors, and the children’s perceptions of their behavioral changes could predict their post-treatment adjustment more strongly than the actual changes measured by adults. Future studies should compare children’s perceptions of their own improvement to parents’ and teachers’ ratings in the months following treatment. Retrospective ratings of symptomatology Taken together, the findings from adults’ and children’s various forms of assessment highlight the need for measurements that do not depend solely on retrospective ratings; the unpacking of children’s complex changes in reactions may require the collection of fine-grained field observations. Two possible explanations for the difficulty of recognizing change through syndromal assessments are the lack of attention to prosociality and an inability to encode contextual specificity. A third possible explanation is that such measures depend on retrospective analysis and the conceptualization of long, yet specific, time periods. It is important to recognize that the children’s responses to questions about their overall behavior rates and social event rates required that they conceptualize their behavior over a specified time period, a task that may have been challenging for children as well as adults. In the current research and in the Metcalfe (2007) and Choukas-Bradley et al. (2008) studies, counselors rated children with the TRF ten days into the treatment program, again two weeks after that, and a third time two weeks after that. If raters confused the time periods (for example, by remembering an event from three weeks ago as having occurred two weeks ago, a mistake that seems easy to make), the difference scores would not accurately reflect behavioral changes. Additionally, assessing behavior every two weeks leaves room for a

Approaches to Studying Change 49 significant amount of information to be forgotten. Moreover, the standard period of time for TRF ratings is two months (Achenbach & Rescorla, 2001), and Achenbach, Dumenci, and Rescorla (2002) report that parents are sometimes asked to consider the past six months in filling out the CBCL. Thus, errors in TRF and CBCL ratings of change may be more pronounced in most studies than in the current work. An advantage of the extensive field observation approach is that it tracks behaviors continuously, so that raters are asked to recall information over the past hour. Such data fulfills the standard called for by trait psychologists who wish to see behaviors recorded over multiple contexts; and yet it preserves information about the social environments in which children’s behavioral changes are embedded. At the end of the summer, global impressions of change and standardized syndromal measures can be compared to numerous observations of behaviors, events, and reactions over all days from the summer. It must be noted that global change impression ratings such as the CGI share this one limitation of syndromal measures, in that they ask raters to retrospectively assess behaviors over lengthy time periods. However, the CGI does not share the other two limitations of the TRF in assessing behavioral change: As the Wright et al. (2001) study suggests, raters are capable of internalizing information about changes in children’s environments and context-specific reactions, and the CGI asks raters to assess the degree of improvement in prosocial, adaptive functioning, as well as in problem behaviors. Moreover, it seems that the CGI’s prompts for impressions of change allow informants to consider all the information they know about a child, including contextual information about changes in response to specific events, when forming their judgments, whereas the TRF focuses raters’ attention on the task of aggregating over situational information to

Approaches to Studying Change 50 form judgments about act frequencies. The CGI seems to be an especially useful assessment tool when used in conjunction with fine-grained field observations. Information from these hourly observations can help us to disentangle counselors’ posttreatment impressions of change, and the CGI can in turn allay the concern that field observations alone may be too atomistic (Wright & Zakriski, in press). Shortcomings and Future Directions The current research furthers our understanding of how complex children’s behavioral changes can be, and how difficult it can be to measure them. It does not reveal how we can predict which children will transfer their treatment gains to their home and school settings. Parent CGIs as well as parent and teacher reports (CBCLs and TRFs) were collected during the autumn following treatment, but such data were not yet available at the time of this thesis. We need to understand whether long-term adjustment can be predicted by TRF difference scores, CGI impressions, field observations, and/or children’s interview responses. Thus, future research should examine the relationship between various forms of summer data and assessments completed by parents, teachers, and the children themselves in the fall. Of course, challenges in methodology will make change assessment difficult on the other end of treatment as well. A limitation of the current study was its use of a small sample of children for the interview portion of the data. Ideally, the full sample of children would have been interviewed, but the special nature of the setting and population made this impossible for the current year of study. With more interviewers, it might be possible in future years to interview a larger sample of children within two or three days of the TRF administrations. However, a greater number of interviewers would exacerbate a problem

Approaches to Studying Change 51 with the current study—ideally, all children would be interviewed by the same pair of adults. It may never be possible to conduct interviews with a large sample of children using the same two adults within a limited period of time. However, a simpler improvement to the current study design would be to ensure an equal number of older and younger children. Though I was careful to include an equal number of boys and girls in the original sample, the sample included more older children than younger children. Future studies should also examine whether children’s responses to positive interview vignettes change from the first to the second interview administration in ways that reflect the changes revealed in adults’ field observations. Future work should also attempt to use multiple raters to code children’s interview responses, in order to be sure of coding reliability. Finally, future research should analyze the full range of data provided in the children’s vignette responses, including the evoked emotions, the perceived intent of the interactant, and the imagined response of the interactant to the interviewee’s behavior. Conclusions My findings highlight the difficulty of measuring and explaining children’s behavioral changes during residential treatment, and underscore the importance of assessing change with multiple instruments, including context-sensitive assessments that measure adaptive behaviors and do not require retrospective analysis. Connor et al. (2002) called for a “best estimate” approach in the assessment of children’s improvement, which combines all available information from different forms of assessment. I argue that the “best estimate” of change must include context-sensitive measurements, because important changes in narrow but clinically informative areas may help us understand which children change, why and how they change, and whether those

Approaches to Studying Change 52 changes will be transferred to post-treatment home and school settings. The difficulty of measuring change simultaneously calls for a highly sensitive contextual assessment system, and reinforces the tendency of the field to depend on syndromal measurements that are claimed to be “reliable” and “valid.” Such assessments are easy to access and administer, they are cost-effective, and they have been widely used in the field. In contrast, the use of context-sensitive field observations has not been extensively tested and is costly and labor-intensive. However, if we wish to help emotionally and behaviorally disturbed youth learn how to respond appropriately to their surroundings, we must understand which environments pose the greatest challenges, and which types of reactions improve during residential treatment. If we believe that treatment helps children, then we cannot continue to rely on measurements that aggregate over important information about the changes that are occurring, and that often lead to the interpretation that children are worsening in response to our attempts to help.

Approaches to Studying Change 53 References Achenbach, T. M., Dumenci, L., & Rescorla L. A. (2002). Ten-year comparisons of problems and competencies for national samples of youth: self, parent, and teacher reports. Journal of Emotional and Behavioral Disorders, 10, 194-203. Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont. Block, J. (1989). Critique of the act frequency approach to personality. Journal of Personality and Social Psychology, 2, 234-245. Buss, D., & Craik, K. H. (1983). The act frequency approach to personality. Psychological Review, 90, 105-126. Cervone, D., Shadel, W. G., & Jencius, S. (2001). Social-cognitive theory of personality assessment. Personality and Social Psychology Review, 5, 33-50. Choukas-Bradley, S. C., Banducci, A. N., Metcalfe, L. M., Wright, J. C., & Zakriski, A. L. (2008). Reassessing the assessment of change: Disentangling the social interactional processes that mediate behavior change in at-risk youth. Poster presented at the Eastern Psychological Association’s 2008 Conference. Conners, C. K., Sitarenios, G., Parker, J. D. A., & Epstein, J. M. (1998). Revision and restandardization of the Conners Teacher Rating Scale (CTRS-R): Factor structure, reliability, and criterion validity. Journal of Abnormal Child Psychology, 26, 279-291.

Approaches to Studying Change 54 Conners, C. K., Sitarenios, G., Parker, J. D. A., & Epstein, J. M. (1998). The revised Conners’ Parent Rating Scale (CPRS-R): factor structure, reliability, and criterion validity. Journal of Abnormal Child Psychology, 26, 257-268. Connor, D. F., Miller, K. P., Cunningham, J. A., & Melloni, R. H. (2002). What does getting better mean? Child improvement and measures of outcome in residential treatment. American Journal of Orthopsychiatry, 72, 110-117. Costa, P. T., & McCrae, R. R. (1991). Your NEO Summary. Psychological Assessment Resources, Inc. Coy, K., Speltz, M. L., DeKlyen, M., & Jones, K. (2001). Social-cognitive processes in preschool boys with and without oppositional defiant disorder. Journal of Abnormal Child Psychology, 29, 107-119. Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social information processing mechanisms in children’s social adjustment. Psychological Bulletin, 115, 74101. Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, 755-764. Dodge, K. A., Lansford, J. E., Burks, V. S., Bates, J. E., Pettit, G. S., Fontaine, R., & Price, J. M. (2003). Peer rejection and social information-processing factors in the development of aggressive behavior problems in children. Child Development, 74, 374-393. Dunn, S. E., Lochman, J. E., & Colder, C. R. (1997). Social problem-solving skills in boys with conduct and oppositional defiant disorders. Aggressive Behavior, 23, 457-469. Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention

Approaches to Studying Change 55 studies. Journal of Abnormal Child Psychology, 26, 53-69. Guy, W. (1976). Clinical global impressions. ECDEU Assessment Manual for Psychopharmacology, revised. Rockville, MD: National Institutes of Health. Hoagwood, K., Jensen, P. S., Petti, T., & Burns, B. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. American Academy of Child and Adolescent Psychiatry, 35, 1055-1063. Kazdin, A. E., (1991). Effectiveness of psychotherapy with children and adolescents. Journal of Consulting and Clinical Psychology, 59, 785-798. Kazdin, A. E. (2003). Psychotherapy for children and adolescents. Annual Review of Psychology, 54, 253-276. Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362379. McCrae, R. R., & Costa, P. T. (1987). Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52, 81-90. Metcalfe, L. A. (2007). Contextual versus syndromal assessment of child behavior change: Disentangling components of change impression. Unpublished Honors Thesis for the Department of Psychology at Brown University. Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246-268. Orobio de Castro, B., Merk, W., Koops, W., Veerman, J. W., & Bosch, J. D. (2005). Emotions in social information processing and their relations with reactive and proactive aggression in

Approaches to Studying Change 56 referred aggressive boys. Journal of Clinical Child and Adolescent Psychology, 34, 105116. Parker, E. H., Hubbard, J. A., Ramsdem, S. R., Relyea, N., Dearing, K. F., Smithmeyer, C. M., & Schimmel, K. D. (2001). Children’s use and knowledge of display rules for anger following hypothetical vignettes versus following live peer interaction. Social Development, 10, 529-557. Reynolds, C. R., & Kamphaus, R. W. (2002). Behavior Assessment System for Children – Second Edition. Circle Pines, MN: American Guidance Service. Roberts, B. W., & Caspi, A. (2001). Personality development and the person-situation debate: It’s déjà vu all over again. Psychological Inquiry, 12, 104-109. Shoda, Y., Mischel, W., & Wright, J. (1993). The role of situational demands and cognitive competence in behavior organization and personality coherence. Journal of Personality and Social Psychology, 65, 1023-1035. Weiss, B., Caron, A., Ball, S., Tapp, J., Johnson, M., & Weisz, J. R. (2006). Iatrogenic effects of group treatment for antisocial youth. Journal of Consulting and Clinical Psychology, 73, 1036-1044. Wright, J. C., Lindgren, K., & Zakriski, A. L. (2001). Syndromal versus contextualized personality assessment: Differentiating environmental and dispositional determinants of boys’ aggression. Journal of Personality and Social Psychology, 81, 1176-1189. Wright, J. C., & Mischel, W. (1987). A conditional approach to dispositional constructs: The local predictability of social behavior. Journal of Personality and Social Psychology, 53, 1159-1177. Wright, J. C., & Zakriski, A. L. (2003). When syndromal similarity obscures functional

Approaches to Studying Change 57 dissimilarity: Distinctive evoked environments of externalizing and mixed syndrome children. Journal of Consulting and Clinical Psychology, 71, 516-527. Wright, J. C., & Zakriski, A. L. (in press). Reassessing the assessment of change: Conceptualizing behavior change in children as patterns of adaptation. Manuscript under review. Zakriski, A. L., Wright, J. C., & Underwood, M. K. (2005). Gender similarities and differences in children’s social behavior: Finding personality in contextualized patterns of adaptation. Journal of Personality and Social Psychology, 88, 844-855.

Approaches to Studying Change 58 Table 1. Means and repeated measures ANOVA results for TRF

Time 1

Time 2

Time 3

F1

p2

Flinear3

p

Anxiety

62.03

61.71

61.82

<1

ns

<1

ns

Withdrawal

61.82

61.37

61.09

<1

ns

1.69

>.1

Somatic Complaints

55.48

56.61

56.72

2.27

>.1

4.84

<.03

Social Problems

63.84

64.87

64.77

1.67

>.1

3.30

<.08

Thought Disorders

59.35

60.84

60.02

2.48

<.1

1.67

>.1

Attention Problems

59.33

60.58

59.44

3.12

<.06

<1

ns

RuleBreaking

62.31

63.92

63.08

4.59

<.02

2.78

<.1

Aggression

66.97

68.78

68.08

3.65

<.03

3.56

<.07

Internalizing

62.01

61.96

62.19

<1

ns

<1

ns

Externalizing

65.80

67.34

66.77

4.22

<.02

4.39

<.04

Total Problem Behavior

63.68

65.09

65.03

3.28

<.05

7.62

<.007

Note. 1 Overall Fs for within-subjects effects. Df=2, 276. 2 Greenhouse-Geisser adjustments were used for all p-values associated with repeated-measures effects. 3 Fs for linear contrasts testing for increase or decreases over time. Df=1, 138.

Approaches to Studying Change 59 Table 2. Aggressive reactions to events All Children

Interviewed Children

r2

b

F

p

r2

b

F

p

Peer prosocial

.29

-.21

14.22

<.001

.04

-.10

1.34

<.30

Adult prosocial

.35

-.22

18.49

<.001

.07

-.18

2.55

>.10

Peer ask/tell

.05

.05

<1

ns

0.0

-.01

<1

Ns

Adult instruct

0.0

.01

<1

ns

.01

.05

<1

Ns

Peer argue/tease

.12

.18

4.77

<.04

.08

.22

3.02

<.10

Adult warn/disc. .12

.14

4.55

<.04

.21

.30

9.22

<.005

Approaches to Studying Change 60 Table 3. Withdrawn reactions to events All Children

Interviewed Children

r2

b

F

p

r2

b

F

p

Peer prosocial

.21

-.14

8.96

<.005

.11

-.18

4.24

<.05

Adult prosocial

.37

-.20

19.71

<.001

.12

-.19

4.44

<.05

Peer ask/tell

.13

-.12

5.04

<.04

.07

-.21

2.44

>.1

Adult instruct

.19

-.14

7.88

<.008

.13

-.16

4.92

<.04

Peer argue/tease

.02

-.05

<1

ns

.01

-.05

<1

>.6

Adult warn/disc. .15

-.14

5.96

<.02

.26

-.30

11.78

<.002

Approaches to Studying Change 61 Table 4. Prosocial reactions to events All Children

Interviewed Children

r2

b

F

p

r2

b

F

p

Peer prosocial

.40

.24

23.00

<.001

.19

.21

8.05

<.008

Adult prosocial

.57

.29

45.40

<.001

.21

.25

9.17

<.005

Peer ask/tell

.01

.04

<1

ns

.04

.16

1.57

>.2

Adult instruct

.1

.08

3.77

<.06

.02

.07

<1

ns

Peer argue/tease

.14

-.20

5.40

<.03

.08

-.21

2.87

<.1

Adult warn/disc. .01

-.03

<1

ns

.01

-.05

<1

ns

Approaches to Studying Change 62 Table 5. Event rates All Children

Interviewed Children

r2

b

F

p

r2

b

F

p

Peer prosocial

.01

.03

<1

ns

.11

.23

4.22

<.05

Adult prosocial

.01

.04

<1

ns

0.0

.03

<1

ns

Peer ask/tell

.04

.08

1.41

>.2

.15

.21

5.94

<.02

Adult instruct

.17

-.22

7.16

<.02

.01

-.07

<1

ns

Peer tease/boss

.01

-.03

<1

ns

.07

-.15

2.36

>.1

.01

<1

ns

0.0

.03

<1

ns

Adult warn/disc. 0.0

Approaches to Studying Change 63 Figure Captions Figure 1. Mean Clinical Global Impressions (CGI) ratings. Error bars indicate +/- 1 SEM. Reference line indicates the midpoint of “no change.” (A) Adult ratings of all children; (B) Adult ratings for the sub-group of interviewed children; (C) Child interview ratings. Figure 2. Mean Teacher Report Form (TRF) scores for Aggression, Internalizing, Externalizing, and Total Problem Behavior, as a function of time. Error bars indicate +/- 1 SEM. Figure 3. Linear regressions (blue lines) and nonlinear regressions (loess, in black) for overall behavior rates. Error bars indicate 68% confidence intervals. Figure 4. Linear regressions (blue lines) and nonlinear regressions (loess, in black) for aggressive reactions to peer prosocial talk and adult warn/discipline. Error bars indicate 68% confidence intervals. Figure 5. Linear regressions (blue lines) and nonlinear regressions (loess, in black) for withdrawn reactions to peer prosocial talk and adult warn/discipline. Error bars indicate 68% confidence intervals. Figure 6. Linear regressions (blue lines) and nonlinear regressions (loess, in black) for prosocial reactions to peer prosocial talk and adult warn/discipline. Error bars indicate 68% confidence intervals. Figure 7. Children’s interview responses to vignettes. (A) Time 1; (B) Time 2. AGG = aggressive reactions; WDR = withdrawn reactions; PRO = prosocial reactions. Error bars indicate +/- 1 SEM.

Approaches to Studying Change 64 Figure 8. Children’s observed reactions to events at time points corresponding to interviews. (A) Time 1; (B) Time 2. AGG = aggressive reactions; WDR = withdrawn reactions; PRO = prosocial reactions. Error bars indicate +/- 1 SEM.

Figure 1. B) CGI Means – Adult Ratings for the Interviewed Children 7.00

6.00

6.00

5.00

5.00

Mean

7.00

4.00

4.00

3.00

3.00

2.00

2.00

1.00

1.00 global

adults

peers

agg

wdr

pro

overall

global

adults

CGI ITEM

peers

agg

CGI ITEM

C) CGI Means – Child Ratings

7.00

6.00

5.00

Mean

Mean

A) CGI Means – Adult Ratings of All Children

4.00

3.00

2.00

1.00 global

adults

peers

agg

CGI ITEM

wdr

pro

overall

wdr

pro

overall

Figure 2. Mean T Scores for TRF Scales

70

Mean T Score

68

Aggression Externalizing

66 Total Problems

64 62

Internalizing

60 Time 1

Time 2

Time 3

0 10 20

Days 30 40 -0.3

-0.3

-0.3

0.1

0.1

0 10 20

Days 30 40

0.1

0.3

0.3

0.3

Aggression

-0.1

Mean Z Score

-0.1

Mean Z Score

-0.1

Mean Z Score

Figure 3.

Withdrawal Prosocial

0 10 20

Days

30 40

Figure 4.

0.1 -0.3

-0.1

Mean Z Score

0.1 -0.1 -0.3

Mean Z Score

0.3

Aggression to Adult Warn/Discipline

0.3

Aggression to Peer Prosocial

0

10

20 Days

30

40

0

10

20 Days

30

40

Figure 5.

0.1 -0.3

-0.1

Mean Z Score

0.1 -0.1 -0.3

Mean Z Score

0.3

Withdrawal to Adult Warn/Discipline

0.3

Withdrawal to Peer Prosocial

0

10

20 Days

30

40

0

10

20 Days

30

40

Figure 6.

0.1 -0.3

-0.1

Mean Z Score

0.1 -0.1 -0.3

Mean Z Score

0.3

Prosocial to Adult Warn/Discipline

0.3

Prosocial to Peer Prosocial

0

10

20 Days

30

40

0

10

20 Days

30

40

Figure 7.

Responses to interview vignettes

Figure 7A: Interview Responses at Time 1 0.9 0.8

Mean Response

0.7 0.6 AGG

0.5

WDR 0.4

PRO

0.3 0.2 0.1 0 Argue/Tease

Warn/Disc.

Ask/Tell

Instruct

Vignette

Figure 7B: Interview Responses at Time 2 0.9 0.8

Mean Response

0.7 0.6 AGG

0.5

WDR 0.4

PRO

0.3 0.2 0.1 0 Argue/Tease

Warn/Disc.

Ask/Tell

Vignette

Instruct

Figure 8. Reactions to events at corresponding time points

Figure 8A: Palm Reactions at Time 1 0.45 0.4

Mean Response

0.35 0.3 AGG

0.25

WDR 0.2

PRO

0.15 0.1 0.05 0 Argue/Tease

Warn/Disc.

Ask/Tell

Instruct

Event

Figure 8B: Palm Reactions at Time 2 0.4 0.35

Mean Response

0.3 0.25 AGG WDR

0.2

PRO 0.15 0.1 0.05 0 Argue/Tease

Warn/Disc. Event

Ask/Tell

Instruct

Appendix A: WBOS Coding Manual

Overview of the WBOS

Page 2 of 16

Introduction to the 2007 Wediko Behavior Observation System Wediko is now in the second year of an NIH-funded project designed to improve our understanding of how children’s social interactions change over the summer, and how these changes are linked to their adjustment at home and school. The summer component of the project involves extensive observations of children’s interactions with peers and adults. Each day, over 70 PDAs, programmed with the schedule for that day, will be distributed to counselors and teachers. At the end of certain activities, staff will use the PDAs to record information about children’s interactions during that period. The research team will collect the PDAs each day, upload their data, and prepare them for the next day. Analyses of children’s behavior change will be provided to supervisors for possible review in clinical supervision meetings. The Wediko Behavior Observation System (or WBOS) has a long history. In the early 1980’s, staff completed weekly “checklists” on the children in their groups. This evolved into a system that focused on specific social events children experienced throughout the day (e.g., being teased) and how he or she responded to those events (e.g., withdraw, counterattack). A standardized version of the WBOS provided observations of over 600 children in a five-year study. Much of what we have learned and published about the behavior of aggressive, withdrawn, and “mixed” or comorbid children is based on these and related data collected elsewhere. The PDA version of the WBOS incorporates the strengths of the older system and avoids its weaknesses. At the end of an activity, the counselor provides information about events a target child encountered, how that child responded to those events, and how the other person involved in the interaction responded next. For example, suppose Tommy is the target child, and he was observed interacting with another child, Charlie. The interaction might look like this: Charlie teases Tommy Æ Tommy threatens Charlie Æ Charlie hits Tommy The PDA allows you to record this kind of information efficiently and quickly. It also obtains other information that is needed to interpret children’s behaviors. It does this by prompting you with the relevant questions and by providing the names of the children and counselors present in your activity. Your answers are recorded with simple “clicks” on the PDA screen and automatically stored for you. The best way to learn how to use the PDA is to practice with one. We will do this soon. In the meantime, this document provides an overview of the main elements of the system, including definitions of the codes and rating scales that will be used.

Overview of the WBOS

Page 3 of 16

Overview of Observation Schedule and Guidelines for PDA Use The observation schedule varies slightly over developmental programs. On Sundays with special activities, no data are collected. These and other details will be explained in program-specific meetings. For now, we provide a general overview of the schedule. 1. Pickup. At breakfast, designated counselors from each group or activity will pick up a WBOS fanny pack in Ames Hall. The pack will contain one or more PDAs. Each PDA has a unique ID number and is assigned to a specific group or activity. The PDAs cannot be exchanged. 2. Time of coding. Immediately following each “coded” activity, staff will complete a coding session for that period (see schedule below). It is important that the PDA coding not be done later in the day. If an emergency prevents you from completing the coding immediately, you should note this in the log book provided with the PDA. Do not try to “reconstruct” what happened later in the day, even if you think you remember. 3. Activities to be coded. A typical day is outlined at right. In general, children will be coded in two group activity periods, two sign-up periods, one Think City period, and in group therapy. Coding will be divided up among the adults present. This includes activity counselors, residential counselors (including lead staff), and Think City teachers. 4. Drop-off. By dinner, all PDAs should be returned to the research station in Ames Hall. Some staff may prefer to drop off their PDAs in the afternoon. Overnight, the PDAs will be prepared for another day of coding. If you cannot return your PDA at dinner, you should contact a member of the research team. It is difficult for the team to prepare the PDAs for the next day when they are returned later in the evening. 5. Guidelines for using PDAs. The PDAs are valuable and difficult to replace. We have done everything we can to protect them, but we need your help. Please follow the guidelines at left. Report any problems you have with the PDAs using the logbooks provided in your pack. The research team will respond to your question as soon as possible. Thank you in advance for your cooperation.

Overview of the WBOS

Page 4 of 16

6. Answering children’s questions about PDAs. Children may ask questions about the PDAs and may want to use them. If a child asks about what you are doing with a PDA, you should explain that we study how children interact with peers and adults to help us learn more about how children change over the summer. If the child wants more details, you could explain that you rate everyday behaviors like talking, arguing, teasing, and so on. If a child asks, you might also explain that the information in the PDA is used only to help us learn about children; it is never used to determine group or individual incentives, and is not part of children’s contracts or clinical checklists. All parents have been informed about the project and how the data will be used. Children must not be allowed to play with the PDAs. Games have been removed as far as possible and PDAs are difficult to replace if damaged. 7. Features of the Palm Z-22 ™. Most of the features of the Palm that you will need are shown below. The graffiti pad allows you to “write” using special characters that the Palm recognizes. This will be demonstrated to you in small-group sessions.

Stylus (behind)

Home Keyboard Power

Graffiti Pad (you can “write” here)

Overview of the WBOS

Page 5 of 16

Procedure for Completing a WBOS Session The best way to familiarize yourself with coding is by practicing with a PDA. You will have an opportunity to do that later. This overview will get you oriented and provide a reference that you can use later. Please note the screens on the PDAs may look slightly different from the reproductions below. The balloon callouts will not appear on your PDA. Step 1: Starting a session. On your PDA, click the “W” icon to launch the WBOS program. Page 1 shows the beginning of a session. It shows the day the PDA is prepared to code. The PDA can only be used for the “sign up” date indicated. Please check to make sure the date is correct. On page 2, enter your user name and password. Returning staff will note that this step is simpler and faster than it was last year. The “graffiti” pad at the bottom of the screen also allows you to write your password more quickly. This will be demonstrated in small group sessions. Page 3 asks you to start a new activity or resume an activity that is already started. In the example shown, Think City coding has already been started; if you were in that activity, you could resume coding for it. To start coding a new activity, select “New” at the bottom. You will proceed to the next page.

Page 4 shows the activities you (the coder, or “Adam—Otters (A)” in this example) are

expected to attend for this day. For returning staff, this step has also been simplified to make it faster. Simply select the activity you are going to code. If the activity you need does not appear in the list, select “Other” at the bottom. If you do this, you will be prompted for details about this activity on the next screen.

Overview of the WBOS

Page 6 of 16

Page 4b appears if you click “Other” on the previous page. Often, you will not need this page because you selected a scheduled activity on the previous screen. If you did pick “Other”, you will need to specify several fields using drop-down menus: the Program, Slot (time period of the activity), the Activity Type, and the specific Activity. The options that will appear in certain menus will depend on previous options you have selected. For example, the activity schedule for Crossroads is different from the one for Mastery. Returning staff will note that this screen closely resembles the main login page from last year. Once you have selected an activity, on Page 5 you will select the child who will be the focus of your coding. This is known as the “target” child. The (C) indicates the person is a child. This helps you keep children separate from adults, who will be labeled with an (A). If the child you need to code does not appear in the list, select “Add” to add that child. Once a child has been coded, he/she is marked “done” on this screen and cannot be coded again for this activity. Step 2: Overall assessment of target child’s behaviors. Page 6 is an instruction page; it does not require any information. Note that you will rate how often the target child showed certain behaviors during the activity period using the 0-3 scale shown. On Page 7 the name of the target child appears at the top and a list of 8 behaviors appears. Select the box that corresponds to the rating that best describes the child’s behavior during this period. The PDA will require an answer to each item. Note that the code labels are short due to the small screen size. It is important to use the complete code definitions that appear later in this document.

Overview of the WBOS

Page 7 of 16

Step 3: Overall assessment of social events that happened to the target child. Page 8 is an instruction page similar to the one you saw earlier. Page 9 asks you to rate how often 4 PEER events and 4 ADULT events happened to the target child. As before, the target child is identified at the top of the page. Remember, your ratings should describe events that occurred during this period. The code labels are short because the screen is small. It is essential to use the complete definitions of the events are provided later in this manual.

The PDA will require a rating for each of the events (0, 1, 2, or 3). It will also require that at least 2 events get a rating of “1” or higher. You will note that list includes events that commonly occur in children’s interactions with peers and adults. If you did not observe at least 2 of these events, the PDA will instruct you to abort the coding session for the target child and find another staff who can code him or her. If this is not possible, you will be instructed to make a note in your PDA logbook. The research team will contact you about these aborted sessions. Once you have made your ratings of the events, there may be a slight pause in the PDA while it prepares material for the next page. Please wait if this happens. Do not click with the stylus or turn the screen off. Step 4: Behavior chains. The PDA will choose which event to probe based on your previous answers. It will then ask you to rate how the child responded to that event. Here, we assume the PDA selected being “Teased / bossed” by a peer. Page 10 asks you to indicate which peer “teased / bossed” the target. Again, note that this is a brief code label; full definitions appear later in this manual. If this event happened more than once, focus on the event you most clearly remember. Typically, this is the most recent event—that is, the one closest to the time of coding. Here, assume you selected Joey as the child that “teased / bossed” Allen. Page 11 is an instruction page that explains the scale used to rate the target child’s reactions.

Overview of the WBOS

Page 8 of 16

Page 12 asks you to rate how the target child (Allen) reacted when Joey “teased / bossed” him. “React” means the most salient behavior(s) the child displayed in response within a minute or so after the event occurred. So, if Allen “teased / bossed” by Joey, then Allen “hit / pushed / attacked” Joey within a minute or so after the tease occurred, then you would record that reaction by Allen. If Allen pushed Joey 30 minutes after the tease occurred, this should not be coded as a reaction (even if you believe Allen was “getting back” at the other child). Note that a child may react with more than one behavior to an event. Also note that “0” does not appear. If a reaction occurred, give it a rating of 1, 2, or 3. If it did not occur, nothing is checked. Page 13 asks you to rate how the other child (Joey) responded after the target (Allen) showed the behaviors described on the last page. The rules are

the same as before: A response must occur within a minute or so after the target behaviors described on the last page. Also note that the scale is the same as the one used on the last page. Step 4: Behavior chains, continued. The remaining pages in this session repeat the 4 preceding pages. In this example, the PDA has selected an adult event (“gave instructions”). You will then identify the adult who most recently gave instructions to the target. Next, the instruction page will appear again. As before, you will be asked to rate how the target child responded when the counselor showed the behavior in question. Again, if a particular response did not occur, nothing is checked for that response.

Overview of the WBOS

Page 9 of 16

As the final step in the chain, rate how the counselor responded to the behaviors of the target child. The coding rules for counselors’ responses are the same as the rules for children’s responses.

After two behavior chains have been coded, the PDA will return you to Page 5. The child you just completed will be marked as “done”, and you will either select another child or quit the coding session. When you quit, data you have entered are automatically stored in the PDA until it is docked by the research team.

Overview of the WBOS

Page 10 of 16

Code Definitions This section provides code definitions and examples. The following general points about the codes should be noted: 1. Some codes are used more than once during the coding process (see above description of behavior chains). The codes are grouped below based on where they are used in the coding system (e.g., target child behaviors, social event ratings, peer or adult responses to child in behavior chains). 2. Some codes are defined separately for peers and adults even though they are related. For example, to preserve role distinctions between adults and children, “ask / told” is used for peers, and “gave instructions” is used for adults. 3. Some codes have two or more parts, divided by a slash (e.g., “tease / boss”, “warn / reprimand”). For these codes, if either part of the code occurs, then code the behavior as an instance of the code. 4. A single “burst” of behavior may be coded with one or more codes, depending on the behavior. For example, a child might say, “If you don’t shut up you’ll regret it!” and then pushes the other child, all within a few seconds. This includes a threat (so code as “tease / boss / threat”) and a push (so code as “hit / push / attack”).

Target child and peer behaviors used throughout the WBOS “Talked prosocially” This code covers a range of positive verbal behaviors for children. It refers to age-appropriate attempts to engage in conversation. This can include non-aversive statements, questions, providing information to others, requesting information, and making non-hostile comments about activities, events, or people. “Talked prosocially” emphasizes normal everyday conversing, without an attempt to achieve a particular behavioral response from the target. Note that “ask / tell” would be used when a child asks or tells someone to do something other than talk. Examples John makes a comment to another child about how many fish he caught. Jamal tries to explain to an adult why he was in the off-limits area. Susan asks an adult if she could use some activity supplies. Devon says, “Can you tell me what you know about today’s activities?” This would be coded as talked prosocially, as it is primarily conversational. Non-examples Steve talks or mumbles to himself rather than another person. Crystal argues with another child about who will be first in line (code as “argued / quarreled”) Devon says, “Can you hand me that ball?” Code as “ask / tell”, as it emphasizes a specific action to be performed. “Complied / cooperated” Used when a child acts in accordance with expectations or appropriate requests made by peers or adults. This includes adherence to daily routine without prompting or in response to an instruction or reminder. Yielding or submissive behavior in response to peer bullying is NOT coded as comply/cooperate. Examples When a counselor asks Mika to get back in line, Mika says OK and gets back in line. Davon returns from swim and hands up his towel in response to a reminder. At the end of lunch, Jake starts stacking the plates and clearing the table without being asked. Sasha hands the ball to Crystal when Crystal asks her for it. Non-examples John throws a rock at the cabin after a peer told him to do it in a threatening way. (Code as “yielded / gave in”.) James told another kid to hang up his towel. (Code as “asked / told”.) “Hit / pushed / attacked” Used when a child harms or attempts to harm someone physically, or when a child attempts to physically control someone in a rough or hostile way. Clear attempts to inflict harm should be coded

Overview of the WBOS

Page 11 of 16

with this code, even if the attempt failed (e.g., the child throws an object at someone, but misses). Accidental physical contact, even if harm results, does not fit this code. Examples John pushes another child in line in a rough way. Susan elbows another child while she is being escorted out of the activity. Susan swings at another girl but misses. Non-examples John and another child are engaged in rough-and-tumble play, with no harm intended. Susan rushes out of the door and bumps into another child by accident. An adult restrains a child during an intervention (code as “physically controlled”) “Teased / bossed.” Tease refers to verbal or non-verbal behavior that is intended to taunt, pester, provoke, or embarrass someone. Non-verbal teasing includes mocking or provoking facial or hand gestures. Boss refers to a child commanding someone to do something in a coercive way. If a threat or implied threat occurs, code as boss even if the threat is not carried out. Direct instructions that are not coercive should be coded as “ask / tell”. Examples John says, “You’re a stupid jerk!” Susan mimics another child’s voice in a sarcastic way. John says, “You’d better get off my bed or else!” Susan says, “Get out of my way! Non-examples In a friendly, clearly joking way, John says, “You goofball.” (Code as “talked prosocially.”) In a helpful way, John says, “That’s not how the lock works.” (Code as “talked prosocially.”) In a non-coercive way, Devon says, “Hand me the red bow next to you.” (Code as “asked / told”.) “Argued / quarreled.” Use when the child verbally resists instructions, requests, or statements by others in an oppositional manner. To be coded as such, the behavior must be contrary and quarrelsome. Expression of dissatisfaction with something in an age-appropriate, non-oppositional way should not be coded with this code. Examples When clean-up time is announced, John says, “I’m not gonna clean up my space.” When given a time out, Susan says, “I’m not taking any stinking time out.” When a child asks for the ball, John says, “I’m not sharing anything with you.” Non-examples John says, “If you don’t leave me alone, I’m gonna get you!”. (Code as “teased / bossed”.) When told to take a time out, Susan throws something at the adult. (Code as “hit / pushed / attacked”.) When told to swim, John says, “I can’t because my ear is infected.” (Code as “talked prosocially”.) “Whined / complained.” This code is used when the child uses a babyish, plaintive, or high-pitched voice that is age-inappropriate or immature. The code is also used when the child fusses, gripes, or expresses dissatisfaction, discontent, or resentment in a tiresome way. Examples When swim time is announced, John says plaintively, “I don’t waaaaaaaaaaaaaana go to swim.” At lunchtime, Susan says in a babyish voice, “We never have anything good to eat in this place.” After a minor scratch, John fusses in an exaggerated way about needing to go see the nurse. Non-examples John argues, but not in a babyish way, with an adult about an “unfair” rule. (Code as “argued / quarreled”.) Susan talks about how sad she is that she won’t have visitors on visitors’ day. (Code as “talked prosocially”.)

Overview of the WBOS

Page 12 of 16

“Looked sad / cried.” Looked sad occurs when a child shows non-verbal expressions of unhappiness or grief, appears downcast, somber, or depressed. Cried refers to shedding tears out of sadness, sorrow, or being hurt, either psychologically or physically. Examples Susan begins to cry when she learns that she won’t have visitors on visitors’ day. John’s shoulders slump and he looks down dejectedly when no one picks him for their athletic games. Non-examples Susan cries tears of joy when she learns that her mother will arrive on visitors’ day. You think a child may be feeling sad about a letter he got from home, even though there is no immediate expression of sadness. “Withdrew / isolated.” Used when a child avoids contact with others, physically isolates self from others, or pulls back when someone attempts to interact with him/her. The pulling back from social contact is more important than the physical distance per se. A child can withdraw from interactions even when peers are playing nearby. Examples John sits on the periphery of the group during an activity and does nothing. Susan sits on her bed during free time and stares at the ceiling of the cabin. When another child asks John to play a game, he returns to reading his comic book. When praised by an adult, Susan says nothing and turns away. Non-examples John sits on his bunk during quite time, as instructed by adults. (Code as “comply / cooperate”) Susan works on her clay pot by herself while other children do the same.

Additional codes used for social event ratings and behavior chains “Asked / told.” Used when a child makes a request or tells a peer to do something in a noncoercive way. It can refer to a request for specific help or specific information from a peer, or a more controlling action such as telling a child to follow the rules. A peer asking another child to do something in a coercive way would be coded as peer boss/tease (see above). (See “gave instructions” for the adult version of this code.) See also “talked prosocially” for distinctions from that code. “Asked / told” emphasizes a request for a specific action; “talked prosocially” emphasizes normal conversation, without a request for a specific action. Examples Wendy says to a peer, “Can you get off my bunk?” Gabriella says to another child, “You better hurry up or we will be late for Think City.” A child asks, “How do I get to Ames Hall?” (A specific action or type of information is requested.) Non-examples Maria demands, “You’d better get off my bed or else!” (Code as peer “teased/bossed”.) Bobby shouts, “Get out of my way!” (Code as peer “teased/bossed”.) Wanda asks another peer, “Do you like my art project?” (Code as peer “talked prosocially”,) “Gave instructions.” Use when an adult tells an individual child to do something, either immediately or in the very near future. This code does not refer to instructions about how to do something (e.g., how to bait a hook) or general suggestions with no specific action (e.g., telling a child to work hard on having a good summer). Instructions are distinct from warnings (see below). Imposing a time out is handled by a separate code (see disciplined/punished). Examples An adult says to a specific child, “Please go sit on your bunk.” An adult says, “Joey, let Nathan have a turn now.” An adult requests, “Sasha, tell the group the best thing that you did today in activities.”

Overview of the WBOS

Page 13 of 16

Non-examples “If you don’t stop talking, you’ll get a time out.” (Code as “warned/reprimanded”.) “Go take a time out on the rock.” (Code as “disciplined/punished”.) “Think about your goals, Danny, so you can make good choices today” (Code as “talked prosocially”.) “[Adult] Talked Prosocially” This code covers a range of positive verbal behavior directed at the target child by adults. Prosocial talk refers to positive, constructive, non-aversive statements or questions, including: providing information to others, requesting information from others, making comments about activities, events, or people, or otherwise attempting to engage in conversation with others. Examples An adult tells a child he did a good job in cleaning up his bunk. An adult asks a child if he has ever used a kayak before. An adult describes the daily schedule to a child. Non-examples An adult instructs a child to stop pushing in line (Code as “gave instructions”.) An adult tells the group what a good job they did in group therapy today. (Not directed to an individual child.) “Warned / reprimanded.” Used when an adult states to a particular child that some negative consequence will follow if the child does something. Warn may be used if the contingency is implicit, provided it is still clear to the child. Reprimand refers to an adult scolding, admonishing, or telling the child that some behavior is inappropriate. Examples “If you don’t do your chore, you will miss first activity.” “You’d better be out of the water and on the bench by 10.” (A consequence is implied, but clear at swimtime) “It’s not OK to tease, Joey.” Non-examples “Gerald, you need to clean up now and get ready for the next activity.” (Code as “gave instructions”.) “The Walkers are clearly showing me that they aren’t ready to go to the WIT social tonight.” “Disciplined / punished.” This code refers to adults giving a consequence to a particular child for some behavior, including loss of privileges, a time out, or time away from the group. The consequence must be clear and specific and declared at the time. It does not include physical restraint (see “physically controlled”). However, if adult imposes a consequence during or after a restraint, that would be coded as “disciplined/punished.” Examples An adult says, “You need to sit on the time out rock for 1 minute.” An adult tells the child, “Because you could not stop arguing, you must give up your turn.” An adult says, “Because you were 10 minutes late, you’ll have to sit out for 10 minutes of our next activity.” Non-examples An adult restrains a child who is physically attacking another child. (Code as “physically controlled”.) An adult announces to the group, “Because the group was so rowdy, we will all have a minute of quiet time.”

Codes used exclusively in behavior chains “Ignored / no response.” Used to indicate that an interactant (peer or adult) intentionally disregarded, did not acknowledge, or did not respond to a behavior of the target child. Note that ignore does not refer to situations in which it is unclear whether the individual heard or saw the child do the behavior that is being “ignored”. Examples An adult purposefully did not acknowledge Jerry when he argued about cleaning up. A peer did not say or do anything when Denisha asked to have the crayons.

Overview of the WBOS

Page 14 of 16

Non-examples An adult says, “I can’t hear you when you talk to me in that whiny voice.” (code as “warn / reprimand”) A peer smiles but says nothing when Vivian helps her tie her shoe. (code peer as “showing positive emotion”) “Showed positive emotion.” Use to indicate a nonverbal affective response showing warmth, affection, connection, enjoyment, pleasure, etc. The positive emotion must be directed at the target child being coded. Examples A peer responds to Greg’s helpful behavior by smiling. An adult responds to Iliana’s cooperative behavior during clean-up by giving her a high-five. A peer responds to Kyle’s cooperation by doing a celebration dance. Non-examples A peer laughs when Henry gives in to his teasing and starts to cry. (code peer as “teased/bossed”) An adult says to the Falcons, “I am so happy to report that everyone has earned the incentive.” “Yielded/gave in” This code should be used to indicate a submissive response by a peer to a controlling behavior by a child. It should also be used to indicate that an adult or peer has abandoned a recent request or intention. This abandonment is usually the result of noncooperative behavior by the target child. This could include abandoning attempts to redirect a child or not following through on stated consequence (not recommended practice). Examples Sammy gives the ball to a peer who was threatening to fight him for it. Tiana stops trying to defend herself to a peer falsely accusing her of stealing and gives her the candy. An adult who warned Ursula to get down notices her still on the rock but doesn’t do anything about it. Exasperated, an adult gives up on her attempts to redirect Vinny’s rude behavior at lunch. Non-examples An adult ignores Jerry when the child complains about clean up time. (code adult as “ignored/no response”) Denisha lets a peer go first at archery because she didn’t get a turn last time. (code peer “comply / cooperate”) “Physically controlled” This code should be used to indicate adult use of physical intervention or restraint. This includes any physical intervention with a child designed to de-escalate, avoid a potential crisis, or keep a child safe. It can include an adult physically guiding a child away from an escalating argument with a peer, blocking a child from someone with his/her body, basket-holding, or the use of other holding techniques. The physical control must be direct and be targeted at a specific child. Examples An adult takes Jamiel by the arm and walks him away from a heated argument with a peer. An adult uses a basket-hold on Kevin after the child hits someone to help him calm down and control his body. Non-examples After two children start fighting, an adult moves the rest of the group along so others don’t get involved. An adult says, “If you don’t put the rock down I will have to hold your hands.” (code as “warned/disciplined”)

Overview of the WBOS

Page 15 of 16

Quick Guide to Codes Codes used for target child and peer behaviors “Talked prosocially” Covers a range of positive verbal behaviors, including age-appropriate attempts to engage in conversation. Emphasizes normal everyday conversing, without an attempt to achieve particular behavioral response. Use “ask / tell” when child asks or tells someone to do something other than talk. “Complied / cooperated” Used when child acts in accordance with expectations or appropriate requests made by peers or adults. Includes adherence to daily routine without prompting or in response to an instruction or reminder. Yielding or submissive behavior in response to peer bullying is NOT coded as comply/cooperate. “Hit / pushed / attacked” Used when a child harms or attempts to harm someone physically, or when a child attempts to physically control someone in a rough or hostile way. Clear attempts to inflict harm should be coded with this code, even if the attempt failed. Accidental physical contact does not fit this code. “Teased / bossed.” Tease covers behavior that taunts, pesters, provokes, or embarrasses someone, including nonverbal facial or hand gestures. Boss covers commanding someone to do something in a coercive way. If a threat or implied threat occurs, code as “teased / bossed”. Use “ask / tell” for instructions that are not coercive. “Argued / quarreled.” Use when child verbally resists instructions, requests, or statements by others in an oppositional manner. Behavior must be contrary and quarrelsome. Expression of dissatisfaction with something in an age-appropriate, non-oppositional way should not be coded with this code. “Whined / complained.” Refers to child using a babyish, plaintive, or high-pitched voice that is age-inappropriate or immature. Code is also used when the child fusses, gripes, or expresses dissatisfaction, discontent, or resentment in a tiresome way. “Looked sad / cried.” Looked sad occurs when a child shows non-verbal expressions of unhappiness or grief, appears downcast, somber, or depressed. Cried refers to shedding tears out of sadness, sorrow, or being hurt, either psychologically or physically. “Withdrew / isolated.” Use when a child avoids contact with others, physically isolates self from others, or pulls back when someone attempts to interact with him/her. Pulling back from social contact is more important than physical distance per se. Child can withdraw from interactions even when peers are playing nearby.

Additional codes used for social event ratings and behavior chains “Asked / told.” Use when child makes request or tells a peer to do something in a noncoercive way. See “tease / boss” when child tells someone to do something in a coercive way. See “gave instructions” for adult version of this code. See “talked prosocially” for attempts to engage in normal conversation, without a request for a specific action. “Gave instructions.” Use when adult tells an individual child to do something, either immediately or in the very near future. Does not refer to instructions about how to do something or general suggestions with no specific action. Instructions are distinct from warnings (see below). See “disciplined / punished for time outs. “[Adult] Talked Prosocially” Covers a range of positive verbal behaviors, including attempts to engage in conversation. Emphasizes normal everyday conversing, without an attempt to achieve particular behavioral response. Use “ask / tell” when adult asks or tells child to perform specific action. “Warned / reprimanded.” Used when an adult states to a particular child that some negative consequence will follow if the child does something. Warn may be used if the contingency is implicit, provided it is still clear to the child. Reprimand refers to an adult scolding, admonishing, or telling the child that some behavior is inappropriate.

Overview of the WBOS

Page 16 of 16

“Disciplined / punished.” Use when adult gives a consequence to a particular child for some behavior, including loss of privileges, a time out, or time away from the group. The consequence must be clear and specific and declared at the time. It does not include physical restraint (see “physically controlled”).

Codes used exclusively in behavior chains “Ignored / no response.” Used to indicate that an interactant (peer or adult) intentionally disregarded, did not acknowledge, or did not respond to a behavior of the target child. Does not refer to situations in which it is unclear whether the individual heard or saw the child do the behavior that is being “ignored”. “Showed positive emotion.” Used to indicate a nonverbal affective response showing warmth, affection, connection, enjoyment, or pleasure. The positive emotion must be directed at the target child being coded. “Yielded/gave in” Used to indicate submissive response by a peer to a controlling behavior by a child. Also used to indicate an adult or peer has abandoned a recent request or intention, usually as a result of noncooperative behavior by the target child. Includes not following through on stated consequence. “Physically controlled” Used when adult employs physical intervention or restraint. Can include an adult physically guiding child away from a peer, blocking child from someone with his/her body, basket-holding, or the use of other holding techniques. The physical control must be direct and be targeted at a specific child.

Appendix B: Modified Clinical Global Impressions of Change (CGI)

Rater name: _________________________ Group name: ___________ ________ Date: ______

Clinical Global Impressions Please rate each child based on all of your knowledge of him/her to help us evaluate overall change over the course of treatment. Please read the scale point labels carefully and indicate your rating by circling. OVERALL ASSESSMENT 1. Over the course of the summer, how much has this child changed in his/her overall adjustment? Very much Worse

1 1 1 1 1 1 1 1

Much Worse

2 2 2 2 2 2 2 2

Minimally Worse

3 3 3 3 3 3 3 3

No Change

4 4 4 4 4 4 4 4

Minimally Improved

5 5 5 5 5 5 5 5

Much Improved

6 6 6 6 6 6 6 6

Very Much Improved

7 7 7 7 7 7 7 7

SOCIAL ASSESSMENT 2. Over the course of the summer, how much has this child changed in his/her relationships and interactions with adults? Very much Worse

1 1 1 1 1 1 1 1

Much Worse

2 2 2 2 2 2 2 2

Minimally Worse

3 3 3 3 3 3 3 3

No Change

4 4 4 4 4 4 4 4

Minimally Improved

5 5 5 5 5 5 5 5

Much Improved

6 6 6 6 6 6 6 6

Very Much Improved

7 7 7 7 7 7 7 7

3. Over the course of the summer, how much has this child changed in his/her relationships and interactions with peers? Very much Worse

1 1 1 1 1 1 1 1

Much Worse

2 2 2 2 2 2 2 2

Minimally Worse

3 3 3 3 3 3 3 3

No Change

4 4 4 4 4 4 4 4

Minimally Improved

5 5 5 5 5 5 5 5

Much Improved

6 6 6 6 6 6 6 6

Very Much Improved

7 7 7 7 7 7 7 7

BEHAVIORAL ASSESSMENT Please rate behavioral change for each child based on all of your knowledge of him/her over the entire summer. Please read the scale points carefully and indicate your rating by circling. For the purposes of this assessment please use the following definitions: Aggressive behavior refers to arguing, teasing, bossing, bullying, threatening, pushing, and/or hitting. Withdrawn behavior refers to isolating self from others, looking sad, and/or crying. Prosocial/friendly behavior refers to talking, attending, and/or showing positive emotion to others. 4. Over the course of the summer, how much has this child changed in his/her aggressive behavior? Very much Increased

1 1 1 1 1 1 1 1

Much Increased

2 2 2 2 2 2 2 2

Minimally Increased

3 3 3 3 3 3 3 3

No Change

4 4 4 4 4 4 4 4

Minimally Much Decreased Decreased

5 5 5 5 5 5 5 5

6 6 6 6 6 6 6 6

Very Much Decreased

7 7 7 7 7 7 7 7

5. Over the course of the summer, how much has this child changed in his/her withdrawn behavior? Very much Increased

1 1 1 1 1 1 1 1

Much Increased

2 2 2 2 2 2 2 2

Minimally Increased

3 3 3 3 3 3 3 3

No Change

4 4 4 4 4 4 4 4

Minimally Much Decreased Decreased

5 5 5 5 5 5 5 5

6 6 6 6 6 6 6 6

Very Much Decreased

7 7 7 7 7 7 7 7

6. Over the course of the summer, how much has this child changed in his/her prosocial/friendly behavior? Very much Increased

1 1 1 1 1 1 1 1

Much Increased

2 2 2 2 2 2 2 2

Minimally Increased

3 3 3 3 3 3 3 3

No Change

4 4 4 4 4 4 4 4

Minimally Much Decreased Decreased

5 5 5 5 5 5 5 5

6 6 6 6 6 6 6 6

Very Much Decreased

7 7 7 7 7 7 7 7

Appendix C: Child Interview Protocol

Child Interview Protocol

Interviewer: _______________________ Recorded by: _______________________

Child first name: _________________ Child group: ____________________

Date: ___________________

Time start: ___________________ Time end: ____________________

Instructions to Interviewer Please read the instructions that appear below to the student you are interviewing. Instructions to Student “I am going to tell you about things kids and adults might say or do to you. Then I’m going to ask you how those things might make you feel and what you might do next. My helper will write down what you say, but we are not going to show your answers to any kids or other staff members in your group. “There are no right or wrong answers to any of the questions I will ask you. I really want to learn more about what you think and feel. “Do you have any questions?”

Instructions to Interviewer: Note child’s questions and your answers, then continue to the next page.

Child Interview Protocol Interviewer: _______________________

Child first name: _________________

Vignette 1 Imagine you’re with your group at an activity you like. You’re sitting at the table waiting for the activity to start. Another kid comes over to you and says, “Hey loser. You better get out of that seat or else.” If a kid/student did something like this to you, how would it make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Why would a kid/student do something like this to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ If a kid/student did something like this to you, what would you do next? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say you did the things you just said you would do (Interviewer: Summarize response to previous question…). What would the other kid/student do next to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say the other kid did the things you just said he/she would do (Interviewer: summarize response to previous question…). How would that make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________

Child Interview Protocol Interviewer: _______________________

Child first name: _________________

Vignette 2 Imagine you’re with your group at an activity you like. You say something to another kid while an adult is explaining the rules. The adult comes over to you and says, “If you don’t stop talking, you’ll lose your turn.” If an adult did something like this to you, how would it make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Why would an adult do something like this to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ If an adult did something like this to you, what would you do next? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say you did the things you just said you would do (Interviewer: Summarize response to previous question…). What would the adult do next to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say the adult did the things you just said the adult would do (Interviewer: Summarize response to previous question…). How would that make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________

Child Interview Protocol Interviewer: _______________________

Child first name: _________________

Vignette 3 Imagine you’re with your group at an activity you enjoy. Your things are spread out all around you. Another kid comes over to you and says, “Could you move your things out of my way?” If a kid/student did something like this to you, how would it make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Why would a kid/student do something like this to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ If a kid/student did something like this to you, what would you do next? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say you did the things you just said you would do (Interviewer: Summarize response to previous question…). What would the other kid/student do next to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say the other kid did the things you just said he/she would do (Interviewer: Summarize response to previous question…). How would that make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________

Child Interview Protocol Interviewer: _______________________

Child first name: _________________

Vignette 4 Imagine you’re with your group at an activity you enjoy. You aren’t ready to leave yet. An adult comes over to you and says, “Please put all your things away and clean up your area right now.” If an adult did something like this to you, how would it make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Why would an adult do something like this to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ If an adult did something like this to you, what would you do next? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say you did the things you just said you would do (Interviewer: Summarize response to previous question…). What would the adult do next to you? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________ Let’s say the adult did the things you just said the adult would do (Interviewer: Summarize response to previous question…). How would that make you feel? (Interviewer: Prompt as necessary to elicit details.) ______________________________________________________________________________________ ______________________________________________________________________________________

Overall Event Ratings

Instructions to student “Now I am going to ask you how often kids and adults have done certain things to you over the last two weeks. “To help you answer, we’re going to use this scale (display scale). For example, suppose I ask you, “Over the last two weeks, how often was it noisy in the dining hall during mealtime?” You could answer by saying “almost never”, “sometimes”, “often”, “very often,” or “almost always.” You could say the answer or you could point to the answer. “Do you have any questions?” Instructions to interviewer: Note child’s questions and your answers here, then begin reading the questions. 1. Over the last two weeks, how often have other kids teased, bossed, or threatened you? Almost never

Sometimes

Often

Very Often

Almost Always

2. Over the last two weeks, how often have other kids argued or quarreled with you? Almost never

Sometimes

Often

Very Often

Almost Always

3. Over the last two weeks, how often have other kids asked or told you to do something? Almost never

Sometimes

Often

Very Often

Almost Always

4. Over the last two weeks, how often have other kids talked to you in a friendly or nice way? Almost never

Sometimes

Often

Very Often

Almost Always

5. Over the last two weeks, how often have adults disciplined or punished you? Almost never

Sometimes

Often

Very Often

Almost Always

6. Over the last two weeks, how often have adults warned or scolded you? Almost never

Sometimes

Often

Very Often

Almost Always

7. Over the last two weeks, how often have adults asked or told you to do something? Almost never

Sometimes

Often

Very Often

Almost Always

8. Over the last two weeks, how often have adults talked to you in a friendly or nice way? Almost never

Sometimes

Often

Very Often

Almost Always

Overall Behavior Ratings

Instructions to student “Now I am going to ask you how often you have done certain things over the last two weeks. You can answer the same way you did before, using the same scale. “Do you have any questions?” Instructions to interviewer: Note child’s questions and your answers here, then begin reading the questions. 1. Over the last two weeks, how often have you stayed by yourself or stayed away from other people? Almost never

Sometimes

Often

Very Often

Almost Always

2. Over the last two weeks, how often have you looked sad or cried when other people could see or hear you? Almost never

Sometimes

Often

Very Often

Almost Always

3. Over the last two weeks, how often have you whined or complained to other people? Almost never

Sometimes

Often

Very Often

Almost Always

4. Over the last two weeks, how often have you argued or quarreled with other people? Almost never

Sometimes

Often

Very Often

Almost Always

5. Over the last two weeks, how often have you teased, bossed, or threatened other people? Almost never

Sometimes

Often

Very Often

Almost Always

6. Over the last two weeks, how often have you hit, pushed, or attacked other people? Almost never

Sometimes

Often

Very Often

Almost Always

7. Over the last two weeks, how often have you talked to other people in a friendly or nice way? Almost never

Sometimes

Often

Very Often

Almost Always

8. Over the last two weeks, how often have you cooperated with other people or done something you were asked to do? Almost never

Sometimes

Often

Very Often

Almost Always

Perceptions of Change Instructions to student “Now I’m going to ask you to think about how your behavior may have changed this summer. “To help you answer, we’re going to use this scale (display scale). For example, suppose I ask you, “Since you came to Wediko, how has your swimming changed?” You could answer by picking one of these answers (point to scale). You could say: “Very much worse”, “Much worse”, “A little worse”, “No change”, “A little improved,” “Much improved”, or “Very much improved”. You could say the answer or you could point to the answer. “Do you have any questions?” Instructions to interviewer: Note child’s questions and your answers here, then begin reading the questions. 1. Since you came to Wediko this summer, how have you changed in your overall behavior? VMW MW LW

NC

LI

MI

VMI

2. Since you came to Wediko this summer, how have you changed in how you get along with adults? VMW MW LW

NC

LI

MI

VMI

3. Since you came to Wediko this summer, how have you changed in how you get along with other kids? VMW MW LW

NC

LI

MI

VMI

4. Since you came to Wediko this summer, how have you changed in behaviors like arguing, teasing, bossing, threatening, pushing, or hitting? VMW MW LW

NC

LI

MI

VMI

5. Since you came to Wediko this summer, how much have you changed in behaviors like staying by yourself, staying away from people, looking sad, or crying? VMW MW LW

NC

LI

MI

VMI

6. Since you came to Wediko this summer, how much have you changed in behaviors like being friendly, talking to people, or showing positive feelings to other people? VMW MW LW

NC

LI

MI

VMI

Appendix D: Teacher Report Form (TRF)

An electronic copy of the TRF is not available.