Psychological Medicine, 2003, 33, 623–635. f 2003 Cambridge University Press DOI : 10.1017/S0033291703007669 Printed in the United Kingdom

The Social Phobia Diagnostic Questionnaire : preliminary validation of a new self-report diagnostic measure of social phobia M. G. N E W M A N,1 K. E. K A C H I N , A. R. Z U E L L I G , M. J. C O N S T A N T I N O A N D L. C A S H M A N-M C G R A T H From the Department of Psychology, The Pennsylvania State University, University Park, PA, USA

ABSTRACT Background. The development and validation of the Social Phobia Diagnostic Questionnaire (SPDQ), a new self-report diagnostic instrument for social phobia is described in three separate studies. Study 1. The participants were 125 undergraduates seeking help for an anxiety disorder of whom 60 had social phobia. Receiver operating characteristics (ROC) analysis was conducted comparing SPDQ diagnoses and clinician-based Anxiety Disorder Interview Schedule-IV (ADIS-IV) diagnoses of social phobia. Diagnoses made by the SPDQ showed an 85 % specificity, an 82% sensitivity and kappa agreement with the ADIS-IV of 0.66. Study 2. The participants were 462 undergraduates who completed the SPDQ and a battery of additional questionnaires. The SPDQ had good internal consistency (a=0.95), good split-half reliability (r=0.90) and strong convergent and discriminant validity. Study 3. The participants were 145 undergraduates who completed the SPDQ at two time points separated by 2 weeks as well as several additional questionnaires. Scores on the SAD, FNE and SISST of SPDQ categorized undergraduates were also compared to scores on these measures from 35 clinical community participants to determine the clinical validity of the SPDQ. The SPDQ had strong 2-week test–retest reliability and good convergent and discriminant validity. Undergraduates diagnosed with social phobia by the SPDQ were not significantly different on the SAD, FNE and SISST from the socially phobic community sample, but both groups had significantly higher scores than undergraduates identified by the SPDQ as not meeting criteria for social phobia, demonstrating clinical validity of the SPDQ. Conclusions. These three studies provide preliminary evidence of the strong psychometric properties of the SPDQ as a measure to identify socially phobic participants.

INTRODUCTION The publication of the third edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III) (American Psychiatric Association, 1980) marked a commitment by the American Psychiatric Association to revise 1 Address for correspondence: Dr Michelle G. Newman, Department of Psychology, The Pennsylvania State University, 310 Moore Building, University Park, PA 16902-3103, USA.

its nosological categories based on empirical research. DSM-III also introduced social phobia as a diagnostic entity. These two events triggered the beginning of research into the reliable identification of socially phobic individuals and the reliable measurement of improvement or remission in social phobia symptoms (Beidel et al. 1989; Turner et al. 1993; McNeil et al. 1995). With the introduction of the diagnosis, came the development of a number of measures of

623

624

M. G. Newman and others

social phobia. However, validation studies of self-report diagnostic screens for social phobia only began to be published after 1997. Such screens include the Anxiety Disorders Screening Day Instrument (ADSDI) (Olfson et al. 2000), Anxiety Screening Questionnaire (ASQ) (Wittchen & Boyer, 1998), the Social Phobia Questionnaire (SPQ) (Stein et al. 1999), the Social Phobia Inventory (SPIN) (Connor et al. 2000), the Mini-Social Phobia Inventory (Mini-SPIN) (Connor et al. 2001) and the social phobia subscale of the Psychiatric Diagnostic Screening Questionnaire (PDSQ) (Zimmerman & Mattia, 2001a, b). Although all of these measures have some strong psychometric properties, each of these scales also has some limitations. Such limitations include a failure of all of the studies to assess the inter-rater agreement with the structured interviews to which the diagnostic screens were compared for accuracy (Wittchen & Boyer, 1998; Stein et al. 1999 ; Connor et al. 2000, 2001 ; McQuaid, 2000 ; Olfson et al. 2000 ; Zimmerman & Mattia, 2001b), a failure of all screens but the Mini-SPIN to find a cut-off that optimizes sensitivity and specificity such that both are at least 0.8 or above, low positive predictive power of the ADSDI, ASQ, SPQ, Mini-SPIN, and PDSQ social phobia scale (Wittchen & Boyer, 1998 ; Stein et al. 1999 ; Connor et al. 2000, 2001 ; McQuaid et al. 2000 ; Olfson et al. 2000; Zimmerman & Mattia, 2001 b), and a failure of the MINI-SPIN study to determine if the scale discriminates socially phobic participants from those with other anxiety disorders (Connor et al. 2001). In addition, test–retest reliability was not evaluated for the ADSDI, Mini-SPIN and SPQ (McQuaid et al. 2000 ; Olfson et al. 2000 ; Connor et al. 2001) and convergent and discriminant validity with other self-report measures of relevant diagnoses were not assessed for the ADSDI, ASQ and Mini-SPIN (Connor et al. 2001; Olfson et al. 2000 ; Wittchen & Boyer, 1998). It should also be noted that the SPQ did not demonstrate discriminant validity when compared to a measure of depression (McQuaid et al. 2000). Moreover, the Mini-SPIN was validated as a scale to select generalized socially phobic participants and it is unclear whether this scale is able to detect participants with non-generalized social phobia (Connor et al. 2001).

Thus, the goal of the present study was to develop a self-report diagnostic screen for social phobia with good sensitivity, specificity, convergent and discriminant validity, test–retest reliability, and positive predictive power. To improve upon methodological limitations of prior studies, reliability between the comparison structured interviews was calculated. Preparation of a measure designed to assess social phobia according to DSM-IV criteria requires consideration of whether an individual experiences excessive fearfulness in social, observational and evaluative situations, as well as fear of embarrassment and/or of being viewed critically by others. Such a scale should also account for the number and type of situations that are feared and/or avoided, level of impairment in functioning, and degree of distress. This type of scale would add to the already useful social phobia measures by providing a low-cost tool that might track explicit social phobia symptomatology across time. Also, if it showed a high rate of agreement with structured interviews it would be a useful screening device that could decrease some of the time and expense required conducting such interviews. Moreover, such a measure could be used to select quickly undergraduates meeting social phobia symptom criteria for experimental studies before any attempt was made to replicate findings in the more difficult to obtain clinical community samples. The Social Phobia Diagnostic Questionnaire (SPDQ) was designed to assess social phobia directly according to DSM-IV criteria. The style of the SPDQ (i.e. a series of yes/no easy to answer questions as well as symptom rating scales) was modelled after the social phobia module of the Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV) (Brown et al. 1994). It includes two yes/no questions that assess excessive fearfulness in social, observational and evaluative situations, as well as fear of embarrassing oneself, and/or being viewed critically by others. In addition, it includes a question about whether the individual tries to avoid social situations. Next, it includes a list of 16 social situations for which fear and avoidance are rated on a Likert Scale from ‘ 0’ to ‘4 ’: with ‘0 ’ equaling ‘ no fear or avoidance ’ and ‘ 4’ equaling ‘very severe fear or consistent avoidance ’. Following the ratings of individual social situations are two yes/no questions to determine

The Social Phobia Diagnostic Questionnaire

whether the fear comes on as soon as the person encounters the feared social situation and every time that they are in such feared situations. An additional question assesses whether the person views the fear as unreasonable. Finally, it includes 4-point Likert Scale ratings of level of severity, and overall impairment in functioning arising from social fears (see Appendix). This article describes the preliminary assessment of the psychometric properties of the SPDQ.

STUDY I: SENSITIVITY, SPECIFICITY AND ABILITY TO DISCRIMINATE KNOWN GROUPS Method Participants Participants were 125 undergraduates recruited as part of two separate assessment studies. Of the 125 participants, 83 were individuals seeking assessment for an anxiety disorder with the potential of being referred for free treatment as part of separate treatment studies taking place at the Center for Treatment of Anxiety and Depression at The Pennsylvania State University. Sixty of the 125 participants met DSM-IV criteria for social phobia as a primary or secondary diagnosis based on the ADIS-IV, Lifetime Version (ADIS-IV-L) (Di Nardo et al. 1994) or the ADIS-IV (Brown et al. 1994). Of those who did not meet criteria for social phobia, 17 had primary panic disorder, six had primary generalized anxiety disorder (GAD) and 42 did not meet criteria for any anxiety diagnoses. Additional co-morbid diagnoses of the 60 socially phobic participants included obsessive– compulsive disorder (OCD) (2 %), major depression (10 %), dysthymia (7 %), post-traumatic stress disorder (PTSD) (10 %), specific phobia (22 %), panic disorder (20 %), agoraphobia (8 %) and generalized anxiety disorder (GAD) (32 %). Within the GAD group co-morbid diagnoses included panic disorder (17 %), major depression (17 %), PTSD (17 %), specific phobia (17 %) and dysthymia (17 %). Within the panic disorder cohort, 18 % had agoraphobia, 6 % had dysthymia, 24% had GAD, 29% had specific phobia and 12% had major depression. Ninetyfive (76 %) participants were female and 30 (24 %) were male. Five (4 %) were AfricanAmerican, 8 (6 %) were Asian-American, 5 (4 %)

625

were Hispanic, 99 (79 %) were Caucasian and 8 (6 %) identified themselves as other. The average age was 21, with a range of 18 to 41. Chi-square analyses showed no significant differences between the social phobia and comparison groups on ethnicity or gender distribution. An analysis of variance (ANOVA) also found no differences in mean age. Measures The Panic Disorder Self-Report (PDSR) The PDSR (Newman et al. 1998, 2003a) is a 22-item self-report measure designed to diagnose panic disorder based on DSM-IV criteria. The PDSR has demonstrated good test–retest reliability in a college sample (r=0.92). Both convergent and discriminant validity have also been demonstrated. The validity of the PDSR was supported by comparisons between PDSR diagnoses and clinician-based ADIS-IV-L diagnoses of individuals with panic disorder, GAD, social phobia and a non-anxious comparison group. The PDSR showed a 100 % specificity and 89 % sensitivity. Diagnoses made by the PDSR yielded a 0% false positive rate and an 11 % false negative rate. This questionnaire was used as an initial screening measure to select individuals who might meet criteria for panic disorder and controls. The Generalized Anxiety Disorder Questionnaire-IV (GAD-Q-IV ) The GAD-Q-IV (Newman et al. 2002) is a 9item self-report measure designed to diagnose GAD based on DSM-IV criteria. Using receiver operating characteristic (ROC) analyses, the GAD-Q-IV showed 89 % specificity and 83 % sensitivity when compared to structured interview diagnoses of individuals with GAD, social phobia, panic disorder and non-anxious controls. The GAD-Q-IV also demonstrated test– retest reliability, convergent and discriminant validity and kappa agreement of 0.67 with a structured interview. Students diagnosed GAD by the GAD-Q-IV were not significantly different on two measures than a GAD community sample, but both groups had significantly higher scores than students identified as not meeting criteria for GAD, demonstrating clinical validity of the GAD-Q-IV.

626

M. G. Newman and others

Procedure Participants received the SPDQ (see Appendix) as part of a battery of self-report measures administered during a mass-testing situation that also included the PDSR and the GADQ-IV. Data were pooled from two separate studies ; however, both studies took place during the same time period. Fifty of the participants (10 non-anxious controls, 17 panic disordered participants, six with GAD and 17 of the socially phobic participants) were recruited as part of an ongoing study examining childhood diagnoses in adult anxiety disorder clients (Zuellig & Newman, 1996). In this sample, individuals who endorsed several symptoms for GAD on the GAD-Q-IV, or who endorsed several symptoms of panic disorder on the PDSR, or who appeared not to meet criteria for any anxiety disorder were invited to participate in a face to face structured interview. The administration of the SPDQ and the interview always took place at least 10 days apart. The 50 participants were interviewed by a trained doctoral student using the ADIS-IV-L. All of these interviews were audiotaped. A randomly selected sample of audiotaped interveiws (45 % of entire sample ; 45% of GAD sample, 45% of the socially phobic sample, 45% of panic disordered participants, 45% of controls) was assessed by a second trained blind interviewer. Inter-rater reliability revealed a 94% overall concordance rate for GAD, panic disorder, social phobia, or an absence of these diagnoses. Given the high diagnostic concordance between the first interviewer and the second, blind interviewer, the diagnosis of the first interviewer was used to determine concordance estimates between the SPDQ and ADIS-IV-L. The other 75 participants were recruited for a study examining interpersonal differences between social phobia subgroups (Kachin et al. 2001). In this study, participants who either endorsed several symptoms for social phobia on the SPDQ or who appeared not to meet social phobia criteria were invited to participate in a structured interview. As with the first study, the administration of the SPDQ and the interview always took place at least 10 days apart. These participants (43 socially phobic participants and 32 non-anxious controls) were interviewed twice by independent assessors using the ADIS-IV.

The first assessor served as a screener and any individuals not meeting criteria for social phobia (except those who demonstrated a clear absence of social phobia) were not invited back for a second interview. The second interviewer was uninformed of the diagnostic assignment of the first interviewer and was encouraged to use his own judgement about whether the individual met diagnostic criteria for any disorder. In cases where initial disagreement occurred (13 % of the cases), the two assessors met and diagnosis was determined by consensus. If consensus could not be reached, the participant was excluded from the study. Reliability of the SPDQ was examined by comparing the SPDQ to the ADISIV consensus diagnosis. Although two different structured interviews were used in these studies, it is important to note that the interviews are identical in terms of assessment of current DSM-IV criteria. The only difference between the interviews is that the ADIS-IV-L queries the diagnoses throughout the individual’s life whereas the ADIS-IV focuses more centrally on current diagnoses. The SPDQ was scored by using a sum total response. This scoring system was devised in an attempt to create a score that would best enable detection of the presence of social phobia. To create a total score, all yes answers were coded as 1 and all no answers as 0 (e.g. items : (1) Nervous or fearful of social situations ; (2) Overly worried that you may embarrass yourself; (3) Do you try to avoid social situations ?; (20) Experience fear each time ; (21) Fear comes on as soon as you encounter the situation; (22) Is social fear excessive or unreasonable ?). Additional items (e.g. 4a, 5a, 6a, 7a, 8a, 9a, 10a, 11a, 12a, 13a, 14a, 15a, 16a, 17a, 18a, 19a) were each divided by four, whereas distress and interference items (i.e. 23 and 24) were divided by two and these numbers were added to the total. Total scores ranged from 0 to 27. Data analyses To determine whether there were demographic differences between the samples from the two studies on ethnicity and gender x2 analyses were used. To determine differences between the studies in age and in total SPDQ scores unvariate ANOVAs were conducted. In order to determine the ability of the SPDQ, to discriminate people with social phobia from

627

The Social Phobia Diagnostic Questionnaire

those with GAD, panic disorder, and from non-anxious control participants, ROCs of this measure were analysed comparing the SPDQ to ADIS-IV diagnosis. All participants with anxiety disorders other than social phobia as well as controls were grouped together and compared to individuals with either primary or secondary social phobia. An ROC analysis determines the sensitivity and specificity of all possible cut-off points on the SPDQ. We sought the optimal cut-off score for the SPDQ such that both sensitivity and specificity were maximized. An ROC curve was created by plotting sensitivity against one minus specificity for each possible cut-off point. This curve provides a visual representation of the accuracy of the SPDQ to detect socially phobic persons. The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the test. The closer the curve comes to the diagonal of the ROC space, the less accurate the test. The resulting area under the curve (AUC) was then calculated. The AUC represents the ability of the SPDQ to discriminate socially phobic participants from the comparison sample. An area of one represents perfect discrimination whereas an area of 0.5 represents poor discrimination. Results Chi-square analyses indicated no differences between the samples of the two studies in ethinicity or gender. ANOVAs indicated no difference between the studies in age of participants, in total SPDQ scores of persons diagnosed socially phobic, or in total SPDQ scores of individuals not meeting criteria for an anxiety disorder. As a result, the data sets of the two studies were combined and analysed jointly. Fig. 1 shows the ROC curve of the SPDQ. The area under the curve was 0.89 (S.E. 0.03, P<0.001), suggesting that the probability is 89 % that someone with social phobia will have a higher score on the SPDQ than someone without social phobia. The 95% CI of the area under the curve ranged from 0.83 to 0.95. Table 1 shows the sensitivity and specificity for the various cut-off points of the SPDQ. The optimal balance between sensitivity and specificity is achieved with a cut-off point of 7.38. This cut-off leads to a sensitivity of 82 %

1 0·9 0·8 0·7 0·6 0·5 0·4 0·3 0·2 0·1 0

0

0·2

0·4

0·6

0·8

1

FIG. 1. Receiver operating characteristic curve (x) of the sensitivity and specificity of the SPDQ to detect the presence of social phobia. (- - - -, No discrimination.)

(49 of 60) an a specificity of 85 % (55 of 65). Thus, using this cut-off, the rate of false positive diagnoses by the SPDQ was 15% and the rate of false negative diagnoses assigned by the SPDQ was 18 %. Kappa agreement between the ADIS-IV and the SPDQ was 0.66 with 83 % of participants correctly classified. (When avoidance and fear items were included together in the scoring system and when avoidance items were included rather than fear items, the sensitivity and specificity were less impressive.)

STUDY 2: CONCURRENT VALIDITY Method Participants Four hundred and sixty-two undergraduate students (147 (32 %) males and 314 (68 %) females) participated. Age of participants ranged from 17 to 32 years, with a mean of 19. Fourteen (3 %) of the participants defined themselves as African-American, 18 (4 %) as AsianAmerican, 20 (4 %) as Hispanic, three (1 %) as Native American and 407 (88 %) as Caucasian.

628

M. G. Newman and others

Table 1. SPDQ cut-off score

Sensitivity %

6.38 6.75 7.13 7.38 7.63 8.00 8.38 8.75 9.13 9.50 9.88 10.13 10.50 10.88 11.13 11.38 11.75 12.13

87 85 82 82 80 73 70 67 63 63 63 60 57 55 53 47 45 45

Sensitivity and specificity of the SPDQ

Specificity %

Correctly classified %

Positive predictive power %

Negative predictive power %

79 80 82 85 85 85 85 85 85 86 89 92 92 92 92 92 92 94

82 82 82 83 82 79 78 76 74 75 77 77 75 74 74 70 70 70

79 80 80 83 83 82 81 80 79 81 84 88 87 87 87 85 84 87

86 85 83 83 82 78 75 73 71 72 73 71 70 69 68 65 65 65

The row in bold-type indicates the cut-off that achieves the optimal balance between sensitivity and specificity.

Individuals received class credit in exchange for their participation. Procedure Participants attended a group questionnaire completion session where they filled out the battery of questionnaires listed below as well as the SPDQ and GAD-Q-IV. The questionnaire selected to assess convergent validity was the Social Interaction Anxiety Scale (SIAS) (Mattick & Clarke, 1998). Questionnaires selected to assess discriminant validity included the PTSD checklist (Weathers et al. 1991), the Civilian Mississippi Scale (CMS) (Keane et al. 1988), the Penn State Worry Questionnaire (PSWQ) (Meyer et al. 1990) and the GAD-Q-IV (Newman et al. 2002b). Measures SIAS This 20-item scale assesses general fears of social interaction. The scale possesses high levels of internal consistency and test–retest reliability. It has also been shown to discriminate socially phobic participants from those with panic disorder with or without agoraphobia, simple phobia samples, and non-anxious samples (Mattick & Clarke, 1998; Peters, 2000). Furthermore, the

SIAS demonstrated strong convergent validity with self-report measures and with negative and positive thoughts in speech and conversation (Cox et al. 1998; Ries et al. 1998). It also demonstrated discriminant validity when compared to established measures of depression, state and trait anxiety, and social desirability (Mattick & Clarke, 1998) and was shown to be sensitive to change arising from treatment (Cox et al. 1998; Mattick & Clarke, 1998 ; Ries et al. 1998). PSWQ The PSWQ is a 16 item self-report measure of the frequency and intensity of worry. Factor analysis indicated that the PSWQ assesses a unidimensional construct with an internal consistency coefficient of 0.91 (Meyer et al. 1990). High test–retest reliability was also demonstrated (Meyer et al. 1990). The PSWQ has also been shown to distinguish individuals with GAD from each of the other anxiety disorder groups (Brown et al. 1992). Correlations between the PSWQ and measures of anxiety, depression, and emotional control supported the convergent and discriminant validity of the measure (Brown et al. 1992). In addition, this measure discriminated samples that (1) met all, some, or no DSMIII-R diagnostic criteria for GAD and (2) met criteria for GAD versus PTSD (Meyer et al.

The Social Phobia Diagnostic Questionnaire

1990). The PSWQ has also demonstrated sensitivity to change in response to psychotherapy (Meyer et al. 1990). PTSD checklist (PCL) The PCL (Weathers et al. 1991) was developed to assess PTSD symptoms in civilian populations and consists of 17 items that correspond to DSM-IV symptoms of PTSD. In a sample of Vietnam veterans, PCL scores demonstrated a coefficient alpha of 0.93 and convergent validity with other PTSD scales. The cut-off score for this scale has also demonstrated a diagnostic sensitivity of 0.82 and a specificity of 0.83 (Weathers et al. 1991b). Inter-rater agreement with a structured interview for current PTSD was 0.74. Reliability and validity of this scale has also been demonstrated in patients with PTSD arising from treatment of breast cancer (Andrykowski et al. 1998) and from a motor vehicle accident (Blanchard et al. 1996). CMS This is a civilian form of the Mississippi Scale for Combat-Related PTSD (MSCRP). The MSCRP has been shown to have good test–retest reliability over a 1-week period and a sensitivity of 0.93 and specificity of 0.89 in differentiating between a PTSD group and two non-PTSD comparison groups (Keane et al. 1988). The CMS has been shown to have a raw score distribution that is roughly symmetric, with an acceptable degree of dispersion and a reasonably high internal consistency reliability coefficient. High crosslanguage stability has been demonstrated, and both English and Spanish versions have shown high internal consistency (Norris & Perilla, 1996). Four hundred and four victims of Hurricane Andrew, provided additional evidence of scale reliability and also showed that the scale correlates in meaningful ways with known traumatic stressors (Norris & Perilla, 1996). Another study showed that the scale demonstrates strong convergent validity with other PTSD measures (Gold & Cardena, 1998). Data analysis To determine internal consistency of the SPDQ, Cronbach’s alpha and Guttman split-half reliability were conducted. The 7.38 SPDQ cut-off from the ROC analysis was then used to classify

629

participants as socially phobic or non-socially phobic. Next, we examined convergent and discriminant validity of the SPDQ by conducting point biserial correlations between the SPDQ classifications and SIAS, CMS, PSWQ, and GAD-Q-IV. To determine whether the convergent validity correlation between the SPDQ and SIAS was significantly higher than the discriminant validity correlations between the SPDQ and CMS, PSWQ, and GAD-Q-IV we applied the formula recommended by Cohen & Cohen (1975) to test the significance of the difference between two correlations from the same sample. RESULTS Results of Cronbach’s alpha showed that the SPDQ was highly internally consistent (N=462, a=0.92). Guttman split-half reliability also showed high internal consistency r(462)=0.89. Point biserial correlations showed that the SPDQ was more highly correlated with the SIAS (r=0.64) than it was with the PCL (r=0.29), the CMS (r=0.34), the PSWQ (r=0.32) or the GADQ-IV (r=0.29). Calculation of the significance of the difference between the correlation coefficients showed that the correlation between the SPDQ and SIAS was significantly higher than the correlation between the SPDQ and PCL, t(462)=7.91, P<0.01, and between the SPDQ and the CMS, t(462)=8.14, P<0.01. Similarly, the correlation between the SPDQ and SIAS was higher than the correlation between the SPDQ and the PSWQ, t(462)=7.37, P< 0.01 and between the SPDQ and the GADQ-IV, t(462)=7.76, P<0.01. STUDY 3: RELIABILITY, CONVERGENT, DISCRIMINANT AND CLINICAL VALIDITY Method Participants Two samples of participants were included in this study. The first sample consisted of 145 undergraduates (102 (70 %) females and 43 (30 %) males). Age of participants ranged from 18 to 40 years, with a mean of 19.45 (S.D.=6.17) years. Four (2.9%) of the participants defined themselves as African-American, seven (5 %) as Asian-American, six (4.3 %) as other and 128

630

M. G. Newman and others

(88 %) as Caucasian. Individuals received class credit in exchange for their participation. The second sample consisted of 35 clinical community participants from a prior social phobia therapy outcome study (Newman et al. 1994). These clients averaged 47 years of age (S.D.=10) and gender distribution was 18 females (50 %) and 18 males (50 %). Half of these participants were diagnosed as meeting criteria for avoidant personality disorder. Procedure Undergraduate students attended two questionnaire completion sessions exactly 2 weeks apart where they completed a battery of selfreport questionnaires. The questionnaire selected to assess convergent validity was the Social Avoidance and Distress Scale (SADS) (Watson & Friend, 1969). Questionnaires selected to assess discriminant validity included the Beck Depression Inventory (BDI) (Beck et al. 1961), the panic disorder severity scale (PDSS) (Shear et al. 1997), the PSWQ (Meyer et al. 1990) and the GAD-Q-IV. The clinical community sample was diagnosed using the Structured Clinical Interview for DSMIII-R (Spitzer et al. 1989) and an unstructured interview using DSM-III-R (American Psychiatric Association, 1987) criteria for avoidant personality disorder (APD). These interviews gathered sufficient information for making DSM-III-R diagnoses of anxiety disorders, and APD (Hofmann et al. 1995). Diagnostic interviews were audiotaped and a blind independent rater listened to a random selection of half of the tapes (14 presumed social phobics) in conjunction with tapes of control participants (N=11) not included in this study. Agreement between the blind rater and the interviewer was 100% for the diagnosis of social phobia and 93 % for the diagnosis of APD. Participants completed a battery of questionnaires including the FNE, SAD, and SISST prior to receiving therapy. Measures Social Avoidance and Distress Scale (SAD) (Watson & Friend, 1969) This scale measures discomfort in and avoidance of interpersonal interactions. Internal consistency, test–retest reliability and known groups validity were demonstrated for this scale (Watson & Friend, 1969).

Fear of Negative Evaluation Scale (FNE) (Watson & Friend, 1969) This measure taps into one’s apprehension about other’s evaluations. Internal consistency, 1-month test–retest reliability, known groups validity, criterion validity, and sensitivity to change from treatment have been demonstrated for this measure (Watson & Friend, 1969 ; Butler, 1985 ; Mattick et al. 1989 ; Heimberg et al. 1990). Panic Disorder Severity Scale (PDSS) (Shear et al. 1997) This 7-item scale assesses panic disorder severity. Internal consistency, convergent and discriminant validity and sensitivity to change have been demonstrated for this scale (Shear et al. 1997). Originally designed as a clinician administered device, it has subsequently exhibited sensitivity to change when used as a self-report measure (Penava et al. 1998). In the current sample, test–retest reliability was 0.84 and this scale demonstrated discriminant validity with the PSWQ (r=0.21), and the BDI (r=0.29). Beck Depression Inventory (BDI ) (Beck et al. 1961) The BDI assesses the presence and severity of depression symptoms. This scale has been shown to have high internal consistency, good test– retest reliability and high concurrent validity with other measures of depression (Beck et al. 1988). Social Interaction Self-Statement Test (SISST ) (Glass et al. 1982) This instrument assesses positive and negative self-statements during stressful social interactions. The scale has good internal consistency with split-half reliability coefficients of 0.73 for the positive and 0.86 for the negative subscales. Good concurrent validity was also demonstrated for this scale and the subscales have been shown to distinguish persons with high social anxiety from those with low social anxiety (Glass et al. 1982). Data analyses Based on the 7.38 cut-off score suggested by the ROC analysis, undergraduates were categorized into those who did and those who did not meet

The Social Phobia Diagnostic Questionnaire

social phobia criteria at Times 1 and 2. Using these classifications, logistic regression was then conducted to determine the likelihood of diagnostic status remaining stable across time. As a complementary way to determine stability, we also ascertained kappa agreement between Time 1 and Time 2. Convergent and discriminant validity was ascertained via point biserial correlations comparing SPDQ categorizations to the SAD, BDI, PDSS, PSWQ, and GAD-Q-IV. To determine whether the convergent validity correlation between the SPDQ and SAD was significantly higher than the discriminant validity correlations between the SPDQ, and BDI, PDSS, PSWQ, and GAD-Q-IV, we applied the formula recommended by Cohen & Cohen (1975) to test the significance of the difference between two correlations from the same sample. In addition, because the SAD was not normed on a clinical sample of socially phobic individuals we also conducted analyses to determine whether the convergent validity correlation between the SIAS and SPDQ obtained from the student sample in Study 2 was significantly higher than the discriminant validity correlations from this student sample. To do this, we applied the formula recommended by Cohen & Cohen (1975) to test the significance of the difference between two correlations from independent samples. To test the clinical relevance of student participants who met or failed to meet social phobia criteria based on the SPDQ (using the cut-off of 7.38), univariate ANOVAs were conducted comparing the FNE, SAD and SISST scores of the SPDQ identified socially phobic (N=31) and non-socially phobic (N=112) undergraduates in this study and the pre-therapy scores of the 35 clinical community participants. Significant results were followed up with post hoc Bonferroni corrections. Results Results of the logistic regression showed that the SPDQ score at Time 2 was reliably predicted by Time 1 score (x2 (1, N=142)=47.6, P< 0.001) and that 88% (125/142) of the sample showed stability across time in terms of their categorization. Odds ratios indicated that someone classified as meeting criteria for social phobia at Time 1 is 30 times more likely to be

631

classified as meeting criteria for social phobia at Time 2 than someone not classified as meeting criteria for social phobia at Time 1. Kappa agreement between Time 1 and Time 2 was 0.63. These statistics suggest that the SPDQ has good test–retest reliability. Comparisons of the convergent and discriminant correlation coefficients obtained from this sample showed that the correlation between the SPDQ and SAD (r=0.61) was significantly higher than the correlation between the SPDQ and PDSS ((r=0.31), t(145)=4.02, P<0.01), higher than the correlation between the SPDQ and GAD-Q-IV ((r=0.28), t(145)=4.19, P< 0.01), higher than the correlation between the SPDQ and BDI ((r=0.32), t(145)=3.81, P< 0.01) as well as the SPDQ and the PSWQ ((r= 0.38), t(145)=3.01, P<0.01). Results comparing the convergent validity coefficient from Study 2 to the discriminant validity coefficients from this sample showed that the SPDQ was more highly correlated with the SIAS than it was with the PDSS (t(145)=4.44, P< 0.001), the GAD-Q-IV (t(145)=4.77, P<0.001), the BDI (t(145)=4.33, P<0.001) and the PSWQ (t(145)=3.64, P<0.001). ANOVAs comparing SPDQ identified undergraduates to clinical community participants showed significant main effects on the SISST positive subscale (F(2, 176)=15.58, P<0.001), the SISST negative subscale (F(2, 175)=14.23, P<0.001), FNE (F(2, 171)=29.13, P<0.001) and SAD (F(2, 177)=38.89, P<0.001). Post hoc Bonferroni corrections indicated that on each of these measures, whereas the SPDQ identified non-socially phobic group differed significantly from both the student socially phobic group and the clinical community cohort (all values for P<0.01), the SPDQ identified student socially phobic group was not significantly different from the clinical community group on either of the measures (Table 2).

DISCUSSION These results provide preliminary evidence that the SPDQ may be an effective way to initially screen for social phobia. The SPDQ showed very good accuracy at detecting the presence of social phobia (with a specificity of 85%), good accuracy at detecting the absence of social

632

M. G. Newman and others

Table 2. Means and standard deviations of FNE and SAD scores for clinical community socially phobic clients. SPDQ identified socially phobic participants, and SPDQ identified nonsocially phobic participants SPDQ selected

Measure SAD FNE SISST negative SISST positive

Non-socially phobic (N=113)

Socially phobic (N=31)

Clinical community socially phobic (N=35)

4.08a (3.71) 10.33a (7.60) 30.03a (8.80) 52.88a (10.04)

10.65b (5.64) 19.02b (7.04) 38.85b (10.61) 46.97b (7.16)

11.44b (7.90) 19.34b (7.13) 37.26b (11.35) 43.06b (10.17)

Differing superscripts across rows indicate significant differences between means with all values of P<0.001.

phobia (with an 82% sensitivity) and a good overall rate of agreement with the ADIS-IV (k=0.66). Evaluation of the SPDQ also showed that it had excellent test–retest reliability over a 2-week period, strong internal consistency, as well as strong convergent and discriminant validity. The clinical validity of this measure was also demonstrated by the finding that students diagnosed with social phobia by the SPDQ were not significantly different on several measures than a socially phobic community sample, but both groups had significantly higher scores than students identified as not meeting criteria for social phobia. One potential criticism of the 7.38 cut-off is that it allows for the possibility that participants may be determined to meet criteria for social phobia even though they have not endorsed all of the social phobia diagnostic criteria on the SPDQ. Initially, we did try classifying participants as socially phobic only when they endorsed all of the SPDQ social phobia criteria. However, even though this approach led to 95% specificity, sensitivity was only 57 %, meaning that roughly half of the participants who met social phobia criteria were being missed. Given that the SPDQ was developed for use as a screening measure, it was as important not to miss people who met social phobia criteria, as it was to be certain that only people who met full criteria were identified. Thus, it seemed appropriate to

determine empirically which cut-off would provide the best balance between sensitivity and specificity. The 7.38 cut-off achieves this balance by producing a low false positive rate and a low false negative rate. Nonetheless, if someone desired greater certainty that only persons with social phobia were included in their sample, they might choose to use a higher cut-off (e.g. 10.13). The latter cut-off would provide a smaller (8 % v. 15 %) rate of false positives than the 7.38 cutoff although it also has a higher (40 % v. 18%) rate of false negatives. The finding that the SPDQ was sensitive to distinguishing between individuals with social phobia, GAD, panic disorder and non-anxious controls, was particularly encouraging, given the high rates of co-morbidity and substantial overlap between these conditions (Barlow, 1988; Sanderson & Barlow, 1990; Okasha et al. 1994; Borkovec et al. 1995 ; Roemer et al. 1997 ; Newman et al. 2003b). It is also important to note that like most clinical community samples, the socially phobic, GAD and panic disordered students in the current study had a number of overlapping diagnoses. The fact that the SPDQ was able to distinguish those with primary or secondary social phobia from those with mixed diagnoses other than social phobia provides additional support for its validity. The use of this questionnaire may reduce the number of individuals ruled out following the more costly structured interview. Moreover, this questionnaire may be valuable as a means to select quickly undergraduates meeting social phobia symptom criteria for experimental studies examining the processes related to social phobia before attempting to replicate findings in more difficult to obtain clinical community samples. Even though the current study found that undergraduate participants selected with the SPDQ were similar to clinical community samples on externally valid criteria, the sensitivity and specificity rates found here may not apply to community clinic samples. Thus, this measure should be tested using such a sample. Several additional limitations should be noted. First, inter-rater agreement related to the ADIS-IV-L diagnosis of 50 participants in Study 1 was based on a second rater listening to an audiotape of the structured interview. A more stringent means to determine the accuracy of the

633

The Social Phobia Diagnostic Questionnaire

diagnosis would have been to use a second independent rater as was done for the other 75 participants in Study 1. The study samples were also limited by a lack of ethnic diversity. Future research should attempt to replicate these findings with a more ethnically diverse sample.

We would like to thank Dr Louis G. Castonguay for his helpful comments on an earlier draft of this manuscript. Preparation of this article was supported in part by National Institute of Mental Health Research Grant MH-58593.

APPENDIX SPDQ 1. In social situations where it is possible that you will be noticed or evaluated by other people, do you feel excessively nervous, fearful or uncomfortable ? 2. Do you tend to be overly worried that you may act in a way that might embarrass or humiliate yourself in front of other people, or that others may not think well of you? 3. Do you try to avoid social situations ?

Yes ____ No ____ Yes ____ No ____ Yes ____ No ____

Below is a list of some situations that are fear provoking for some people. Rate the severity of your anxiety and avoidance on the following scales : 0=No fear 1=Mild fear 2=Moderate fear 3=Severe fear 4=Very severe

0=Never avoid 1=Rarely avoid 2=Sometimes avoid 3=Often avoid 4=Always avoid

(a) Fear (b) Avoidance Parties 0 1 2 3 4 0 1 2 3 4 Meetings 0 1 2 3 4 0 1 2 3 4 Becoming the focus of attention 0 1 2 3 4 0 1 2 3 4 Dating circumstances 0 1 2 3 4 0 1 2 3 4 Meeting people in authority 0 1 2 3 4 0 1 2 3 4 Speaking with people in authority 0 1 2 3 4 0 1 2 3 4 Saying ‘ no’ to unreasonable requests 0 1 2 3 4 0 1 2 3 4 A first date 0 1 2 3 4 0 1 2 3 4 Asking others to do something differently 0 1 2 3 4 0 1 2 3 4 Being introduced 0 1 2 3 4 0 1 2 3 4 Initiating a conversation 0 1 2 3 4 0 1 2 3 4 Keeping a conversation going 0 1 2 3 4 0 1 2 3 4 Giving a speech 0 1 2 3 4 0 1 2 3 4 Others judging you 0 1 2 3 4 0 1 2 3 4 Being under observation by others 0 1 2 3 4 0 1 2 3 4 Being teased 0 1 2 3 4 0 1 2 3 4 Do you tend to experience fear each time you are in feared social situations? Yes ____ No ____ Does the fear come on as soon as you encounter feared social situations ? Yes ____ No ____ Would you say that your social fear is excessive or unreasonable? Yes ____ No ____ Circle the degree to which your social fear interferes with your life, work, social activities, family, etc ? 0 1 2 3 4 No Interference Mild Moderate Severe Very Severe/Disabling 24. How distressing do you find your social fear ? (Circle one) 0 1 2 3 4 Not Distressing Mildly Moderately Severely Very Severely 25. Has what you have been able to achieve in your job or in school been negatively effected by your social fear ? Yes ____ No ____

4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.

634

M. G. Newman and others

REFERENCES American Psychiatric Association (1980). Diagnostic and Statistical Manual of Mental Disorders, 3rd edn. American Psychiatric Association : Washington, DC. American Psychiatric Association (1987). Diagnostic and Statistical Manual, 3rd edn. Revised. American Psychiatric Association: Washington, DC. Andrykowski, M. A., Cordova, M. J., Studts, J. L. & Miller, T. W. (1998). Posttraumatic stress disorder after treatment for breast cancer: prevalence of diagnosis and use of the PTSD Checklist – Civilian Version (PCL–C) as a screening instrument. Journal of Consulting and Clinical Psychology 66, 586–590. Barlow, D. H. (1988). Anxiety and Its Disorders : The Nature and Treatment of Anxiety and Panic. Guilford Press : New York. Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry 41, 561–571. Beck, A. T., Steer, R. A. & Garbin, M. G. (1988). Psychometric properties of the beck depression inventory : twenty-five years of evaluation. Clinical Psychology Review 8, 77–100. Beidel, D. C., Turner, S. M., Jacob, R. G. & Cooley, M. R. (1989). Assessment of social phobia : reliability of an impromptu speech task. Journal of Anxiety Disorders 3, 149–158. Blanchard, E. B., Jones-Alexander, J., Buckley, T. C. & Forneris, C. A. (1996). Psychometric properties of the PTSD checklist (PCL). Behaviour Research and Therapy 34, 669–673. Borkovec, T. D., Abel, J. L. & Newman, H. (1995). The effects of therapy on comorbid conditions in generalized anxiety disorder. Journal of Consulting and Clinical Psychology 63, 479–483. Brown, T. A., Antony, M. M. & Barlow, D. H. (1992). Psychometric properties of the Penn State Worry Questionnaire in a clinical anxiety disorders sample. Behaviour Research and Therapy 30, 33–38. Brown, T. A., Di Nardo, P. A. & Barlow, D. H. (1994). Anxiety Disorders Interview Schedule for DSM-IV. The Psychological Corporation : San Antonio TX. Butler, G. (1985). Exposure as a treatment for social phobia : some instructional difficulties. Behaviour Research and Therapy 23, 651–657. Cohen, J. & Cohen, P. (1975). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Lawrence Erlbaum : Hillsdale, NJ. Connor, K. M., Davidson, J. R., Churchill, L. E., Sherwood, A., Foa, E. B. & Weisler, R. H. B. J. P. (2000). Psychometric properties of the Social Phobia Inventory (SPIN): a new self rating scale. British Journal of Psychiatry 176, 379–386. Connor, K. M., Kobak, K. A., Churchill, L. E., Katzelnick, D. & Davidson, J. R. T. (2001). Mini SPIN : a brief screening assessment for generalized social anxiety disorder. Depression and Anxiety 14, 137–140. Cox, B. J., Ross, L., Swinson, R. P. & Direnfeld, D. M. (1998). A comparison of social phobia outcome measures in cognitivebehavioral group therapy. Behavior Modification 22, 285–297. Di Nardo, P. A., Brown, T. A. & Barlow, D. H. (1994). Anxiety Disorders Interview Schedule for DSM-IV : Lifetime Version (ADIS-IV-L). The Psychological Corporation : San Antonio, TX. Glass, C. R., Merluzzi, T. V., Biever, J. L. & Larson, K. H. (1982). Cognitive assessment of social anxiety : development and validation of a self-statement questionnaire. Cognitive Therapy and Research 6, 37–55. Gold, J. W. & Cardena, E. (1998). Convergent validity of three posttraumatic symptoms inventories among adult sexual abuse survivors. Journal of Traumatic Stress 11, 173–180. Heimberg, R. G., Dodge, C. S., Hope, D. A., Kennedy, C. R. & Zollo, L. J. (1990). Cognitive behavioral group treatment for social phobia: comparison with a credible placebo control. Cognitive Therapy and Research 14, 1–23. Hofmann, S. G., Newman, M. G., Taylor, C. B. & Roth, W. T. (1995). Social phobia with and without avoidant personality disorder : preliminary behavior therapy outcome findings. Journal of Anxiety Disorders 9, 427–438.

Kachin, K. E., Newman, M. G. & Pincus, A. L. (2001). An interpersonal approach to the classification of social phobia subtypes. Behavior Therapy 32, 479–501. Keane, T. M., Caddell, J. M. & Taylor, K. L. (1988). Mississippi Scale for Combat-Related Posttraumatic Stress Disorder : three studies in reliability and validity. Journal of Consulting and Clinical Psychology 56, 85–90. McNeil, D. W., Ries, B. J. & Turk, C. L. (1995). Behavioral assessment : self-report, physiology, and overt behavior. In Social Phobia : Diagnosis, Assessment, and Treatment (ed. R. G. Heimberg, L. M. R. Hope, D. A. Hope and F. R. Schneier), pp. 202–231. Guilford Press: New York. McQuaid, J. R., Stein, M. B., McCahill, M. E., Laffaye, C. & Ramel, W. (2000). Use of brief psychiatric screening measures in a primary care sample. Depression and Anxiety 12, 21–29. Mattick, R. P. & Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behaviour Research and Therapy 36, 455–470. Mattick, R. P., Peters, L. & Clarke, J. C. (1989). Exposure and cognitive restructuring for social phobia. Behavior Therapy 20, 3–23. Meyer, T. J., Miller, M. L., Metzger, R. L. & Borkovec, T. D. (1990). Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy 28, 487–495. Newman, M. G., Hofmann, S. G., Trabert, W., Roth, W. T. & Taylor, C. B. (1994). Does behavioral treatment of social phobia lead to cognitive changes ? Behavior Therapy 25, 503–517. Newman, M. G., Zuellig, A. R., Kachin, K. E. & Constantino, M. J. (1998). Examination of the reliability and validity of the PDSR : a self-report measure of panic disorder. Paper presented at the 32nd Annual meeting of the Association for Advancement of Behavior Therapy, Miami, FL, November 1998. Newman, M. G., Zuellig, A. R., Kachin, K. E., Constantino, M. J., Przeworski, A., Erickson, T. & Cashman-McGrath, L. (2002). Preliminary reliability and validity of the GAD-Q-IV : a revised self-report diagnostic measure of generalized anxiety disorder. Behavior Therapy 33, 215–233. Newman, M. G., Przeworski, A., Zuellig, A. R., Kachin, K. E. & Constantino, M. J. (2003 a). The reliability and validity of the panic disorder self-report (PDSR) : a new diagnostic measure of panic disorder. (Manuscript in preparation.) Newman, M. G., Castonguay, L. G., Borkovec, T. D. & Molnar, C. (2003 b). Integrative Psychotherapy for Generalized Anxiety Disorder. In The Nature and Treatment of Generalized Anxiety Disorder (ed. R. Heimberg, D. Mennin and C. Turk). Guilford Press : New York. Norris, F. H. & Perilla, J. L. (1996). The revised Civilian Mississippi Scale for PTSD : reliability, validity, and cross-language stability. Journal of Traumatic Stress 9, 285–298. Okasha, A., Bishry, Z., Khalil, A. H., Darwish, T. A., Seif El Dawla, A. & Shohdy, A. (1994). Panic disorder: an overlapping or independent entity ? British Journal of Psychiatry 164, 818–825. Olfson, M., Guardino, M., Struening, F., Schneier, F. R., Hellman, F. & Klein, D. F. (2000). Barriers to the treatment of social anxiety. American Journal of Psychiatry 157, 521–527. Penava, S. J., Otto, M. W., Maki, K. M. & Pollack, M. H. (1998). Rate of improvement during cognitive-behavioral group treatment for panic disorder. Behaviour Research and Therapy 36, 665–673. Peters, L. (2000). Discriminant validity of the Social Phobia and Anxiety Inventory (SPAI), the Social Phobia Scale (SPS) and the Social Interaction Anxiety Scale (SIAS). Behaviour Research and Therapy 38, 943–950. Ries, B. J., McNeil, D. W., Boone, M. L., Turk, C. L., Carter, L. E. & Heimberg, R. G. (1998). Assessment of contemporary social phobia verbal report instruments. Behaviour Research and Therapy 36, 983–994. Roemer, L., Molina, S. & Borkovec, T. D. (1997). An investigation of worry content among generally anxious individuals. Journal of Nervous and Mental Disease 185, 314–319.

The Social Phobia Diagnostic Questionnaire Sanderson, W. C. & Barlow, D. H. (1990). A description of patients diagnosed with DSM-III revised generalized anxiety disorder. Journal of Nervous and Mental Disease 178, 588–591. Shear, M. K., Brown, T. A., Barlow, D. H., Money, R., Sholomskas, D. E., Woods, S. W., Gorman, J. M. & Papp, L. A. (1997). Multicenter collaborative Panic Disorder Severity Scale. American Journal of Psychiatry 154, 1571–1575. Spitzer, R. L., Williams, J. B., Gibbon, M. & First, M. B. (1989). Structured Clinical Interview for DSM-III-R – Sandoz Version (SCID). American Psychiatric Press : Washington, DC. Stein, M. B., McQuaid, J., Laffaye, C. & McCahill, M. E. (1999). Social phobia in the primary care medical setting. Journal of Family Practice 48, 514–519. Turner, S. M., Beidel, D. C., Long, P. J., Turner, M. W. & Townsley, R. M. (1993). A composite measure to determine the functional status of treated social phobics : the social phobia endstate functioning index. Behavior Therapy 24, 265–275. Watson, D. & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology 33, 448–457. Weathers, F. W., Litz, B. T., Herman, D. S., Huska, J. A. & Keane, T. M. (1991a). The PTSD checklist (PCL) : Reliability, validity

635

and diagnostic utility. Paper presented at the International Society for Traumatic Stress Studies, San Antonio TX. Weathers, F. W., Huska, J. A. & Keane, T. M. (1991b). The PTSD Checklist-Civilian, version (PCL-C). Available from F. W. Weathers, National Center for PTSD, Boston Veterans Affairs Medical Center, 150 S. Huntington Avenue, Boston, MA 02130, USA. Wittchen, H. U. & Boyer, P. (1998). Screening for anxiety disorders. Sensitivity and specificity of the Anxiety Screening Questionnaire (ASQ 15). British Journal of Psychiatry 34, 10–17. Zimmerman, M. & Mattia, J. I. (2001 a). The Psychiatric Diagnostic Screening Questionnaire : Development, reliability and validity. Comprehensive Psychiatry 42, 175–189. Zimmerman, M. & Mattia, J. I. (2001b). A self-report scale to help make psychiatric diagnoses : the Psychiatric Diagnostic Screening Questionnaire. Archives of General Psychiatry 58, 787–794. Zuellig, A. R. & Newman, M.G. (1996). Childhood anxiety disorders in adults diagnosed with generalized anxiety disorder or panic disorder. Paper presented at the 30th Annual Meeting of the Association for the Advancement Of Behavior Therapy, New York, NY, November 1996.

The Social Phobia Diagnostic Questionnaire

additional support for its validity. The use of this questionnaire may reduce ..... Therapy, Miami, FL, November 1998. Newman, M. G., Zuellig, A. R., Kachin, K. E., ...

131KB Sizes 2 Downloads 447 Views

Recommend Documents

The Social Phobia Diagnostic Questionnaire
Diagnostic and Statistical Manual of Mental. Disorders ... predictive power of the ADSDI, ASQ, SPQ,. Mini-SPIN ... reliability, and positive predictive power. To.

Candidate!Questionnaire!
Indiana'currently'funds'vouchers'for'private'and'parochial'schools'and' ... The'Center'for'Education'and'Career'Innovation'costs'taxpayers'an'excess'of'3'million ...

The Coach–Athlete Relationship Questionnaire
This cluster of items shows that although Co-orientation forms a small part ...... meaning system. In: Canary DJ,. Stafford L, eds. Communication and. Relational ...

The VARK Questionnaire - portuguese.pdf
Whoops! There was a problem loading more pages. Retrying... The VARK Questionnaire - portuguese.pdf. The VARK Questionnaire - portuguese.pdf. Open. Extract. Open with. Sign In. Main menu. Displaying The VARK Questionnaire - portuguese.pdf.

Pediatric Sleep Questionnaire -
Pediatric Sleep Questionnaire. Patient Name: Date: ______. Drs. Chmura would like you to complete this form as accurately as honestly as possible. In our.

Questionnaire design banner.pdf
Page 1 of 3. Questionnaire design. Genre. To find out how successful my action genre is I could ask the following. question. On a scale of 1 to 10, 1 being not ...

questionnaire 2 Spiderwick.pdf
Page 3 of 15. Tasked with inventing a social network,. we addressed the needs of long- distance relationships of all kinds! hakuna. Page 3 of 15. Page 4 of 15. CQ-5101U. 4. Customer Services Directory. U.S.A.. Customer Services Directory. (United Sta

Functional Requirements Questionnaire (FRQ) Overview.pdf ...
Blue Prism Group plc, Centrix House, Crow Lane East, Newton-le-Willows, WA12 9UY, United Kingdom. Registered in England: Reg. No. 4260035. Tel: +44 870 ...

Logo Design Questionnaire - JUST™ Creative
Is your deadline fixed or flexible? Email address. Date. Design deadline. Country. Jacob Cass | http://justcreativedesign.com | jacobcass@justcreativedesign.