Phobic Disorders - Issues of Reliability and Validity

AO1	AO2
Reliability refers to the consistency of a measuring instrument. Reliability can be measured in terms of:           Inter-rater reliability           Test-retest reliability Inter-rater - Whether two independent assessors give similar scores. Skyre et al (1991) - assessed inter-rater reliability for diagnosing social phobia by asking three clinicians to assess 54 patient interviews. There was high inter-rater agreement showing that the diagnosis of phobia is reliable.	Research evidence - Reliability has not always been found to be high. Kendler et al (1999) - used face-to-face and telephone interviews to assess individuals with phobias. Over a one month interval, they found a mean agreement of +.46. Reliability over the long term was even lower at +.30. Picon et al (2005) - however, found good test-retest reliability (better than +.80) with a Portuguese version of the SPAI over a 14 day interval. This indicates that reliability can be good at least in the short term.
	Reasons for low reliability - Kendler et al (1999) - suggest that the low reliability found in their study might be due to several factors: Test-retest reliability might be due to poor recall by participants of their fears, for example, people tend to over exaggerate fears when recalling previous distress. Low inter-rater reliability might be due to the different decisions made by interviewers when deciding if the severity of a symptom does or does not exceed the clinical threshold for a symptom.
Test-retest - Whether the test items are consistent. Scales such as SCID take 1-2 hours to complete. The alternative is to use shorter, structured, self-administered scales. These are popular for specific phobias. Hiller et al (1990) - reported satisfactory to excellent diagnostic agreement in a test-retest study using the MDC.	Cultural differences in diagnosis - Cultural norms about 'normal functioning' and 'normal' fears are likely to affect any diagnosis rendering the diagnosis 'invalid'. Judgements may also vary with respect to culturally-relative disorders. A good example of this is provided by taijin-kyofusho (TKS), a culturally distinctive phobia recognised in Japan. This is a social phobia where an individual has a fear of embarrassing others in social situations.  An individual with such a condition would not be diagnosed in this country as having a social phobia, indicating the effect of cultural experiences on the diagnosis of a disorder.
	Diagnosis by computer - Reliability may be improved through the use of computerised scales for assessing phobic and other disorders. Kobak et al (1993) - suggest that increased reliability occurs because there is less opportunity for the administrator to affect th responses that are given. In addition people with social phobias may prefer to answer without the presence of another person because of their fears of negative evaluation.
Validity refers to the extent that a diagnosis represents something that is real and distinct from other disorders and the extent that a classification system measures what it claims to measure. Comorbidity - Is an important issue for the validity of diagnosis. It refers to the extent that two or more conditions co-occur. Research has found high levels of comorbidity between social phobias, animal phobias, generalised anxiety disorder and depression. Such comorbidity suggests that these conditions are not separate entities and therefore the diagnostic category is not very useful.	Comorbidity - The findings on comorbidity have been supported in many other studies. Eysenck (1997) - reported that up to 66% of patients with one anxiety disorder are also diagnosed with another anxiety disorder. The implication is that a diagnosis should simply be 'anxiety disorder' rather than a phobia or OCD.
	The implications of low reliability and/or validity - In order to conduct research on the effectiveness of treatments for phobic disorders researchers require a reliable and valid means of assessing the disorders in the first place.
Concurrent validity - Establishes the value of a new measure of phobic symptoms by correlating it with an existing one. Herbert et al (1991) - tested concurrent validity of the Social Phobia Anxiety Inventory by giving the test and various other standard measures to 23 social phobics. The SPAI correlated well with the other measures. Construct validity - Assesses the diagnostic questionnaires and interviews. This measures the extent that a test for phobic disorders really does measure a target construct of phobias. To do this, clinicians identify possible target behaviours that we would expect in someone with a phobic disorder and see if people who score high on the test for phobic disorders also exhibit the target behaviour.	Support for concurrent validity - Mattick and Clarke (1998) - showed that their Social Phobia Scale (SPS) correlated well with other standard measures. This indicates that there are methods of diagnosis that agree and therefore appear to be measuring something real.
	Support for construct validity - The SPAI correlates well with behavioural measures of social phobia. It also doesn't correlate with behaviours related to other anxiety disorders. However, perhaps this is not surprising because the inventory includes questions about cognitions and behaviours across a range of fear-producing situations. This means that it is likely to correlate with behaviours associated with social phobias.