The dimensionality and validity of the Older Americans Resources and Services (OARS) Activities of Daily Living (ADL) Scale. (Doble & Fisher, 1998)

Doble, S. E., & Fisher, A. G. (1998). The dimensionality and validity of the Older Americans Resources and Services (OARS) Activities of Daily Living (ADL) Scale. Journal of Outcome Measurement, 2(1), 4-24.

School of Occupational Therapy, Dalhousie University, Halifax, Nova Scotia, Canada.

The psychometric properties of the OARS ADL scale, comprised of seven physical activities of daily living (PADL) and seven instrumental activities of daily living (IADL) items, were examined using a Rasch measurement approach. Two of the PADL items failed to demonstrate acceptable goodness-of-fit with the measurement model but the remaining 12 items could be combined into a single measure of ADL ability. Although the OARS ADL scale was designed to identify those community-dwelling elderly who need supports and services to continue to live in the community, the scale items were found to be poorly targeted to community-dwelling elderly since almost half of our sample received maximal scores. Rasch analysis identified how we might improve the sensitivity of the OARS ADL scale but its utility in outcome and longitudinal studies remains questionable.

Notes:

Literature review

1. The Older Americans Resources and Services (OARS) Activities of Daily Living (ADL) Scale
   
   
   1. 7 Physical Activities of Daily Living (PADL)
      
   2. 7 Instrumental Activities of Daily Living (IADL)
      
2. The importance of PADL and ADL; the dependency in PADL and IADL are associated with
   
   
   1. Poor quality of life
      
   2. Increased risk of nursing home placement
      
   3. Increased risk of death
      
3. 2 major problems associated with self- and proxy-based assessment of PADL and IADL
   
   
   1. Evaluating PADL and ADL with few items
      
      
      1. Ceiling and floor effect
         
      2. Compromised reliability
         
      3. In a study conducted by Suurmeijer et al (1994), it is found that when PADL and IADL items are included in a single scale, the unidimensionality of the scale items is compromised. However, results of other researchers supported the unidimensionality of the combined scale (Finch, Kane, & Philp, 1995; Kempen & Suurmeijer, 1990, Siu, Reuben & Hays, 1990; Silverstein, Fisher, Kilgore, Harley, & Harvey, 1992; Spector et al., 1987)..
         
   2. Summing ordinal rating
      
      
      1. Compromise the true measure
         
      2. Guttman scaling is a deterministic model and it requires the data to maintain an absolutely rigid structure, which is rarely the case in social science. In addition, this model holds that people will pass all items lower than their ability level and fail all items higher than their ability level. As a result, the differences have to be big between item difficulties in order for a scale to confirm to this expectation. The resulting trade-off is that it might reduce the sensitivity to identifying changes within an individual over time or small changes across individuals.
         
4. Rasch model
   
   
   1. Difficulty level of items are based on the Likelihood of the item being passed
      
   2. Rasch analysis could generates item-parameter related statistics
      
      
      1. a calibrated difficulty level for each test item
         
      2. Mean square residual (MnSq) with an expected value of 1. MnSq greater than 1.4 is no good
         
      3. A standardized goodness-of-fit statistics (z) with an expected value of 0. z greater or equal to 2 is no good.
         
   3. Rasch model also provided us with estimation for person-related parameter, theta, and the associated statistics. Greater separation suggests higher sensitivity.
      

Subjects

Elderly, at least 60 years old and at least one health condition affecting daily living

Analyses

1. Conducted using BIGSTEPS:
   
   
   1. Item separation statistics: whether subjects separate items into levels of difficulty
      
   2. Person separation statistics: whether items separate subjects into levels of ability
      
   3. Item difficulty measure
      
   4. Person ability measure
      
   5. Goodness-of-fit statistics
      
   6. Unidimensionality of the ADL scale was determined by examining MnSq and associated z fit statistic for each item
      
2. Greater separation suggests increased sensitivity of the scale
   
3. If more than 5% of the subjects' failed to demonstrate goodness-of-fit with the measurement model when z is set at 2, then there might be problem with the validity of the scale
   

Results

1. Fit of items to the measurement model p. 12-17
   
   
   1. Item separation statistic
      
   2. Items located within .25 logitsè actual order might be different because the distance is too small
      
   3. Item order (Finch et al., 1995, Fillenbaum, 1988; Kempen & Suurmeijer, 1990; Siu et al., 1990; Silverstein et al., 1992; Spector et al., 1987; Suurmeijer et al., 1994; Lawton and Brody, 1969)
      
   4. Check for large gaps to see whether we might be missing items to measure certain ability level
      
   5. Check for items with large error estimates
      
   6. Check goodness-of-fit statistics
      
   7. Get rid of the bad items and redo the analyses
      

1. Fit of the persons to the measurement model P. 17-20
   
   


1. Person separation statistics
   


2. Check the number of groups
   

Doble and Fisher (1998) conducted a study to evaluate the psychometric properties of the Older Americans Resources and Services (OARS) Activities of Daily Living (ADL) Scale, which contains 7 ADL and 7 IADL items. Participants were community-dwelling elderly adults in either Canada and United States age 60 and above. Most subjects were interviewed directly through self-report while, for certain participants', such as those having dementia or memory impairments, information was obtained through proxy report. All except for the continence item were rated based on a 3 point scale with 0 indicating the subject is completely incapable of performing the task and 2 indicating no problem at all. The authors used the program BIGSTEPS to run the Rasch model analysis. Their results found that, after dropping two misfitting items (i.e., bathing and continence), the remaining items were found to be hierarchically ordered and measure the same underlying construct (e.g., perceived ADL ability). However, based on the parameter estimates of the ability score, the authors concluded that the OARS ADL items have limited applications for the community-dwelling individuals they were designed for.

A validation study of the daily activities questionnaire: an activities of daily living assessment for people with Alzheimer's disease. (Oakley F, 1999)

Oakley F, L. J., Sunderland T. (1999). A validation study of the daily activities questionnaire: an activities of daily living assessment for people with Alzheimer's disease. Journal of outcome measurement, 3(4), 297-307.

National Institutes of Health, Warren G. Magnuson Clinical Center, Bethesda, MD 20892-1604, USA. Fran_Oakley@nih.gov

Abstract: The Daily Activities Questionnaire (DAQ) was developed to assess activities of daily living (ADL) independence in people with Alzheimer's disease. After administering it to 276 people diagnosed with Alzheimer's disease, we examined the quality of the rating scale and its structure using a Rasch measurement approach. Results indicated that the original 10-point rating scale should be restructured to a 5-point rating scale to improve the quality of the instrument. In addition, we found that all but two ADL items defined the same construct and could be combined into a single summary measure of ADL independence. The remaining items were positioned along a hierarchical continuum, with IADL tasks more difficult than PADL tasks. Furthermore, the tasks were logically ordered by difficulty. We therefore report that the DAQ is a valid scale and conclude that it is a viable measure of ADL independence for studies of Alzheimer's disease.

http://www.ncbi.nlm.nih.gov/pubmed/10572383

My Note:

Daily activities questionnaire (Oakley, et al., 1991)

• PADL assess independence in individual: bathing, dressing, toileting, walking, eating and grooming
• IADL: Home care, cooking, shopping, finances, and phone use.

Some interesting rule of thumbs for Rasch analysis:

"Rasch analysis determines the unidimensionality of the instrument by examining response patterns of the items in the instrument, which are demonstrated by infit Mean Square (MnSq) statistics. The MnSq is the ratio between observed and expected variance (Wright and Masters, 1982).

An acceptable range for an infit MnSq value for a rating scale is between .6 and 1.4 (Wright and Masters, 1994). An item with a MnSq value smaller than 0.6 indicates that the item does not provide additional information beyond the rest of items on the scale. If the MnSq value for an item is greater than 1.4, either the item does not define the same construct as the rest of the items in the instrument or it is ambiguously defined.

Rasch analysis also provides a significance test for the MnSq value. A Zstd value greater than 2.0 indicates that the corresponding MnSq value is significant at the .05 level."

Due to the disordered steps in the 10 category scenario, the authors converted the original 10-point rating scale to a 5-point rating scale.

One final word for this study…. The authors consider the combined scale of ADL/IADL as unidimensional….

http://www.rasch.org/rmt/rmt82a.htm

Oakley and his associated conducted a study to assess the Validity of the Daily Activities Questionnaire (DAQ) (Oakley et al 1991). The DAQ is an 14-item instrument which was designed to assess independence in PADL (ADL) and IADL. The participants of the study were individuals diagnosed with Alzheimer's disease and who were institutionalized. The authors used the FACETS software to assess the fit of Rasch Model for the data. After the initial analyses, the authors the 10-point scale into a 5-point scale because they found the original 10-point rating scale to have limited ability in discriminating among patients' ability; they also dropped walking as an unfit item from the final analyses. Results of the Rasch analysis supports the notion of unidimensionality or an underlying ADL construct; in addition, based on results about the level of endorsability of the items, it is found that the DAQ items are spread across a hierarchical linear continuum where IADL items were found to be more difficult than PADL (ADL) items at least for patients with Alzheimer disease..