IRT rules of thumb

One problem I often encounter when writing academic paper is to look for "rules of thumb" for the statistical analyses...

Since I am a newbie to IRT and Rasch model, I think the methodology and the rules of thumb employed in the AIM ADL study as well as the associated citations might come handy at some point...

http://ratologydisabled.blogspot.com/2009/03/activities-of-daily-living-adl-and.html

Activities of daily living (ADL) and instrumental activities of daily living (IADL) items were stable over time in a longitudinal study on aging. (Finlayson, Mallinson, & Barbosa, 2005)

Finlayson, M., Mallinson, T., & Barbosa, V. M. (2005). Activities of daily living (ADL) and instrumental activities of daily living (IADL) items were stable over time in a longitudinal study on aging. Journal of Clinical Epidemiology, 58(4), 338-349.

Department of Occupational Therapy, University of Illinois at Chicago (MC 811), 1919 West Taylor Street, Chicago, IL, 60612, USA. marciaf@uic.edu

OBJECTIVE: The purpose of this analysis was to examine the stability over time of the activities of daily living (ADL) and instrumental activities of daily living (IADL) items in the Aging in Manitoba (AIM) Longitudinal Study and to evaluate the existence of differential item functioning across settings (home, nursing home).

STUDY DESIGN AND SETTING: The study used data from 607 participants of the AIM Longitudinal Study who were more than 85 years of age in 1996 and who had complete data from 1983, 1990, and 1996 for all ADL and IADL items. Rasch analysis was used to examine how the rating scale of the ADL and IADL items was used by participants, and to determine if the ordering of items remained stable across three time periods (1983, 1990, 1996) and the two different settings (home, nursing home).

RESULTS: The rating scale worked best when dichotomized into "received no assistance" and "receives assistance." Except for four items (making tea, making meals, doing nursing care, and going outside in any weather), the items were stable across administration periods, and across settings.

CONCLUSION: The AIM can be used to evaluate changes in disability over time and may have the potential to identify those at risk for transitions in care.

http://www.ncbi.nlm.nih.gov/pubmed/15862719

My notes

There were approximately 50 longitudinal studies on aging internationally as of 2005.

One reason the aging studies receive interest is in the pattern and predictors of long term care use.

Advances in technologies have shifted the definition of independence in the past 30 something years. For instance, people might no longer report having problem doing laundry, shopping for groceries or cooking because they could use washers and driers, online shopping and microwave oven to perform the tasks they otherwise might not be able to perform. In other words, advances in technologies might inevitably result in the "observed higher functioning of individual" while such observations are not the real change in the persons' ability.

Comparing TCC and Rasch model

"Rasch measurement focuses on the performance of items. Rasch measurement takes ordinal level data and converts the probability of endorsing an item to a logti or log of odds ratio."

In Rasch model, when the ordering of item difficulties are stable (β), we can use the data to analyze the ability or functional status for the individuals (θ). In this concept, "stability" does not refer to test-retest reliability in CTT. Stability has to do with how functional status is defined by the respondent.

For the Aging in Manitoba (AIM) study, in calculating a functional status score for the AIM, higher scores equal receipt of more help.

Methods of this study

• Use Winstep Program
• Testing the assumptions of Rasch model: local independence and unidimensionality
  
  • Item fit
  • Item discrimination
  • Principal component analysis
• Item fit: Item infit statistics are sensitive to sensitive to unexpected responses. Values greater than 1.0 indicate departures from unidimensionality while values smaller than 1.0 indicates redundancy and the potential for violating local independence. An item infit ranging between .8 to 1.2 is considered good while that ranging between .7 to 1.3 considered acceptable.
• Item discrimination: Item discrimination estimates describe the slope for the ICC. Rasch model assumes all ICC slopes have a value of 1.0. Linarcre showed that a value between .5 and 1.7 generally equate to the infit statistics of 1.2 to 0.8—it also suggest that "ICC in this range do not unduly impact the assumptions of the Rasch model and have no meaningful impact on measurement.
• Stability within settings: The authors assess the stability of item calibration by examining the order of items was constant over the 3 measurement periods separately for the home and nursing home setting. Items with the logit values 2 standard deviation apart are considered as significantly different.
• Stability across settings: Calculate item calibration across all three measurement period for home and nursing home. Again > 2 SE indicates significant differences

Results section

5-point scale was rescaled into a 2-step dichotomous scale due to the disordered steps

Checking Rasch model assumptions: Smith notes that infit statistics can be susceptible to test variance and sample size and recommended the use of standardized residual (zstd) in conjunction with the infit mean square (MnSq) and with common sense.

Item discriminations >.5 indicates that data for these items did not fit Rasch model assumption

The assumption of unidimensionalty was supported using PCA. A minimum of 10% contribution of additional variance accounted for is usually considered the cutoff point to suggest an additional factor in the factor structure.

The disordered step might be an artifact of the intent of AIM study focusing on measuring the unmet needs rather than functional status per se.

People who made a significant transition such as from a home to a nursing home showed a more variable pattern in how they endorse ADL and IADL items over time--- as shown by the floating squares shown on figure 3

Future researchers might wish to complete similar analyses on younger individuals

Finlayson and his associates conducted a study to examine the stability of responses for ADL and IADL items over years. The data used this this study contained a subset of data collected through the Aging in Manitoba (AIM) longitudinal Study. All participants were born in 1911 or ealier with the average age of around 90. Rasch model analysis was conducted using the WINSTEPS program. Originally, there were 5 response categories including: 1. Do it myself; 2. my spouse or other people in the household does it; 3. Someone from outside the house does it; 4 A formal service does it; and 5. Someone from a facility does it. Because results obtained through the 5-point scale led to disordered steps. As a result, items were collapsed into 2 categories (i.e., receives assistance, not receives assistance). The ensuing analyses were conducted using the two-point rating scale and the data was found to adequately met the assumptions of Rasch model (e.g., local independence, unidimensionality). In addition, the scale was fairly stable within setting (e.g., being at home without services, living in a nursing home) over time as well as cross settings.

Methodological challenges in measurements of functional ability in gerontological research. A review. (Avlund, 1997)

Avlund, K. (1997). Methodological challenges in measurements of functional ability in gerontological research. A review. Aging, 9(3), 164-174.

Department of Social Medicine and Psychosocial Health, University of Copenhagen, Denmark.

This article addresses two important challenges in the measurement of functional ability in gerontological research: the first challenge is to connect measurements to a theoretical frame of reference which enhances our understanding and interpretation of the collected data; the second relates to validity in all stages of the research from operationalization to meaningful follow-up measurements in longitudinal studies. Advantages and disadvantages in different methods to do the measurements of functional ability are described with main focus on frame of reference, operationalization, practical procedure, validity, discriminatory power, and responsiveness. In measures of functional ability it is recommended: 1) always to consider the theoretical frame of reference as part of the validation process (e.g., the theory of "The Disablement Process"; 2) always to assess whether the included activities and categories are meaningful to all people in the study population before they are combined into an index and before tests for construct validity; 3) not to combine mobility, PADL and IADL in the same index/scale; 4) not to use IADL as a health-related functional ability measure or, if used, to ask whether problems with IADL or non-performance of IADL are caused by health-related factors; 5) always to make analyses of functional ability for men and women separately as patterns of functional ability and patterns of associations between other variables and functional ability often vary for men and women; and 6) to exclude the dead in analyses of change in functional ability if the focus is on predictors of deterioration in functional ability.

http://www.ncbi.nlm.nih.gov/pubmed/9258374

My note:

Disability refers to the expression of a functional limitation in a social context.

For a given ability, disability occurs when there is a gap between personal capability and the demands of the activity. Disability can be alleviated from either side, by increasing capability or by reducing demand.

* Interesting thought about the standardized evaluation of disability: Someone who cannot manage shopping, because he or she lives too far away from the shopping center is disabled. But when a person moves to a place closer to the shopping center, she or he become non-disabled by this definition. 8-O

ADL is often used synonymously with PADL (physical activities of daily living).

How could functionality and disability be defined during interview and questionnaires?

• With or without difficulty
• With or without personal help
• Pain
• Reduced speed
• Tiredness
• Initiative
• With or without technical aids

The main issues relating to the measurement of ADL and IADL

• Disagreement about which activities goes where because whether a person perform a task is sometimes dependent on gender, culture, house condition etc. For instance, concerning disability involving cooking, up to the point of this writing, elder women seem to cook more than man. When a person report to not perform the cooking activity, it is because there is a lack of opportunities or because this person really have problem performing the cooking task. Similarly, when I said that I had not gone out for social functions since I had the spinal injuries, is it because the physical condition stopped me from attending these functions or is it because nobody asked? 8-O lol
• Second, should we measure maximal capacity or actual functional status
• Third, how do people evaluate the importance of being able to perform individual activities? For instance, is it more important for me to be able to walk around like ordinary people or to regain, first, my ability to lift up to 25 bls of weight?

Measurement procedure for functional ability

• Observation
• Performance
• Self-report
• Structured interview

Content validity

1. Content validity has to do with how well an instrument covers the full range of the domain
2. Most ADL/PADL measures are further development of Katz's Index of ADL
3. Most measures of IADL came from Lawton & Brody, who, recommended one 8-item scale for female and 5 for male.
4. A Canadian study showed the following reason for not performing the IADL tasks
   
   1. No need (e.g., no animal or garden for me to take care of)
   2. No need because someone else does it (e.g., my mama did cooking, grocery shopping)
   3. Doesn't know how to do it or not motivated to do it… (e.g., I take public transit in NYC… not driving)
   4. Physical inability (e.g., going up and down the hill)
   5. Fear of falling
   6. Environmental obstacles (e.g., the floor is slanted… not flat)
      
5. Old men don't cook? More associated with gender-role rather than functional disability
6. IADL differences were found among old people in different areas and countries. For instance, the reported performance of light household between people in Kuwait and West-Berlin. The plausible reason…. The report of "disability" might not due to the physical inability rather than where you live.

Construct validity

1. When the result of a measurement agrees with the theoretical expectations… we got construct validity.
2. Construct validity of a measure could be tested by the use of Factor analyses, Guttman Scalogram Analysis, and, Rasch model
3. Even though both PADL and IADL measure disability, it might cause some problems to combine by ADL and IADL in the same index.
4. Confounding results:
   
   1. Studies showing mobility forms a separate dimension
   2. ADL and IADL do not belong to the same dimension
   3. Unidimensional structure of ADL and IADL
5. The contradictory findings might be caused by differences in
   
   1. Questions
   2. Categories
   3. Statistic methods
   4. Sample population
6. Some studies distinguish functional disability into four categories
   
   1. able without difficulties
   2. Able with difficulty
   3. Able with help
   4. Not able at all

Discriminatory power

• The aim of a method may be the ability to distinguish
  
• The high functioning elderly from the medium or low
• Those with preclinical disability from these with real disability (? Is there such thing called fake disability? 8-O)
• Elderly people in different service setting
• Elderly people with different diseases
• Show variations within the single group such as the differential level of functioning among elderlies in senior homes
  
• Some functional ability scales are able to show differential discriminative effect for those with poorer functioning (sounds like the concept of α in Item Response theory to me)
• Is it possible the use a scale to assess the all range of functioning? (Item bank maybe?)
• What qualities are we trying to distinguish? Speed? Tiredness? Or Help?

The categories used are important. For instance able/not able" distinguish less than categories such as "able/able with difficulties/not able"…

Responsiveness

• Defined as the ability to detect change
  
  • IADL might be better suited for the detection of change
  • Mobility might be better suite in detecting changes in elderly than young people
• Stats related problem
• Ceiling or floor effect concerning improvement or decrease in functionality
• Inclusion of the dead in the measuring of change in functional ability…. 8-O
• Usefulness of functional ability measure
  
  • To measure consequences of the disease or condition
  • To detect risk group for earlier or preventative intervention
  • To evaluation the effect of intervention

• The types of intervention that directly affect demands and disability could be
• Modification of environment
• Activity accommodations
• External support
• Psycho-social factors
• Coping
  
• Recommendations of the authors about the measures of functional ability
  
  • There has to be a theoretical framework
  • Assess content validity before jumping into construct validity
  • Do not combine mobility, ADL/PADL and IADL in the same index
  • Always make sure that the performance problems associated with IADL are due to "physical disability"
  • Look for other differentiating factors such as gender or culture
  • Think about the impact of the dead in the study results

Activities of daily living instruments: optimizing scales for neurologic assessments. (Lindeboom, Vermeulen, Holman, & De Haan, 2003)

Lindeboom, R., Vermeulen, M., Holman, R., & De Haan, R. J. (2003). Activities of daily living instruments: optimizing scales for neurologic assessments. Neurology, 60(5), 738-742.

Department of Clinical Epidemiology and Biostatistics, Academic Medical Center, Amsterdam, The Netherlands.

The ability to perform activities of daily living (ADL) is an important part of assessment in neurologic patients. A literature search was carried out to identify multi-item ADL scales developed for the assessment of neurologic patients, comparing item content, range, and detail of ADL scales. Of the 113 ADL scales identified, 27 (24%) were designed for use in neurology. In the basic ADL (BADL) domains (basic mobility and self-care), individual items were present in 44% to 81% of instruments. In the extended ADL (EADL) domains (e.g., outdoor mobility, housekeeping), items were present in up to 67% of the instruments identified. A typical trade-off was observed between the range, the detail (number of items), and hence the practicality of a scale. In general, scales focus on either BADL or EADL domains or, on occasion, some of both, rather than measuring the full range of functioning. There are many ADL scales in neurology, with much overlap in item content, leading to redundancy. New scales developed with the traditional methods will not solve the existing difficulties associated with range and detail, ordinal scale scores, and cross-instrument comparability. The possibilities of a modern psychometric method known as item response theory that was designed to solve these problems are discussed.

http://www.ncbi.nlm.nih.gov/pubmed/12630419
http://www.neurology.org/cgi/content/full/60/5/738?cookietest=yes&eaf

My Notes

Functional outcome measure

1. Basic activities of daily living (BADL) contains item domains
   
   1. Basic mobility (e.g., transfers, walking in door)
   2. Self-care activities (toilet, hygiene, dressing, and feeding)
2. Extended ADL (EADL)—House keeping or shopping
   
   1. Househod activities (cleaning, laundry, meals)
   2. Community activities (shop, transportation)
   3. Social/recreational activities (sports, clubs, outings
   4. Cognitive activities (communicate, paperwork, finances)

The authors of this paper examined the redundancy and adequacy of neurologic ADL scales.

1. Redundancy refers to the overlap in item contents among scales
2. Adequacy refers to the comprehensiveness of a scale in term of the number of domains included.

The authors located 113 scales and retained only 27 scales that were originally developed and evaluated in the field of neurology for conditions such as stroke, dimension, dystonia, MS and brain trauma. Check the table for the reviews and comparisons for the 27 scales.

The "bandwidth fidelity problem" refers to a trade-off between the range, the detail and the practicality of a scale.

After the authors did a review on existing scales, they tried to ran analyses using Rasch model using information captured in an item bank containing 190 items to generate estimates for the item difficulties and individual ability.

Lindeboom and his associates (2003) conducted a study to examine the redundancy and adequacy of neurological ADL scales. In this study, redundancy refers to the overlaps between item contents while adequacy has to do with the comprehensiveness of the scale or the range of the symptoms the scale covers. Among the 113 ADL scales they identified through the literature, 27 of them were designed and evaluated for the field of clinical neurology. Their review found a lot of overlaps in the item contents for both the BADL (ADL) and EADL (IADL) domains, which resulted in redundancy. On the other hand, since different perspectives of functioning are more important for patients with different diagnosis, the gain in "fidelity" is often at the expense of comprehensiveness-- the "bandwidth fidelity problem." Later in this paper, the authors spoke of the promise of Item Response Theory and CAT.

A validation study of the daily activities questionnaire: an activities of daily living assessment for people with Alzheimer's disease. (Oakley F, 1999)

Oakley F, L. J., Sunderland T. (1999). A validation study of the daily activities questionnaire: an activities of daily living assessment for people with Alzheimer's disease. Journal of outcome measurement, 3(4), 297-307.

National Institutes of Health, Warren G. Magnuson Clinical Center, Bethesda, MD 20892-1604, USA. Fran_Oakley@nih.gov

Abstract: The Daily Activities Questionnaire (DAQ) was developed to assess activities of daily living (ADL) independence in people with Alzheimer's disease. After administering it to 276 people diagnosed with Alzheimer's disease, we examined the quality of the rating scale and its structure using a Rasch measurement approach. Results indicated that the original 10-point rating scale should be restructured to a 5-point rating scale to improve the quality of the instrument. In addition, we found that all but two ADL items defined the same construct and could be combined into a single summary measure of ADL independence. The remaining items were positioned along a hierarchical continuum, with IADL tasks more difficult than PADL tasks. Furthermore, the tasks were logically ordered by difficulty. We therefore report that the DAQ is a valid scale and conclude that it is a viable measure of ADL independence for studies of Alzheimer's disease.

http://www.ncbi.nlm.nih.gov/pubmed/10572383

My Note:

Daily activities questionnaire (Oakley, et al., 1991)

• PADL assess independence in individual: bathing, dressing, toileting, walking, eating and grooming
• IADL: Home care, cooking, shopping, finances, and phone use.

Some interesting rule of thumbs for Rasch analysis:

"Rasch analysis determines the unidimensionality of the instrument by examining response patterns of the items in the instrument, which are demonstrated by infit Mean Square (MnSq) statistics. The MnSq is the ratio between observed and expected variance (Wright and Masters, 1982).

An acceptable range for an infit MnSq value for a rating scale is between .6 and 1.4 (Wright and Masters, 1994). An item with a MnSq value smaller than 0.6 indicates that the item does not provide additional information beyond the rest of items on the scale. If the MnSq value for an item is greater than 1.4, either the item does not define the same construct as the rest of the items in the instrument or it is ambiguously defined.

Rasch analysis also provides a significance test for the MnSq value. A Zstd value greater than 2.0 indicates that the corresponding MnSq value is significant at the .05 level."

Due to the disordered steps in the 10 category scenario, the authors converted the original 10-point rating scale to a 5-point rating scale.

One final word for this study…. The authors consider the combined scale of ADL/IADL as unidimensional….

http://www.rasch.org/rmt/rmt82a.htm

Oakley and his associated conducted a study to assess the Validity of the Daily Activities Questionnaire (DAQ) (Oakley et al 1991). The DAQ is an 14-item instrument which was designed to assess independence in PADL (ADL) and IADL. The participants of the study were individuals diagnosed with Alzheimer's disease and who were institutionalized. The authors used the FACETS software to assess the fit of Rasch Model for the data. After the initial analyses, the authors the 10-point scale into a 5-point scale because they found the original 10-point rating scale to have limited ability in discriminating among patients' ability; they also dropped walking as an unfit item from the final analyses. Results of the Rasch analysis supports the notion of unidimensionality or an underlying ADL construct; in addition, based on results about the level of endorsability of the items, it is found that the DAQ items are spread across a hierarchical linear continuum where IADL items were found to be more difficult than PADL (ADL) items at least for patients with Alzheimer disease..

Change in MO

Starting from the following posting, I will be posting notes that I find useful for myself rather than more detailed notes for the whole paper..

http://ratologydisabled.blogspot.com/2009/03/variation-in-sources-of-clinician-rated.html

Variation in sources of clinician-rated and self-rated instrumental activities of daily living disability (Albertm et al., 2006)

Albertm, S. M., Bear-Lehman, J., Burkhardt, A., Merete-Roa, B., Noboa-Lemonier, R., & Teresi, J. (2006). Variation in sources of clinician-rated and self-rated instrumental activities of daily living disability. The Journal of Gerontology, 61A(8), 826-831.

Department of Behavioral and Community Health Sciences, University of Pittsburgh, Graduate School of Public Health, 130 DeSoto Street, Pittsburgh, PA 15261, USA. smalbert@pitt.edu

BACKGROUND: It is unclear how well self-reports and clinician ratings of performance in the instrumental activities of daily living (IADLs; household maintenance tasks) correspond and why they may differ.

METHODS: We assessed clinician-rated IADL performance using an occupational therapy protocol, the Assessment of Motor and Process Skills (AMPS). AMPS and self-rated IADL disability were compared in two groups of nondemented elderly persons without ADL limitation: a group with functional limitation only (self-reported difficulty in some area of upper or lower body function, n = 139) and a group that reported functional limitation plus IADL disability (difficulty in at least one IADL task, n = 49). Occupational therapists were blind to self-reports, and all assessments were conducted in respondent homes.

RESULTS: Self-rated IADL disability was significantly associated with the AMPS motor skill score (r = -.34, p <.001), but the motor skill score was only moderately sensitive (61%) and specific (67%) in identifying self-rated disability. In adjusted logistic regression models, clinician-rated performance and self-rated IADL disability shared some physical predictors, but only clinician-rated performance was related to cognitive status. AMPS process skill scores did not relate to self-rated IADL disability or physical or cognitive status.

CONCLUSIONS: In this sample of older adults without dementia or ADL disability, clinician ratings of IADL motor skill and self-rated IADL disability were correlated. Physical deficits appear to be more salient in self-ratings than is cognitive ability, because cognitive ability (in particular, verbal fluency) was associated only with clinician-rated IADL performance.

http://www.ncbi.nlm.nih.gov/pubmed/16912100

My Notes:

An important aspect of the AMPS is its use of a Rasch measurement model to separately estimate the parameters for the ability level and difficulty level/

What I really need from this study are the examples concerning the discordances between clinician reports and the self-reports of the patients. For instance, despite the low discriminability of self-reports on functional limitation, people reporting "difficulty carrying something as heavy as 10 lbs" are more likely to report having problems in IADLs and perform below the age norm. In addition, slower gait is found to be associated with higher risks in self-reported disability and poorer clinician-rated motor skill; for each second required in walking 4-meters increase 20% of the risk.

Another interesting finding of this study is that the gender and ethnic differences are observed in self-reported IADL disability but not in the rating of clinicians. The authors suggested that such a differential finding might have something to do with the response bias intrinsic to the self-reports.

I guess, when in dasein, it is just fairly inevitable for us to see our world as a dasein-defined world...

In addition, concerning the following quote:

"Physical deficits appear to be more salient in self-ratings than is cognitive ability"....

Well, believe me, it is much easier for me to use my head to tell that I could move less than 10 steps a time (physical deficits) than to figure out how delusional I am (mental deficits)....

Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. (Katz, 1983)

Katz, S. (1983). Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. Journal of the American Geriatrics Society, 31(12), 721-727.

Abstract: The aging of the population of the United States and a concern for the well-being of older people have hastened the emergence of measures of functional health. Among these, measures of basic activities of daily living, mobility, and instrumental activities of daily living have been particularly useful and are now widely available. Many are defined in similar terms and are built into available comprehensive instruments. Although studies of reliability and validity continue to be needed, especially of predictive validity, there is documented evidence that these measures of self-maintaining function can be reliably used in clinical evaluations as well as in program evaluations and in planning. Current scientific evidence indicates that evaluation by these measures helps to identify problems that require treatment or care. Such evaluation also produces useful information about prognosis and is important in monitoring the health and illness of elderly people.

http://www.ncbi.nlm.nih.gov/pubmed/6418786

My notes:

Interesting historical facts…..

1. Late 1800s to early 1900s: information reported was often in terms of days of sickness and sick day
2. 1940: Hierarchically ordered classifications of disability were used
3. 1950 in a Canadian Survey, a disability ratio was introduced (e.g, days of hospitalization per 100 days)
4. 1959-1980: the range of available measures of physical, mental, and social functions have expanded

For Lawton, activities of daily living covered self-maintenance functions from multiple domains. His model also includes the ordered hierarchy of six functions of Katz (bathing, dressing, toileting, transfer, continence, and feeding). The construct validity of the measure was established by observing the recovering patients passing through stages parallel to the ordinary developmental stages faced by children.

Lawton considered IADL to be more complex than IADL. The IADL functions are concerned with people's ability to cope with their environment and to adapt to the tasks that they have to perform. These including shopping, cooking, housekeeping, laundry, use of transportation, managing money, managing medication and using telephone.

(In Lawton, M.P. Assessing the competence of older people, in Kent D. Kastenbaum R, Sherwood S (EDS.): Research Planning and Action for the Elderly, New York, Behavioral Publications, 1972, PP. 122-143.)

The domain of ADL could be considered as including three self-maintenance components:

• basic ADL
• mobility
• IADL.

Some evaluation systems include all three components while others include more than one component.

Katz and his associate (1981) conducted an evaluation study checking the compatibility of 24 assessing instruments. Higher degree of compatibility is found between the basic ADL and mobility. IADL is only acceptably compatible with others.

1. The reliability and validity
   
   1. Reliability: internal consistency (relationship between items and the total scale), observer consistency (inter and intra rater) and respondent consistency (test-retest reliability)
2. Usefulness
1. Some studies found the ADL instruments useful. These are studies which purpose is to evaluate the effectiveness of interventions and to document the longitudinal projection of illness. Information provided by these studies further established the validity of the given ADL instruments.

A fairly MDS perspective concerning the 3 self-maintenance components including ADL: basic ADL, mobility and IADL…

IME and discrepancies

When I first read about how the experts found discrepancies between the observations of the others and patients’ self-report concerning their functionality and disability, it only sounded like another piece of information to be included in the measurement-related issue concerning ADL/IADL.... Good for the literature review section...

Then, I thought of how after my visit to the IME (Independent Medical Examiner who was paid for by the Workers' Compensation Insurance Company), days after I dropped off from work again, how the IME decided that I could get back to work with the light duties partially defined as lifting no more than 25 bls... while I could barely move my own butt...

That almighty IME... and the alike...

So I thought... if not then, at least now... you better not be in my shoe because... then... you will have to live through the experiences called "discrepancies reported between the observations of the others and patients’ self-report".... and... what you do to the others... now you will see... (OMG, did I just sin? Or, perhaps, just pondering about the implications of constructivist learning approach? 8-O lol)

For... I wouldn't have known.... and did not know how this line of research would have anything to do with me... until tonight... when something reminds me of IME...

A critical review of scales of activities of daily living (Law & Letts, 1989)
Evaluating activities of daily living: directions for the future (Law, 1993)

This posting is cross-posted in the Ratology Reloaded blog.

A critical review of scales of activities of daily living (Law & Letts, 1989)

Law, M., & Letts, L. (1989). A critical review of scales of activities of daily living. American Journal of Occupational Therapy, 43(8), 522-528.

Department of Medicine, McMaster University, Hamilton, Ontario, Canada.

Abstract: Occupational therapists routinely perform activities of daily living (ADL) assessments. Although the literature contains many ADL scales, few sources summarize and review the measurement properties of such scales. In this paper, standard criteria are used to review scales of basic self-care. Each scale is critically appraised regarding its purpose, clinical utility, construction, standardization, reliability, and validity. Recommendations are made regarding the ADL scales that are most suitable for describing, predicting, or evaluating ADL function. This review is intended to help therapists in selecting the most appropriate ADL measure to use in their clinical practice.

http://www.ncbi.nlm.nih.gov/pubmed/2672821

My notes:

This is a review paper for ADL scales.

The problems associated with ADL scale is that there are too many of them but not too many of them have been validated. In addition, many authors would simply select the items they want to include from existing instruments which causes problems in scale reliability and validity.

In addition, the instruments are "symptom-centered" rather than "client-centered." In other words, items included in the scale might not reflect what the patients really want…

There is a lack of a universal disability framework although there does exist a diagnostic framework in that most scales are often constructed and validated for patients of specific age groups or with certain diagnosis.

The scales were reviewed based on 6 main questions:

1. What is the purpose of the scale?
   
   1. Description, prediction or evaluation? see http://ratologydisabled.blogspot.com/2009/03/evaluating-activities-of-daily-living.html
2. Is the scale clinically useful?
   
   1. Some scales are too short and others either too complex or too long
   2. Existing research found that incongruence between observations of the professionals and those reported by the patients.
      
      1. Klein-Parris, Clermount-Michel, and O'Neill (1986) found that higher accuracy of assessment was associated with high functional patients and low complexity in the items
      2. McGinnis, Seward, DeJong, and Osberg (1986) made comparison between the rating of professionals and the patients using the Barthel Index. They found self-reported scores to be lower.
3. Was the scale construction adequate?
   
   1. Are items selected based on expert opinions or statistical properties for their discriminability, predictability or evaluate-ability?
   2. Level of measurement? Nominal, ordinal, interval or ratio?
   3. Item weighting? And, does the weighting make any difference?
4. Is the scale standardized?
   
   1. Has there been research done to establish the reliability and validity of the scale?
5. Is the scale reliable?
   
   1. How is the reliability derived? Is the statistics appropriate?
6. Is the scale valid?
   
   1. Content validity? Does the instrument cover the whole domain (i.e., disability)?
   2. Construct validity? Are you really measuring what you thought you are meaning?
   3. Criterion? Is there any criterion measure that have been determined as valid measure in the domain?

Please check Table 1 in the main article for the results of the analyses and the concluding paragraphs for the authors' recommendations concerning these ADL scales reviewed in this article.

Evaluating activities of daily living: directions for the future (Law, 1993)

Law, M. (1993). Evaluating activities of daily living: directions for the future. American Journal of Occupational Therapy, 47(3), 233-237.

School of Occupational Therapy and Physiotherapy, McMaster University, Hamilton, Ontario, Canada.

Abstract: Current assessments of simple activities of daily living (ADL) and more complex, instrumental activities of daily living (IADL) could be improved. These assessments are criticized because there are so many different tests for various diagnostic populations, because they rely on self-report rather than observation, because they are based on such varied conceptual frameworks, because they are often cumbersome and lengthy to administer, and because they often rely on outdated or specific cultural perspectives. Improvement of ADL and IADL assessment lies in making them more contextual and client specific, (i.e., by addressing clients' needs in real-life contexts that consider roles, culture, varying environments, and developmental stage.

http://www.ncbi.nlm.nih.gov/pubmed/8456923

My Notes

Interesting historical facts (really cool to know...):

• The actual term activities of daily living or ADL has not always been used in occupational therapy. The first reference to an "assessment of every day activities" was reported by Sheldon in the journal of health and physical education in 1935 (Feinstein, Josephy, & Wells, 1986)
• Edith Buchwalk (1949), a physiotherapist, was the first to publish the term activities of daily living; she used it to refer to an assessment checklist.
  

Currently ADL & IADL evaluation issues

• Predictive ADL and IADL tools attempt to predict an event or functional status in another situation on the basis of the client's current level of functional performance.
• Evaluative ADL and IADL scales are used to evaluate client functional performance over time, thereby allowing the evaluation of process and the effectiveness of the occupational therapy intervention—norm-reference tool.
• Many ADL assessments have been developed but not those for IADL.

Analysis of current evaluation practices

• There are too many ADL assessments developed and validated for different populations. Feinstein et al. (1986) did a review and found at least 43 different instruments developed in a period of over 40 years.
• There is a significant trend for developing self-report instruments. However, there are significant differences in the observations made through self-report and reports by professionals.
• ADL and IADL instruments are often developed with different conceptual framework, including the inclusions of different impairment-related items and scaling methods. For instance, most instruments use ordinal scaling which results in the issue of incompatibility between the scores because the distance between scores are not equal (Merbits, Morris, & Grip, 1989).
• Another issue concern the administration of ADL and IADL because the instruments are either too long or too short.
• A fifth issue related to the instruments concerning the philosophical and ethical basis. For instance, following the philosophy of the medical model, improvements are made through changes in the clients. By reframing the problem as a function of the interaction between the individual and the environment, the resolution would involve the change in the environment.
• Finally, the biggest limitation of the current instruments lies in their nature of decontextualization. With most evaluations, factors such as the patient's cultural background, roles and developmental stage were not taken into consideration.

Essentially, there is a consensus among the experts that there should be better methods of development and validation for ADL and IADL instruments (Feinstein et al., 1986; Fisher, 1990; Law & Letts, 1989)

What should be evaluated?

• From the occupational perspective, assessments and their process should focus on the daily activities endorsed by the clients. Feinstein et al (1986) identified three advantages of client-centered assessment: clear goals, interventions are valued by the clients and intervention outcomes are enhanced.
• Other than focusing solely on the changes in the clients, there is also a need to modify contextual factors such as physical, cultural, social, economic or institutional environment.
• In addition, the intervention outcome should be the contextualized occupational performance of the ADLs rather than the decontextualized performance of the components.

Methodological issues concerning the development of the instruments:

• Testing the reliability and validity of the instruments (Classical Testing Theory CTT)
• Investigating the use of new methodologies such as Rasch Model (IRT)
• Investigating the use of alternative scaling models.

Disability: a model and measurement technique. (Williams, Johnston, Willis, & Bennett, 1976)

Williams, R. G., Johnston, M., Willis, L. A., & Bennett, A. E. (1976). Disability: a model and measurement technique. British Journal of Preventive & Social Medicine, 30(2), 71-78.

Abstract: Current methods of ranking or scoring disability tend to be arbitrary. A new method is put forward on the hypothesis that disability progresses in regular, cumulative patterns. A model of disability is defined and tested with the use of Guttman scale analysis. Its validity is indicated on data from a survey in the community and from postsurgical patients, and some factors involved in scale variation are identified. The model provides a simple measurement technique and has implications for the assessment of individual disadvantage, for the prediction of progress in recovery or deterioration, and for evaluation of the outcome of treatment regimes.
http://jech.bmj.com/cgi/content/abstract/30/2/71

My notes:

• Katz et al. (1963)—hypothesize that the relearning of very basic abilities such as continence, feeding, and washing followed the same pattern taken by learning in infancy
• Carroll (1962) explored similar orderings in the recovery of stroke patients
• Harris and Luck (1972) explored cumulative orderings of activities

The authors' work assumed that disability would fit the cumulative model called Guttman Scaling (Guttman, 1950)

Example:

1. Getting up
2. Going out of the house
3. Going back to work
   

With 0 means nondisabled and 1 means disabled

Scale Type 111 011 001 000

Nonscale Type 101 110 100 010

Different types of plausible error: measurement error, sampling error and random variation among individuals

Two coefficients are used to represent the conventional level of acceptable errors which allows us to draw inferences that there exists a valid cumulative and unidimensional Guttman scale:

• Coefficient of reproducibility: Acceptable when >.90. It reflects the variation in the proportion of disabled people under each item
• The coefficient of scalability: Acceptable when > .6. It tells us the proportion of the remaining response that could be correctly predicted using our hypothesis.

The authors then went on analyzing two sets of data using the Cuttman scale hypotheses.
For me, the most interesting part of their finding is in how the ordering of occurrences for certain disabilities is dependent on gender and SES. For instance, men were slower in relinquishing their independent mobility than cooking for themselves… while women, the contrary… 8-O

According to the authors, the usefulness of this model is that it helps us to make prediction about the deterioration and recovery processes.

Please see the following link for more information about Guttman Scale: http://en.wikipedia.org/wiki/Guttman_scale

A critical review of 12 ADL scales (Bruett & Overs, 1969)

Bruett, T. L., & Overs, R. P. (1969). A critical review of 12 ADL scales. Physical Therapy, 49(8), 857-862.

Abstract: Twelve ADL scales reported in the literature since 1951 are analyzed according to activities measured, number of scoring steps, and type of scaling used. The problems associated with each and the type of professional worker for which each is most useful are discussed. It is suggested that mobility be expressed in natural environment as well as clinical measurement terminology.

My Notes

Comparison of 12 ADL scales (I decided to not embed the google doc spreadsheet because it causes errors in the HTML code)

3 general purposes for the ADL scales

• To inventory severity of disability in patient populations and to measure the effectiveness of treatment.
• To construct patient profile and to provide information concerning intervention
• To appraise changes in goal attainment

Scaling: Nominal, ordinal and interval

The purposes for which the measurement is being made should determine the measurement units in which the outcome is expressed.

"Should an investigator attempt to build a unidimensional scale from which a Guttman-type scale may be constructed" even though there seem to be much evidence showing the multidimensional nature of the disability indicators?

What is the relationship between patient's self-report and the report of significant others?

Replication of the multidimensionality of activities of daily living. (Fitzgerald, Smith, Martin, Freedman, & Wolinsky, 1993)

Fitzgerald, J. F., Smith, D. M., Martin, D. K., Freedman, J. A., & Wolinsky, F. D. (1993). Replication of the multidimensionality of activities of daily living. Journal of Gerontology, 48(1), 4.

Department of Medicine, Indiana University School of Medicine.

Abstract: We attempted to replicate the three-dimensional factor structure of a previously proposed ADL scale and demonstrate an association between the advanced ADL dimension and cognitive function. Data used in these analyses were baseline assessments of health and functional status of hospitalized patients enrolled in a randomized controlled trial of case managers as a means of reducing health care utilization. We submitted 14 items from the OARS to a two-stage process of principal components factor analysis. Four significant dimensions emerged that were remarkably similar to the advanced, basic, and household ADL dimensions reported by Wolinsky and Johnson (1991). In this sample of hospitalized patients, however, incontinence emerged as a weak fourth dimension. Multiple regression of SPMSQ mental status examination scores on these ADL dimensions demonstrates the association between cognitive function and the advanced ADL dimension. These data confirm that the underlying structure of ADLs consists of at least three separate dimensions, one of which is aligned with cognitive capacity.

http://geronj.oxfordjournals.org/cgi/reprint/48/1/S28

My notes:

Some authors thought Kat's et al.'s ADL and Lawton & Brody's IADL should be modeled separately. Spector et al & Kempen and Suurmeijer suggested a unidimensional structure. Wolinsky et all suggested a three dimensional model: basic ADL, Household ADL and Advanced ADL.

The authors of this study used principal Component analyses with a different dataset, trying to replicate the same 3 factor structure obtained in Wolinsky & Johnson (1991). The initial results of their analyses resulted in a 4-factor structure. In addition to Basic ADL, Household ADL and Advanced (cognitive) ADL factors, the fourth factor concerning incontinence emerged in their analysis.

Although results of this report did successfully replicate the 3-factor structure of ADLs as proposed by Wolinsky & Johnson (1991), one comment I have is that, for the purpose of this study, wouldn't confirmatory factor analysis (CFA) be more pertinent? At the same time, based on the analyses I ran using NLTC survey data, I also came upon the cognitive factor...

The Use of Health Services by Older Adults. Wolinsky, F. D., & Johnson, R. J. (1991).

Wolinsky, F. D., & Johnson, R. J. (1991). The Use of Health Services by Older Adults. Journal of Gerontology, 46(6), 12.

Department of Medicine, Indiana University School of Medicine.

Abstract: Using baseline data on the 5,151 respondents surveyed as part of the panel design of the Longitudinal Study on Aging (LSOA), this article estimates, cross-sectionally, the relationships hypothesized in the behavioral model of health services utilization. In addition to the traditional indicators of the predisposing, enabling, and need characteristics, the richness of the LSOA permits the inclusion of measures of multigenerational living arrangements, kin and nonkin social supports, health worries and the sense of health control, health insurance coverage, residential stability, and several multiple-item scales of functional limitations. Despite these innovations, the ability of the behavioral model to accurately predict the use of health services by older adults remains relatively unchanged. Important conceptual clarifications involving the hypothesized relationships, however, are identified and discussed.
http://www.ncbi.nlm.nih.gov/pubmed/1940101

My notes:

6 instruments were used to measure need characteristics in this study. One of these is a standard dichotomous question asking the patients to rate their overall health. The remaining 21 items include:

• The Activities of Daily Living (ADL) scale by Katz et al. (1963)
• The Instrumental Activities of Daily Living (IADL) scale by Duke University Center for the Study of Aging and Human Development (1978)
• Nagi's (1976) disability scale

5 unidimensional scales emerged from the principal component analyses

1. Basic activities of daily living scale (Basic ADL; alpha=.827): containing 5 items from ADL, including bathing, dressing, getting out of bed, walking, and toileting
2. Household activities of daily living scale (Household ADL; alpha=.828): containing 4 items from IADL including meal preparation, shopping, light housework and heavy housework
3. Advanced activities of daily living scale (Advanced ADL; alpha=.641): containing 3 items from ADL and IADL, including managing money, telephoning and eating

The other 2 scales contain items from Nagi's (1976) disability scale with one scale concerns lower and the other concerning upper body limitations.

1. Lower body limitation scale (alpha=.863) includes difficulties in walking a quarter of a mile, walking up 10 steps without rest, standing or being on one's feet for 2 hours, stooping, crouching, kneeling and lifting or carry 25 pounds.
2. Upper body limitation scale (alpha=.588) includes difficulties in sitting for 2 hours, reaching up over one's head, reaching out as if to shake hands, and using fingers to grasp objects.

In this article, the authors also found that, among the 3 ADL scales, having difficulties with the advanced ADLs was the single significant predictor of events such as taking the bed-disability days, hospital contact and mortality. Advanced ADLs, at the same time, are also instrumental in the detection of cognitive impairments.

On the other hand, lower body limitations, as opposed to those of the upper body, seem to be a better determinant for the utilization of health services.

Disability Blog

I will be posting articles concerning disability under this blog...