KT Library Descriptor Scales


In the Knowledge Translation process it is important for the results of research to be described in terms of their readiness for conversion to "knowledge" and the degree to which that knowledge is appropriate for application or use. Repositories of information that have been developed through NIDILRR funding are often asked about the degree to which particular products in their holdings are "evidence-based." This term generally addresses (1) the degree to which the information/product is based upon the results of research activities/findings and (2) the degree to which those research findings confidently predict what will happen if the information/product is used in intended ways.

It is important for people with disabilities, their families, service providers, and others to be able to access useful research-based information and to understand its appropriate use. A standard descriptor scale can be a useful tool that reflects both the degree of evidence supporting information/products as well as the readiness/confidence that the reader can place in immediate use with predictable results. The KTDRR's descriptor scales provide information regarding the strength of evidence, consumer orientation, and readability of the articles and publications reviewed.

 

Strength of Evidence

The strength of evidence is examined in terms of the rigor of the research supporting the informational material and its recommendations. The following scale of 1-5 describes the supporting evidence observed during the review of the product.

Scale Evidence

1

Supporting evidence is based on opinion of the author(s).

2

Supporting evidence is based on expert opinions of expert panels, committees, professional associations, or consumer organizations or other stakeholder groups.

3

Supporting evidence is based on qualitative or quantitative research study designs that are non-randomized, controlled, cohort based, case series, case-controlled, cross sectional, survey or descriptive/
exploratory in nature. Innovative technologies/devices are supported by consumer efficacy reviews and evaluations.

4

Supporting evidence is based on mixed method research designs and/or randomized controlled trials with small sample sizes showing statistically non-significant trends and may show false negative results. Technologies and devices are tested rigorously by substantial number of consumers demonstrating similar consistent results when used.

5

Supporting evidence is based on a large, long-term randomized controlled trial or systematic reviews/meta-analyses of randomized controlled trials of smaller sample sizes that yield sufficient power to confidently predict cause and effect or answer the questions "what works." Technologies/devices are supported by user data demonstrating measured effectiveness and benefit sufficient to support predictable widespread use or technology transfer.

 


Consumer Orientation

This scale describes the degree to which the product acknowledges and aligns with known survey or other results targeting needs, interests, desires of the target group for use of the product/innovation.

 

Scale Consumer Orientation

A

Significantly oriented around consumer data

B

Includes some consumer data

C

Does not include consumer data information




Readability

Readability describes the degree to which the product is presented using language, a reading level, and a level of technicality that promotes understanding for a specific audience. The results/outcomes of recommendations, interventions, or use are clear and user-friendly. The following scale is based on the Flesch-Kincaid grade level score.

 

Scale (Flesch-Kincaid)

I

Low reading level (grade 6 and below)

II

Average reading level (grades 7-11)

III

High reading level (grade 12 and over)

The Flesch-Kincaid Grade Level score rates text on a U.S. grade school level. For example, a score of 8.0 means that an eighth grader can understand the document. For most standard documents, aim for a score of approximately 7.0 to 8.0.

The formula for the Flesch-Kincaid Grade Level score is:
(.39 x ASL) + (11.8 x ASW) – 15.59
Where:
ASL = average sentence length (the number of words divided by the number of sentences)
ASW = average number of syllables per word (the number of syllables divided by the number of words)

While Microsoft Word™ checks spelling and grammar, it can display information about the reading level of the document, including the following readability scores. Each readability score bases its rating on the average number of syllables per word and words per sentence.

To display readability statistics in Microsoft Word™:

1. On the Word™ Edit menu, click Preferences, and then click Spelling and Grammar.
2. Select the Check grammar with spelling check box.
3. Select the Show readability statistics check box, and then click OK.

Next go to the Word™ Tools menu, and select Spelling or Spelling and Grammar.

When Word™ finishes checking spelling and grammar, it displays the readability statistics for the document (See example below).

 

Readability Statistics
Counts

Words

14,384

Characters

67,841

Paragraphs

208

Sentences

630

Averages

Sentences per Paragraph

3.2

Words per sentence

22.6

Characters per Word

4.6

Readability

Passive Sentences

9%

Flesch Reading ease

58.0

Flesch-Kincaid Grade Level

10.4

(Source: Microsoft Office™ Help documentation)


Example of KTDRR's descriptor scales

Descriptor Scales

Evidence: 2 - Expert opinions
Consumer Orientation: C - No data
Readability: Grade 12 or above