Evaluation of the Home Environment Assessment for the Visually Impaired (HEAVI): an instrument designed to quantify fall-related hazards in the visually impaired

Background To (1) develop and refine the Home Environment Assessment for the Visually Impaired (HEAVI), and (2) determine the interrater reliability of this instrument, which was designed to quantify the number of fall-related hazards in the homes of individuals with visual impairment. Methods Twenty homes of community-dwelling adults were included in this study. Each home was graded by an occupational therapist (OT) and two non-expert (NE) graders. Seventy-three HEAVI items were evaluated in eight rooms, for a total of 185 potential hazards per home (some items were assessed in multiple rooms). Pairwise and three-way agreement between graders was evaluated at the item, room, and home level using Krippendorff’s alpha and intraclass correlation coefficients (ICC). Additionally, the most hazardous home locations and items were determined by comparing the mean and standard deviation of the number of hazards by room and grader. Results Of the 73 items, 45 (62%) demonstrated at least moderate agreement overall and for each OT/NE pair (Krippendorff’s alpha >0.4), and remained in the final instrument (a total of 119 potential hazards per home as some items were assessed in multiple rooms). Of these 119 potential hazards, an average of 35.7, 33.2, and 33.3 hazards per home were identified by the OT and NE graders, respectively. Moderate to almost perfect agreement on the number of hazards per home and number of hazards per room, except the dining room, was found (ICCs of 0.58 to 0.93). Bathroom items were most often classified as hazards (>40% of items for all graders). The item classes most commonly graded as hazardous were handrails and lighting (>30% of items). Conclusion Our results indicate that NE graders can accurately administer the HEAVI tool to identify fall-related hazards. Items in the bathroom and those related to handrails and lighting were most often identified as hazards, making these areas and items important targets for interventions when addressing falls.


Background
Falls affect one third of adults over the age of 65 annually [1] and are the leading cause of fatal and nonfatal injuries in this age group [2]. Many older adults who fall develop fear of falling, and up to 40% will restrict their activities of daily living, leading to a decline in physical activity and social interactions [3]. Fall risk factors can include features of the home environment such as loose rugs, floor clutter, and poor lighting [4,5], which are possible targets for fall interventions. The majority of falls occur in or near the home [6][7][8], highlighting the importance of developing and evaluating tools that assess fall risk in the home environment. As studies have shown that home modifications are effective in reducing falls, a reliable tool is important to inform such interventions [9][10][11][12][13].
The home environment may pose particular risk for older adults with visual impairments given difficulty with hazard perception. Impaired vision can manifest as visual field loss, decreased visual acuity, and/or decreased contrast sensitivity, and each is associated with increased fall risk [14][15][16][17][18][19][20][21][22][23][24]. In the United States, 2.4 million adults have low vision, and an additional 937,000 are blind; these numbers are projected to increase to 3.9 and 1.6 million, respectively, by the year 2020 [18]. The high prevalence of visual impairment, along with the important role that environmental hazards may play with regard to falls, point to a need for home assessment tools that detect hazards that are specifically problematic for older adults with vision loss.
Multiple home assessments have been developed to evaluate the home safety and fall risk of older individuals, though none have focused on individuals with visual impairments [25][26][27][28][29]. Many of these home assessments evaluate general indoor and outdoor lighting but do not adequately assess home hazards that are likely to be problematic for individuals with poor vision, such as lighting (wattage, placement, switch access, window coverings), color contrast, visual distractions, and glare [30]. There has also been an overreliance on subjective assessments of home environmental conditions in previous tools [30], as well as required grading by experts such as occupational therapists. Low vision rehabilitation has increasingly focused on home-based services, including evaluation and modification of the home environment; however, the lack of a standardized assessment for this population makes it impossible to compare outcomes of evaluations and interventions across practices. This represents an unmet need for a vision-centric home assessment tool that can be easily and reliably administered by individuals with varied training and areas of expertise [31].
This study reports on a novel tool, the Home Environment Assessment for the Visually Impaired (HEAVI), designed to identify specific hazardous items that may contribute to fall risk in persons with poor vision. The aim of this work was to: (1) develop and refine the HEAVI, and (2) determine the interrater reliability of this instrument. Items for the tool and the approach to measurement were derived from the literature and based on the Home Environmental Assessment Protocol (HEAP), an assessment for persons with dementia living at home that has been shown previously to have adequate content validity and interrater reliability [25]. Here, we assess the interrater reliability of HEAVI items based on agreement between an expert grader (a licensed occupational therapist with a clinical practice focused on assessing homes for safety) and two non-expert graders (non-occupational therapists without specialized training in environmental assessments). The goal of creating this tool is to support further research to identify which home hazards contribute most to falls, predict who is at greatest risk of falling, and develop targeted interventions for this population.

Study participants
The study protocol was approved by the Johns Hopkins Medicine Institutional Review Board. A convenience sample of 20 participants was recruited from the Baltimore/DC area from various sources including home-and clinic-based patients and word-of-mouth referrals, and informed consent was obtained from each participant. These participants were not recruited based on visual impairment status or overall health status.

Development of the HEAVI
This home assessment tool adapted many items from the HEAP [25], but excluded items specific to individuals with dementia and caregivers (e.g., lack of safety latches on cabinet doors, lack of safety adaptations to oven/stove). A total of 18 items related to poor vision were initially added based on a literature review and the professional experience of an occupational therapist (OT) with expertise working with individuals with vision loss (VG). The literature review used search terms related to lighting, object contrast, stairs, and visual impairment; was conducted in PubMed and Google Scholar; and included literature from the past 30 years that was published in English. The references from sentinel work and conference proceedings in environmental modification, falls, and occupational therapy related to lighting, contrast, stairs, and visual impairment were reviewed for citations not captured by the original review. Additionally, relevant government documents were reviewed to identify safety guidelines for hazard classification cut points, such as stair riser height.
Items added to the HEAVI included 12 items assessing lighting and six assessing contrast between objects, which are particularly important for those with vision loss, as lack of appropriate lighting and contrast may make those with visual impairment at greater risk of falling . [32,33].
Items were chosen and worded such that no specific training would be needed to complete the home assessment. Items were reviewed and refined twice based on review and discussion within the research team.
The 73 distinct items in the initial instrument were subcategorized into five classes: 1) handrails, 2) lighting, 3) floors, 4) furniture, and 5) other. These items were graded in up to eight rooms of the home: 1) entryway, 2) living room, 3) dining room, 4) kitchen, 5) bedroom, 6) bathroom, 7) stairs, and 8) hallways. Of the 73 distinct items, several (such as ambient lighting) were graded in multiple rooms, for a total of 185 possible hazards per home. Table 2 summarizes the items, shows the item subcategorization, defines the threshold for classifying items as hazardous, and outlines the potential rooms where each item was assessed.

Training of graders
Prior to grading, the two non-expert graders received instruction on how to complete the home assessment from the OT expert grader (VG). The non-expert graders were both ophthalmic research technicians familiar with interacting with visually impaired patients, but with no prior experience in home assessment. This group was chosen, as we wanted to be able to assess a non-expert group of graders. The expert grader reviewed the home assessment tool with both non-expert graders, explained how each item should be assessed, and provided basic definitions of items on the form (e.g. what constitutes a threshold), as well as training on measuring items such as ambient lighting. This training lasted approximately 2 h. The OT confirmed with each non-expert grader that the assessment process was understood. All three graders independently assessed each room of the first test house. After the first home was assessed, all three graders discussed and compared their results. The OT discussed and explained the proper procedure(s) for each discrepancy grading between the two non-expert graders and the OT. A second house was assessed, and the same process was repeated, with each session lasting approximately 2 h. Neither of the homes used for training were included in these analyses. During the data collection of the 20 homes included in these analyses, the graders did not discuss or compare results.

Home assessment procedures
A single home visit was scheduled for each study participant. Participants were instructed to arrange all rooms, including lighting, as they normally would, and asked not to clean or tidy prior to evaluation of the home. Study participants were typically in the home at the time of the visit, but not necessarily in the individual rooms being evaluated. For each home assessment, all three graders independently assessed the home and agreed on the specific entrances and rooms to evaluate, but each grader evaluated rooms independently and in separate rooms. There was no discussion to address whether each grader evaluated the same items within each room.
For each categorical item, answer choices included "Yes," "No," or "Not Assessed," and items graded as "Yes" were classified as hazards. Clutter was defined as items on the floor, excluding furniture, that cover potential walkable space and was categorized as "None," "1-25%," "26-50%," "51-75%," or "76-100%." Light intensity indoors and outdoors was measured in lux using a digital light meter (Dr. Meter model LX1330B). All size and distance measurements were rounded to the nearest half-inch. These continuous measures were then reclassified as hazardous or non-hazardous based on guidelines from the Americans with Disabilities Act [34], International Residential Code [35], Lighting Research Center at Rensselaer Polytechnic Institute [36], and Illuminating Engineering Society of North America [37]. Demographic information such as age (in years), race (white or African-American), years of education completed, cohabitation status (living alone or with others), marital status (married or single/divorced/widowed), and employment status (employed or retired/disabled/unemployed) were collected from all participants. Participants also provided information on their homes, including dwelling type (apartment or house), ownership status (rent or own), number of floors, and total number of rooms in the home. Home square footage was obtained from Zillow, an online real estate database (http://www.zillow.com), to characterize the size of the homes being graded.

Statistical analyses
Refinement of the HEAVI Item grading was aggregated to calculate: 1) the total number of rooms per home with each type of hazard (for example, the number of rooms with an ambient lighting hazard), 2) the total number of hazards in each room, and 3) the total number of hazards in the entire home. Interrater agreement was calculated for each of these variables by comparing responses from each of the non-expert graders with the expert grader, responses between the non-expert graders, and responses across all three graders as a three-way comparison. Agreement on the grading of each item was calculated using Krippendorff's alpha, a standard measure of agreement that accommodates missing data [38]. Based on these results, the HEAVI was refined by removing items with poor agreement using criteria from Landis and Koch [39]: 1) poor agreement (<0.00), 2) slight agreement (0.00 -0.20), 3) fair agreement (0.21 -0.40), 4) moderate agreement (0.41 -0.60), 5) substantial agreement (0.61 -0.80), and 6) almost perfect agreement (0.81 -1.00). Items with less than moderate agreement (≤0.40) were removed from the instrument prior to further analyses.

Assessment of interrater agreement
Using the refined tool described above, we aimed to determine if individuals without specialized occupational therapy training can reliably use this instrument. To do so, intraclass correlation coefficients (ICC(2,1)) and 95% confidence intervals (CI) were used to calculate the absolute interrater agreement for each of the 20 homes. To do so, we aggregated the number of rooms with each hazard and number of hazards per room and home. Intraclass correlation coefficients were qualitatively evaluated using criteria from Landis and Koch [39], described above.
Bland-Altman plots were constructed to determine if agreement was constant across the number of noted hazards. [40] Lastly, the most hazardous home locations and item classes were determined by comparing the mean and standard deviation (SD) of the number of hazards by room and grader. All data were analyzed using Stata 14.0 (StataCorp, LP, College Station, TX).

Results
This study includes 20 participants and their homes. The average age of study participants was 63.1 years (SD = 18.8). Approximately 25% were male, and 90% were white (Table 1). Most participants lived in a home (65%) as opposed to an apartment (35%), with a mean of 2.1 floors (SD = 0.9), 4.4 rooms (SD = 1.2), and 2,202 sq. ft. across residences.

Refinement of the HEAVI
Krippendorff's alpha values were calculated for each item across all rooms to determine agreement between all three graders ( Table 2). Levels of three-way agreement ranged from low levels of agreement (−0.06) to perfect agreement (1.00). Among the 73 items graded, 48 (66%) demonstrated moderate to perfect agreement (three-way Krippendorff's alpha >0.4) [39], while 25 (34%) demonstrated fair to poor agreement and were not included in the final instrument. Among the 48 items showing moderate to perfect agreement, five (10%) failed to demonstrated moderate to perfect agreement in one or both pairwise expert/non-expert values. Two of these five (uneven dirt path on public sidewalk, no light source in bathtub/shower with curtains closed) demonstrated a Krippendorff's alpha of at least 0.3 in each expert/non-expert comparison and were included in the final item list, while the remaining three (cracked pavement on private path, sloped ground on private path, stair nosing >1.5 inches) did not meet this standard and were excluded, leaving 45 of the original 73 items in the final instrument, for a total of 119 possible hazards per home (reduced from the original instrument that contained 185 total possible hazards). Based on these results, the HEAVI was refined such that 12 of the original 16 objective measures (75%) and 33 of the 57 subjective measures (58%) comprised the final instrument (p = 0.22).

Assessment of interrater agreement
Using this final instrument, the average number of hazards identified per home did not differ by grader. Out of a total of 119 possible hazards per home, an average of 35.7 (SD = 9.1) were identified by the expert grader, and 33.2 (SD = 9.5) and 32.3 (SD = 8.9) for the two nonexpert graders (p >0.20 for each pair-wise comparison of expert and non-expert graders) ( Table 3). There were no significant differences in the number of hazards identified in each of the eight rooms by the expert and each non-expert grader (p >0.20 for all comparisons).
Agreement on the total number of hazards per room and per home was evaluated using the final instrument. Agreement on the total number of hazards per home was 0.83 across all three graders; pairwise agreement between the expert and each non-expert grader were 0.84 and 0.81, and agreement between the two non-expert graders was 0.85, indicating almost perfect agreement for each comparison (ICC >0.81) (Table 4). Likewise, agreement on the number of hazards per room was substantial to almost perfect (ICC >0.61) for all three-way comparisons (Table 4). In pairwise comparisons of the       Step stool required to get onto bed Differences in the number of hazards identified by the expert and each non-expert grader are shown in Bland-Altman plots ( Fig. 1a and b). On average, there was no significant difference between the number of hazards identified by the expert grader as compared to the nonexpert graders (non-expert 1 vs. expert: +2.5, 95% CI = −7.1 -12.1, p = 0.40; non-expert 2 vs. expert: +3.5, 95% CI =−5.8 -12.7, p = 0.23). Additionally, the difference in the number of hazards identified by the non-expert and expert graders did not vary across the observed range for average number of hazards identified by each expert/non-expert pair (p = 0.73 for Fig. 1a and p = 0.91 for Fig. 1b).

Home assessment results
We also examined our data to identify which home locations and item classes were most hazardous. Of the rooms graded, the bathroom was found to be the most hazardous as judged by the total number of hazardous items and the percentage of items graded as hazardous (Table 3). There was a total of 16 items graded in the bathroom for each home, and the mean number classified as hazardous ranged from 6.6 to 7.4 for all three graders. Of the five types of hazards (handrails, floors, lighting, furniture, and other, as shown in Table 2), all three graders most often identified items related to handrails and lighting as hazardous. There were 10 handrail items graded per home, and the mean number graded as hazardous was ≥3.1 (>30%) for all three graders. Of the 37 lighting items graded per home, the mean number identified as hazardous was ≥11.5 (>30%) by all three graders. Additionally, all three graders reported that ≥60% of rooms had less than  300 lux of ambient light (the classification of ambient lighting hazard). All three graders also found that most rooms had an average ambient lighting level below 300 lux across the whole home, with the bathroom serving as an exception (where average lighting was measured as >321 lux by all three graders).

Discussion
Here, we develop and refine the HEAVI tool, a novel instrument designed to: (1) assess fall-related hazards specific to older adults with visual impairments; (2) be administered by a non-expert grader evaluating the home environment; (3) incorporate several elements regarding lighting, contrast, and changes in elevation which may be particularly hazardous for persons with poor vision; and (4) objectively measure home features when possible.
While 73 items were initially evaluated as part of this screening tool, only 45 of these demonstrated acceptable agreement between an expert and two non-expert graders. Many of these items were graded in multiple rooms, and therefore the original instrument, which included a total of 185 potential hazards, was reduced to 119 in this final instrument. When these final 45 items were assessed, good agreement was noted between the expert and non-expert grader with regard to overall number of home hazards and the number of hazards in each of the eight rooms graded. A larger proportion of objective items was retained in the final instrument as compared to subjective items, but this difference was not significant (p = 0.22) suggesting a trend towards objective measures being more reliably graded. Several other home assessments have been previously validated, [25][26][27][28][29] though the current instrument was designed to fill a specific niche. First, it was primarily geared towards the visually impaired, with an emphasis on lighting, contrast, and other items which could generate particular problems for those with visual impairment. It is quite possible that these same hazards would pose risks to persons without visual impairment as wella point which requires further study. It is unclear which specific types of visual impairment would be best served by the HEAVI, though multiple visual measures including visual field loss, contrast sensitivity, stereo acuity, and visual acuity have all been associated with higher rates of falls, [14][15][16][17][18][19][20][21][22][23][24] suggesting that most eye diseases will produce higher rates of falls which could potentially be modified via a safer home environment. Second, the instrument was designed to be geared specifically to fall-related hazards and did not evaluate other home hazards that may also be relevant to the visually impaired (i.e. dangers related to food preparation). Finally, the assessment tool was designed to be administered by a person without specific skills in home assessments, as most low vision services are delivered by persons without expertise in modifying homes to reduce fall risk.
Our results indicate that non-expert graders can reliably administer this home assessment tool, as we observed good interrater agreement. We observed substantial to almost perfect agreement (>0.60) between all three graders for the number of hazards found in all rooms, with the exception of the dining room where agreement was moderate (0.41 -0.60) ( Table 4). Agreement on the number of hazards in dining rooms was almost perfect between the two non-expert graders, which suggests that while the non-expert graders had a low level of agreement with the expert grader, their ratings were consistent with each other. The items graded in the dining room that had fair to poor agreement between the expert and each non-expert grader (ICCs ranging from −0.13 to 0.36), but moderate agreement between the two non-expert graders (ICCs ranging from 0.47 to 0.54) were assessments of windows without sheer coverings and exposed light bulbs. High agreement (>0.80) was observed for the number of bathroom, entrance, and total home hazards found by the three graders (Table 4). This high level of agreement between graders may reflect the inclusion of objectively graded items (such as stair riser height and number of handrails) not present in other existing home assessments. Additionally, the non-expert graders received training from the same expert grader to whom they were being compared. Only two homes (not included in this study) were used for training, though it is possible that some degree of training from a home-assessment professional may be needed to achieve the high level of interrater agreement in future use of the tool.
We did not find a difference between the total number of hazards rated between the expert and the two nonexpert graders ( Fig. 1a and b). Additionally, differences in the number of graded hazards were not influenced by the presence of high or low number of hazards within a given home. Over the eight rooms graded, the observed differences amount to less than one hazard per room. While this difference was not statistically significant, it is difficult to determine the practical implications of this difference from these analyses, as each potential hazard may pose a different level of fall risk. It is possible that additional training, for example increasing training to three or more homes, may reduce this difference.
In addition to creating a final instrument, we examined the types and location of hazards found in this study population. The room with the highest percentage of items graded as hazardous was the bathroom (Table 3), while handrail and lighting items were most often hazardous in the entire home. These results suggest that these areas/ classes of items may be particularly important when addressing falls. Poor ambient room lighting (<300 lux) was among the most commonly reported hazards, and only lighting in the bathroom was, on average, greater than 300 lux. This suggests that inadequate lighting is a pervasive in-home factor contributing to falls, especially among those with vision loss. As lighting is an easily modifiable environmental condition, this study provides preliminary evidence that home lighting is an important intervention target.
There are limitations to this study that need to be considered. First, assessments were only made at one time point, and therefore our reliability analyses do not assess parameters of individual grader repeatability. Second, a convenience sample of participants that included those with and without visual impairment was used to evaluate this instrument. While we do not expect the performance of the in-home assessment tool to differ by degree and type of vision loss, this will need to be explored further in a larger study population. Third, this instrument does not include an assessment of the individual's physical or mental abilities, and therefore does not assess the individual's interaction with their environment. It is possible that some identified hazards may not be hazardous for a particular individual, or vice versa. Fourth, we compared two non-expert graders to one expert for these analyses. To determine the clinical utility of this instrument, further comparisons with a broader group of graders (both expert and non-expert) will be required. Fifth, we only focused on one owner/occupant of the home given the logistical challenges of getting consent from all occupants, which may bias the demographic data that was collected. However, the intent of this work was not to characterize the owners of the home, but to see if the graders could grade homes similarly. Sixth, the items added to the HEAP were added largely based on the experience of the OT (VG), though additions were supported by a systematic review of the literature. Additionally, the expert grader who helped to develop the HEAVI also trained the non-expert graders. It is possible that this could introduce bias, but it is unknown if our expert grader is superior to other OTs in getting non-expert graders to produce the same grading. Lastly, this study does not determine how the number of hazards relates to fall risk; however, this work is currently underway, and prior studies have found a relationship between home hazards and risk of home injury [12,41].
While the developed home assessment tool was based on a prior instrument, the HEAP, this assessment tool is unique. The final assessment includes 10 items assessing lighting and two items assessing contrast between objects. Both are particularly important for those with vision loss, [32,33] and lack of appropriate lighting and contrast places those with visual impairment at greater risk of falling. Additionally, this tool includes a greater emphasis on objective measurements, which are typically not evaluated in existing home assessments. Inclusion of objective measures was shown to facilitate agreement and repeatability between graders, which in turn may reduce the need for specialized expertise to accurately administer this in-home assessment tool. Lastly, this instrument is intended to grade the home, and not the person within the home. To that extent, it may be relevant when setting up residences for older adults (where the person living there may not be known) and for common spaces where multiple people will be living.

Conclusion
The development of this tool has important clinical implications. While multiple in-home fall hazard assessments have been developed, none have specifically included items that assess fall risk among those with vision loss. Both the high rate of falls and the increasing number of visually impaired older adults highlight the need for home assessment tools to better capture fall hazards that are specifically problematic for individuals with vision impairment. Additionally, risk factors important for those with vision loss may also impact older adults with normal or near-normal vision, such as poor visual perception due to cognitive decline. [42,43] Further research will explore if this assessment is associated with greater risk of in-home falls in a visually impaired population and help determine which specific home features increase fall risk in older adults.
Abbreviations CI: Confidence interval; HEAP: Home environmental assessment protocol; HEAVI: Home environment assessment for the visually impaired; ICC: Intraclass correlation coefficients; OT: Occupational therapist; SD: Standard deviation