The eSMAF: a software for the assessment and follow-up of functional autonomy in geriatrics

Abstract

Background

Functional status or disability forms the core of most assessment instruments used to identify mix and level of resources and services needed by older adults who possess common characteristics. The Functional Autonomy Measurement System (SMAF) is a 29-item scale measuring functional ability in five different areas. It has been recommended for use for home care, for allocation of chronic beds, for developing care plans in institutional settings and for epidemiological and evaluative studies. The SMAF can also be used with a case-mix classification system (Iso-SMAF) to allocate resources based on patients' functional autonomy characteristics. The objective of this project was to develop a software version of the SMAF to facilitate the evaluation of the functional status of older adults in health services research and to optimize the clinical decision-making process.

Results

The eSMAF was developed over an 24-month period using a modified waterfall software engineering process. Requirements and functional specifications were determined using focus groups of stakeholders. Different versions of the software were iteratively field-tested in clinical and research environments and software adaptations made accordingly. User documentation and online help were created to assist the deployment of the software. The software is available in French or English versions under a 30-day unregistered demonstration license or a free restricted registered academic license. It can be used locally on a Windows-based PC or over a network to input SMAF data into a database, search and aggregate client data according to clinical and/or administrative criteria, and generate summary or detailed reports of selected data sets for print or export to another database.

Conclusion

In the last year, the software has been successfully deployed in the clinical workflow of different institutions in research and clinical applications. The software performed relatively well in terms of stability and performance. Barriers to implementation included antiquated computer hardware, low computer literacy and access to IT support. Key factors for the deployment of the software included standardization of the workflow, user training and support.

Background

Demographic changes are leading to increased demand for services and resources in the continuum of geriatric care. With the emergence of Integrated Service Delivery Systems to improve continuity of care between settings, clinicians and managers need reliable assessment instruments capable of identifying elders' needs, promoting the provision of the appropriate resources, and providing standardized data that can match elders and providers across the continuum [1]. Functional status or disability forms the core of most assessment instruments used to identify mix and level of resources and services needed by older adults who possess common characteristics.

In the province of Quebec, Canada, a battery of assessment tools was implemented in 2001 across the entire continuum of geriatric care. Included in this package, the Functional Autonomy Measurement System (SMAF) [2] is used in clinical settings for the assessment and follow-up of elderly disabled patients [3, 4]. The SMAF [see Additional File 1] is a 29-item scale based on the World Health Organization classification of disabilities [5]. It measures functional ability in five areas: activities of daily living (ADL) (7 items), mobility (6 items), communication (3 items), mental functions (5 items), and instrumental activities of daily living (IADL) (8 items). Each item is scored on a 4-point scale from 0 (independent) to 3 (dependent) for a maximum score of 87. For every item that has a rating of 1 or higher (i.e. not independent), the human resources (help or supervision) required to overcome the disability in this specific area and the stability of these resources is evaluated. The SMAF must be administered by a trained health professional, who scores the individual after obtaining the information by questioning the subject and proxies or by observing the subject. The theoretical framework of functional autonomy used for the development of the SMAF is presented elsewhere [6]. A newer theoretical framework taking into account the social aspects of disability and documenting the impact of the social and physical environment on a person's functioning has been proposed since [7, 8]. The impact of social roles on functional autonomy is currently not included in the SMAF evaluation. A subscale assessing social functioning has been developed [9] and is currently being evaluated for addition to the SMAF.

The SMAF has been recommended for use for home care, for allocation of chronic beds, for developing care plans in institutional settings and for epidemiological and evaluative studies [10]. The reliability, validity and responsiveness of the SMAF has been established extensively [11–14]. Recently, a new case-mix classification was developed to better allocate resources based on the functional autonomy characteristics of patients as indicated by the SMAF [15, 16]. The classification [see Additional File 2] matches client results on the 5 dimensions of the SMAF scale to 14 mutually exclusive profiles (Iso-SMAF) using Euclidean distance measure. SMAF and Iso-SMAF profiles are associated with a specific amount of nursing and support services and also with the cost of services according to the type of facility [17].

Health service researchers have routinely used these instruments as outcome measures in program evaluation and economic evaluation of geriatric health services [10, 18–21]. Traditionally, this is done using paper-based clinical pathways where information from paper forms is converted into a format suitable for computerized quantitative data analysis by manual data entry. Both SMAF and Iso-SMAF are currently incorporated in dedicated clinical information systems. However, current clinical information systems that incorporate SMAF and Iso-SMAF data are not easily available to clinicians or researchers outside large public institutions (i.e. private seniors' homes, assisted living facilities). These systems offer limited interoperability and user-based data management functions with no way to search and view aggregated SMAF and Iso-SMAF data for a given client or group of clients. In order to address these specific needs, a stand-alone software was developed to increase the accessibility of these tools to the research and academic community. This paper describes the development of the software, its functions and how it is currently being used. The software is available in French or English versions under a 30-day unregistered demonstration license or a free restricted registered academic license through the Sherbrooke Health Expertise Center. Download instructions are listed in the availability and requirements section at the end of the paper.

Methods

Software engineering and implementation

The development of the eSMAF followed a modified waterfall model with feedback loops from users. The modified waterfall model is a software development model in which development is seen as flowing steadily downwards (like a waterfall) through the phases of requirements analysis and functional specifications, design and implementation of software architecture, software testing and validation, and software deployment and maintenance [22]. Each phase has a well-defined starting and ending point with a closed feedback loop with identifiable deliverables to the next phase. The eSMAF project started in 2003 and culminated in the last year with the commercial deployment of the software in several organizations and institutions. The timeline for the development phases of the eSMAF is illustrated in Figure 1. The details of the development, implementation and deployment of the software are described below.

Figure 1

Timeline in the last 3 years for the development, implementation and deployment of the eSMAF. Specific steps within the modified waterfall model are color-coded as indicated in the legend on the right.

Requirements analysis and functional specifications

At the outset of this project, the requirements for the proposed software were to deliver in a timely manner (15 months) and with limited financial resources (about US$30 K) a low maintenance software application to: 1) facilitate SMAF data collection at the point of care, and 2) provide a way for clinicians, managers and researchers to easily access, search and view aggregated SMAF and Iso-SMAF data for a given client or group of clients. The priority was to develop a French version of the software first and then consider multi-language versions (the SMAF evaluation has been translated into 7 languages). The software was to be used under different research and clinical environments with limited IT support by single or multiple users (less than 25 users at the same time). Functional specifications for the software were established through focus groups (n = 3, 20 individuals total) of stakeholders (MD, nurses, PT, OT, department head, hospital managers, health service researchers) and prioritized by expert consensus with the development team according to the initial requirements list, analyses of feasibility (level of difficulty to implement) and development cost of each feature, and overall impact on the software architecture.

Design and implementation of software architecture

After taking into consideration the complexities of the information flow of the SMAF data (from data entry to consultation to report and data extraction by clinical personnel, managers and researchers) and its variations depending on the context of use (homecare, hospital care or research), very early in the project we abandoned the idea of embedding the eSMAF in existing proprietary clinical information systems currently in use. In order to transmit information from this stand-alone system back to the patient's electronic medical record or other databases, we chose static merging of individual or aggregated group data from the eSMAF database as CVS files using an export function. An overview of the software architecture for the eSMAF is summarized in Figure 2. In keeping with the initial requirements list and projected use of the software, we chose as the basis for the software architecture 1) Microsoft Access as a back end (database) to store information, and 2) a Visual Basic client to act as a front end (software interface) for easy data insertion, retrieval and reporting. The format of the database was selected according to projected deployment constraints (limited availability of on-site and remote technological support and cost issues) and ease of installation (no server required).

Figure 2

Software architecture of the eSMAF. a) Microsoft Access database is used as a back end to the software; b) eSMAF shell programmed in Visual Basic is used as a front end to display and enter data in the database; c) Hierarchical levels within the eSMAF shell; and d) Functions available in the software depending on the hierarchical levels.

The software architecture is based on five overlapping hierarchical levels (group, subgroup, client, client test, test detail). Each level provides a different view of the database (ex: global aggregated results at the group level to detailed individual information at the test detail level) from which specific functions such as user and workflow management, client record management, online help, graphs, reports, data extraction or printing can be executed. The core of the software architecture is the "Clients" level where a unique subject identifier is created, tests or data entry masks are linked to this unique identifier (Client Test level) and data entry functions (Test edition) are applied. The unique identifier at the "Clients" level can be associated with "Groups" or "Subgroups" to aggregate clients' records in user-defined categories, thus facilitating navigation and viewing of records in the database. For example, a group could be used to represent a health care center with the subgroups representing the different units in this center. In this way, the user can view individual patient records for a subgroup of clients in an institution. The group and subgroup can also be used to limit access to records to a specific category of user.

Software testing and validation

Testing and validation of the software was done iteratively throughout the software development process. Feedback from users regarding the usability of the software was elicited at the different stages of development and testing of the eSMAF software. Usability is defined as the extent to which a product can be used by specified users to achieve specific goals effectively, efficiently, and with satisfaction in a specified context of use [23]. The first version was completed at the end of 2003 and tested in-house over a 2-month period at the Sherbrooke Geriatric University Institute with a selected group of beta testers (n = 12 nurses) as they evaluated 600 residents in the long-term care units of two facilities. Beta testers were asked to keep a journal logging usability issues and any bugs. They were then debriefed as a group or individually by the project manager to identify software instability (i.e. bugs) and obtain suggestions for improvements or additional features.

A list of changes to be implemented was formulated and the software went through another design phase at the beginning of 2004. Existing features were optimized and new ones added. At the end of 2004, a new version was built and distributed to a larger group of beta testers. Software robustness was tested through a large-scale deployment in 19 separate institutions (approximately 200 users in total) as part of a 2-year research project on health services [24]. During this testing phase, usability issue and bugs identified by the users in the 19 institutions taking part in the trial (see discussion section) were transmitted to the development team by the field project coordinator and changes were made periodically until a stable version was obtained. In 2005, the need for a multi-language (French and English) version of the eSMAF arose. This version, which was to become the last, was developed and tested in a small group of testers. As part of the software testing and validation, the need for user guides and training material (tutorials) was identified. A user guide with screen shots was written and incorporated as part of the online help function in the software or as a stand-alone document. The user guide was translated for the multi-language (French and English) version of the eSMAF.

Deployment and maintenance

In January 2006, the software was released for institutional users. Licensing and training on the software is supported by the Sherbrooke Health Expertise Center. User support is offered on a contractual basis through a partnership agreement with a private company. As of July 2006, approximately 35 site licenses for a total of 250 workstations have been distributed to institutions in Canada and France. Institutional licensing efforts are still ongoing.

Results

Software functions of eSMAF

The principal functions of the eSMAF software are summarized below and include: (1) Navigation interface and records management utility; (2) Users and workflow management; (3) Client record module and data entry; (4) Report function and care planning; (5) Search functions; and (6) Data extraction function and online help.

Navigation interface and records management utility

A standard interface is used throughout the hierarchical levels of the software. The layout of the interface is shown in Figure 3. There are three main work areas: the main area, the toolbar area and the navigation toolbar area. The main area is used to display or edit information. The toolbar area provides access to the software functions, depending on the hierarchical level the user is on. At the group level, functions include data management functions, report and search functions, and the software management control panel. The interface at the group and client level consists of buttons and list views which can be used to display patients or tests in different modes: icons, list and detailed view. The detailed view includes a sorting option that allows users to display records according to specific criteria (ex: date of creation or Iso-SMAF profile). The navigation toolbar allows users to navigate across hierarchical levels of the software.

Figure 3

Main interface organization of eSMAF shell. a) Main area; b) Toolbar area for records management; c) Toolbar area for software functions; d) Navigation toolbar area where individual buttons are used to go back, switch display mode and refresh the display; e) Icons record view; f) Detailed record view; and g) Sorting function in detailed record view.

Users and workflow management

Workflow management options (administrator and user privileges, records management, etc.) are set through a control panel accessible from the group level interface. Figure 4 shows the interface of this control panel and the principal workflow management options. Access privileges for users are defined according to user groups created by the administrator users. Each user is associated with one user group. The user group determines what kind of access the user in question has (logging into the database, accessing groups or subgroups in the client records, viewing and/or editing records or portions of records). A "Utilities/Options" panel accessible only to users with administrative privileges is used to configure options for the software (ex: path to the location of the database). Traceability features (ex: what was changed in a given record and when) can be viewed through a software log tab as all user actions are logged in the database.

Figure 4

Users and workflow management. a) Access to users and workflow management; b) Control panel of the software divided into 5 sections; c) Workflow management functions enabling users to classify patient records into user-defined groups; d) Software options and preferences; and f) Management of user groups and subgroups access and editing privileges within the database.

Client record module and data entry

The client record module consists of a data repository of evaluations completed for a given client, allowing users with administrative privileges to generate new evaluation masks. Each evaluation can then be filed and edited. The main evaluation used in the software is the SMAF test, but it is also possible to fill out an information record containing nominal information, a clinical note and a care table (see Report function and care planning below). The SMAF test interface is shown in Figure 5. A test can be locked and unlocked, depending on the access the user has. Data are entered by accessing questions using the overall questionnaire view or using the navigation functions in the question view. Complementary information for a given item can be entered through tab navigation in the question view.

Figure 5

Data entry mask for SMAF evaluation. a) Upon choosing a SMAF test, the user is presented with a summary of the test; b) By activating a specific item, a data entry window appears for the selected item; c) The user can fill out the data from the test; d) More details and relevant information for this particular question can be entered using the tabs; and e) Navigation between questions and sections.

Report function and care planning

In order to continue support for paper-based clinical pathways, a report module was incorporated to allow users to generate paper summary reports or detailed reports on any given client or group of clients. Reports can take several forms and include: individualized summary, distribution graphs on specific indices (ex: Iso-SMAF profiles) of selected groups of patients, line graphs charting the evolution of a specific category of data for an individual across multiple evaluations, and bar charts showing specific changes between the two last evaluations of a given individual. Report functions can be applied at the client level or aggregate clients level by manually selecting them in the navigation interface and records management utilities or through selected outputs from the search engine. The principal report function outputs are illustrated in Figure 6.

Figure 6

Report function. a) Buttons activating report functions; b) Data from the SMAF for a given individual or several individuals are tabulated and illustrated according to 7 types of summaries; c) and d) Evolution of a given individual's or several individuals' clinical scores can be viewed in different formats.

In addition to these reporting features, users can automatically generate, for a given client or clients, a color-coded graphical care table that, for each section of the SMAF, summarizes the level of assistance needed by this or these individuals. The care table can be generated individually using data entered on the SMAF scale and printed. An illustration of a care table is presented in Figure 7. For a specific sphere of activities (ex: ADL), colors represent the level of assistance needed by the client for a given task (ex: bathing). The printed care table is then displayed in the client's room and serves as a visual reference to help guide the care provided to the client by nursing staff and orderlies.

Figure 7

Care table generated from a SMAF evaluation.

Search functions

Search functions were implemented at the aggregate client level to allow users to search and parse information from the database according to user-defined parameters. The search functions were implemented so that it would be possible to generate user-defined queries on all levels of the database (tests, patient information, groups, etc.) and execute report and data extraction functions on a subset of specific records. The interface for the search functions is illustrated in Figure 8. To do a search, search parameters need to be specified. There are no limits to the number of parameters that can be specified. Individual parameters are specified for a category of information (ex: score on a given section or question of the SMAF). Strings of parameters can be grouped together with the logical operator "And" or "Or" and saved as a specific search that can be reapplied at any time. Upon execution of a search, records (clients) matching the search parameters are clustered together as a group of records. Report functions and data extraction functions on this group of records or on individual records can then be applied.

Figure 8

Search function. a) Activation of search function; b) Queries in the search function can be formulated on 5 categories of information; c) Queries can be linked together with logical operators, and d) saved as a specific search to be executed later.

Data extraction function and online help

Individual or aggregated group data can be transferred and merged with other datasets using the export function. Selected records or sections of records within the database can be exported as CVS files. The CVS file can be imported to another database, spreadsheet files and statistical applications such as SPSS. The data extraction function is illustrated in Figure 9. Online help in the eSMAF can be consulted to address issues with the software itself and to crosscheck scoring procedures on the SMAF evaluation. The software help system can be accessed from anywhere in the software. The SMAF help is only available when editing a SMAF test, and provides information on how to complete a specific question on the test. The help function is illustrated in Figure 10.

Figure 9

Data extraction function. a) Data extraction function; b) Data from the SMAF or c) specific section of the SMAF for a given individual or several individuals can be extracted to d) a CVS file.

Figure 10

Online help function for scoring the SMAF.

Discussion

Intended use of the software

The eSMAF's principal intended use is to streamline the collection of SMAF and Iso-SMAF data in clinical settings while offering a way to control the source and quality of the data entered, facilitate the aggregation, viewing and extraction of these data according to user-defined needs, and empower users to easily merge this information with other databases or data sources. For example, in the context of program evaluation research, the eSMAF was successfully deployed to collect SMAF outcome measures periodically over a 2-year period at 19 long-term public home care institutions in an administrative region of Quebec, Canada [24]. Medical archives personnel (n = 57 users) were trained on the use of the software prior to the start of the project. A key individual was identified in each facility to manage user questions and technical problems. This person was in contact with a district coordinator, who was the contact point for the research team. Clinical assessments in the eSMAF database from each institution were merged with administrative databases in order to objectively describe the relationship between the needs of older adults as established from Iso-SMAF profiles and the actual services provided to them by home care programs. SMAF data from 8000 patient records over a 2-year period were collected. As an example of the accessibility of these data, retrospective analysis of the data clearly illustrated the presence of a gap between the levels of home care services provided and the needs of disabled individuals, thus providing both researchers and clinical managers with arguments for changes in budget allocation for services in this administrative region. The data continue to be collected and used by administrative personnel to benchmark and document the autonomy of the client population and the services offered.

In this deployment, some barriers to implementation were encountered. The first was related to the use of small monitors and low resolution screens by some of the users. At the time many of the users were working with a screen resolution of 800 × 600 on 15-inch monitors. As the software interface was designed to work with a screen resolution of at least 1024 × 768, this caused a major problem. This problem was resolved by making the software automatically switch to the required resolution in the operating system (i.e. Windows) when launching, then switching back when it was closed. Another issue arose when the eSMAF was installed in many different network configurations in different physical locations, and regular updates were required because of its development stage. While the IT support needed to install and maintain the software in a server environment is minimal, this support was not readily available at most sites as IT support was contracted out. This introduced delays in granting user administrative and access privileges to the database. Communications with each site's IT support department was also difficult. When an update to the software was needed, it was not installed at the same time at each location. This caused problems when trying to find if a reported bug was related to a new or an old version and required close follow-up of installed versions in each physical location. Once installed and configured, the eSMAF performed as expected.

For clinical purposes, it is also possible to incorporate and analyze SMAF data and Iso-SMAF profiles on a regular basis, thereby rendering them accessible for decision-making in day-to-day operations (define eligibility criteria, support planning and evaluation processes, organize resource allocations) at different clinical and administrative levels. For example, from patient SMAF evaluations using the search function, department heads or hospital unit managers can estimate the clinical work load associated with a number of beds in a unit by looking at specific functional autonomy indicators (incontinence, dressing and eating) of patients in this unit. Care plans can also be circulated more freely and updated more easily. Prior to the development of the eSMAF, the clinical staff generated care tables for each client manually by putting color-coded stickers on each section of the care table and they had to crosscheck that they corresponded with the current SMAF data on record. The eSMAF's care table function offers an automated means to efficiently generate care tables for large numbers of clients while insuring that the outputs in these care tables correspond to the clients' on-record functional status.

Planned future developments

Using feedback from current users, we are looking at optimizing the data management features of the eSMAF to facilitate its integration in institutional clinical workflow and/or its use in a health services research context. While the software in its current version has been used successfully in the context of multiple users (one database stored on a local area network with multiple controlled access to the database), the underlying database infrastructure is not optimized for large-scale networked use (i.e. database with more than 5000 records with simultaneous access from more than 40 users from multiple geographically separate sites). Initial user needs assessment and budgetary constraints governed the choice of the database infrastructure to a readily available non-commercial grade database format. The existing database platform (Microsoft Access 2000) is not robust enough to operate under such conditions. Therefore, to expand the usability of the software under this context of use, a Web-based eSMAF supported with a commercial grade database format (ex: My SQL) is being considered.

Dynamic integration of the eSMAF software with existing clinical information systems will also a play a critical role in its successful deployment in large-scale hospital settings. This integration is possible with the existing software architecture but depends to a large extent on establishing communication between systems and establishing database synchronization rules that fit the workflow and needs of the application. Depending on user interest, another development area will be to offer other language versions. As the SMAF instrument has been translated into 7 languages, the eSMAF could be ported to other languages. The underlying programming architecture related to the operation and use of the software is not language-specific. Indeed the programming architecture is such that most of the user interface shell (data entry mask of the SMAF) can be customized rapidly while still maintaining the current database structure using tables containing content in any language.

Conclusion

The eSMAF software was developed with limited resources to address specific needs. It can be used in different research and clinical contexts to optimize and facilitate the assessment and follow-up of elderly disabled patients with the Functional Autonomy Measurement System and Iso-SMAF profiles. The robustness and usability of the software were tested through iterative field trials with users. An academic license for the software is now available to qualified users. Maintenance of the existing version of the software will be executed on a limited basis as permitted by the availability of new financial resources through institutional licensing.

Availability and requirements

Project name: eSMAF

Operating system(s): Microsoft 2000, XP

Programming language: Visual Basic

Other requirements: Monitor resolution of 1024 × 768

Licenses: Full version 30-day demonstration, restricted academic license through e-mail activation of demonstration version. Commercial license needed for non-academic or institutional use.

Demonstration license: Time-limited demonstration version through direct download from BMC geriatrics. English (Additional file 3; French (Additional file 4)

Registered academic license: Restricted free academic licenses can be obtained by contacting the Sherbrooke Health Expertise Center at info@expertise-sante.com.

http://www.expertise-sante.com

Commercial license: For institutional and/or commercial use, licensing inquiries can also be made to the Sherbrooke Health Expertise Center at info@expertise-sante.com.

http://www.expertise-sante.com