Skip to content
Q.REC
Quality of Refractive Error Care Manual
Last edited 561 days ago by Anthea Burnett.
This manual was developed by Anthea Burnett PhD and Ling Lee PhD, with major contributions by Myra McGuiness PhD, Beatrice Varga, Yadira Perez Hazel and Suit May Ho.
This work was funded by The Fred Hollows Foundation with support from the Australia Government through the Australian NGO Cooperation Program (ANCP). In collaboration with The University of New South Wales
Cover photos: Sam Phelps
Key Contacts
Anthea Burnett
Consultant, The Fred Hollows Foundation
Ling Lee
Consultant, The Fred Hollows Foundation
Glossary
Dispensed spectacles: The glasses that have been prescribed, made and provided to a USP from an optical service.
Intra-service correlation: Describes how strongly the USP outcomes are from the same services.
Optical service: any place that provides refractive error services including refraction and optical dispensing. This might include privately owned stores, franchisee stores, vision centres, community health centre services, eye hospitals or secondary and tertiary hospitals.
Unannounced standardised patient (USP): An individual that has been trained to visit optical services in a standardised manner.
Index of acronyms & abbreviations
ANOVA
Analysis of variance
D
Dioptre
DC
Dioptre cylinder
DS
Dioptre sphere
HREC
Human research ethics committee
IRB
Institutional Review Board
MoE
Margin of Error
Q.REC
Quality of Refractive Error Care
USP
Unannounced standardised patient
WHO
World Health Organization
△
Prism dioptres
There are no rows in this table
Q.REC: A manual for investigating the quality of refractive error care
Introduction
Uncorrected refractive errors are the leading global cause of vision impairment, with 161 million people and an additional 510 million people living with distance and near vision impairment, respectively.1 In 2021, the United National General Assembly adopted a resolution committing the international community to provide eye health for all people living with vision impairment by 2030. To achieve such a target, the World Health Organization (WHO) stated that a commitment to Universal Health Coverage using an integrated people-centred eye care approach is the model of choice for quality and affordable eye care services, and that quality eye care services need to be provided according to population needs.2 To monitor progress and drive change towards Universal Health Coverage that includes the quality of eye care services, a broader set of measurement indicators are required.
Indicators should provide insights to shape change and stimulate action, track outcome progress and the quality of an intervention. Indicators have recently been established, to assess the ‘real world’ quality of refractive care, where care is a combination of prescribing and dispensing for refractive error. The Quality of Refractive Error Care (Q.REC) indicator provides information on the proportion of prescribed spectacles that are clinically optimal for patients, by comparing a baseline prescription to dispensed spectacles. The Q.REC study can also be used to assess core dimensions of quality, such as whether refractive error services are effective, equitable, safe and people-centred, and have the potential to identify aspects of clinical practice that require improvement, or further training.
Rationale
A Q.REC study employs simulated patients — the gold standard for evaluating quality in clinical practice3 — and can be easily administered in urban settings. Simulated patients, or Unannounced Standardised Patients (USPs), are ‘actors’ who are trained to act covertly as patients in a standardised fashion, while observing clinical techniques and services provided. USPs have been used extensively in low- and middle-income countries, often in evaluating family planning, pharmaceutical dispensing patterns, and clinical prescribing patterns.4 Studies employing USPs have also previously been conducted to evaluate refractive error outcomes.5-7
Advantages
It is anticipated that a Q.REC study will identify the proportion of people that are prescribed and dispensed spectacles appropriate for their refractive error needs. It can identify adherence to national guidelines or best practice and identify specific opportunities for quality improvement which can then be translated into policy changes or quality improvement initiatives. Q.REC studies can also be used to monitor ongoing delivery of quality refractive error care services, within the context of integrated people-centred eye care.
If a Q.REC study is executed well, the distinct advantage of employing USPs is that observation bias is minimised, as refractive error services are likely to modify their behaviours if they feel that they are being observed or assessed.
Disadvantages
This protocol is unlikely to provide specific information for each facility as only a few USPs visit each service. Also, as all USPs will be adults, the findings might not be applicable to the quality of children’s refractive error care.
There is a possibility that USPs might be detected, and if so, the data should be excluded that consequently might broaden the range on Q.REC measurement. If there is a significant proportion of USP visits detect (i.e. >5%), and the USP continues to complete the visit and data is included, then there could be an overestimation on the Q.REC. USPs might also inadvertently mislead optical service staff to either perform better or poorer than usual behaviour. To reduce these potential biases, well-delivered USP training is essential.
Phase 1: Planning a Q.REC Project
Feasibility assessment
The following sections help to determine whether it is feasible to conduct a Q.REC Project. The main things to consider are how many optical services can be visited, the amount of study personnel required, the available equipment for recruitment, training and data collection, the time available of all personnel, and the support from relevant stakeholders.
Study Personnel
Additional support can be sought:
We recommend the study coordinator and study optometrist (who can be one person responsible for both roles) should be the key people/person that familiarises themselves with all the study materials including protocol, sample size calculator, budgeting template, ethics application, recruitment forms, optometrist training materials, USP training materials, data collections forms, and data quality control procedures.
The ideal study personnel to be involved with USP training, monitoring USP data collection and also completing some data collection would be eye health professionals with significant experience in refraction techniques and dispensing, likely optometrists or refractionists. Additional desirable skills would include teaching and research experience. Three optometrists/refractionists are required at optometrist and USP training, however, based on budget, time and availability, only one study optometrist might only be required during data collection. An additional optometrist will need to be on standby to conduct any additional baseline refractions required.
The number of USPs to recruit depends on the number of optical services that need to be visited and the available time to complete data collection. More information on ideal USPs is available in USP recruitment, (see USP Recruitment).
Provider mapping
Mapping out all the optical services within the study’s area of interest is important for determining the potential sample size, identifying the potential travel required by USPs and therefore budget implications. Determining a comprehensive list of all the optical services can be approached in multiple ways and is likely context-specific. This might include requesting information from governmental departments of business or health, using Google Maps or equivalent, or even travelling around to manually capture all services.
When conducting this exercise, be aware of the time in between mapping and beginning data collection. Optical services can be dynamic businesses where some will close and move location, while other new services might open. This has potential impact on the logistics and organising the USPs visiting services.
Equipment and resources required
The majority of information required to develop a protocol can be obtained from this document. Several equipment and resource lists can be developed for the different components of the Q.REC study. This includes:
translating documents
Consider which documents and forms will need translation to your local language. We expect for settings where English is not regularly used by optical services and potentially USPs, the following documents will require translation:
The above list is likely to be the minimum documents that are likely to need translation.
Once the documents have been translated, we also recommend back-translation into English by a different translator. This is to ensure the initial translation is appropriate. Both translators should have a least basic knowledge of refraction and refractive error terms.
Advertising for USPs
Consider how many and where flyers might be placed and online advertisements. Any advertisement will most likely need to be approved by the ethics committee
Screening and recruiting USPs
Consider where the potential USPs be screened, and by whom. The basic equipment needed is likely:
Training study optometrists
At least a day of training optometrists in the Q.REC protocol so they are capable of providing USP training, baseline data collection and post-visit data collection Paper, laptops or mobile devices, an appropriate facility and per diems are likely required when training optometrists in the Q.REC protocol.
Training USPs including baseline data collection
Similar to training the study optometrist(s), an appropriate facility, paper, laptops or mobile devices and per diems for all study personnel and USPs should be considered. It is likely that USPs will be using their own mobile devices during field data collection, therefore, provision of mobile data also need to be considered.
Depending on your local context and targeted optical services, some optical services, as part of their standard practice, might conduct eye tests that are not related to prescribing for refractive error. For example, in Australia, optometrists in optical stores most commonly prescribe glasses to patients. However, when patients attend optical stores in Australia it is also common for the optometrist as standard practice to provide an eye test that include tests that check the health of the eyes. Tests might include slit lamp examination for the front of the eye, examining pupil reactions, checking eye pressures with contact or non-contact tonometry, or even checking the health of the retina with fundoscopy or photography. It is important to consider the extra tests unrelated to refraction that USPs can undergo to minimise their detection, and identify the tests that USPs can safely and appropriately refuse.
This planning exercise (Annex 1: Q.REC Planning) is to be completed by the study coordinator, refractionists/optometrists and those who have experience with visiting the optical services in the selected area. The exercise aims to identify all the potential eye examination techniques and equipment USPs might encounter at the optical service visits. This will then inform the equipment and space requirements when training the USPs. It is possible that your own facilities have limited equipment, however it is important that if the external services have other equipment, consider borrowing from other sources to demonstrate the USPs. As a last resort, images and videos can be shown to the USPs during training.
Post-visits data collection
At a minimum, a distance and near VA chart, a focimeter, ruler and laptop/mobile device to enter the data are required. Please review Form D: Assessment of Glasses, and consider whether there is quality control checking or refresher refraction training with USPs will be conducted to determine if any additional equipment is required.
Practical considerations
While the protocol can appear to be straightforward, do not underestimate the time planning and logistics might take to execute a Q.REC project without creating unintended bias. This might include:
Calculation of sample size
The Q.REC Sample Size Calculator is used for estimating the proportion of spectacles that meet the selected/desired spectacle quality indicator for the targeted population. The calculation takes into account that there is clustering at the service level and clustering of the spectacles as USPs visit multiple stores. The Q.REC sample size calculator can be found here:
There are a few assumptions or considerations that need to be taken when calculating sample size:
Figure 1 Example of Q.REC Sample Size Calculator
If there is an intention to compare sub-groups, for example, urban versus rural or private versus public services, the following calculator might not be sufficient. Additional sample size calculations will be required to ensure the sample size is sufficient to detect a differences between groups. However, clustering effects should also be taken into consideration.
Budgeting
The Q.REC Budget Template (Figure 2) can be used as a template for determining the study budget. The template will firstly require the number of spectacles per store, the number of optical services and total number of spectacles required (obtained from Q.REC Sample Size Calculator). There are a few assumptions or considerations that need to be taken when determining the study budget:
Note: Unlike the Q.REC Sample Size Calculator, the Budget Template will need to be downloaded to be modifiable (shown in Figure 3).
Figure 2: Example of Q.REC Budget Template
Figure 3: How to download Q.REC Budget Template
Approvals, support and buy-in
Once the feasibility of the Q.REC study and an approximate plan has been determined, consider the stakeholders that are needed for approval and/or support for the project. For the Q.REC study to be recognised and findings to be trusted, support and buy-in from the Department/Ministry of Health, optical associations or refraction training institutions should be considered.
Ethical considerations
Applying for ethics approval should be submitted as soon as possible. All USPs should provide written consent to accept being trained to act as a standardised patient and allow their refractive error and demographics information to be collected and reported. There are three approaches when applying for ethics approval to collect information from optical service visits:
The opt-out approach is recommended. This approach involves providing a participant information statement to all services (not just the ones selected) to inform the study is occurring in their area, they might be selected and then should be offered the option to opt-out of participation. An example Participant Information Statement and Opt-out form is provided in Annex 2: Templates: Optical Services.
This option reduces the risk of USPs being detected and enables to actively review USP detection after all visits have been completed.
A disadvantage of using the opt-out approach is the potential delay of USP visits as enabling services to have enough time to decide to opt-out is required. We recommend a minimum of two weeks. Furthermore, if opt-out slips are received during USP visits, previous data already collected from the stores may have to be discarded. To avoid having to discard data, opt-out forms can be designed to allow for already collected information to be kept unless the services explicitly request for data removal.
Depending on your human research ethics committee (HREC) or institutional review board (IRB), they are significantly more likely to grant approval with this approach compared to not seeking any consent or notification.
The main advantage of not seeking consent from optical services is that there is less likelihood for USPs to be detected and optical services to change their behaviour. There is also less preparation required before training USPs and begin data collection. However, depending on the HREC or IRB applied to, they may or may not grant approval with this approach.
If the HREC/IRB insist on seeking informed signed consent, we would recommend the following:
Phase 2: Preparing for data management
REDCap customisation
The basic Q.REC data collection tools have been developed using REDCap (Research Electronic Data Capture), a secure, web-based software platform designed to support data capture for research studies.8, 9 We identify these data collection tools as the Q.REC REDCap project. If REDCap is unavailable, this study can be conducted on paper.
Uploading the Q.REC REDCap Project
The Q.REC REDCap project is available as an XML file that can be uploaded to your institution’s hosted REDCap for use. Guidance on how to create a new project and upload the data collection tools can be reviewed in the video here:
For those familiar or interested in pursuing this research with REDCap, we understand that each project might slightly differ, therefore there are opportunities to customise the project within REDCap. Several of the potential customisations to be considered are presented in the Project Setup section of the project (Figure 4).
Figure 4: REDCap Project Setup Screen
REDCAP team roles
At a minimum, the study coordinator and study optometrist(s) should have an account to log into Redcap. Other co-investigators can be added. USPs are not required to have a Redcap account.
To add the study team and their individual accounts to the REDCap project, you will need to first request each of them to access to REDCap through your IT administration team that is hosting REDCap. Once provided access, each team member should receive an authentication link to activate their REDCap account. Once activated, then you can add them to the specific Q.REC REDCap project. Guidance on how to add study team members can be found here:
The principal investigator can decide who within the study team has access to some or all ‘Instruments’/forms, as well as what capabilities they have within the project. Limiting what team members can see or do can sometimes help with making it simpler to use. To update each study team’s roles and capabilities, use the User Rights application (Figure 5 – black outline). Guidance on how to use the User Rights application can be found here:
Figure 5: Relevant Applications in REDCap
Multi-site vs Single-site project
For a multi-site project, it is sometimes easier to keep track by entering the location(s) the USP has been allocated. A field within the “Project Admin” Instrument has been developed but will need to be updated with the location names.
If multiple sites have separate data collection teams, and each data collection team should only have access to the USP data for their site (recommended), then use the “DAGs” (Data Access Groups) application (Figure 5 – green outline). Guidance on how to use the DAGs application can be found here:
Translating within Redcap
To translate the USP patient information statement and consent form, data collection forms, and survey settings, use the Multi-Language Management application (Figure 5 – orange outline).
Guidance on how to use the Multi-Language Management application can be found here:
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.