Article Text

Methodological variations and their effects on reported medication administration error rates
  1. Monsey Chan McLeod,
  2. Nick Barber,
  3. Bryony Dean Franklin
  1. Centre for Medication Safety and Service Quality, UCL School of Pharmacy and Imperial College Healthcare NHS Trust, London, UK
  1. Correspondence to Monsey Chan McLeod,Centre for Medication Safety and Service Quality, Pharmacy Department, Imperial College Healthcare NHS Trust,Charing Cross Hospital, London W6 8RF, UK; monsey.mcleod{at}imperial.nhs.uk

Abstract

Background Medication administration errors (MAEs) are a problem, yet methodological variation between studies presents a potential barrier to understanding how best to increase safety. Using the UK as a case-study, we systematically summarised methodological variations in MAE studies, and their effects on reported MAE rates.

Methods Nine healthcare databases were searched for quantitative observational MAE studies in UK hospitals. Methodological variations were analysed and meta-analysis of MAE rates performed using studies that used the same definitions. Odds ratios (OR) were calculated to compare MAE rates between intravenous (IV) and non-IV doses, and between paediatric and adult doses.

Results We identified 16 unique studies reporting three MAE definitions, 44 MAE subcategories and four different denominators. Overall adult MAE rates were 5.6% of a total of 21 533 non-IV opportunities for error (OE) (95% CI 4.6% to 6.7%) and 35% of a total of 154 IV OEs (95% CI 2% to 68%). MAEs were five times more likely in IV than non-IV doses (pooled OR 5.1; 95% CI 3.5 to 7.5). Including timing errors of ±30 min increased the MAE rate from 27% to 69% of 320 IV doses in one study. Five studies were unclear as to whether the denominator included dose omissions; omissions accounted for 0%–13% of IV doses and 1.8%–5.1% of non-IV doses.

Conclusions Wide methodological variations exist even within one country, some with significant effects on reported MAE rates. We have made recommendations for future MAE studies; these may be applied both within and outside the UK.

  • Medication safety
  • Patient safety
  • Health services research

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Medication errors that result in patient harm, also called preventable adverse drug events, are estimated to occur in 1–2% of hospital inpatients.1 ,2 Of all types of medication errors, medication administration errors (MAEs) are least likely to be intercepted before they reach the patient.3 Most hospital inpatients also receive more administrations than prescriptions, thus increasing the opportunities for error (OE). According to UK medication incident reports, errors at the administration stage account for the majority of patient harm and deaths.4 These data suggest that greater efforts are needed to prevent errors at administration.

To prevent errors, we must identify, measure and understand the problem. However, studies of MAEs can be challenging and resource intensive as direct observation is generally required.5 Methodological variations between studies are well known,5–7 and these can limit interpretation of findings. Inconsistent MAE definitions, MAE subcategories, denominator definitions and MAE rate calculations exist; these present a potential barrier to interpreting and evaluating the transferability of interventions to reduce MAEs. The types of dose studied are also likely to affect the MAE rate. Intravenous (IV) doses are widely perceived to be higher risk for MAEs compared with non-IV doses; a recent UK report identified MAE rates of 3–8% of non-IV doses and 49–94% of IV doses.8 However, the true extent of the difference in error rates between IV and non-IV doses is unknown as studies used different methods and definitions. It is also widely believed that MAEs are more prevalent in children than in adults, but no direct comparison exists.9 Consequently, the effects of such commonly accepted risk factors on reported MAE rates have yet to be quantified.

There are also important differences between countries in how medication is prescribed, dispensed and administered, which can hinder the interpretation of study findings. For example, in the UK, nursing staff are responsible for preparing the majority of doses, including IV doses, on the ward. By contrast, in the USA, pharmacy staff typically prepare the majority of doses and supply these as patient-specific unit-doses. Thus MAEs in the UK would include errors made by the nurse at the preparation stage while such preparation-related errors in the USA are more likely to have been inherited from the earlier dispensing stage.

As MAE studies are important but time-consuming and costly to conduct, to maximise their value, we conducted a systematic review of UK MAE studies to: (1) summarise the variation in MAE definitions, MAE subcategories and denominator definitions, (2) quantify their effect on reported MAE rates, (3) use comparable MAE and denominator definitions to determine overall non-IV and IV MAE rates for adult and paediatric doses, and (4) quantify the effect of including IV and paediatric doses on reported MAE rates. We used one country as a case-study because differences between countries in how medications are prescribed, dispensed and administered, are also likely to affect the prevalence and types of MAEs identified, and thus any exploration of the effects of methodological variation on reported MAE rates.10–12

Methods

Setting

In the UK, medications for hospital inpatients are typically prescribed and administered from paper drug charts.13 Electronic prescribing is currently rare for hospital inpatients (although common for discharge and primary care prescribing), few hospitals use barcode verification at the point of administration and unit-dose drug distribution is not used. Instead, nurses administer medications using ward-based stocks, patient-specific multi-dose supplies from the hospital pharmacy, and/or patients’ own drugs brought in from home.

Search strategy

Nine electronic databases were initially searched for published studies up to and including May 2010: British Nursing Index (from 1985), Cumulative Index to Nursing and Allied Health Literature (from 1981), Embase (from 1980), Health Management Information Consortium (from 1983), International Pharmaceutical Abstracts (from 1970), Medline (from 1950), Pharmline (from 1978), Science Citation Index Expanded (from 1970), Social Science Citation Index (from 1970). The search terms were (medication* OR medicine* OR drug* OR ‘near miss’ OR ‘near misses’) AND (error* OR discrepan*) AND adminis* AND (prevalence OR incidence OR harm OR severity OR mortality OR morbidity OR ‘adverse event’ OR ‘adverse events’ OR ‘adverse drug event’ ‘ OR ‘adverse drug events’ OR caus*). ‘Medication error’ was also included as a mapped thesaurus term in Medline and Embase. Studies were limited to those conducted in humans and published in English. The search was repeated in October 2012 to identify papers published since May 2010, however Pharmline was excluded as it was archived shortly after May 2010.

Study selection

For both searches, one reviewer (MCM) initially screened all titles and available abstracts. A random 10% sample was screened by a second reviewer (BDF) to assess reliability. Only studies reporting empirical MAE rates detected using observation methods in UK National Health Service (NHS) hospitals were included; observation is generally considered to be the gold standard.5 14 Conference abstracts, case-reports, and studies focusing only on anaesthesia, nutrition or a specific type of MAE were excluded. Full papers of selected studies were retrieved and further examined, including their reference lists. A shortlist of studies was produced; both reviewers screened these and the final set of studies confirmed through discussion.

Data extraction and quality assessment

The two reviewers independently extracted data using standardised forms and discrepancies resolved through discussion. A third reviewer was available if agreement could not be reached. Where necessary, authors were contacted for missing information. The quality of each study was independently assessed by the two reviewers using published criteria which are specific to studies measuring MAE rates.5 We also added criteria for reporting: (1) whether or not IV administrations were included and (2) whether or not paediatric doses were included, as pilot work indicated that not all studies reported this information.

Data analysis

MAE definitions, subcategories and denominator definitions were compared and summarised descriptively. The effect of specific MAE definitions, MAE categorisation and denominator definitions on reported MAE rates was calculated where data were available. A meta-analysis15 of reported MAE rates from studies that used the same MAE and denominator definition was conducted using a random-effects model. An overall MAE rate was calculated separately for non-IV and IV data; for studies that included both types of doses, separate MAE rates were extracted where possible. For studies conducted in multiple countries, only UK data were extracted. Heterogeneity was assessed by calculating the I2 index.15 OR were calculated to assess the effect of IV versus non-IV doses and paediatric versus adult doses on MAE rates, where the same error and denominator definitions were used.

Results

We identified 2025 studies: 109 full articles were retrieved and 24 potentially relevant studies were subsequently shortlisted. There was 100% agreement between the two reviewers on initial inclusion versus exclusion of a 10% sample (n=203 studies). Of the 24 shortlisted studies, four were excluded because: an MAE rate could not be extracted from two studies,16 ,17 one was conducted in a non-NHS hospital,18 and the method of MAE detection could not be ascertained in another.19 Twenty studies10 ,11 ,14 ,20–36 therefore met our inclusion criteria. Of these, four14 ,33 ,34 ,36 analysed data from previous studies;26 ,28 ,30 ,35 a final 16 unique studies were included. A third reviewer was not required.

Characteristics and quality of studies

The characteristics of the 16 included studies are outlined in figure 1 and table 1. The majority were descriptive and conducted in adult settings. Generalisability was limited in eight studies as these were conducted in: (1) only one or two wards,22 ,23 ,25–27 ,29 ,30 (2) wards that received a hospital-specific intervention,29 ,30 or (3) an unknown number and type of wards.11

Table 1

Characteristics of 16 UK observational studies and reported medication administration error (MAE) rates

Figure 1

Characteristics of 16 observational studies of medication administration errors. Weighted font sizes have been used to illustrate approximate proportion of studies between groups that contain more than two studies. aOne study of parenteral administrations was included as all doses observed for intravenous (IV) doses except for one intramuscular and one subcutaneous dose. bThree of the 12 studies also presented comparisons with other countries.

In relation to the quality criteria, ten studies reported clear definitions and methods for determining the MAE rate; six did not. Specifically, the following were unclear: (1) the number of MAEs possible per dose,20 ,21 ,31 (2) whether or not dose omissions were included in the denominator,11 ,20 ,21 ,28 ,31 and (3) whether or not ‘extra doses’(as defined by Allan and Barker),5 were included in the denominator.11 ,20 ,24 ,28 ,31 Participants were told the study objectives in three studies, were not informed in three and partially informed in ten. Observers were pharmacists in 14 studies, a pharmacist and pharmacy technician in one 31 and a nurse in another.35 Data were collected by one observer in nine studies, two observers in six,10 ,20 ,25 ,26 ,29 ,31 and four pharmacists in another.30 Of the seven studies with more than one observer, one26 assessed inter-observer reliability (reported in a separate paper),14 one reported that ‘detection of medication errors was comparable between the two observers’,10 and five did not report whether or not inter-observer reliability was assessed.20 ,25 ,29–31 Potential sources of variation were explored in some studies: observations at specific times of day,22 ,24 ,26 days of the week,22 ,26 time-point of inpatient stay,22 timing of administration in relation to when the medication was prescribed,22 and nurse-specific variation.26 All papers reported whether or not IV doses were studied; three studied both dose types but did not report error rates for these separately.20 ,31 ,32 Ten papers did not specify whether adults, paediatrics, or both, were studied, however all were confirmed as being conducted in adult settings by the relevant authors.

Clinical severity of MAEs was assessed in eight studies: five25 ,26 ,28 ,30 ,35 used the validated method of Dean and Barber,37 one used an earlier method developed by Dean,38 one involved an unreported number of clinical pharmacists and the researcher reaching consensus on whether each MAE was minor, moderate or major,24 and one used the judgement of an experienced pharmacist researcher to classify each MAE as either minor or potentially serious.29 All severity assessments were based on potential (rather than actual) harm.

No obvious trend in MAE rates over time was identified, table 1. A forest plot of non-IV studies which used the same MAE definition and denominator also showed no apparent trend in MAE rates over time (see online supplementary Appendix 1). A scatterplot of the same studies revealed no discernible correlation between MAE rates and sample size (see online supplementary Appendix 2).

MAE definitions

Three different overall MAE definitions were identified. Fourteen studies10 ,11 ,20–23 25–30 ,32 ,35 used Allan and Barker's definition:5 ‘a deviation from the physician's medication order as written on the patient's chart’. Of these, three11 ,28 ,32 expanded this US-based definition to include ‘any deviation from standard hospital policy or the manufacturer's instructions’, and one35 included three additional guidance to evaluate the ‘appropriateness of administration’. These made the definition more specific for studying IV doses, paediatric doses and doses administered to patients with dysphagia in the UK. One study24 used a circular definition: ‘error in an administered dose or an omitted dose’, and one31 used an outcome-based but general definition: ‘preventable events that may cause or lead to inappropriate medication use or patient harm’. Observers intervened to prevent all identified MAEs in three studies23 ,24 ,31 and only for potentially serious errors in the remaining 13; all interventions were included as MAEs. We found inconsistencies in what was included as an MAE, even when the same definition was used. Four specific variations were identified, table 2. The most significant and divisive among researchers was ‘wrong time’ errors. Based on data reported in one single-centre study, including wrong time errors of over 30 min from the time for which the dose was due increased the MAE rate from 27% to 69% of 320 IV doses.24 The effect of including wrong time errors in non-IV doses was not assessed as relevant studies did not report the number of doses with wrong time errors only. However, including wrong time errors is likely to substantially increase the reported MAE rate as doses administered over 60 min from the time for which the dose was due occurred in 13–50% of a total of 9054 non-IV doses.10 ,22 ,23 ,35

Table 2

Summary of variations associated with medication administration error (MAE) inclusion/exclusion criteria in 16 UK observational studies and their effect on the reported MAE rate

MAE subcategories

Forty-four different MAE subcategories were identified, with a median of 11 per study (range 3–16), table 3. In some cases, differences in subcategories reflect different ways of classifying the same errors. For example, MAE subcategories such as ‘wrong diluent’ and ‘wrong solvent’ can be considered more detailed subcategories of a broader subcategory: ‘wrong preparation technique’. Furthermore, in studies where only one MAE was allowed for each dose, none specified the hierarchy used to decide how the subcategory was allocated if more than one error was observed for the same dose. Although the classification should not affect the overall MAE rate, care should be taken when comparing specific MAE subcategories across studies.

Table 3

Medication administration error (MAE) subcategories included in 16 UK observational studies

We also found different researchers used the same term to mean different things and different terms to mean the same thing. This mainly concerned ‘unordered drug’ errors (also known as ‘unauthorised drug’ and ‘unprescribed drug’). One study used ‘unauthorised drug’ to include ‘wrong drug’, ‘wrong patient’ and ‘administration of a drug without a valid prescription’.23 However, other studies differentiated ‘unauthorised drug’ from ‘wrong drug’ by stating that the former involves the administration of a drug where no medication order exists, while the latter involves administration of a different drug against an existing medication order.11 ,25 ,26 ,29 ,32 ,35 An ‘unauthorised drug’ error is also generally differentiated from an ‘extra dose’ error which is administering an extra dose of a prescribed drug, for example giving a medication twice a day instead of once a day.5

In some cases, differences in MAE subcategories used may reflect disparities in the types of error included. One study considered dose omissions, a common MAE subcategory, as a ‘violation of procedure’ and differentiated these from MAEs.31 Including dose omissions in this study would increase the MAE rate from 1.2% to 5.6% of 742 drug administrations. There were also studies that included some procedural violations within established MAE subcategories, for example not wearing gloves was included in a ‘wrong preparation technique’ subcategory,24 ,27 but was not included as an MAE in other studies. However, data were not reported separately in the relevant studies so we were unable to determine their effect on the reported MAE rate.

Several studies additionally reported a breakdown of specific MAE subcategories based on the reason for error. Although the causes of MAE are outside the scope of this review, these additional subcategories were frequently reported and provide an important role for understanding MAE rates. For example, ‘omission due to unavailability’ was commonly included as a subset of omissions and accounted for 52–67% of omissions from a total of 12,993 non-IV doses10 ,21 ,22 ,26

Denominators used to determine MAE rates

We identified four denominators and three main differences between them which may affect interpretation of reported MAE rates. The four denominators were the total number of: (1) OE defined as the ‘sum of all doses ordered plus all the unordered doses given’,5 (2) ‘drug administrations’, (3) ‘prepared and/or administered doses’, and (4) ‘prescribed doses’. The first difference between the denominators is whether or not dose omissions were included. All ten studies10 ,22 ,23 ,25–27 ,29 ,30 ,32 ,35 that used OE and one24 that used ‘prescribed doses’ as the denominator included dose omissions, while it was unclear in the remaining five11 ,20 ,21 ,28 ,31 whether or not dose omissions were included. Dose omissions accounted for 0–13% of a total of 934 IV doses11 ,24 ,27 ,28 and 1.8–5.1% of a total of 16 465 non-IV doses,10 ,22 ,23 ,25 ,26 ,35 therefore excluding dose omissions from the denominator will inflate the reported MAE rate. The second difference is whether or not extra doses were included: 10 studies that used OE included extra doses in the denominator, but it was unclear in the remaining six. Despite this variation, extra doses are relatively rare and therefore unlikely to substantially affect the reported MAE rate. The third difference is whether or not each dose was split into preparation and administration. This was generally a feature of studies that included IV and/or paediatric doses. Seven of the 10 studies that used an OE as the denominator counted one OE per dose10 ,22 ,23 ,25 ,26 ,29 ,35 (all were studies of non-IV doses) and three allowed up to two OEs per dose27 ,30 ,32 (all included IV doses, two in adults and one in paediatrics). In the paediatric study where up to two OEs were possible per dose, researchers reported that MAEs occurred in 19.1% of OE and 27.6% of doses.32 The effect of allowing up to two OEs per dose in this study therefore resulted in a lower calculated MAE rate.

MAE rates for non-IV versus IV doses

A meta-analysis of 21 533 adult non-IV OEs from eight studies10 ,22 ,23 ,25 ,26 ,29 ,30 ,35 revealed MAEs occurred in 5.6% of non-IV OEs (95% CI 4.6% to 6.7%). One other MAE rate for non-IV doses was extracted but excluded from the meta-analysis as a different denominator was used (MAE rate 3.2% of 2000 drugs administered).21 Heterogeneity between studies was relatively low (random effects model I2=19%).

Nine MAE rates for IV doses were extracted; however, we conducted a meta-analysis of three MAE rates from two studies only as it was inappropriate to include studies that used different error and denominator definitions. MAEs occurred in 35% of a total of 156 adult OEs (95% CI 2% to 68%).27 ,29 Heterogeneity between studies was low (random effects model I2=0); however, this was based on a small sample of IV OEs which resulted in a wide 95% CI. Based on these limited data, we estimated IV doses to be five times more likely to be associated with an MAE than non-IV doses (pooled OR 5.1; 95% CI 3.5 to 7.5).

MAE rates for adult versus paediatric doses

Of the three studies that included paediatric doses, two reported IV and non-IV data together31 ,32 and one study combined adult and paediatric IV doses.28 It was thus inappropriate to perform a meta-analysis of paediatric MAE rates for comparison with adult MAE rates.

Discussion

Methodological variations

While methodological variations between studies are widely known, no review has systematically summarised and quantified their effects on reported MAE rates. Using the UK as a case-study, we found wide methodological variation even within one country. Some differences reflect the objectives of specific studies; the rationale for other differences was less clear. We also quantified the effect of some methodological variations on the reported MAE rate. Notably, IV doses were five times more likely to be associated with a MAE than non-IV doses. While we recognise the 95% CI for the pooled MAE rates for non-IV and IV doses overlap, the 95% CI for the OR does not cross zero, suggesting that the odds of error was significantly different for non-IV and IV doses.

Our findings highlight the importance of considering a number of methodological details when interpreting studies of MAE rates. More research is required to quantify other methodological effects on reported MAE rates, for example: (1) whether or not nurse participants were fully, partially or not informed of the study objectives, (2) type of observer for example, pharmacist and/or nurse, and (3) the type of medication order included in studies, for example regular and/or ‘when required’ medication orders.

MAE rates and practical implications

Our meta-analysis revealed an overall MAE rate of 5.6% of non-IV OEs and 35% of IV OEs in UK hospitals. Our pooled estimate of the MAE rate for non-IV doses was based on a relatively homogenous, large sample of OEs in adult patients from a wide range of settings and therefore may be useful for benchmarking and monitoring UK hospital MAE rates. By contrast, there was a limited sample and wide CI for IV doses.

Sub-analysis of MAE rates for non-IV doses showed no apparent trends over the past 15 years. However, interpretation is limited as studies cannot be compared directly due to methodological variations that exist. Studies measuring MAE rates at frequent regular intervals using consistent methods are required to monitor long-term trends; this may require coordination at a local and national level in order to maximise the utility of the data collected beyond that of a ‘standalone descriptive study’.

Limited numbers of UK studies and insufficient reporting in all three paediatric studies prevented calculation of overall MAE rates for paediatric non-IV and IV doses separately. Future studies measuring and reporting separate MAE rates for non-IV and IV doses in paediatrics are required to assess the effect of including paediatric doses on reported MAE rates.

Suggestions for future studies of MAEs

Based on our findings, we suggest definitions and methods for measuring MAEs be based on those used previously, to allow comparison with past findings as well as capturing new errors that arise. For studies that include IVs, paediatrics and other doses that require multiple manipulations, for example for patients with dysphagia, we suggest building on the work by Taxis et al28 by separating MAE subcategories according to preparation and administration stage. This will develop our understanding of where MAEs occur and allow comparisons to be made across different medication doses and systems.

Based on Allan and Barker's MAE definition5 we propose subcategories be assigned from the perspective of the medication order where practical (rather than the patient's perspective). Although the patient's perspective plays a vital role in assessing the quality of healthcare in many cases, we advocate the medication order viewpoint to provide a practical approach to categorising MAEs which also allows for better comparison with previous studies. The perspective is important to distinguish between errors such as administering a ‘wrong drug’ to the right patient (patient perspective) and administering the right drug (according to the medication order used at the time of administration) to the ‘wrong patient’; the former is an error at the preparation stage and the latter is an error at the administration stage.

To improve the clarity of ‘unordered/unauthorised drug’ errors, we recommend splitting this subcategory into three: ‘wrong drug’, ‘wrong patient’ and ‘administration without a medication order’. A ‘wrong drug’ error occurs when the incorrect drug is selected against an existing medication order, a ‘wrong patient’ error occurs when the correct drug is selected but administered to a different patient and ‘administration without a medication order’ is giving a drug to a patient against no existing medication order (eg, giving a dose before it has been prescribed on the drug chart).

The use of OE as the denominator has been advocated for determining medication error rates in general39 and for MAE rates specifically.5 For calculating MAE rates, we recommend using the proportion of OE with at least one MAE as we found it to be the most practical and easily interpretable. We therefore advocate this calculation be used alone or in addition to other MAE rate calculations. In studies where each dose may be associated with more than one OE, the proportion of doses given (or omitted) with at least one MAE should also be reported where possible.

Finally, based on our findings in this review and our experience in conducting observation studies, we propose a set of reporting guidance to support future researchers, table 4. This is intended for use in conjunction with standard good practice for reporting, and is designed to be non-prescriptive as a considerable part of a study's design and subsequent reporting will depend on the objectives. Further work is needed to evaluate this.

Table 4

Suggested reporting criteria for future studies that involve measuring medication administration error (MAE) rates adapted from Allan and Barker5

Limitations

Few studies of IV doses and substantial heterogeneity meant findings from only two studies were used to calculate the overall MAE rate for IV doses and we were also unable to explore the differences between adult and paediatric MAE rates. Only UK-based studies were included and therefore the overall MAE rates cannot be extrapolated to other countries. Finally, 11 of the 16 included studies were co-authored by authors of this review. We report this as a potential limitation as we recognise that this may be perceived as a source of bias. However, we believe this to be also one of the strengths of our review, as our experience has allowed us to review the studies to a greater level of detail.

Conclusion

We have used one country's literature to summarise methodological variations between studies within one country and evaluated their effect on reported MAE rates. Our review was based on the UK literature, however our methodological and reporting recommendations can be applied to other countries. Overall, our findings can be used by researchers to make future MAE studies more transparent and comparable.

Acknowledgments

The authors thank Professor Ann Jacklin for her comments on this manuscript.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Contributors All authors contributed to the conception of the review. MCM prepared the protocol with contribution from BDF and NB. Papers were reviewed for inclusion and exclusion by MCM and a sample of these papers were also reviewed by BDF. Data were extracted by MCM and BDF. Data analysis and interpretation were conducted by MCM with contribution from BDF and NB. MCM wrote the manuscript with contributions from BDF and NB. All authors reviewed and approved the final version of the manuscript.

  • Funding This paper represents independent research supported by the National Institute for Health Research (NIHR) Patient Safety Translational Research Centre. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.