- Research
- Open access
- Published:
Virtual, augmented, mixed, and extended reality interventions in healthcare: a systematic review of health economic evaluations and cost-effectiveness
BMC Digital Health volume 1, Article number: 53 (2023)
Abstract
Introduction
Health economic evaluations are required to best understand the value of interventions to the health economy. As extended reality technologies (an umbrella term including virtual, augmented, and mixed reality) become cheaper and more accessible it is likely that they will be used more within healthcare.
Objective
The aim of this study was to systematically review common practices within health economic evaluations of extended reality interventions in healthcare and to discuss the methods, outcomes, and methodological quality to inform future HEEs.
Methods
MEDLINE, Embase, NHSEED, PubMed, and the ACM Digital Library were searched, and studies retrieved and screened. We extracted descriptions of the population, intervention, comparator, outcomes, context, costs, and economic evaluation data from studies that fit our criteria. We included studies that involved healthcare patients who were provided extended reality interventions versus standard care, other types of care, or another extended reality application within the same setting where the outcome included both health outcomes and health economic evaluations.
Results
The search identified 1,693 records in total, of which 1,271 were excluded after title and abstract screening. A total of 422 articles were retrieved and screened and the majority (n = 233) were excluded as they did not contain a health economic analysis or cost data. Fourteen articles were included in this review, all of which found that extended reality health interventions could provide cost savings. Our findings showed considerable heterogeneity between studies and a lack of clear descriptions of XR interventions, limiting their use within procurement.
Conclusion
Extended reality in healthcare has the potential to offer significant clinical benefits and research has shown it to be promising at delivering cost-savings. We make recommendations based on the findings of our review for future health economic analyses to help ensure that health economic analyses can support decision-makers in procuring these technologies.
Trial registration
PROSPERO 2022 CRD42022342110.
Introduction
Health services have finite resources; it is therefore important to be able to demonstrate that new interventions will bring patient benefits without substantial additional costs. To really understand the value an intervention has to the health system it is essential we understand both its clinical effectiveness and its cost-effectiveness. For some interventions the lower cost and greater potential for adoption might mitigate slightly lower outcomes and, likewise, an intervention that is very clinically effective may not be implementable due to its high costs. Health systems in different countries often adopt different models that direct their investment. For example, within the UK the National Institute for Health and Care Excellence (NICE) has a willingness to pay (WTP) of £20–30,000 per quality adjusted life year (QALY), a measure of value given to a particular health state. This means that more expensive treatments will not be reimbursed as standard, with exceptions sometimes made for rarer diseases [1]. To analyse the cost of using and distributing healthcare resources, particularly where there are several alternatives or options, health economists conduct health economic evaluations (HEEs) to assess which interventions or resources are the best value for money.
One area that has garnered particular attention in recent years for its potential to offer cost savings to the health system is the application of extended reality (XR) technologies in healthcare [2]. XR refers to technologies that offer some level of immersion, a digital environment that extends our senses into the virtual. The most common examples include virtual environments (VEs), virtual reality (VR) where you are fully immersed in a virtual environment, and augmented reality (AR) where a virtual environment is overlayed onto the real world. The cost of XR has decreased as the functionality has increased leading to its wider deployment on a variety of digital platforms. AR and VR applications can be found on smartphones and VR headsets used for gaming in peoples’ homes; they have become ubiquitous in the lives of many. An XR product can consist either of a purpose-built system or can take advantage of existing systems including commercially available products. The latter may be less costly as it does not require development of the device. However, the development of software for XR interventions still requires specialist knowledge that is not yet widely available or affordable [3, 4]. Despite this, it does appear that the value for money of digital health interventions may be more closely linked to clinical benefits rather than the cost of the intervention [5].
Its availability and relatively low cost have led some to champion it as superior to standard treatments, for example in the case of exposure therapy for social anxiety disorder [6]. Within exposure therapy, treatments can be delivered by smaller teams and in less time, simulating environments that are too difficult, unsafe, or costly to expose people to in-vivo [7, 8]. For rehabilitation, XR can offer benefits beyond standard interventions in upper limb treatments [9], Parkinson’s Disease [10], and stroke [11]. It can offer relief from chronic pain [12] and burn wound care [13]. Its clinical efficacy in these areas, and others, has consistently been demonstrated yet its cost-effectiveness is still relatively unknown.
Considering the savings that might be made utilising existing XR technologies such as VR or AR, which are decreasing in cost each year, these might be an attractive way of providing resource efficiencies in healthcare. Likewise, the potential to develop unique and/or more effective treatments using the functionality of XR might lend itself to innovative new interventions. However, without better understanding the challenges, and both clinical and cost-effectiveness, XR may not have the opportunity to realise its full potential within healthcare. Even if it is demonstrated to be more effective for certain outcomes, additional requirements such as equipment, training, or space may limit its applicability within real-world settings. XR interventions for healthcare are already available to private payers but there are limited pathways to provision within healthcare systems, for example through reimbursement [14], which is also seen for other digital health products.
Previous reviews have often called for evaluations of cost-effectiveness and noted how few have yet been done [11, 15,16,17]. No systematic reviews of the cost-effectiveness of XR have been published and a preliminary search of PROSPERO, the Cochrane Database of Systematic Reviews, and the JBI Database of Systematic Reviews and Implementation Reports found none currently being done or underway. A broad review across XR and healthcare, whilst not providing a definitive assessment of cost-effectiveness, can highlight the opportunities and challenges for XR HEEs in healthcare for future researchers when undertaking HEEs.
This study aimed to systematically review common practices within HEEs of XR in healthcare and to discuss their methods, outcomes, and reported challenges to inform researchers, policy makers, and developers. It focused on costs and health economic evaluations. Methods, reporting quality, and outcomes are discussed, and recommendations made for future research and the implementation of XR in healthcare.
Methods
Search strategy
The search strategy was developed alongside a Senior Information Specialist at Nottinghamshire Healthcare NHS Foundation Trust, who also ran the search. An initial limited search of Ovid MEDLINE was undertaken to identify articles on the topic. The text words contained in the titles and abstracts of relevant articles, and the database specific subject headings used to describe the articles were used to develop a full search strategy for Ovid MEDLINE ALL, Ovid Embase, NHS Economic Evaluation Database (NHSEED), PubMed, and the ACM Digital Library (see Supplementary files). The search strategy, including all identified keywords and subject headings, was adapted for each included database. The reference list of all included sources of evidence was screened for additional studies. Studies published in any language were included and for all time. Inclusion criteria are shown in Table 1.
Study selection
Following the search, all identified citations were collated and uploaded into EndNote 20 and duplicates removed. Following a pilot test, titles and abstracts were screened by two or more independent reviewers for assessment against the inclusion criteria for the review. Potentially relevant studies were retrieved in full, and their citation details imported into EndNote. The full text of selected citations were assessed in detail against the inclusion criteria. Reasons for exclusion of papers at full text that did not meet the inclusion criteria were recorded and are reported. Any disagreements that arose between the reviewers at each stage of the selection process were resolved through discussion. The results of the search and the study inclusion process are presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram [18].
Assessment of methodological quality
Selected studies were appraised for methodological quality in the review using the Joanna Briggs Institute checklist for economic evaluations [19]. This checklist considers eleven areas of quality and asks whether these have been included within the study write-up. They can be answered as Yes, No, Unclear, or Not Applicable. Higher quality studies adhere to more areas of quality, i.e. there are more affirmative responses. Generally, this checklist is used to decide what studies to include in a review. However, all studies regardless of their methodological quality underwent data extraction and synthesis (where possible) as the objective of this review is to consider the reporting of economic evaluations rather than the validity of the economic evaluation.
Data extraction
The data extracted included descriptive data about the intervention/s and comparator/s examined, study population/participants and context, study methods; secondly, results for the resource use, cost and cost-effectiveness measures; and thirdly, where possible author conclusions about factors that promote/impede cost-effectiveness of the intervention.
Data synthesis
Data extracted from included studies were analysed and summarised to address the review objective using a formal narrative synthesis. This included a textual analysis of the relationships between and within the different interventions, methodologies, clinical outcomes, and economic evaluation data. Due to the heterogeneity of the studies the review synthesised the data qualitatively rather than as a quantitative analysis. Our synthesised results were reported using the Synthesis without Meta-Analysis [20] and PRISMA guidelines [18].
Definitions
Due to the heterogeneity of health economic evaluations across global health systems, countries, and academic disciplines, we will detail here our definitions that were used in our data extraction and synthesis. For health economic analyses, we have used the definitions adopted by the National Institute for Health and Care Excellence (NICE) [21]. We use the term ‘cost-effectiveness’ to describe value for money and the following terms for different analyses.
Cost–benefit analysis
Where costs and benefits are measured using the same monetary units to see whether the benefits exceed the costs [21].
Cost-consequence analysis
Where costs and consequences (such as health outcomes) of an intervention are compared with a suitable alternative. Outcomes are not summarised in a single measure or in financial terms but are shown in their natural units [21].
Cost-minimisation analysis
Where the costs of different interventions that provide the same benefits are compared. If they are equally effective only costs are compared, and the cheapest will be most cost-effective [21].
Cost-utility analysis
Where benefits are assessed, both in terms of quality and duration of life, and expressed as quality-adjusted life years (QALYs) [21].
We also include the commonly used definition of cost-analysis to account for analyses where only costs were measured and outcomes were not considered [21].
We have defined payer and societal perspectives as follows:
Payer perspective
Where the value for money is considered from the viewpoint of the healthcare payer (e.g., an individual, an institution, a service, or a system) [21].
Societal perspective
Where the value for money is considered within the wider economy, e.g., where benefits might be felt by employers, family members, etc. [21].
Results
The search identified 1,693 records in total after excluding duplicates, of which 1,271 were excluded after title and abstract screening based on the exclusion criteria. A total of 422 articles were retrieved and screened by one author (AGB) with a sample of 100 checked by another author (MC). The majority (n = 233) were excluded as they did not contain a health economic analysis or cost data. Others were excluded as they did not fit the inclusion criteria (see PRISMA flow diagram, Fig. 1). Fourteen articles were included in this review.
PRISMA flow diagram [18]
Funding
A total of seven studies received public funding [22,23,24,25,26,27,28], three from either a not-for-profit organisation or charity [29, 30] and one from a commercial organisation [31]. Three were unclear or did not report funding [32,33,34]. Commercial affiliations were declared in three articles [27, 28, 31] and were unclear in five [26, 29, 32,33,34]. The rest declared none.
Types of journals
Most articles were published in medical technology journals (n = 7; [24, 26, 27, 29, 33,34,35]) followed by medical specific (n = 6; [22, 28, 30,31,32]) and one within a medical general journal [25].
Country
Studies were conducted in the USA [26, 28, 29, 31, 32], Spain [25, 30], Nigeria [22], Taiwan [23], Mexico [33], Netherlands [35], France [34] and the UK [27]. One was conducted in multiple countries – Norway, Denmark, and Belgium [24].
Currency
The currency reported included a single currency—USD (n = 3; [24, 29, 32], Euro [30, 35], Taiwan Dollar [23] – or two currencies (USD and Nigerian Naira [22]; USD and Mexican Peso [33]). Five did not specify the currency but used a dollar symbol [25, 26, 28, 31, 34].
Publication year
Articles were published between 2009 and 2022. One was published in 2009 [26], one in 2011 [34], and one in 2015 [25]. Two were published in 2018 [29, 32], four in 2019 [23, 24, 28, 33], two in 2020 [31, 35] and three in 2022 [22, 27, 30].
Medical field
Four studies were applicable to pain management or distraction—within inpatient settings [29], wound care [32], and surgical [28, 33]. Six related to rehabilitation in patients who had experienced back pain [22], renal failure [30], stroke [23,24,25], and after knee surgery [31]. Four related to psychiatric treatments for patients diagnosed with a psychotic disorder [27, 35] or post-traumatic stress disorder [26, 34].
Description of XR interventions
Five interventions are classed as low immersive as they used an MS Kinect to simulate or respond to the patient’s movements in a virtual environment on a screen [22, 23, 25, 29, 30]. Four used a head mounted display, indicating that they were fully immersive [27, 28, 33, 35]. These included various commercially available devices including the Google Cardboard [33], the Sony HMZ-T3W 3D movie goggles [28, 36], Cinimizer goggles [28], Samsung Gear VR System [28], and the HTC Vive Pro [27]. Six did not provide sufficient information to ascertain the level of immersiveness [24, 26, 29, 31, 32, 34].
These were mainly designed to be delivered in a clinical setting [22,23,24,25, 28,29,30, 32, 33, 35]. One was also delivered at home [25], one only at home [31], and three reported this information elsewhere [26, 27, 34]. For those that identified a commercial product, four companies remain active [24, 27, 29, 35] and one appears to no longer operate [31].
Description of HEAs
Table 2 provides an outline of the included studies. Many of the articles did not specify what type of HEA they conducted (n = 8; [23, 25, 26, 28, 30, 32,33,34]). Two conducted both a cost-utility analysis and a cost-consequence analysis [27, 35] and one a cost-utility analysis only [22]. Three conducted a cost-consequence analysis [23, 29, 30], six a cost-minimisation analysis [24, 25, 28, 31, 33, 34], and one conducted a cost–benefit analysis [26]. One conducted a cost-analysis and reported a cost saving but did not consider outcomes [32]. Eight did not report explicitly on the perspective [23, 25, 26, 28, 30, 32,33,34]. Ten took a payer perspective [22,23,24, 28,29,30,31,32,33,34], one took a societal perspective [26] and three took both [25, 27, 35].
Studies mainly used usual care to compare costs [22, 23, 35, 24, 27, 29,30,31,32,33,34]. They also compared delivering the intervention at home and at a clinic [25] or compared the cost of the intervention with the cost of anaesthesia [28]. One did not report a comparator [26]. Studies either replaced existing interventions [22, 26, 28, 31, 33, 34] or provided XR in addition to treatment as usual or conventional approaches [23,24,25, 27, 29, 30, 32, 35].
Costs
Two studies used micro-costing [27, 30]. One also considered the costs incurred for adverse events related to the XR device including minor (e.g., motion sickness) and major (e.g. hospitalisation) [29]. The cost incurred using XR was not reported by eight studies [23, 26, 28, 30,31,32,33]. Two provided a maximum cost for the XR based on the outcomes of the health economic analysis [27, 34]. Of those that did report costs, two provided a complete cost including e.g., equipment, consumables, support [24, 29, 35]. One provided a cost for the equipment, the development of the intervention, and variable costs such as consumables which they calculated as a cost per patient [22]. One reported on the cost of the equipment only [25].
Most resource costs were related to service use (n = 6; [22, 23, 27, 30, 31, 35]) with two identifying medication costs [28, 32] and one including both medication costs and hospitalisation as well as reimbursement for improved patient satisfaction [29]. Staff time was costed by five studies [25, 27, 32, 34, 35] and medical consumables by four [22, 30, 32, 35]. Finally, the treatment offered as a comparator was provided with an overall cost by five [24,25,26, 31, 33]. An additional study considered unpaid care [27].
Quality adjusted life years (QALYs)
Three studies included QALY gains [22, 27, 35]. In the first, these were calculated by using Oswestry Disability index scores mapped to the SF-6D [37] and then multiplied by the duration of time [22]. In another, QALYs were derived from the Green Paranoid Thoughts Scale where the mean score was converted into the standard mean difference by dividing the raw mean change scores by the standard deviation of the Green Paranoid Thoughts Scale at baseline in control [38]. A conversion factor using a time trade-off was then used to identify the corresponding utility change [35]. Finally, QALYs were calculated from health state utilities derived from EQ-5D-5L [39] and ReQoL-20 scores that had been converted into utility scores at baseline [40, 41], 6 weeks, and 6 months using the area under the curve approach [27, 42].
Outcomes
One study reported that there were no cost savings from the XR intervention within the clinical trial as it did not lead to a reduction in therapist time and due to the cost of the equipment may actually lead to an increase in cost [24]. However, a cost saving was reported when therapist time was reduced to 25% and scaled up for delivery to a larger population. Studies attributed savings to the XR intervention reducing length of hospitalisation [23, 29] or rehospitalisation [31], reducing the number of tests and outpatient visits needed [30], reducing the use of anaesthesia and other procedures [28, 32, 33], and reducing staff time [25, 26, 31]. Three studies considered the economic viability of the XR intervention, finding it to be less costly and more effective than a control [22, 27, 35].
Quality appraisal
The quality of six was very low, with between 0 and 4 confirmed quality indicators reported within the article [23, 26, 28, 32,33,34]. Seven had between 7 and 10 confirmed quality indicators [22, 24, 25, 27, 29,30,31] and only one provided all 11 within their article [35]. The majority did not report on incremental or sensitivity analyses.
The results of the JBI quality appraisal are shown in Table 3.
Discussion
Our systematic review identified fourteen studies reporting health economic analyses or cost-effectiveness of an XR healthcare intervention. The most common health economic analyses were cost-minimisation and cost-consequence analyses and perspectives were mainly payer. Usual care was frequently used as a comparator. Costs typically focused on resource costs such as service use, medication, staff time, medical consumables, and treatment costs with many providing limited or no information about XR costs (e.g., equipment). Three studies included QALYs. Under certain conditions, the use of XR was found to be cost saving in all studies. These conditions included where the use of XR led to reductions in hospitalisation, number of medical tests or procedures needed, number of outpatient visits, and staff time. Many also indicated cost savings when delivery was considered over a longer period and larger populations.
The majority of studies received public funding with no reported commercial affiliations, were published in medical technology journals, and conducted within the USA with the most frequently reported currency being USD. The articles were published between 2009 and 2022, covering diverse medical fields such as pain management, rehabilitation, and psychiatric treatment. The interventions only utilised non-immersive and immersive virtual or augmented reality; none adopted mixed reality or metaverse technologies. Most interventions were designed for clinical settings, while some were delivered at home. Our findings appear to suggest the economic viability of XR healthcare interventions across a range of disciplines and settings, and in comparison to both usual care and alternative interventions. However, as most did not include the cost or related costs of the equipment, it is difficult to confirm that cost savings were made in these cases.
Eight studies within this review did not provide a cost for the XR equipment with two using the health economic analysis to provide a maximum cost based on willingness to pay. Although XR costs are dropping, they are not yet free and so without knowing the cost saving in consideration of the equipment, it is difficult to assess whether it is economically viable. Where costs were reported these were not always clear. Six studies also did not report sufficient information about the XR intervention to understand the level of immersiveness offered. This limits providers in replicating outcomes in real-world practice, particularly where less immersive, and therefore more affordable, devices have been used.
There are additional medical fields where clinical effectiveness has been demonstrated that were not included in this review, such as ophthalmology [43], and in certain populations that were not specifically targeted, such as older adults [44]. This highlights the heterogeneity of cost-effectiveness studies within XR for healthcare where the focus has traditionally been on the areas covered within this review – pain management, psychiatric care, and rehabilitation.
The case for XR in healthcare
The pace of technological innovation has often meant that cost-effectiveness research lags behind clinical effectiveness, for example in artificial intelligence healthcare systems [45]. However, it is interesting to note within our review that costs have been considered alongside health outcomes from as early as 2009 in the field of XR. This most likely reflects the long history of XR being used to simulate real world environments in a safer and more controlled way so as to deliver exposure therapy for various psychiatric conditions [7, 8]. It is promising, therefore, that our review has also found several other medical fields where XR shows promise, including rehabilitation and pain management.
The use of XR to deliver clinical and cost effectiveness has huge potential in healthcare. Using immersive environments has the potential to support patients, including younger populations, in overcoming the pain, discomfort, and boredom associated with repetitive rehabilitative tasks, e.g., through gamifying physical therapy [46], and reducing pain perception through directing attention away from painful stimuli [47]. It can also be used to deliver powerful and effective psychological treatments [48].
Within our review, we found that XR can be both clinically and cost effective in replacing existing healthcare interventions [22, 26, 28, 31, 33, 34]. This can lead to reduced staff time and savings based on the replacement of medical tests or procedures. XR as an add-on intervention, where it is considered in addition to treatment as usual or conventional approaches, was also found to be cost-effective [23,24,25, 27, 29, 30, 32, 35]. As an add-on, XR can also demonstrate improved QALYs, shorter lengths of stay in hospital or fewer hospitalisations, and reductions in medical tests or procedures.
Previous reviews have addressed the lack of evaluations within real-world settings [16, 49]. It is promising that, of the five studies which used commercial products, four of the companies identified are still operational. The implementation of XR interventions requires not just research to demonstrate clinical and cost effectiveness, but also sustainable business practices.
XR equipment is becoming more affordable, and performance is improving. This could also mean that underserved communities can access healthcare and treatments that were either unavailable or difficult to deliver [33, 43].
Recommendations
The move towards ‘software as a medical device’ within regulation now means that relatively inexpensive devices can be purchased as capital by healthcare services with it then being possible to run multiple software for various interventions from one device. It is likely that costs will remain low, more so if there is high utilisation although the lifetime of XR devices is somewhat uncertain as this will depend on amount of use, whilst the longevity of manufacturer support may be limited if older models are deemed obsolete or are discontinued (e.g., Kinect).
More clarity is needed where the costs associated with XR are provided. Descriptions should provide sufficient information on what is and isn’t included e.g., ongoing development or personalisation, equipment and its replacement, maintenance, or technical support. Additionally, if providing a one-off cost this should also include the lifespan.
Potentially, where there is a clear understanding of the willingness to pay threshold, the maximum cost could be calculated based on the findings of the analysis [27, 34]. Alternatively, sensitivity analyses should be done to consider the potential savings of scaling up to more patients or what variables might lead to more savings. This is particularly important in XR where the cost of the devices change over time or where how it is delivered can lead to significant savings, e.g., by reducing staff time [24].
Where XR was demonstrated to reduce lengths of hospital stay or fewer medical treatments, researchers should consider the reasons for this in more detail. It has long been understood that XR can lead to increased adherence to treatment [3] which could lead to more effectiveness in usual care. If the mechanisms are better understood, we could explore how its application within other areas of healthcare where XR may not have a direct effect but could be used to increase adherence.
We must also begin to consider cost-effectiveness across multiple health and care systems and perspectives. XR interventions can have a global reach and be available in different countries.
Strengths and limitations
This is the first review to consider the methods and results of studies addressing the cost-effectiveness of XR in healthcare. The quality of included studies was considered, and recommendations for future health economic evaluations were made. As this was a review of current practice within health economic analyses and cost-effectiveness studies for XR in healthcare we did not exclude low quality studies. However, this may have led to some areas of this review highlighting common practices that are not common within studies with a higher quality rating. Despite including all languages in our search, we did not identify any relevant publications during screening that were in languages other than English. After screening full text articles, we found that whilst many identified cost-effectiveness within their abstracts, they did not conduct a health economic analysis to justify these claims. As a result, we had a high exclusion rate.
Implications for practice and future research
This review has addressed health economic evaluations of XR in healthcare. However, the use of XR in healthcare is still nascent in many areas and we must adopt a much broader consideration of its use than just cost-effectiveness. A recent survey found that the acceptability of VR therapies in mental health amongst the general population was higher in those more familiar with VR, suggesting that increasing adoption of consumer XR technologies could lead to wider spread adoption within health and social care [50]. However, the resources needed for these technologies (e.g., infrastructure, connectivity, training) may require additional investment that is not included in health economic evaluations. Additionally, as with all technologies, the introduction of XR must consider the wider ethical implications. For example, if cost savings are from reduced staff time, what is the impact on patient experience? It is promising that one of the studies included ratings of patient satisfaction in its evaluation [29]. Their introduction into health and social care may also lead to the exclusion of certain populations due to e.g., accessibility issues or digital literacy. Moving forward, health economic evaluations of more complex interventions should aim to be done in tandem with evaluations of the broader impact, in line with Medical Research Council guidelines for developing and evaluating complex interventions [51].
Conclusion
XR in healthcare has the potential to offer significant clinical benefits and research has shown it to be promising at delivering cost-savings. We make recommendations based on the findings of our review for future health economic analyses to help ensure that health economic analyses can support decision-makers in procuring these technologies.
Availability of data and materials
The datasets used and analysed during the current study are either included in this published article or are available from the corresponding author on reasonable request.
References
Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.
Craven M, Bergin A. Health Economics. In: Kilkelly F, O’Brien R, Ticho S, editors. The growing value of XR in healthcare in the United Kingdom. 2021. p. 46–57. Available from: https://www.xrhealthuk.org/.
Rutkowska A, Salvalaggio S, Rutkowski S, Turolla A. Use of virtual reality-based therapy in patients with urinary incontinence: a systematic review with meta-analysis. Int J Environ Res Public Health. 2022;19(10):6155.
Macedo M, Marques A, Queirós C. Realidade virtual na avaliação e no tratamento da esquizofrenia: Uma revisão sistemática. J Bras Psiquiatr. 2015;64(1):70–81.
Gega L, Jankovic D, Saramago P, Marshall D, Dawson S, Brabyn S, et al. Digital interventions in mental health: evidence syntheses and economic modelling. Health Technol Assess (Rockv). 2022;26(1):i–181.
Caponnetto P, Triscari S, Maglia M, Quattropani MC. The simulation game—virtual reality therapy for the treatment of social anxiety disorder: a systematic review. Int J Environ Res Public Health. 2021;18(24):13209.
Eshuis LV, van Gelderen MJ, van Zuiden M, Nijdam MJ, Vermetten E, Olff M, et al. Efficacy of immersive PTSD treatments: a systematic review of virtual and augmented reality exposure therapy and a meta-analysis of virtual reality exposure therapy. J Psychiatr Res. 2021;143:516–27. https://doi.org/10.1016/j.jpsychires.2020.11.030.
Eshuis LV, Gelderen MJ Van, Zuiden M Van, Nijdam MJ, Vermetten E. Efficacy of immersive PTSD treatments : a systematic review of virtual and augmented reality exposure therapy and a meta-analysis of virtual reality exposure therapy. J Psychiatr Res. 2022. https://doi.org/10.1016/j.jpsychires.2020.11.030.
Baeza-Barragán MR, Labajos Manzanares MT, Vergara CR, Casuso-Holgado MJ, Martín-Valero R. The use of virtual reality technologies in the treatment of duchenne muscular dystrophy: systematic review. JMIR mHealth uHealth. 2020;8(12):1–13.
Wang W, Wong SSL, Lai FHY. The effect of virtual reality rehabilitation on balance in patients with parkinson’s disease: a systematic review and meta-analysis. Electron. 2021;10(9):1003.
Ogourtsova T, Souza Silva W, Archambault PS, Lamontagne A. Virtual reality treatment and assessments for post-stroke unilateral spatial neglect: a systematic literature review. Neuropsychol Rehabil. 2017;27(3):409–54. https://doi.org/10.1080/09602011.2015.1113187.
Grassini S. Virtual reality assisted non-pharmacological treatments in chronic pain management: a systematic review and quantitative meta-analysis. Int J Environ Res Public Health. 2022;19(7):4071.
Scapin S, Echevarría-Guanilo ME, Boeira Fuculo Junior PR, Gonçalves N, Rocha PK, Coimbra R. Virtual Reality in the treatment of burn patients: a systematic review. Burns. 2018;44(6):1403–16. https://doi.org/10.1016/j.burns.2017.11.002.
Vincent C, Eberts M, Naik T, Gulick V, O’Hayer CV. Provider experiences of virtual reality in clinical treatment. PLoS One. 2021;16(10):1–14. https://doi.org/10.1371/journal.pone.0259364.
Wu J, Zeng A, Chen Z, Wei Y, Huang K, Chen J, et al. Effects of virtual reality training on upper limb function and balance in stroke patients: Systematic review and meta-meta-analysis. J Med Internet Res. 2021;23(10):e31051.
Zanatta F, Giardini A, Pierobon A, D’Addario M, Steca P. A systematic review on the usability of robotic and virtual reality devices in neuromotor rehabilitation: patients’ and healthcare professionals’ perspective. BMC Health Serv Res. 2022;22(1):1–16. https://doi.org/10.1186/s12913-022-07821-w.
Segawa T, Baudry T, Bourla A, Blanc JV, Peretti CS, Mouchabac S, et al. Virtual Reality (VR) in assessment and treatment of addictive disorders: a systematic review. Front Neurosci. 2020;13:1409.
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
The Joanna Briggs Institute. JBI critical appraisal tools for use in JBI systematic reviews: checklist for economic evaluations. Available from https://jbi.global/sites/default/files/2019-05/JBI_Critical_Appraisal-Checklist_for_Economic_Evaluations2017_0.pdf.
Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020;368:1–6.
Glossary. National Institute for Health and Care Excellence. Available from: https://www.nice.org.uk/Glossary.
Fatoye F, Gebrye T, Mbada CE, Fatoye CT, Makinde MO, Ayomide S, et al. Cost effectiveness of virtual reality game compared to clinic based McKenzie extension therapy for chronic non-specific low back pain. Br J Pain. 2022;0(0):204946372211091.
Ho TH, Yang FC, Lin RC, Chien WC, Chung CH, Chiang SL, et al. Impact of virtual reality-based rehabilitation on functional outcomes in patients with acute stroke: a retrospective case-matched study. J Neurol. 2019;266(3):589–97. https://doi.org/10.1007/s00415-018-09171-2.
Islam MK, Brunner I. Cost-analysis of virtual reality training based on the virtual reality for upper extremity in subacute stroke (VIRTUES) trial. Int J Technol Assess Health Care. 2019;35(5):373–8.
Lloréns R, Noé E, Colomer C, Alcañiz M. Effectiveness, usability, and cost-benefit of a virtual reality-based telerehabilitation program for balance recovery after stroke: a randomized controlled trial. Arch Phys Med Rehabil. 2015;96(3):418–425.e2.
Wood D, Murphy J, McLay R, Koffman R, Spira J, Obrecht R, et al. Cost effectiveness of virtual reality graded exposure therapy with physiological monitoring for the treatment of combat related post traumatic stress disorder. Annu Rev Cybertherapy Telemed. 2009;7:223–9.
Altunkaya J, Craven M, Lambe S, Beckley A, Rosebrock L, Dudley R, et al. Estimating the economic value of automated virtual reality cognitive therapy for treating agoraphobic avoidance in patients with psychosis: findings from the gameChange randomized controlled clinical trial. J Med Internet Res. 2022;24(11):e39248. Available from: https://www.jmir.org/2022/11/e39248. Cited 2023 Apr 3.
Nguyen N, Lavery WJ, Capocelli KE, Smith C, DeBoer EM, Deterding R, et al. Transnasal endoscopy in unsedated children with eosinophilic esophagitis using virtual reality video goggles Nathalie. Clin Gastroenterol Hepatol. 2019;17(12):2455.
Delshad SD, Almario C V., Fuller G, Luong D, Spiegel BMR. Economic analysis of implementing virtual reality therapy for pain among hospitalized patients. npj Digit Med. 2018;1(1). https://doi.org/10.1038/s41746-018-0026-4.
García Testal A, José F, Olmos M, Antonio J, Gómez G, Coca JV, et al. Impact of an intradialysis virtual ‑ reality ‑ based exercise program on healthcare resources expenditure : a micro ‑ costing analysis. BMC Nephrol. 2022;1–7. https://doi.org/10.1186/s12882-022-02859-8.
Prvu Bettger J, Green CL, Holmes DN, Chokshi A, Mather RC, Hoch BT, et al. Effects of virtual exercise rehabilitation in-home therapy compared with traditional care after total knee arthroplasty: VERITAS, a randomized controlled trial. J Bone Jt Surg. 2020;102(2):101–9.
Drever SA, Soltani M, Sharar SR, Wiechman SA, Patterson DR. Virtual reality hypnosis for pain control during wound care in a patient with burn injuries: a potential cost-savings intervention. J Burn Care Res. 2018;39(suppl_1):S101–S101.
Mosso Vazquez JL, Tomas Obrador G, Mosso Lara D, Luis Mosso Lara J, Wiederhold BK, Lara Vaca V, et al. Pain reduction with VR in indigenous vs urban patients in ambulatory surgery. Annu Rev Cybertherapy Telemed. 2019;17:99–104.
Robillard G, Bouchard S, Dumoulin S, Guitard T. The development of the SWEAT questionnaire: a scale measuring costs and efforts inherent to conducting exposure sessions. Annu Rev Cybertherapy Telemed. 2011. Available from: http://w3.uqo.ca/cyberpsy/docs/qaires/sweat/2011Robillardet alSWEATARCTT2011.pdf. Cited 2018 Oct 22.
Pot-Kolder R, Veling W, Geraets C, Lokkerbol J, Smit F, Jongeneel A, et al. Cost-effectiveness of virtual reality cognitive behavioral therapy for psychosis: health-economic evaluation within a randomized controlled trial. J Med Internet Res. 2020;22(5):1–13.
Pot-Kolder R, Veling W, Geraets C, van der Gaag M. Effect of virtual reality exposure therapy on social participation in people with a psychotic disorder (VRETp): Study protocol for a randomized controlled trial. Trials. 2016;17(1):25. Available from: https://com-mendeley-prod-publicsharing-pdfstore.s3.eu-west-1.amazonaws.com/8fb9-PUBMED/10.1186/s13063-015-1140-0/13063_2015_Article_1140_pdf.pdf?X-Amz-Security-Token=FQoGZXIvYXdzEEQaDMuH1sUnykzIK7ru6iKfBMRifXWhvyvAca5IcHt5RAKyVavm33eM1h%2Fp0gBkNI9vqpyFdZ. Cited 2018 Nov 28.
Carreon LY, Glassman SD, Mcdonough CM, Rampersaud R, Berven S, Shainline M, et al. Predicting SF-6D utility scores from the Oswestry disability index and numeric rating scales for back and leg pain $watermark-text $watermark-text $watermark-text. Spine (Phila Pa 1976). 1976;34(19):2085–9.
Green CEL, Freeman D, Kuipers E, Bebbington P, Fowler D, Dunn G, et al. Measuring ideas of persecution and social reference: the Green et al. Paranoid Thought Scales (GPTS). Psychol Med. 2008;38(1):101–11.
EuroQol Research Foundation. EQ-5D-3L U. 2018.
Keetharuth AD, Rowen D, Bjorner JB, Brazier J. Estimating a preference-based index for mental health from the recovering quality of life measure: valuation of recovering quality of life utility index. Value Heal. 2021;24(2):281–90. https://doi.org/10.1016/j.jval.2020.10.012.
Van Hout B, Janssen MF, Feng YS, Kohlmann T, Busschbach J, Golicki D, et al. Interim scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L value sets. Value Heal. 2012;15(5):708–15. https://doi.org/10.1016/j.jval.2012.02.008.
Whitehead SJ, Ali S. Health outcomes in economic evaluation: the QALY and utilities. Available from: https://academic.oup.com/bmb/article/96/1/5/300011.
Ma MKI, Saha C, Poon SHL, Yiu RSW, Shih KC, Chan YK. Virtual reality and augmented reality— emerging screening and diagnostic techniques in ophthalmology: a systematic review. Surv Ophthalmol. 2022. https://doi.org/10.1016/j.survophthal.2022.02.001.
Dermody G, Whitehead L, Wilson G, Glass C. The role of virtual reality in improving health outcomes for community-dwelling older adults: systematic review. J Med Internet Res. 2020;22(6):e17331.
Voets MM, Veltman J, Slump CH, Siesling S, Koffijberg H. Systematic review of health economic evaluations focused on artificial intelligence in healthcare: the tortoise and the cheetah. Value Heal. 2022;25(3):340–9.
Phelan I, Furness PJ, Dunn HD, Carrion-Plaza A, Matsangidou M, DImitri P, et al. Immersive virtual reality in children with upper limb injuries: Findings from a feasibility study. J Pediatr Rehabil Med. 2021;14(3):401–14.
Goudman L, Jansen J, Billot M, Vets N, De Smedt A, Roulaud M, et al. Virtual reality applications in chronic pain management: systematic review and meta-analysis. JMIR Serious Games. 2022;10(2):e34402.
Rizzo AS, Koenig ST, Talbot TB. Clinical virtual reality: emerging opportunities for psychiatry. Focus (Madison). 2018;16(3):266–78. Available from: https://www.researchgate.net/publication/326472495. Cited 2020 Jan 8.
Langener S, Vandernagel J, van Manen J, Markus W, Dijkstra B, Defuentes-merillas L, et al. Clinical relevance of immersive virtual reality in the assessment and treatment of addictive disorders: a systematic review and future perspective. J Clin Med. 2021;10(16):1–27.
Gomez Bergin, AD, Allison, AM, Hazell, CM. Understanding public perceptions of virtual reality psychological therapy: development of the attitudes towards virtual reality therapy (AVRT) Scale. JMIR Mental Health. 2023. Available from: https://preprints.jmir.org/preprint/48537/accepted.
Craig P, Dieppe P, Macintyre S, Mitchie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:979–83. Available from: www.mrc.ac.uk/. Cited 2021 Feb 23.
Acknowledgements
AB, MC acknowledge support of the National Institute for Health Research (NIHR) which funds their employment (see funding statement). This work was supported by the Engineering and Physical Sciences Research Council [grant number EP/Y009800/1]. The views expressed are those of the authors and not necessarily those of the NIHR, the National Health Service or the Department of Health and Social Care. These organisations had no role in study design, including; collection, management, analysis and interpretation of data; writing of the report; and the decision to submit the report for publication. We would also like to thank Naomi Thorpe and Elizabeth Doney from the Nottinghamshire Healthcare NHS Foundation Trust Library Services for their invaluable help.
Funding
This research was funded by the NIHR Nottingham Biomedical Research Centre and NIHR MindTech MedTech Co-operative and supported by the Engineering and Physical Sciences Research Council [grant number EP/Y009800/1].
Author information
Authors and Affiliations
Contributions
AGB screened all articles, extracted data, conducted the quality assessment, and was a major contributor in writing the manuscript. MC provided inter-reporter reliability as second author for screening, data extraction, and quality assessment. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
MC and AGB contributed to Altunkaya et al. [22].
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1.
Search Strategy.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Gómez Bergin, A.D., Craven, M.P. Virtual, augmented, mixed, and extended reality interventions in healthcare: a systematic review of health economic evaluations and cost-effectiveness. BMC Digit Health 1, 53 (2023). https://doi.org/10.1186/s44247-023-00054-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s44247-023-00054-9