Background of the digital health evaluation framework
Evaluation of health intervention requires an understanding of various health evaluation frameworks that are used in digital health intervention. Digital health project requires evaluation to establish effectiveness, safety, risks, efficiency and success of the implementation. This also requires evaluation of the cost and benefits of the implementation process. The digital health project is health monitoring system for patients with COVID-19 in rural area. The following paper explores the evaluation frameworks for digital health project (DHP).
There are many different digital health interventions or projects that have been developed for solving certain issues within the health care services. These interventions are critical for meeting the client needs especially those interventions that aim at reaching out patients in remote areas. There are many reasons for conducting evaluation of digital health care interventions. These digital health information systems require evaluation to establish their feasibility and return on investment (Soobiah et al., 2020).
There is need to understand some of the domains or factors that forms important aspects of digital health projects. Firstly, the domain of usage and accessibility forms an important aspect of digital health and is concerned with usability of the project and accessibility of the project. This is critical for ensuring all stakeholders are using the project according to the intended purpose. Secondly, the digital health project has another area that forms part of evaluation is the risks that are associated with the project. There are some risks that associated with digital health and these forms part of health intervention evaluation. Thirdly, the benefits of the health interventions are necessary for evaluating the project. Fourthly, data system is another area that forms part of the digital health system. This includes various forms of data that are collected, data management and safety of the data collected with the digital health project (Eisner et al., 2019).
Evaluation of digital health projects enables determination of accessibility of these projects by users. Firstly, digital health evaluation frameworks enable determination of improvement with specific focus. Evaluation frameworks are specific and focus on certain areas of the intervention and results in the areas of improvement. This implies that evaluation will determine areas that need to improve within the health intervention (Khoja et al., 2013).
Secondly, evaluation frameworks will attempts to establish the safety or risk that might be associated with digital health intervention. In order to determine the safety of the digital health intervention or project a digital health evaluation framework with focus on risk or safety are used. Moreover, digital health interventions require close monitor and evaluation to determine the safety of these projects to end users. For instance, are these interventions safe or ensure safety of data accumulated. This can be revealed thorough evaluation of digital health intervention.
Scenario
Thirdly, evaluation frameworks are designed to determine the efficiency of the health intervention. Health interventions especially those that work in rural areas are designed to work without challenges. This efficiency can be determined through digital evaluation framework with focus on efficiency (Chan et al., 2015).
Fourthly, personalization is another area that evaluation framework attempts to reveal for effective implementation of the project. Most intervention that are used within health sector requires personalization with view of meeting person centered care. Lastly, evaluation frameworks work to determine the accessibility of the health intervention especially from the user point of view (Bartholomew et al., 2016).
The health scenario involves Happy Valley Base Hospital (HVBH) which has digital health project. The digital health project is developed in response to COVID-19 using hospital in the home care. This is meant to solve the expected increase in COVID-19 patients. The main focuses of digital health information system composing of health monitoring system. The evaluation of the in-home patient care monitoring and information technologies is therefore important for ensuring effective implementation. Project based on 500-bed hospital and with target of 200000 people within the community in the rural area. A comprehensive evaluation framework is necessary for evaluating the health monitoring system for COVID-19 patients in the rural area.
There are many different health interventions evaluation frameworks that can be used to evaluate the health monitoring system. These digital health evaluation frameworks complement each other and enable a comprehensive approach to health intervention evaluation. Some of the proposed health intervention evaluation frameworks are:
- Integrate, Design, Assess, and Share (IDEAS)
- 6 Steps in Quality Intervention Development (6SQuID
- PRECEED-PROCEED model
- Integrate, Design, Assess, and Share (IDEAS)
The Integrate, Design, Assess, and Share (IDEAS) framework brings into the picture design thinking and design approach of the intervention. This framework is chosen since the framework was design for evaluation of digital health intervention and this is also digital health project. The framework is user centered which can be equivalent of person-centered approach which is critical important in health care (Mummah et al., 2016). This forms the basis of using IDEAS in evaluation. The framework has some components that include Integrate, Design, Assess, and Share. The framework is developed to follow project right from the conceptual phase to post implementation evaluation. Framework integrates various components of the project such as user, data, technology, and theory. Health monitoring system needs an integrative approach to bring user and theory together and in a coherent. Design phase of the evaluation take into consideration user feedback to design and redesign the technology for future effectiveness. Assess element of the framework is based on assessment of the project through evaluation and pilot phase of the project. This implies that framework come along with ability to assess both pilot phase or post implementation evaluation. Share component focuses on sharing the intervention and its finding to key stakeholders. This will enable sharing of findings from the project with the executive management team (Fedele et al., 2019).
Evaluation frameworks
Conducting evaluation using IDEAS requires understanding of the whole health information system. Firstly, IDEAS the integrate phase of the design framework will be used to evaluate users of the health monitoring system together with user behaviour and behavior theory. This help to determine the integration of users with the health monitoring system. Evaluation will be conducted through collection of data from the system itself, from users based on survey and feedback from users. Secondly, IDEAS will consider the design process of the health monitoring system in order to establish various design parameters. IDEAS will consider the design of the monitoring technology and this include areas that are data input methods, actual monitoring of the patient and feedback from users. This phase involves evaluation of health monitoring system itself, users and implementation strategies. The evaluation is conducted through collection of feedback from users. Thirdly, the evaluation framework is conducted through pilot program that test on efficacy of the project. The team will conduct pilot project aiming at registering people and testing the system. Lastly, project evaluation will take into account finding from the project where data are collected from users through RCT research.
PRECEED-PROCEED model is another model that is also chosen for evaluating the health monitoring system for COVID-19 project. The framework focuses on needs for designing the intervention, implementation and then evaluation. The framework is chosen to help complement the IDEAS framework that focuses on design thinking. The PRECEED-PROCEED model bring in the factors that come into play to determine the choice of health intervention. The model is divided into two the PRECEED and PROCEED (Skivington et al., 2021). Firstly, PRECEED part of the evaluation framework is focusing on Predisposing, Reinforcing, and Enabling Constructs. These are parameters for assessing factors that influences the need for intervention. In the Happy Valley Base Hospital scenario COVID-19 requires understanding of social, epidemiology, ecology and implementation factors that influence need for health monitoring and implementation. Secondly, PROCEED is an acronym for ‘Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development’. This framework component is crafted for determination of outcomes and implementation of the project. The framework component is designed for implementation of the project, process evaluation, impact evaluation and outcome evaluation. The PRECEED-PROCEED is therefore a comprehensive evaluation framework that evaluates the intervention from many perspectives to ensure effectiveness. This is important for ensuring project considers those factors that influence prevention and management of the COVID-19 pandemic which has become world issue (Crosby & Noar, 2011).
Evaluation using Integrate, Design, Assess, and Share (IDEAS)
Evaluation using PRECEED-PROCEED model
The PRECEED-PROCEED model is used to evaluate health monitoring system by evaluation various components. Firstly, the model is used to evaluate implementation of the project through evaluation of various components of the project. The data for the evaluation will be collected from various stakeholders such as management, medical staff, patients and system administrators. The data will be evaluated to establish efficiency of the system to users. Secondly, the process evaluation is another component of the health intervention that focus on processes within the system itself. For instance, the process of using system or the process of alerting medical staff. Thirdly, the evaluation framework evaluate impact of the health monitoring system. This is conducted through collection of data from the system, from the hospital registry and from users of the system. This data include the number of people using the system, the number patient helped using the system or the number patient hospitable from system.
The 6SQuID is another evaluation framework that can be used to evaluate the case scenario. 6SQuID focuses on the design and feasibility of the intervention which is important for evaluating the health monitoring system with its ability to work in the rural area. In addition, the framework is also seeking to bring about steps to quality intervention development which is essential for health monitoring system (Wight et al., 016). The six steps for quality intervention development include understanding problem and causes, contextual factors, mechanism for change, classifying deliverable, testing and adapting intervention, and evidence collection for evaluation. Following these steps enable prevention of loss of resources during evaluation and implementation of the intervention. The framework brings a view that into design and evaluation of health intervention. Moreover, the framework enables determination of feasibility of the intervention before the project is implemented using the public resources (Christopoulou, Kotsilieris & Anagnostopoulos, 2018).
6SQuID will be used to evaluate the system following some steps. Firstly, 6SQuID enable evaluation of the COVID-19 problem and how health monitoring system come into the problem this is conducted from the expert perspective. Secondly, this step involves identification of modifiable determinants which can be used to change the situation caused by problem. Thirdly, the step involves the decision to change the system and this involves introduction of the health information system to change the situation caused by COVID-19. Fourthly, the step involves evaluation of deliverable from the project and this requires data from system design experts and medical experts. Fifthly, the step involves testing of the health monitoring system to determines its usage and challenges. This is conducted through recruiting users and other system and testing users to determine its uses. Lastly, the last step is the collection of evidence from research and other data on the effectiveness. This is done through collection of data from system users, feedbacks and survey of stakeholder.
PRECEED-PROCEED model
The evaluation of the health monitoring system used in the rural area for COVID-19 patient consider various components of the whole system. Firstly, the technology that is used within the monitoring system which largely consider clinical is setting such as vital signs used in COVID-19 management. Secondly, evaluation will also factor in the user of the system especially the client and medical staff. Thirdly, safety and risk associated with the system is also considered in the evaluation process. Lastly, data management composes of the medical data about the patient that is collected during treatment of patients.
The health monitoring system composes of various software and hardware. This include sensory device that is given to patients within the rural area. This sensory measure temperature, heartbeat and blood pressure. These data is monitored by medical staff through smartphones and computers. The system contains user information and data collected upon registration and usage of the system. The estimated amount for the sensory system is $8.4 and this target more than 100000 patients totaling to $ 84000 from the devices alone.
Conclusion
In conclusion, evaluation of health care intervention enables determination of the success or effectiveness of the intervention. Using evaluation framework to evaluate health monitoring system requires the determination of usability, safety, risk, benefits and data management of the digital health project. This includes the determination of the designing process and steps used to develop the project. There are many different health evaluation frameworks that can be used to evaluate the health monitoring system. Some of the health intervention evaluation frameworks are Integrate, Design, Assess, and Share (IDEAS), 6 Steps in Quality Intervention Development (6SQuID and PRECEED-PROCEED model. these frameworks are essential since they provide evaluation from different perspective.
Reference
Bartholomew, L.K., et al. (2016). Planning health promotion programs; an Intervention Mapping approach. 4th ed. San Francisco, CA: Jossey-Bass. Retrieved from https://scholar.google.com/scholar?q=Bartholomew,%20L.K.,%20et%20al.%20Planning%20health%20promotion%20programs;%20an%20Intervention%20Mapping%20approach%20.%204th%20ed.%20San%20Francisco,%20CA:%20Jossey-Bass,%202016.
Chan, S., et al. (2015). Towards a Framework for Evaluating Mobile Mental Health Apps. Telemedicine Journal and E-Health, 21(12):1038–1041,.10.1089/tmj.2015.0002
Christopoulou, S.C., Kotsilieris, T. & Anagnostopoulos, I.. (2018). Assessment of Health Information Technology Interventions in Evidence-Based Medicine: A Systematic Review by Adopting a Methodological Evaluation Framework. Healthcare, 6(109):1–22,.10.3390/healthcare6030109
Crosby, R. & Noar, S. (2011). PRECEDE-PROCEED Journal of Public Health Dentistry, 71 Suppl 1: S7-15. https://www.ruralhealthinfo.org/toolkits/health-promotion/2/program-models/precede-proceed
Eisner, E., Bucci, S., Berry, N., Emsley, R., Barrowclough, C. & Drake, R.J. (2019). Feasibility of using a smartphone app to assess early signs, basic symptoms and psychotic symptoms over six months: a preliminary report. Schizophr Res. 208:105–13. https://doi.org/10.1016%2Fj.schres.2019.04.003
Fedele, D.A., et al. (2019). Design Considerations When Creating Pediatric Mobile Health Interventions: Applying the IDEAS Framework. Journal of Pediatric Psychology, 44(3):343–348,.10.1093/jpepsy/jsy084
Khoja, S., et al. (2013). Conceptual Framework for Development of Comprehensive e-Health Evaluation Tool. Telemedicine and e-Health, 19(1):48–53,.10.1089/tmj.2012.0073
Mummah, S. A., Robinson, T. N., King, A. C., Gardner, C. D., & Sutton, S. (2016). IDEAS (Integrate, Design, Assess, and Share): A Framework and Toolkit of Strategies for the Development of More Effective Digital Interventions to Change Health Behavior. Journal of medical Internet research, 18(12), e317. https://doi.org/10.2196/jmir.5927
Soobiah C, Cooper M, Kishimoto V, et al (2020). Identifying optimal frameworks to implement or evaluate digital health interventions: a scoping review protocol. BMJ Open;10:e037643. doi: 10.1136/bmjopen-2020-037643
Skivington K, Matthews L, Simpson S A, Craig P, Baird J, Blazeby J M et al. (2021). A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance BMJ, 374 :n2061 doi:10.1136/bmj.n2061
Wight, D., Wimbush, E., Jepson, R., et al (2016). Six steps in quality intervention development (6SQuID) J Epidemiol Community Health;70:520-525.
Order an Essay Now & Get These Features For Free:
Turnitin Report
Formatting
Title Page
Citation
Outline
