Usability and Quality Evaluation of the “E- Midwife” Mobile Application for Nurse- Midwives in Obstetric Complications: A Randomized Controlled Trial

INTRODUCTION

A smartphone is a useful and advanced device that was developed using wireless computational devices. Based on the study done in 2017, it was found that 77% of the world mobile device market invested in smartphones. Smartphones are used by more than 32% of the world population. ¹ Technological advancement in mobile devices helps to shift from personal desktop computers to handy mobile smartphones among academicians and students. ²

Globally, the use of mobile devices has increased and acts as an emerging area of interest in the field of usability testing. Nearly 165,000 mobile health (mHealth) apps are currently accessible in the Apple iTunes and Android app stores in the US, and roughly two-thirds (66%) of Americans use mobile applications (apps) to manage their health. ³ The most recent Annual Survey for Education Report says that during the lockdown, the percentage of households using smartphones increased from 36% in 2018 to 74.8% in 2022. According to the report, 88 percent of homes have access to the Internet, and 67.6% of families have smart- phones. Smartphone use has grown rapidly in recent year. According to Madhav Chavan, president and board member of Pratham Education Foundation, mobile phones and smartphones have recently become the new norm for rural families. ⁴ According to a Deloitte report, India will have had 1 billion smartphone users by 2026, with rural areas driving the market for Internet-enabled phones. In India, there are 1.2 billion mobile customers, 750 million of whom use smartphones. Demand for smartphones is predicted to expand as more people use the Internet, driven by the need for e-learning and e-health. Due to its many apps, including remote healthcare and high-speed gaming, 5G is anticipated to become the fastest-adopted mobile technology. ^{3 , 4} Health care professionals can benefit from mobile devices and apps in a variety of ways, but probably the most important benefit is enhanced access to point-of-care tools, which has been shown to support better clinical decision-making and patient outcomes. ⁵

Smartphones have powerful computing capabilities, capacious memory, a huge screen, and an open operating system that promotes the development of apps. The most recent generation of smart- phones is increasingly seen as a handheld computer rather than a mobile phone. ⁶ At a rate of over two billion apps per month, customers were downloading more than 800 apps every second from the App Store. Additionally, the use of medical apps increased simultaneously. ⁷

The method of delivering education through electronic devices like smartphones or tablets is referred to as mobile learning apps. They are the best and unique alternative to monotonous classroom lecture learning and are getting popular so quickly. According to a survey, there is 4.7 billion growth per year in the development of mobile learning apps. ⁸ In the last few years, mobile phone networking has spread its usage extensively in India. As there is extensive use of mobile facilities, there is an explicit range of ways to optimize the use of mobile technology for improving maternal wellbeing and care in the field of mobile health. This intervention proposed a potential solution to many of the challenges in developing countries. ⁹ These mHealth apps help find solutions for workforce shortages, lack of health information, limited training for health workers, and patient tracking difficulties. ¹⁰

Ineffectively made apps that are widely used may be challenging to use, misapplied, or underutilized, and will ultimately fall short of their objectives. Therefore, it is crucial that apps offer the necessary functionality and guarantee quality. This emphasizes the significance of app usability testing at every stage of development, both before and after prototyping. ¹¹ Usability testing of a mobile health app is highly challenging, as each user is unique with diverse needs and expectations from the health apps. Likewise, each mobile app is unique and has its own educational purpose. The success of these apps depends on their usefulness and feasibility. ¹²

The main generic usability attributes of testing mobile apps are learning aptitude, competency, memorability, user contentment, efficacy, simplicity, lucidity, faults, and performance in learning. ¹³ This model is used to set up important frameworks for testing and assessment. ¹⁴

Midwifery-related apps are an important part of health care apps, which play an effective role in delivering proper maternal and child health care. In spite of various ongoing efforts made in developing countries to improve maternal and child health, maternal death rates are much higher in these countries. The risk of maternal death ranges from 1:60 in developing countries to about 1:3700 in developed countries. ¹⁵ In a survey done in 2022, the majority of the samples stated that many health care apps had a constructive impact on medical practice and delivery of patient care. ¹⁶

The main advantage of using mobile technology is its cost-effectiveness and availability for the delivery of proper health care. ⁷ However, there are only a few midwifery apps in Indian app stores with dynamic features. Many nurse-midwives stated that they really desired to utilize some useful midwifery-specialty apps. Thus, it is vital to propose and develop more useful midwifery apps in India. Moreover, there is almost no midwifery app focusing on the management of obstetric complications in India. ¹¹

Researchers introduced the “E-Midwife” mobile app as a digital health teaching tool with a focus on South Asian culture to improve knowledge in the management of obstetric complications. It uses an e-learning platform with instructional videos, management techniques, medications, and other resources to help nurse-midwives. Users of Android devices can get the app for free from the Google Play Store. This main goal of mobile app is to enable the users to conveniently access pertinent information about obstetric problems from any place at any time. The aim of this study was to test the usability and quality of the “E-Midwife” mobile app for nurse-midwives after its development, which provides information about obstetric problems.

MATERIALS AND METHODS

This study is a part of a randomized controlled trial which focused on the usability of interventional digital material as an mHealth app. This trial was registered in Clinical Trial Registry of India. (Ref. No.: CTRI/2022/03/040857). The study was carried out in Puducherry, India, from June 2022 to December 2022. The study participants were nurse-midwives recruited using random allocation of samples. Nurse-midwives who were using android phones, were willing to participate in the study, were working in hospital settings, and were available during the period of data collection were included in the study. Nursing students and those who were not willing to continue their participation during the study were excluded.

This study followed a four-step approach including content development phase, app design phase, app development phase, and usability testing phase.

1. Content Development Phase

Before preparing the content of the app, an integrative literature study was conducted. The following steps were completed: theme identification; research question selection; inclusion and exclusion criteria definition; study evaluation; and knowledge synthesis.

The integrated management of pregnancy, labor, and childbirth module served as the foundation for developing the contents. The primary signs and symptoms, diagnosis, and treatment of obstetric problems constitute the instructional content. To help nurse-midwives grow more knowledgeable, was created this software. To provide current information about obstetric problems, we reviewed many publications, midwifery and nursing textbooks, and other sources. The diagnosis and management of obstetric problems were spread throughout the static and dynamic contents of the app after full substantiation of the hospital and ward standard operating procedures. Four nursing faculties in midwifery and two administrators from the nursing and midwifery fields assessed the material. It was verified by four nurse-midwives and three obstetricians working in the labor room. Although most studies indicate a minimum of six experts, it is agreed that the lowest acceptable number of experts for the validation and review procedure is two. ¹⁷

Eleven professionals examined the contents of the mobile app while taking the suggestions into account. Two assessors were employed to validate the content of the app. Participants were invited via a letter, which included a personal introduction from the researcher, an explanation of the research question, and the significance of the role played by the healthcare providers in this study. The invitation, the mobile app evaluation questionnaire, and the informed consent forms were distributed through email or in person.

2. App Design Phase

The choice of app tools, definition of the navigation structure, and configuration planning were all parts of the app development process. For the Android operating system, a specialized mobile app was created. The process of developing an app included preparation of the content, including evidence-based management criteria, need-supportive communication methods, and consultation with software developers on the design features and content of the app. Regular meetings, virtual workspaces, file sharing platforms, emails, and messages were used to maintain ongoing follow-up and contact between the peer leaders and end users. Digital prototypes were utilized throughout the design process.

3. App Development Phase

Android-compatible phones can utilize the mobile app with the following link: . The nurse-midwives will find it simple to understand the text, graphic-based images, videos, and other contents in this app. Based on the authorized subject matter, we arranged an app development team to create the app model. The purpose of the initial testing was to find potential issues, such as delayed image opening, inoperable buttons, and a minimized image screen. During the initial testing, frozen pages, frozen images, display issues, spelling errors, and syntax errors were identified and corrected.

The planning and structuring of instructional materials, definitions, the writing of the procedures involved, the choice of media, and interface design were all parts of the design of the app. Hypertexts were used to link the texts included in the topics and figures. The app requests a password and username. The password issued by the researcher should be used by the nurse-midwives to access the app. The 14 topics that make up the content of the mobile app are as follows: normal labor, partograph, eclampsia, fetal distress, preterm labor, poorly progressing labor, prolonged labor, cord prolapse, shoulder dystocia, postpartum hemorrhage, retained placenta, perineal tear, emergency cesarean section preparation, and drugs used by midwives. The major categories for the app view were core signs and symptoms, diagnosis of obstetric problems, and immediate management.

4. Usability Testing Phase

User satisfaction plays an important role in the overall success of utilizing a health care delivery system. The sample size was calculated using two population means of power analysis with an alpha error of 5%, power 80%¸ and effect size 0.5033. The following formula was used to determine the sample size:

$N=\frac{(S_1^2+S_2^2){(Z_{1-\mathrm{\alpha /2}}+Z_{\mathrm{1-\beta }})}^2}{({\mu }_1-{\mu }_2)}$

Sample size was calculated using two population mean of power analysis. Standard deviation in the 1^st group (S₁) was 5.061 whereas standard deviation in the 2^nd group (S₂) was 4.714. Mean difference between the 1^st and 2^nd samples was 2.46, whereas $Z_{1-\mathrm{\alpha /2}}$ was 1.96 and $Z_{\mathrm{1-\beta }}$ as 0.842.

The sample size was determined 62, with 10% drop out rate calculated as 68. Hence, the sample size rounded as 70 in each group. Among the 196 nurse midwives included in the study, 140 subjects were randomly allocated to the intervention and control groups. The participants in the intervention group utilized and tested the app, and the control group did not use the app (Figure 1).

Figure 1. CONSORT flowchart of the study participants

The app link was given to the users, and they were asked to use it for 8 weeks continuously. After using the mobile app for 8 weeks by the intervention group, they were asked to complete the usability testing scale and quality rating scale. The participants’ opinions of the mobile system were gathered. The usability of the system was evaluated using the System Usability Scale (SUS), and its quality was evaluated using the Mobile Application Rating Scale (MARS). There are 10 questions on the SUS questionnaire. A 5-point Likert scale is used to score each question with a range of 1=strongly disagree and 5=strongly agree, respectively. To find the SUS overall value, we multiply the total of the scores by 2.5. The SUS ratings include a range between 0 and 100. Engagement, functionality, aesthetics, information, and subjective quality are the five main quality areas of the SUS questionnaire for testing the usability of the app.

The MARS is used to rank the mHealth apps. All categories include 20 individual questions. Each question has 5 possible answers, with 5 indicating “excellent” and 1 indicating “inadequate.” Thus, the MARS score is a number between 1 and 5. An expert team examined each questionnaire for content validity and reliability.

Written consent to participate in this study was obtained from each participant, and personal information about the participants was kept confidential at all times. Institutional Research Committee approval (IRC202104) and Institutional Ethical Committee approval (322/IEC/32/PP-2/2021) were obtained from the study setting (IGMC & RI, Puducherry, India).

SPSS statistical software, version 17, was used to analyze the data. The mean and standard deviation are used to provide quantitative Likert scale data. After the mobile system was used, the participants’ views towards using the app for managing obstetric complications were assessed.

RESULTS

Among 196 nurse-midwives, 70 were randomly allocated to the usability testing of the app. The SUS scale consists of 10 questions. The median response to each question is 2, which denotes neither agreement nor disagreement (it’s a neutral response). In our investigation, the average score of each question was greater than 2 (Table 1). The overall value of SUS for this mobile system was 82.75 0.68 (it was calculated by multiplying the sum of each score by 2.5).

There were five key quality categories, including engagement, functionality, aesthetics, information, and app subjective quality. Table 2 shows the mean scores of the app quality items which were then averaged to get the final app quality mean score (3.77±0.86).

DISCUSSION

The demand for health services can no longer be satisfied by the conventional healthcare model. Healthcare professionals and computer scientists are increasingly interested in using Internet technology to offer patients and health care professionals more effective and advanced healthcare services. We may uncover a lot of articles concerning midwifery healthcare applications by searching PubMed. ¹⁸ According to the data, there were close to 4,929,926,187 internet users worldwide on September 30, 2020. In Asia, there were 2,555,636,255 Internet users, which constitute 51.8%, and the data keeps growing over time. ¹⁹

The veracity of the information in many healthcare applications has come under increased scrutiny over the last few years. In certain investigations, the expert involvement rate of the applications was as low as 9%, while their adherence to medical evidence was low. Currently, the general population uses Android more frequently than any other mobile operating system. ²⁰ As a result, we created the program to be downloaded and installed on Android-powered smartphones. The aim of this study was to create and validate the “E-Midwife” mobile application for nurse-midwives, which provides information about obstetric problems. Technological developments enhance obstetric care, emphasizing its application to the care of the mother and fetus. ²¹ In the healthcare setting where they work, nurse-midwives are in charge of creating and using clinical guidelines, protocols, applications, and algorithms related to obstetric problems. Based on the data, this should be accomplished through connecting clinical practice with knowledge, skill, technology, and innovation. ²² The overall result of E-Midwife’s usability testing indicated that it had good usability. The mobile app was viewed as user-friendly and professional overall by the participants. The use of bullet points rather than paragraphs, raising the size of images, and increasing the font size of a few topics were some suggestions offered to enhance certain topics by the participants. The feasibility and expense of implementing the improvements were taken into consideration when making decisions about the mobile app. All workable recommendations were carried out.

This data aids in the development of a mobile app that supports clinical judgment by nurse-midwives during health assessment, early diagnosis, accurate detection of problems, and fetal health promotion. It is beneficial to deliver the baby safely and without causing injury to the mother or the fetus. An in-depth review of the literature was completed before the creation of the “E-Midwife” application. ²³ The articles that were evaluated for quality of evidence were reviewed. This was done in order to provide nurse-midwives with a mobile application that would aid in the early detection of obstetric complications as well as provide the appropriate management to deliver high-quality care. Other research in mHealth and electronic health that used the SUS technique found that while participants believed they had achieved the main purpose of utilizing the applications, the intended goal was not met. ²⁴

Suboptimal usability is a significant barrier to technology adoption, and poor usability is a primary reason for discontinuing the use of mHealth technology. As a result, usability factors must be considered both before and after prototyping to support the quality of the technology and the end-user experience. ²⁵ In our study, nurse-midwives suggested that easy-to- use apps should be developed as complex procedures, and those which escalate the workload of health workers in resource-constrained contexts were more likely to fail. ²⁶ caries in pre-school children remain a significant oral health issue. In an outbreak period such as the Coronavirus disease 2019 (COVID-19 In our research, the E-Midwife mobile app was developed from a midwife’s point of view. This might not only please the users, but also better meet the requirements of the nurse-midwives, and it might also increase professional involvement and nursing practice. Despite their relevance, the majority of mHealth apps are given to the public without adequate scientific effort committed to their design, development, and evaluation. Meanwhile, no clear methodological strategy has been developed to employ a theoretical framework in app usability research. ²⁷ To address this obstacle, we used a variety of usability evaluation approaches at various stages of the development process.

The important strength of this study is its new approach to implementing an educational method that helps nurse-midwives manage labor without or with minimal complications. The limitations of this study include the lack of follow-up after the implementation of the mobile app.

CONCLUSION

The E-midwife mobile application has a high usability and quality which makes its usage effective, and efficient. Nurse-midwives, caretakers, and students may utilize mobile applications as an efficient teaching and learning strategy. Thus, it is recommended that these mobile apps should be used along with other educational methods for educating nurse-midwives. Also, future research with more samples and follow-up is suggested.

ACKNOWLEDGEMENT

We would like to thank the reviewers and nurse-midwives who gave their time and expertise to this study. We extend our thanks to the APJ Abdul Kalam Centre for Excellence and Entrepreneurship, Chennai, and the Department of Science and Technology, New Delhi, for funding this project (Grant number: NewGen4/SR/2022/01/04 dated 01.04.2022).

Conflict of Interest:

Not declared.

References

1. Bangal VB, Borawake SK, Gavhane SP, Aher KH. Use of mobile phone for improvement in maternal health : a randomized control trial. International Journal of Reproduction, Contraception, Obstetrics and Gynecology. 2017; 6:5458-63.
2. Mathur P, Chande SV. Usability Testing Methods for Mobile Learning Applications. International Journal of Computer Science and Mobile Applications. 2017; 5:1-9.
3. Laing SS, Alsayid M, Ocampo C, Baugh S. Mobile Health Technology Knowledge and Practices Among Patients of Safety-Net Health Systems in Washington State and Washington, DC. Journal of Patient-centered Research and Reviews. 2018; 5:204-17.
4. Sun Sh. Smartphone penetration rate in India from 2009 to 2023, with estimates until 2040. Germany: Statista; 2023. [Cited 20 Sep 2023]. Available from: https://www.statista.com/statistics/1229799/india-smartphone-penetration-rate/.
5. Nørgaard SK, Nichum VL, Barfred C, et al. Use of the smartphone application “Pregnant with Diabetes”. Danish Medical Journal. 2017; 64:A5417.
6. Braüner Klokkenga CM, Enemark U, Adanu R, et al. The effect of smartphone training of Ghanaian midwives by the Safe Delivery application on the incidence of postpartum hemorrhage : A cluster randomised controlled trial. Cogent Medicine. 2019; 6:1632016.
7. Shorey S, Lau YY, Dennis CL, et al. A randomized controlled trial to examine the effectiveness of the ‘Home-but not Alone’ mobile health application educational programme on parental outcomes. Journal of Advanced Nursing. 2017; 73:2103-17.
8. Skar JB, Garnweidner-Holme LM, Lukasse M, Terragni L. Women’s experiences with using a smartphone app (the Pregnant+ app) to manage gestational diabetes mellitus in a randomised controlled trial. Midwifery. 2018; 58:102-8.
9. Zhang C, Ran L, Chai Z, et al. The design, development and usability testing of a smartphone-based mobile system for management of children’s oral health. Health Informatics Journal. 2022; 28:14604582221113432.
10. Costa Souza FML, Dos Santos WN, da Costa Santos RS, et al. Effectiveness of mobile applications in pregnant women ’ s adherence to prenatal consultations : randomized clinical trial. Revista Brasileira de Enfermagem. 2021; 74:e20190599.
11. Mramba BP, Kaijage SF. Design of an interacive mobile application for maternal, neonatal and Infant care support for Tanzania. Journal of Software Engineering and Applications. 2018; 11:569-84.
12. Matthews S, Atkins B, Walton N, et al. Development and Use of a Cardiac Clinical Guideline Mobile App in Australia : Acceptability and Multi-Methods Study. JMIR Formative Research. 2022; 6:e35599.
13. Alturki R, Gay V. Recent Trends and Advances in Wireless and IoT-enabled Networks. Springer, Cham: Switzerland; 2019.
14. Borgen I, Småstuen MC, Jacobsen AF, et al. Effect of the Pregnant+ smartphone application in women with gestational diabetes mellitus: A randomised controlled trial in Norway. BMJ Open. 2019; 9:e030884.
15. World Health Organization. Trends in maternal mortality 2000 to 2017: Estimates by WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division: executive summary. World Health Organization: Geneva; 2019.
16. Tuyishime E, Park PH, Rouleau D, et al. Implementing the World Health Organization safe childbirth checklist in a district Hospital in Rwanda: a pre- and post-intervention study. Maternal Health, Neonatology and Perinatology. 2018; 4:7.
17. Yusoff MSB. ABC of Content Validation and Content Validity Index Calculation. Education in Medicine Journal. 2019; 11:49-54.
18. Jackson J, Leziak K, Niznik CM, Yee LM. Health Care Providers’ Utilization of and Perspectives on Mobile Health Technology for Diabetes and Pregnancy Support. Diabetes Spectrum. 2021; 34:257-67.
19. Internet World Stats. World Internet Users and Population Statistics. US: Miniwatts Marketing Group; 2023. [Cited 21 Sep 2023]. Available from: https://www.internetworldstats.com/stats.htm.
20. Seyyedi N, Rahmatnezhad L, Mesgarzadeh M, et al. Effectiveness of a smartphone-based educational intervention to improve breastfeeding. International Breastfeeding Journal. 2021; 16:70.
21. Benski AC, Stancanelli G, Scaringella S, et al. Usability and feasibility of a mobile health system to provide comprehensive antenatal care in low-income countries : PANDA mHealth pilot study in Madagascar. Journal of Telemedicine and Telecare. 2017; 23:536-43.
22. Gilbert S, Mehl A, Baluch A, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020; 10:e040269.
23. Behboudi F, Pouralizadeh M, Yeganeh MR, Khoshrang H. Designing and evaluating an educational mobile application for prevention of foreign body aspiration and measure to open airways of children. International Journal of Africa Nursing Sciences. 2021; 15:100313.
24. Lacroze E, Frühauf A, Nordmann K, et al. Usability and acceptance of a mobile health wallet for pregnancy-related healthcare: A mixed methods study on stakeholders’ perceptions in central Madagascar. PLoS One. 2023; 18:e0279880.
25. Vuong AM, Huber JC, Bolin JN, et al. Factors affecting acceptability and usability of technological approaches to diabetes self-management: A case study. Diabetes Technology & Therapeutics. 2012; 14:1178-82.
26. Muhamat NA, Hasan R, Saddki N, et al. Development and usability testing of mobile application on diet and oral health. PLoS One. 2021; 16:e0257035.
27. Cho H, Yen PY, Dowding D, et al. A multi-level usability evaluation of mobile health applications: A case study. Journal of Biomedical Informatics. 2018; 86:79-89.