Clinical \gls{deterioration} is a critical concern in healthcare, particularly for vulnerable populations such as the elderly and chronically
ill patients. It refers to a decline in a patient's health status and may lead to adverse outcomes, including hospitalization,
longer stays in intensive care units, and increased healthcare costs.
The \Gls{ews} has been widely adopted internationally for preemptive detection of deteriorating patients\cite{downey_strengths_2017}.
A large body of scientific evidence validates the effectiveness of \Glspl{ews} in assessing severity of illness, and in predicting adverse clinical events,
such as severe deterioration, likelihood of \gls{icu} admission, and mortality, both in hospital wards\cite{subbe_validation_2001, buist_association_2004, paterson_prediction_2006, gardner-thorpe_value_2006, alam_exploring_2015, bilben_national_2016, brekke_value_2019}
and in ambulatory care \cite{ehara_effectiveness_2019, burgos-esteban_effectiveness_2022, paganelli_conceptual_2022}.
Two commonly used clinical scores are the \Gls{news2} and the
Traditionally, doctors and nursing staff perform collection and evaluation of the data manually, often inputting data into an EWS-calculator by hand.
However, as Eisenkraft et al. mentioned in 2023, \enquote{some known setbacks of the NEWS and other scales are the frequency of scoring and
response, integration into practice, and miscalculation by healthcare providers\ldots~}\cite{eisenkraft_developing_2023}.
\Gls{rpm} can improve deterioration detection\cite{shaik_remote_2023} by greatly reducing the
amount of human interaction required to take measurements and perform \Gls{ews} calculations.
A number of studies have explored \Gls{rpm} combined with automated \Gls{ews} calculation in hospitals\cite{eisenkraft_developing_2023, filho_iot-based_2021, un_observational_2021, karvounis_hospital_2021}.
With hospitals facing overwhelming patient load during the SARS-CoV-2 pandemic, interest in exploring \Gls{rpm} surged to unprecedented heights,
and \Gls{news2} emerged as an effective tool for predicting severe infection outcomes\cite{filho_iot-based_2021, gidari_predictive_2020, otoom_iot-based_2020, carr_evaluation_2021}
while reducing person-to-person contact during patient monitoring.
In order to examine the current state of scientific knowledge about the use of wearable devices for automated \Gls{ews} monitoring of
patients at home, a comprehensive review of the existing literature was conducted.
By systematically examining and synthesizing the current body of knowledge, this review identified a variety of approaches for
utilizing smart medical devices in post-discharge patient care, as well as existing limitations and challenges in future research
in this rapidly evolving field.
\subsubsection{Search strategy}
A systematic search strategy was implemented on the Scopus database, aimed to encompass a broad spectrum of literature relevant
to the use of smart medical devices for automated early warning score monitoring of patients dismissed from ambulant or hospital care.
The search focused on topics related to the research area, encompassing the examination of \Gls{ews}, hospital admission, care escalation,
and medical emergencies in combination with IT automation, medical wearables and \Gls{iot}.
The Scopus database was chosen for its extensive coverage of scholarly literature across multiple disciplines.
For the search strategy, the following inclusion and exclusion criteria were employed to select relevant articles:
Inclusion criteria:
\begin{itemize}
\item Articles focusing on the utilization of medical wearable devices for remote patient monitoring
\item Articles addressing the automated calculation of early warning scores
\item Articles discussing the application of early warning scores outside of medical care facilities
\end{itemize}
Exclusion criteria:
\begin{itemize}
\item Non-English language articles
\item Publications for which full-text access was not available
\item Duplicate articles
\item Articles outside of the \enquote{Computer Science} subject area
\end{itemize}
The following Scopus query was used to identify relevant literature:
\begin{tcolorbox}[enhanced, center, width=0.95\linewidth, rounded corners=all, colframe=black!75!white, boxrule=0.5pt, colback=black!5!white]
\begin{lstlisting}[language=SQL]
TITLE-ABS-KEY(("patient" OR "clinical" OR "medical") AND ("deterioration" OR "instability" OR "decompensation" OR "admission" OR "hospitalization" OR "escalation" OR "triage" OR "emergency")) OR ("early warning" OR "early warning score" OR "warning" OR "score*" OR "EWS") AND TITLE-ABS-KEY("system" OR "automat*" OR "smart*" OR "wearable*" OR "internet of thing*" OR "iot" OR "digital" OR "sensor*" OR "signal" OR "intelligen*" OR "predict*" OR "monitor*" OR "sreen*" OR "remote" OR "it" OR "comput*" OR "mobile" OR "5G" OR "network" (("vital*" OR "bio*") AND ("marker*" OR "sign*" OR "monitor*"))) AND TITLE-ABS-KEY("home" OR "domestic" OR "community" OR "remote" OR "longterm" OR "nursing" OR "rehabilitation" OR "outof*hospital" OR "telemedicine" OR "ehealth" OR "mhealth")
Internet of things enabled in-home health monitoring system using early warning score\cite{anzanpour_internet_2015}&
Anzanpour 2015 \\
\hline
2 &
Context-Aware Early Warning System for In-Home Healthcare Using Internet-of-Things\cite{anzanpour_context-aware_2016}&
Anzanpour 2016 \\
\hline
3 &
An IoT based system for remote patient monitoring\cite{archip_iot_2016}&
Archip 2016 \\
\hline
4 &
Wireless sensor network-based smart room system for healthcare monitoring\cite{arnil_wireless_2011}&
Arnil 2011 \\
\hline
5 &
Design and Development of IOT Based Multi-Parameter Patient Monitoring System\cite{athira_design_2020}&
Athira 2020 \\
\hline
6 &
Medical warning system based on Internet of Things using fog computing\cite{azimi_medical_2016}&
Azimi 2016 \\
\hline
7 &
Self-aware early warning score system for IoT-based personalized healthcare\cite{azimi_self-aware_2017}&
Azimi 2017 \\
\hline
8 &
Review on IoT based Healthcare systems\cite{b_v_review_2022}&
Krishna 2022 \\
\hline
9 &
Effectiveness of Early Warning Scores for Early Severity Assessment in Outpatient Emergency Care: A Systematic Review\cite{burgos-esteban_effectiveness_2022}&
Burgos-Esteban 2022 \\
\hline
10 &
A QRS Detection and R Point Recognition Method for Wearable Single-Lead ECG Devices\cite{chen_qrs_2017}&
Chen 2017 \\
\hline
11 &
Adopting the Internet of Things technologies in health care systems\cite{chiuchisan_adopting_2014}&
Chiuchisan 2014 \\
\hline
12 &
An Efficient Wireless Health Monitoring System\cite{chowdary_efficient_2018}&
Chowdary 2018 \\
\hline
13 &
DeepSigns: A predictive model based on Deep Learning for the early detection of patient health deterioration\cite{da_silva_deepsigns_2021}&
da Silva 2021 \\
\hline
14 &
Use of ultra-low cost fitness trackers as clinical monitors in low resource emergency departments\cite{dagan_use_2020}&
Dagan 2020 \\
\hline
15 &
A data fusion algorithm for clinically relevant anomaly detection in remote health monitoring\cite{de_mello_dantas_data_2020}&
de Mello Dantas 2020 \\
\hline
16 &
Patient attitudes towards remote continuous vital signs monitoring on general surgery wards: An interview study\cite{downey_strengths_2017}&
Downey 2018 \\
\hline
17 &
Developing a real-time detection tool and an early warning score using a continuous wearable multi-parameter monitor\cite{eisenkraft_developing_2023}&
Eisenkraft 2023 \\
\hline
18 &
An IoT-Based Healthcare Platform for Patients in ICU Beds During the COVID-19 Outbreak\cite{filho_iot-based_2021}&
Filho 2021 \\
\end{tcolorbox}
\caption{\label{tab:inclusion-table-1}List of reviewed articles \textit{(Part 1 of 3)}}
Patient Monitoring System Based on Internet of Things\cite{gomez_patient_2016}&
Gomez 2016 \\
\hline
20 &
Continuous monitoring is superior to manual measurements in detecting vital sign deviations in patients with COVID-19\cite{gronbaek_continuous_2023}&
Gronbaek 2023 \\
\hline
21 &
Secure and lightweight privacy preserving Internet of things integration for remote patient monitoring\cite{imtyaz_ahmed_secure_2022}&
Imtyaz 2022 \\
\hline
22 &
Remote Continuous Health Monitoring System for Patients\cite{jagadish_remote_2018}&
Jagadish 2018 \\
\hline
23 &
Cost utility analysis of continuous and intermittent versus intermittent vital signs monitoring in patients admitted to surgical wards\cite{javanbakht_cost_2020}&
Javanbakht 2020 \\
\hline
24 &
Wearable sensors to improve detection of patient deterioration\cite{joshi_wearable_2019}&
Remote patient monitoring using artificial intelligence: Current state, applications, and challenges\cite{shaik_remote_2023}&
Shaik 2023 \\
\hline
38 &
Prototype development of continuous remote monitoring of ICU patients at home\cite{thippeswamy_prototype_2021}&
Thippeswamy 2021 \\
\hline
39 &
IoT based Smart Healthcare Monitoring Systems: A Review\cite{tiwari_iot_2021}&
Tiwari 2021 \\
\hline
40 &
Observational study on wearable biosensors and machine learning-based remote monitoring of COVID-19 patients\cite{un_observational_2021}&
Un 2021 \\
\hline
41 &
Adaptive threshold-based alarm strategies for continuous vital signs monitoring\cite{van_rossum_adaptive_2022}&
van Rossum 2022 \\
\hline
42 &
A retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach\cite{wu_predicting_2021}&
Wu 2021 \\
\hline
43 &
IoT based Real Time Health Monitoring\cite{yeri_iot_2020}&
Yeri 2020 \\
\hline
44 &
Vital Signs Prediction and Early Warning Score Calculation Based on Continuous Monitoring of Hospitalised Patients Using Wearable Technology\cite{youssef_ali_amer_vital_2020}&
Youssef Ali Amer 2020 \\
\hline
45 &
Features of electronic Early Warning systems which impact clinical decision making\cite{zarabzadeh_features_2012}&
Zarabzadeh 2012 \\
\end{tcolorbox}
\caption{\label{tab:inclusion-table-3}List of reviewed articles \textit{(Part 3 of 3)}}
\end{table}
% TODO for all outcomes, present and compare the findings of each study
however in most of them, no automated EWS calculations were made\cite{archip_iot_2016, azimi_medical_2016, chowdary_efficient_2018, yeri_iot_2020, lee_all-day_2020, athira_design_2020, phaltankar_curaband_2021, thippeswamy_prototype_2021}.
In 2015, Anzanpour et al. developed a monitoring system which collects vitals data and calculates an \Gls{ews}, however due to limited or nonexistent
availability of wireless sensors for all relevant vital signs, the work was limited to using a laboratory prototype
and required manual interaction in transferring vitals data\cite{anzanpour_internet_2015}.
Sahu et al. documented their development of an \Gls{ews}-supported digital early warning system using the PM6750\cite{sahu_internet--things-enabled_2022},
an experimental vitals data monitoring device capable of taking continuous measurements in a laboratory setting\cite{noauthor_pm6750_nodate}.
Recent studies indicate a growing trend towards investigating automated \Gls{ews} calculations in real-world scenarios\cite{downey_strengths_2017, karvounis_hospital_2021, b_v_review_2022, dagan_use_2020}.
Notably, the availability of comprehensive, mobile vital signs monitoring equipment has seen a significant increase, especially in the wake of the SARS-CoV-2
% There is a lack of software calculating MEWS with RPM
The rapid advancements in wearable, networked biosensors have expanded the horizons of \Gls{rpm}.
Still, their integration with \Glspl{ews} remains under-explored, especially for outpatients or those outside traditional medical care settings.
While \Glspl{ews} have proven effective in hospitals and ambulant care facilities, the practicality of implementing them remotely, under real-life conditions,
leveraging state-of-the-art smart medical devices, remains uncertain.
Existing research on \Gls{rpm} predominantly focuses on the technology's capability for vitals monitoring, often sidelining the integration and automated calculation of \Glspl{ews}.
Consequently, there is a knowledge gap concerning the system's effectiveness, feasibility, and challenges when deployed in everyday environments,
particularly in patients' homes or during their daily routines.
\subsection{Research goals}
The objective of this research was to explore \gls{rwm}: remote monitoring of patients dismissed from direct medical care using an \Gls{ews}.
The research questions stated above were pursued by designing, implementing and deploying an \Gls{rwm} system, followed by a usability study and subsequent evaluation
\item\textbf{EWS selection:} identification of an \Gls{ews} which is:
\begin{itemize}
\item widely recognized
\item clinically validated
\item applicable for adult patient evaluation
\item capable of assessing the overall risk of patient deterioration
\item not limited to specific patient populations
\end{itemize}
\item\textbf{Device selection:} procurement of smart medical devices that:
\begin{itemize}
\item are commercially available and have clinical approval for medical use
\item offer user-friendly, non-intrusive measurement procedures, suitable for the patient to use at home and on the go
\item allow secure and automated retrieval of the vitals data needed to calculate the chosen \Gls{ews} via an \Gls{api}
\end{itemize}
\item\textbf{System design and implementation:} development of a software application facilitating \Gls{rwm} using the selected medical devices, ensuring:
\begin{itemize}
\item regular vitals data capture from patients
\item accurate calculation of the chosen \Gls{ews} based on captured data
\end{itemize}
\item\textbf{Real-world usability trial:} conduction of a trial, wherein a test subject utilizes the implemented system in real-world conditions,
collecting data useful for system evaluation.
\item\textbf{Evaluation:} methodical review of the implemented \Gls{rwm} system:
\begin{itemize}
\item Analysis regarding the effectiveness of the system in consistently gathering \Gls{ews} samples
\item Investigation of failure points when \Gls{ews} sample retrieval was unsuccessful
\item Collation of the subject's feedback on user experience
\end{itemize}
\item\textbf{Feasibility and challenge discussion:} reflection on the entire research process to:
\begin{itemize}
\item draw conclusions on the feasibility of \Gls{rwm} using modern smart medical sensors
\item discuss application of the chosen \Gls{ews} for \Gls{rwm}
\item highlight the identified technical and operational challenges
Three widely used \Glspl{ews} for deterioration are the \Gls{pews}\cite{monaghan_detecting_2005}, \Gls{news2}\cite{noauthor_national_2017} and \Gls{mews}\cite{subbe_validation_2001}.
All three are established as being effective in predicting clinical deterioration.
\Gls{pews} was excluded due to its application being limited to pediatric patients.
Among other factors, \Gls{news2} takes into account whether a patient is currently suffering from hypercapnic respiratory failure, and whether or not the
patient is currently being ventilated using supplemental oxygen.
These parameters are generally not applicable to patients dismissed from medical care, hence \Gls{mews} was chosen as the early warning score
for the envisioned \Gls{rwm} system.
To calculate \Gls{mews}, the following vital signs must be captured and processed by the system:
Notably, the AVPU score could not be measured, because it typically necessitates a clinical assessment.
Furthermore, the inclusion of the Withings Scanwatch came with a notable limitation.
Although the device possesses the capability to measure a patient's respiration rate, this functionality is restricted to nocturnal measurements during sleep.
After consulting with several medical professionals, a decision was made to forgo the traditional respiration rate measurement, as well as the AVPU parameter in the \Gls{mews} calculation.
Instead, we introduced a custom \textit{respiration score}, shown in Table~\ref{tab:respiration-score}, which represents any shortness of breath reported by the patient.
To replace the respiration rate in the \Gls{mews} calculation, we took into account the patient's \Gls{spo2}, as well as the described respiration score.
\subsection{Requirements}
Following the selection of an \Gls{ews} and suitable medical sensors, a comprehensive \Gls{rwm} application was conceptualized,
dubbed as the \textit{Mobile Early Deterioration Warning System (Medwings)}.
Prior to its development, several key software requirements for the application were determined.
\subsubsection{Functional Requirements}
\begin{description}
\item[User Authentication:] Medwings must provide a robust user authentication and authorization system to ensure confidentiality of sensitive patient data, while preventing
unauthorized access to restricted information.
\item[Portability:] Patients must be able to access Medwings from their mobile phone. Enabling access from other types of end user devices is desirable.
\item[Data Collection:] The application must be capable of collecting and storing vital sign readings from all three Withings smart medical sensors without manual
interaction. Additionally, it must be able to prompt patients on whether they are experiencing shortness of breath, and store the response.
\item[Measurement prompts:] The application must regularly prompt patients to take vital sign readings.
\item[MEWS Calculation:] When sufficient vital signs for a patient are collected, Medwings must automatically compute and store the \Gls{mews}.
\item[Data Synchronization:] Vitals data, \Gls{mews} measurements and any associated metadata should be synchronized and accessible on
each of the patient's end devices. The captured data should be stored for later analysis.
\item[Data Visualization] Patients should be able to view a history of recorded vitals and \Gls{mews} measurements.
\end{description}
\subsubsection{Non-functional Requirements}
\begin{description}
\item[Usability:] Medwings should be intuitive and user-friendly, requiring minimal technical expertise from the end-user.
User interfaces should be adaptive for display on both mobile displays and larger monitors.
If measurements or user interactions are invalid, error messages should be displayed to the user.
\item[Availability:] The system should be available for patients to use at all times, with minimal downtime.
\item[Data Validity:] Vitals records retrieved from the smart sensor devices must be converted and displayed correctly.
Calculated \Gls{mews} values must be correct and take into account all relevant vital parameters:
\begin{itemize}
\item systolic blood pressure
\item heart rate
\item body temperature
\item\Gls{spo2}
\item respiration score
\end{itemize}
When such a set of vitals measurements is used to calculate a \Gls{mews}, the time of measurement for any two
measurements in the set cannot be further apart than ten minutes
\item[Security:] All personal and medical data should be stored securely and. Data in transit must be encrypted at all times.
The identity of clients and servers must be cryptographically verifiable.
\end{description}
\subsection{Design and Implementation}
Medwings was designed as a web-based application, accessible through a web browser.
Opting for this format offers several advantages: the primary benefit is its inherent cross-platform compatibility, enabling usage
Adhering to the classic client-server paradigm, the Medwings design prioritizes centralized data storage and processing.
This architecture was found to be benificial for ensuring efficient data synchronization, secure authentication,
and high system availability.
Django, a high-level Python web framework, was employed to develop both the frontend and backend components of the Medwings application.
One of the primary motivations for selecting Django was its out-of-the-box user authentication and session management capabilities.
Such features substantially expedited the development process, freeing up time and resources to focus on the unique, medically-oriented functionalities of Medwings.
Furthermore, Django's integrated Object-Relational Mapping (ORM) system greatly simplified the creation, management, and querying of the application's database.
This was pivotal given the essential role of the database in storing vitals data.
While web applications offer many advantages, one limitation is their inability to support push notifications directly to the patient's mobile phone.
To address this gap, a separate push notification microservice was deployed and integrated with Medwings.
Considering the time constraints, this approach proved to be an effective compromise.
Each module defines classes representing backend logic, database schemas and user interface elements pertaining to its specific function.
Implementation details are encapsulated within these classes, while public interfaces are exposed to external program code to provide each module's core functionality.
The \code{core} module forms the backbone of the application.
It encompasses configuration settings, secrets such as private encryption keys or API tokens, and functionalities shared across multiple other modules.
Medwings interfaces with the Withings Cloud through the \code{withings} module.
This includes retrieving vitals data through authenticated requests to the Withings Cloud API, which implements the OAuth 2.0 Authorization Framework.
As per its specification, \enquote{In OAuth, the client requests access to resources controlled by the resource owner and hosted by the resource server\ldots~
Instead of using the resource owner’s credentials to access protected resources, the client obtains an access token\ldots~
The client uses the access token to access the protected resources hosted by the resource server.
}\cite{hardt_oauth_2012}
While this process is largely transparent for the resource owner --- the patient in this case --- the communication between
Medwings as the resource client and Withings as the resource server is complex, and is therefore abstracted by the module.
Aside from OAuth 2.0, \code{withings} also encapsulates fetching, parsing, and storing vitals data retrieved from Withings.
Medwings implements a standalone user authentication system, which is provided by the \code{authentication} module.
Patients must register with a username and password to be able to use the application.
\item The patient grants Medwings the permission to retrieve their health data from Withings in an OAuth2 authorization flow. This process is shown in
Figures \ref{fig:ui-register-init} and \ref{fig:ui-register-oauth2}.
\item A registration form, displayed in Figure~\ref{fig:ui-register-continue}, is shown, prompting the user to choose a username and password, and to enter relevant personal information.
\item The user is shown a confirmation that the account was created successfully, and is asked to download the Gotify app, described below, and log in using their Medwings credentials.
The \code{medwings} module, pivotal to the core functionalty of Medwings, defines the data model used to represent and store the various vital signs handled by the application.
Furthermore, it provides interfaces to access the data, as well as the algorithm used to calculate the MEWS, which is listed in Algorithm~\ref{algo:mews}.
In order to send push notifications to mobile devices, Medwings leverages \textit{Gotify} -- a dedicated notification microservice\cite{noauthor_gotify_nodate}.
Gotify is composed of a web server component, and a mobile app acting as the client software.
The server exposes its own API, which allows external applications like Medwings to dispatch push notifications programmatically.
It uses an independent database for client authentication. The \code{gotify} module ensures synchronization between the user databases of Gotify and Medwings.
In addition, the module provides interfaces to send customized push notifications to specific patients.
A relational database is used to store application data, whereby each Medwings module defines the database schema for the underlying data it is responsible for handling.
Module interdependencies correlate closely with the foreign key references in the data model.
A holistic representation of the Medwings data model is shown in Figure~\ref{fig:datamodel}.
\caption{\label{fig:datamodel}Entity-Relationship diagram (Crow's Foot notation) showing the data model of the Medwings database.}
\end{center}
\end{figure}
At its heart lies the \code{User} entity: it forms the nexus to which all vitals data and user information is linked.
Withings API tokens are stored in the \code{RefreshToken} and \code{AccessToken} entities, while the \code{GotifyUser} and \code{GotifyAccount} entities retain the Gotify API credentials.
To use the smart devices to take measurements, patient users must first install the Withings mobile app on their phone, and use it to create a Withings user account.
Following registration, each device must be linked to the app and configured via Bluetooth.
Some basic configuration is required in order to enable specific device features, such as measurement of \Gls{spo2} on the Scanwatch.
Users are guided through the process by the app's \Gls{gui}.
Being a web application, no installation is necessary to access the Medwings interface, patients simply visit the website in a web browser.
Patients do need to create a Medwings account on the website however, followed by installation and configuration of the Gotify mobile app, as described in the registration
process in Section~\ref{sec:modules}.
The centralized server components, including the Gotify server, a task scheduler used to schedule sending notifications and the Medwings application code itself are deployed
on a publicly accessible web server using a Docker container environment.
Since managing a user in Medwings requires the respective user's state to be mirrored by two other services, Withings and Gotify, keeping user accounts across
all three services in sync presents a challenge.
Particularly during user creation, user accounts must be linked to Withings, created on the Gotify server and finally saved to the Medwings database.
Various integrity checks, such as when the user aborts the registration process midway, were put in place to prevent diverging user states across the three services
and overcome this challenge.
Similarly, vitals records kept in the Medwings database must be synchronized with all records available on the Withings cloud.
Regularly recurring, as well as on-demand data synchronization hooks were implemented to keep the Medwings database up to date,
while database constraints ensure validity of imported data and prevent duplication of existing records.
The non-enterprise Withings API enforces a rate limit of 120 requests per minute.
Medwings polls the API regularly to retrieve the latest health data for patients.
At scale, with many patient users, the rate limit would quickly be reached.
The Withing API does provide functionality to notify client applications upon availability of new data, making it possible to avoid polling.
Given that Medwings was only used by a single patient user during the trial phase, falling back to polling was an acceptable compromise to lower complexity
while still operating within the rate limit.
A MEWS calculation should represent the patient's overall physiological state at -- ideally -- a discrete point in time.
Naturally, there is a delay from when a measurement is taken with a device until it can be retrieved from the API.
The percieved transmission delay in the Medwings implementation was generally consistent with what is stated in the Withings public API documentation:
\enquote{Delays are typically less than two minutes, but it can be longer.}\cite{noauthor_keep_nodate}.
However, in some cases, the measurements taken on a device do not get pushed to the Withings Cloud until much later, or fail to do so at all.
While the cause for these longer than normal delays and missing data points is unknown and outside of the control of Medwings, these edge cases
had to be taken into account.
Furthermore, the time it takes a patient to take measurements using all three devices must also be accounted for.
Therefore, Medwings enforces a maximum allowed time difference of ten minutes between measurements of the different vitals records used to calculate MEWS.
If a set of vitals measurements is, across all records in the set, spaced further apart than ten minutes, no MEWS record is calculated, and the user is shown an
error message, prompting them to repeat the measurements.
\multirow{2}{*}{---}&$P_1$& Patient did not take any measurements \\
\cline{2-3}
&$P_2$& MEWS calculation timed out \\
\end{tcolorbox}
\caption{\label{tab:measurement-failures}Classification of measurement failures during the usability trial}
\end{table}
The Scanwatch and BPM Core are equipped with accellerometers\cite{noauthor_worlds_nodate, noauthor_bpm_nodate}.
If erratic movement is detected, the devices abort the measurement to avoid misinterpretation of sensor readings.
Similarly, upon failure to process captured sensor data into a plausible result, a measurement may be aborted by the device\cite{noauthor_scanwatch_nodate, noauthor_bpm_nodate-1, noauthor_guides_nodate}.
The measurement failure classes $S_1$, $B_1$ and $T_1$ were used to record these kinds of failure for the Scanwatch, BPM Core and Thermo respectively.
Following an $S_1$, $B_1$ or $T_1$ failure, the subject repeatedly carried out measurements using the affected device until a valid reading could be obtained.
Subsequent failures caused by the device aborting measurements were also recorded.
The count of \enquote{device aborted measurement}-failures of each device was compared to the total number of measurement attempts using that device.
Depending on whether the Scanwatch, BPM Core or Thermo failed to synchronize its data within the allowed time, an $S_2$, $B_2$ or $T_2$ failure was recorded respectively.
The number of measurement synchronization failures which occurred was compared to the number of successfully synchronized measurements for each device.
Following an $S_2$, $B_2$ or $T_2$ failure, the measurement process was not repeated until the next notification.
For each notification to which the subject responded, the duration between when the notification was dispatched and when the patient took the first vitals measurement was recorded.
Additionally, the average time taken to complete all three vitals measurements was noted.
If the subject did not visit the Medwings website or carry out any vitals measurements despite being prompted by a notification, a $P_1$ failure was noted.
Finally, if the patient failed to carry out all three required vitals measurements within the ten minute time limit, a $P_2$ failure was recorded.
Preceeding each MEWS measurement, metrics quantifying the quality of the connection between the subject's end device and other devices across the internet were measured.
This was done by sampling and averaging the data transmission rates, both uplink and downlink, as well as the connection round trip times from the end device to a distant reference server,
the location of which was kept constant throughout the trial.
The collected connection metrics were compared with the occurrence of measurement synchronization failures.
Out of $84$ successful individual vitals measurements across all devices throughout the trial, $18\%$ took longer than premitted by Medwings to synchronize with the Withings Cloud.
Particularly while on the go, synchronization was prone to taking too long: $25\%$ of measurements resulted in synchronization failure, compared
to $11\%$ at home.
Especially the BPM Core and Thermo devices suffered from slow synchronization times: in a total of $15$ synchronization timeouts, $n_{B_2}=7$ were caused
by the blood pressure meter, and $n_{T_2}=7$ by the thermometer.
The likelihood of each device aborting a measurement due to inconclusive sensor data was examined and is visualized in Figure~\ref{fig:measurement-repeats}.
For the BPM Core, $15\%$ of attempted measurements had to be repeated ($n_{B_1}=5$).
\caption{\label{fig:measurement-repeats}Number of measurement attempts and aborted measurements for each smart device.}
\end{center}
\end{figure}
Figure~\ref{fig:connection-boxplot} illustrates the comparative boxplots for the \gls{downlink-datarate}, \gls{uplink-datarate}, and \Gls{rtt} connection metrics when
the patient was at home versus on the go.
While there are evident differences in the distributions of these metrics between the two environments, the points representing synchronization failures do not
predominantly cluster around areas of low datarate or high \Gls{rtt}.
when the patient was at home, compared to when they were out of the house ($\overline{t_{r,\text{on the go}}}=68\text{ min}$, $M_{t_{r,\text{on the go}}}=70\text{ min}$).
The patient reported feeling fatigued by the regularity of the notifications from the second trial day onwards.
The average time it took the patient to carry out all three measurements was $4.5$ minutes, with no significant difference between the \enquote{at home} and
The system was successful in retrieving a wide range of vital signs from the patient at regular intervals, and detected abnormal readings
using the \Gls{mews}.
All \Gls{mews} calculations carried out by Medwings resulted in the correct value based on the recorded vitals data.
Both at home and on the go, the patient's vital parameters could be monitored using the system.
While at home, the test subject was able to adhere well to the measurement schedule, missing measurements only twice throughout the week.
With a three-hour window to respond to each notification, the average response time of 36 minutes and the minimal number of $P_1$ failures show that the
4.5-minute measurements were manageable within the home setting.
On the go however, a significant portion of the received notifications were not followed up by vitals measurements.
When leaving the house for extended periods of time, the subject was not always able to bring the whole array of medical devices with them.
Additionally, taking measurements could not always be done discreetly in public spaces, and finding a private area to take measurements
was not always possible.
This lead to a high rate of $P_1$ failures on the go, coupled with a comparatively long average reaction time of $\overline{t_{r,\text{on the go}}}=68\text{ minutes}$.
The rate at which device synchronization failures occured was high, with $39\%$ of successful measurements not being pushed to the Withings cloud in time
for a \Gls{mews} calculation to be valid.
A combination of three factors was determined to be the cause for this.
Firstly, while the Scanwatch is constantly connected to the patient's phone, the BPM Core and Thermo devices only establish
their bluetooth connection intermittently.
Presumably, measurement data updates from these devices are sent to the phone less frequently than for the Scanwatch.
This is strongly underlined by the fact that $93\%$ of all synchronization timeouts were caused by the BPM Core or Thermo.
The second factor becomes apparent after examining the likelihood of each device aborting its measurement due to inconclusive sensor data,
as displayed in Figure~\ref{fig:measurement-repeats}.
Although the aborted measurements did not cause synchronization failures directly, the time taken to repeat measurements impacted
the likelihood of the MEWS calculation timing out before all vitals data was synchronized.
The Scanwatch was particularly prone to prolonging the overall time it took the patient to complete all readings.
The third factor was the time it took the Withings app to push vitals data to the Cloud, and data being available for retrieval through the Withings API.
While no concrete data to quantify this duration could be gathered, it is clear that the two minute delay estimate provided by the manufacturer\cite{noauthor_keep_nodate} was exceeded
substantially in many cases.
The relationship between the connection quality and the likelihood of synchronization failures was examined, as shown in Figure~\ref{fig:connection-boxplot},
but no correlation was found, suggesting that synchronization failures were not majorly influenced by the current connection quality of the subject's end device.
Ultimately, while $61\%$ of notifications to which the patient responded did result in a successful \Gls{mews} calculation, the high rate of device synchronization timeouts
suggests that routing data from the device, to the mobile app, to the Withings cloud, and finally to the Medwings server introduces substantial delays,
thereby affecting the system's usability for its near-real-time calculation requirement.
The Medwings application successfully met its predetermined software requirements.
The system demonstrated robust user authentication mechanisms and offered portability by being accessible on mobile phones.
Additionally, it provided an intuitive interface for data visualization.
Vital signs were collected and stored automatically from all three Withings smart medical sensors, and the respiration score was able to be determined interactively.
The system was consistently available and ensured data validity, while maintaining stringent security protocols.
While certain challenges were encountered during the usability trial, the software's overarching objectives in terms of functional and non-functional
requirements were successfully achieved.
The usability study of the Medwings system provided valuable insights into the system's performance, reliability, and user interaction experience.
Classifying measurement failures into types helped to identify bottlenecks and other areas for improvement.
Overall, the system performed well in obtaining vitals readings regularly from a patient test subject.
The system successfully aggregated data from multiple sources to compute the \Gls{mews}, with all calculations being accurate.
At home, the patient was able to take all vitals measurements quickly and accommodate the measurement process into their daily routine, leading to a
high rate of interaction with the system.
Within the limited trial period, Medwings was able to detect abnormal readings effectively.
A significant portion of recorded vitals data, however, could not successfully be converted into \Gls{mews} records.
These calculation failures pertained to device synchronization delay with the Withings Cloud, highlighting a critical issue.
The BPM Core and Thermo devices synchronize only when turned on through manual interaction.
This leads to longer sync times for these devices, inhibiting Medwings from accessing new vitals data swiftly and performing a timely \Gls{mews} calculation.
A timeout period more lenient than the ten minute window imposed by Medwings may reduce the rate of synchronization failures, but it may also negatively
impact the validity of a \Gls{mews} record if implemented.
The study found that connection metrics such as data transmission rates and round-trip times did not show a significant correlation with synchronization failures.
This suggests that other factors, likely server-side processing delays on the Withings Cloud, contribute to these failures.
The study also revealed that the system is generally more reliable when the patient is at home, as indicated by the low interaction rate while on the go.
One reason was that carrying a a range of vitals sensors while out of the house is not always feasible.
Another was the subject's reluctance to take measurements in public spaces, suggesting a need for more discreet solutions for on the go vitals monitoring.
However, when measurements were made in the mobile environment, Medwings was successful at aggregating the measured data and, aside from the aforementioned
synchronization issues, was able to calculate the subject's \Gls{mews}.
%Patients appreciate the face-to-face aspect of early warning score monitoring as it allows for reassurance, social interaction, and gives them further opportunity to ask questions about their medical care\cite{downey_patient_2018}.
\subsection{Limitations}
Several limitations of the current study and the Medwings system must be noted.
Firstly, the original \Gls{mews} algorithm could not be used in its unaltered form to facilitate automated deterioration scoring.
Measuring the respiration rate and \Gls{avpu} of a mobile patient presented a challenge, which had to be overcome by disregarding
the \Gls{avpu} from the calculation and replacing the respiration rate in favor of a custom respiration score and \Gls{spo2} measurements.
Furthermore, the usability study had only a single test subject, which does not capture the diversity of potential user experiences.
Due to time constraints, the trial period was limited to lasting only one week, reducing the sample size and potentially introducing
bias in the gathered data.
Measurement prompts were dispatched every 3 hours during the day. This allows for just a restricted, intermittent view of a patient's
physiological state, and the chosen sampling period may require adequate ajdustment.
Moreover, although designed and implemented as a multi-user system, Medwings was only tested with one active user.
When used by many more users, the proof-of-concept system may encounter scalability issues, such as exceeding the API rate limits
imposed by Withings.
Lastly, using Medwings and the smart medical devices requires the patient to have internet access, which will negatively impact the
system's effectiveness in some circumstances where a stable connection is not available.
The Medwings system, while successful in its current iteration, offers multiple avenues for improvement.
Its dependence on web-based access to the Withings API to retrieve vitals readings results in significant delays and mandates an internet connection.
A native mobile application offering direct Bluetooth access to sensor measurements would enhance real-time responsiveness and negate the necessity for a stable internet connection.
Unfortunately, Withings do not offer such an option to developers.
Vendor dependence in general introduces a risk to Medwings' continued operation.
Service outages, discontinued device updates, or the vendor ceasing business operations all render the smart devices, and consequently Medwings, unusable.
Vendor-independent access and open-source firmware for the medical devices would mitigate these risks and open up exciting possibilities for further system integrations.
Additionally, expanding the range of monitored vitals could allow implementation of more comprehensive \Gls{rwm} techniques.
The integration of a smart device capable of accurately measuring the respiration rate of a mobile patient would refine the \Gls{mews} calculations,
leading to a more accurate deterioration monitoring system.
The system could also benefit from real-time alerting for emergency situations, rather than relying solely on periodic \Gls{mews} calculations.
This would not only make the system more robust but would also be crucial for immediate medical intervention.
For healthcare providers, a monitoring platform could be developed to allow medical staff to have direct visibility into their patient's vitals and recent developments.
Intermittent vitals monitoring presents an incomplete picture of the patient's health status.
A continuous monitoring system would not only improve the system's efficacy\cite{gronbaek_continuous_2023, shaik_remote_2023}, but could greatly enhance the sampling
frequency and obviate the need for manual patient interaction for taking measurements.
Given more development time, a range of auxiliary features could be integrated into Medwings.
Native mobile notifications, more detailed vitals analysis, and utilization of additional functionalities available in Withings devices
would elevate the system's overall capabilities and usefulness.
As technology advances, future work could explore machine learning models to predict potential health anomalies based on historical data.
Overall, the study has successfully identified key operational successes and challenges, which can be instrumental for future development and refinement of the system.
The Medwings system successfully demonstrated the feasibility of remote patient monitoring using consumer-grade smart devices.
While limitations such as synchronization failures and user compliance exist, the system holds promise for continuous, non-intrusive health monitoring, particularly in a home setting.
Further research and development are required to address its current limitations and expand its capabilities.
% Usability for potential medical staff, if applicable
% Not perfect, but people can get sent home earlier
