\documentclass[12pt, a4paper]{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{DejaVuSerif} % Serif font
\usepackage{ascii} % Monospace font
\usepackage{csquotes}
\usepackage[english]{babel}
\usepackage{graphicx}
\usepackage{parskip}

\usepackage[onehalfspacing]{setspace}
\usepackage{geometry}
\usepackage{titlesec}
\usepackage{fancyhdr}

\usepackage{booktabs} % thick horizontal lines
\usepackage{multirow}
\usepackage{array,tabularx}
\usepackage{colortbl}
\usepackage[bf]{caption} % custom table captions
\usepackage[table]{xcolor}
\usepackage[colorlinks]{hyperref}
\usepackage[most]{tcolorbox}
\pagestyle{plain}
\usepackage{pdfpages} % PDFs in appendix
\usepackage[toc,nopostdot,nonumberlist,style=altlist,acronym]{glossaries}

%\usepackage{algorithm}
\usepackage[vlined]{algorithm2e}
\usepackage{subcaption} % allows multiple figures to share a caption

% Page margins
\geometry{left=3cm, right=2cm, top=3cm, bottom=2cm}

% Correct footnote size
% TODO check if this is working correctly
\renewcommand{\footnotesize}{\fontsize{10pt}{12pt}\selectfont}
\usepackage[bottom]{footmisc}

% Section Headings
\titleformat*{\section}{\fontsize{16}{19}\selectfont\bfseries}
\titleformat*{\subsection}{\fontsize{14}{17}\selectfont\bfseries}
\titleformat*{\subsubsection}{\fontsize{12}{14.4}\selectfont\bfseries}

% Page Numbering
\pagestyle{fancy}
\fancyhf{}
\fancyhead[C]{\thepage} % Page number in header center
\renewcommand{\headrulewidth}{0pt}
\fancypagestyle{plain}{%
  \fancyhf{} % clear all header and footer fields
  \fancyhead[C]{\thepage} 
}
\setlength{\headheight}{14.49998pt}

% Code listing
\usepackage{listings}
\definecolor{bgtinted}{HTML}{efefef}
\definecolor{codegray}{HTML}{111111}
\definecolor{codeorange}{HTML}{91632C}
\definecolor{codegreen}{HTML}{3D5232}
\definecolor{codepurple}{HTML}{4E3A52}
\lstdefinestyle{mystyle}{
    backgroundcolor=\color{black!5!white},
    commentstyle=\color{codepurple},
    keywordstyle=\color{codegray},
    stringstyle=\color{codegreen},
    basicstyle=\asciifamily\color{codegray},
    breakatwhitespace=true,
    breaklines=true,
    captionpos=b,
    keepspaces=true,
    showspaces=false,
    showstringspaces=false,
    showtabs=false,
    tabsize=2
}
\lstset{style=mystyle}

% Inline code snippets
\newcommand{\code}[1]{\tikz[baseline=(X.base)]\node [draw=gray!50,fill=gray!25,semithick,rectangle,inner sep=2.5pt, rounded corners=3pt] (X) {\asciifamily\color{codegray}{#1}};}

% Citations
%\usepackage{cite}
\usepackage[backend=biber, style=vancouver]{biblatex}
\addbibresource{../bibliography/bibliography.bib}

% Colors
\definecolor{PLRI_Rot}{RGB}{190,30,60}
\definecolor{grau}{RGB}{120,110,100}
% A command which generates a TODO message
\newcommand{\todo}[1]{{\fontfamily{lmtt}\selectfont{\color{orange}\small\underline{TODO:}} \textbf{#1}\normalsize \\}}

% Link colors
\definecolor{linkColor}{RGB}{0,42,112}
\definecolor{citeColor}{RGB}{0,112,47}
\hypersetup{
    colorlinks=true,        % true: colored links, false: boxed links
    linkcolor=linkColor,    % color of internal links (change box color with linkbordercolor)
    citecolor=citeColor,    % color of links to bibliography
    filecolor=magenta,      % color of file links
    urlcolor=cyan,          % color of external URLs
    linktocpage=true        % make page numbers, not text, be link on TOC, LOF and LOT
}


\input{./glossary.tex}


\begin{document}


\input{cover.tex}

\pagenumbering{Roman}
\section*{Summary}
\addcontentsline{toc}{section}{Summary}

This research investigated the feasibility of Remote Warning Score Monitoring (RWSM)
for outpatients using commercially available smart medical devices.
A system, named Medwings, was designed, implemented, and evaluated.
A usability study was carried out and found that consumer-grade smart devices can be used to facilitate remote patient monitoring
integrated with Early Warning Scores (EWS).
While the system was largely successful in demonstrating the practicability of RWSM, it also identified
several operational challenges.
The Medwings system shows potential for improving patient quality of life and optimizing healthcare resources.
Despite its current limitations, Medwings opens the door for future research and development,
given the fast-evolving market for advanced smart medical devices.
The study fills a critical knowledge gap and sets the stage for further advancements in the field of remote patient
monitoring with early warning scores.

\newpage
\renewcommand*\contentsname{Table of contents}
\tableofcontents

\newpage
\setcounter{page}{1}
\pagenumbering{arabic}
\section{Introduction}

\subsection{Background}

Clinical \gls{deterioration} is a critical concern in healthcare, particularly for vulnerable populations such as the elderly and chronically
ill patients. It refers to a decline in a patient's health status and may lead to adverse outcomes, including hospitalization,
longer stays in intensive care units, and increased healthcare costs.
The \Gls{ews} has been widely adopted internationally for preemptive detection of deteriorating patients\cite{downey_strengths_2017}.
A large body of scientific evidence validates the effectiveness of \Glspl{ews} in assessing severity of illness, and in predicting adverse clinical events,
such as severe deterioration, likelihood of \gls{icu} admission and mortality, both in hospital wards\cite{subbe_validation_2001, buist_association_2004, paterson_prediction_2006, gardner-thorpe_value_2006, alam_exploring_2015, bilben_national_2016, brekke_value_2019}
and in ambulatory care \cite{ehara_effectiveness_2019, burgos-esteban_effectiveness_2022, paganelli_conceptual_2022}.

Two commonly used clinical scores are the \Gls{news2} and the
\Gls{mews}\cite{burgos-esteban_effectiveness_2022}.
Both are calculated by capturing various vital parameters from the patient at a specific point in time, followed by numerical aggregation of the
captured data according to the score being used\cite{subbe_validation_2001, noauthor_national_2017}.
For \Gls{mews}, each measured physiological parameter is assigned an individual score based on which range it lies in.
The ranges for scoring each parameter are shown in Table \ref{tab:mews-calculation}.
The individual scores are then added together to produce the final \Gls{mews}.
A MEWS value of 5 or above indicates an elevated risk of death, and likelihood of ICU admission\cite{subbe_validation_2001}.

\begin{table}[!ht]
    \centering
    \begin{tcolorbox}[
            enhanced, width=\linewidth, boxrule=2pt, arc=4pt,
            tabularx={
                >{\centering\arraybackslash\footnotesize}X
                >{\centering\arraybackslash\footnotesize\columncolor{red!15}}c
                >{\centering\arraybackslash\footnotesize\columncolor{orange!15}}c
                >{\centering\arraybackslash\footnotesize\columncolor{yellow!15}}c
                >{\centering\arraybackslash\footnotesize\columncolor{green!15}}c
                >{\centering\arraybackslash\footnotesize\columncolor{yellow!15}}c
                >{\centering\arraybackslash\footnotesize\columncolor{orange!15}}c
                >{\centering\arraybackslash\footnotesize\columncolor{red!15}}c
            }
        ]
        \tiny{Individual Score} & $\mathbf{+3}$ & $\mathbf{+2}$ & $\mathbf{+1}$ & $\mathbf{+0}$ & $\mathbf{+1}$ & $\mathbf{+2}$ & $\mathbf{+3}$ \\
        \specialrule{2pt}{0em}{0em}
        \tiny{\textbf{Systolic Blood Pressure} [mmHg]} & $<70$ & $71-80$ & $81-100$ & $101-199$ & ---  & $\geq 200$ & --- \\
        \hline
        \tiny{\textbf{Heart Rate} [bpm]} & --- & $<40$ & $41-50$ & $51-100$ & $101-110$ & $111-129$ & $\geq 130$ \\
        \hline
        \tiny{\textbf{Respiratory Rate} [bpm]} & --- & $<9$ & --- & $9-14$ & $15-20$ & $21-29$ & $\geq 30$ \\
        \hline
        \tiny{\textbf{Temperature} [°C]} & ---  & $<35$ & --- & $35-38.4$ & --- & $\geq 38.5$ & --- \\
        \hline
        \tiny{\textbf{AVPU}} & --- & --- & --- & \tiny{alert} & \tiny{reacting to voice} & \tiny{reacting to pain} & \tiny{unresponsive} \\
    \end{tcolorbox}
    \caption{\label{tab:mews-calculation}MEWS calculation ranges}
\end{table}

Traditionally, doctors and nursing staff perform collection and evaluation of the data manually, often inputting data into \Gls{ews}-calculators by hand.
However, as Eisenkraft et al. mentioned in 2023, \enquote{some known setbacks of the NEWS and other scales are the frequency of scoring and
response, integration into practice, and miscalculation by healthcare providers\ldots~}\cite{eisenkraft_developing_2023}.

\Gls{rpm} can improve deterioration detection\cite{shaik_remote_2023} by greatly reducing the
amount of human interaction required to take measurements and perform \Gls{ews} calculations.
A number of studies have explored \Gls{rpm} combined with automated \Gls{ews} calculation in hospitals\cite{eisenkraft_developing_2023, filho_iot-based_2021, un_observational_2021, karvounis_hospital_2021}.
With hospitals facing overwhelming patient load during the SARS-CoV-2 pandemic, interest in exploring \Gls{rpm} surged to new heights,
and \Gls{news2} emerged as an effective tool for predicting severe infection outcomes\cite{filho_iot-based_2021, gidari_predictive_2020, otoom_iot-based_2020, carr_evaluation_2021}
while reducing person-to-person contact during patient monitoring.

\subsection{Review of existing literature}

In order to examine the current state of scientific knowledge about the use of wearable devices for automated \Gls{ews} monitoring of
patients at home, a comprehensive review of the existing literature was conducted.
By systematically examining and synthesizing the current body of knowledge, this review identified a variety of approaches for
utilizing smart medical devices in post-discharge patient care, as well as existing limitations and challenges in future research
in this rapidly evolving field.

\subsubsection{Search strategy}

A systematic search strategy was implemented on the Scopus database, aimed to encompass a broad spectrum of literature relevant
to the use of smart medical devices for automated early warning score monitoring of outpatients.
The search focused on topics related to the research area, encompassing the examination of \Gls{ews}, hospital admission, care escalation,
and medical emergencies in combination with IT automation, medical wearables and \Gls{iot}.
The Scopus database was chosen for its extensive coverage of scholarly literature across multiple disciplines.

For the search strategy, the following inclusion and exclusion criteria were employed to select relevant articles:

Inclusion criteria:

\begin{itemize}
    \item Articles focusing on the utilization of medical wearable devices for remote patient monitoring
    \item \textit{OR} articles addressing the automated calculation of early warning scores
    \item \textit{OR} articles discussing the application of early warning scores outside of medical care facilities
\end{itemize}

Exclusion criteria:

\begin{itemize}
    \item Non-English language articles
    \item Publications for which full-text access was not available
    \item Duplicate articles
    \item Articles unrelated to the \enquote{Medicine}, \enquote{Medical Informatics} or \enquote{Computer Science} subject areas
\end{itemize}

The following Scopus query was used to identify relevant literature:

\begin{tcolorbox}[enhanced, center, width=0.95\linewidth, rounded corners=all, colframe=black!75!white, boxrule=0.5pt, colback=black!5!white]
\begin{lstlisting}[language=SQL]
TITLE-ABS-KEY(("patient" OR "clinical" OR "medical") AND ("deterioration" OR "instability" OR "decompensation" OR "admission" OR "hospitalization" OR "escalation" OR "triage" OR "emergency")) OR ("early warning" OR "early warning score" OR "warning" OR "score*" OR "EWS") AND TITLE-ABS-KEY("system" OR "automat*" OR "smart*" OR "wearable*" OR "internet of thing*" OR "iot" OR "digital" OR "sensor*" OR "signal" OR "intelligen*" OR "predict*" OR "monitor*" OR "sreen*" OR "remote" OR "it" OR "comput*" OR "mobile" OR "5G" OR "network" (("vital*" OR "bio*") AND ("marker*" OR "sign*" OR "monitor*"))) AND TITLE-ABS-KEY("home" OR "domestic" OR "community" OR "remote" OR "longterm" OR "nursing" OR "rehabilitation" OR "out*of*hospital" OR "telemedicine" OR "ehealth" OR "mhealth")
\end{lstlisting}
\end{tcolorbox}

\subsubsection{Results}

\begin{figure}[h]
    \begin{center}
        \includegraphics[width=.5\textwidth]{./figures/prisma-flowchart.png}
        \caption{\label{prisma-flowchart}PRISMA flowchart showing screening and assessment of identified literature}
    \end{center}
\end{figure}

An initial query on Scopus yielded a total of $N=1997$ records.
After removing duplicates, $N=952$ records were excluded, resulting in $N=1045$ unique records.
Upon screening the titles and abstracts, $N=963$ records did not meet the inclusion criteria, leaving $N=82$ articles to be assessed for
eligibility in full text.
Finally, after a thorough evaluation, $N=45$ articles were included for the literature review, providing insight into the current state of
research on the use of smart medical devices for automated early warning score monitoring in patients transitioning from ambulant or
hospital care.
Figure \ref{prisma-flowchart} shows the screening process. 
The complete list of reviewed literature is shown in Tables \ref{tab:inclusion-table-1}, \ref{tab:inclusion-table-2} and \ref{tab:inclusion-table-3}.

\begin{table}[!ht]
    \centering
    \begin{tcolorbox}[
            enhanced, width=\linewidth, boxrule=2pt, arc=4pt,
            tabularx={
                >{\footnotesize}r
                >{\footnotesize}X
                >{\footnotesize}l
            }
        ]
        \textbf{Number} & \textbf{Title} & \textbf{Author(s), Year} \\
        \specialrule{2pt}{0em}{0em}
        1 &
        Internet of things enabled in-home health monitoring system using early warning score\cite{anzanpour_internet_2015} &
        Anzanpour 2015 \\
        \hline
        2 &
        Context-Aware Early Warning System for In-Home Healthcare Using Internet-of-Things\cite{anzanpour_context-aware_2016} &
        Anzanpour 2016 \\
        \hline
        3 &
        An IoT based system for remote patient monitoring\cite{archip_iot_2016} &
        Archip 2016 \\
        \hline
        4 &
        Wireless sensor network-based smart room system for healthcare monitoring\cite{arnil_wireless_2011} &
        Arnil 2011 \\
        \hline
        5 &
        Design and Development of IOT Based Multi-Parameter Patient Monitoring System\cite{athira_design_2020} &
        Athira 2020 \\
        \hline
        6 &
        Medical warning system based on Internet of Things using fog computing\cite{azimi_medical_2016} &
        Azimi 2016 \\
        \hline
        7 &
        Self-aware early warning score system for IoT-based personalized healthcare\cite{azimi_self-aware_2017} &
        Azimi 2017 \\
        \hline
        8 &
        Review on IoT based Healthcare systems\cite{b_v_review_2022} &
        Krishna 2022 \\
        \hline
        9 &
        Effectiveness of Early Warning Scores for Early Severity Assessment in Outpatient Emergency Care: A Systematic Review\cite{burgos-esteban_effectiveness_2022} &
        Burgos-Esteban 2022 \\
        \hline
        10 &
        A QRS Detection and R Point Recognition Method for Wearable Single-Lead ECG Devices\cite{chen_qrs_2017} &
        Chen 2017 \\
        \hline
        11 &
        Adopting the Internet of Things technologies in health care systems\cite{chiuchisan_adopting_2014} &
        Chiuchisan 2014 \\
        \hline
        12 &
        An Efficient Wireless Health Monitoring System\cite{chowdary_efficient_2018} &
        Chowdary 2018 \\
        \hline
        13 &
        DeepSigns: A predictive model based on Deep Learning for the early detection of patient health deterioration\cite{da_silva_deepsigns_2021} &
        da Silva 2021 \\
        \hline
        14 &
        Use of ultra-low cost fitness trackers as clinical monitors in low resource emergency departments\cite{dagan_use_2020} &
        Dagan 2020 \\
        \hline
        15 &
        A data fusion algorithm for clinically relevant anomaly detection in remote health monitoring\cite{de_mello_dantas_data_2020} &
        de Mello Dantas 2020 \\
        \hline
        16 &
        Patient attitudes towards remote continuous vital signs monitoring on general surgery wards: An interview study\cite{downey_strengths_2017} &
        Downey 2018 \\
        \hline
        17 &
        Developing a real-time detection tool and an early warning score using a continuous wearable multi-parameter monitor\cite{eisenkraft_developing_2023} &
        Eisenkraft 2023 \\
        \hline
        18 &
        An IoT-Based Healthcare Platform for Patients in ICU Beds During the COVID-19 Outbreak\cite{filho_iot-based_2021} &
        Filho 2021 \\
    \end{tcolorbox}
    \caption{\label{tab:inclusion-table-1}List of reviewed articles \textit{(Part 1 of 3)}}
\end{table}

\begin{table}[!ht]
    \centering
    \begin{tcolorbox}[
            enhanced, width=\linewidth, boxrule=2pt, arc=4pt,
            tabularx={
                >{\footnotesize}r
                >{\footnotesize}X
                >{\footnotesize}l
            }
        ]
        \textbf{Number} & \textbf{Title} & \textbf{Author(s), Year} \\
        \specialrule{2pt}{0em}{0em}
        19 &
        Patient Monitoring System Based on Internet of Things\cite{gomez_patient_2016} &
        Gomez 2016 \\
        \hline
        20 &
        Continuous monitoring is superior to manual measurements in detecting vital sign deviations in patients with COVID-19\cite{gronbaek_continuous_2023} &
        Gronbaek 2023 \\
        \hline
        21 &
        Secure and lightweight privacy preserving Internet of things integration for remote patient monitoring\cite{imtyaz_ahmed_secure_2022} &
        Imtyaz 2022 \\
        \hline
        22 &
        Remote Continuous Health Monitoring System for Patients\cite{jagadish_remote_2018} &
        Jagadish 2018 \\
        \hline
        23 &
        Cost utility analysis of continuous and intermittent versus intermittent vital signs monitoring in patients admitted to surgical wards\cite{javanbakht_cost_2020} &
        Javanbakht 2020 \\
        \hline
        24 &
        Wearable sensors to improve detection of patient deterioration\cite{joshi_wearable_2019} &
        Joshi 2019 \\
        \hline
        25 &
        Intelligent Healthcare\cite{kale_intelligent_2021} &
        Kale 2021 \\
        \hline
        26 &
        A Hospital Healthcare Monitoring System Using Internet of Things Technologies\cite{karvounis_hospital_2021} &
        Karvounis 2021 \\
        \hline
        27 &
        All-day mobile healthcare monitoring system based on heterogeneous stretchable sensors for medical emergency\cite{lee_all-day_2020} &
        Lee 2020 \\
        \hline
        28 &
        Analysis of the early warning score to detect critical or high-risk patients in the prehospital setting\cite{martin-rodriguez_analysis_2019} &
        Martin-Rodriguez 2019 \\
        \hline
        29 &
        An IoT-based framework for early identification and monitoring of COVID-19 cases\cite{otoom_iot-based_2020} &
        Otoom 2020 \\
        \hline
        30 &
        A conceptual IoT-based early-warning architecture for remote monitoring of COVID-19 patients in wards and at home\cite{paganelli_conceptual_2022} &
        Paganelli 2022 \\
        \hline
        31 &
        Personalized Mobile Health for Elderly Home Care: A Systematic Review of Benefits and Challenges\cite{pahlevanynejad_personalized_2023} &
        Pahlevanynejad 2023 \\
        \hline
        32 &
        CuraBand: Health Monitoring and Warning System\cite{phaltankar_curaband_2021} &
        Phaltankar 2021 \\
        \hline
        33 &
        Internet of Things in Healthcare, A Literature Review\cite{quraishi_internet_2021} &
        Quraishi 2021 \\
        \hline
        34 &
        Vital Sign Monitoring System for Healthcare Through IoT Based Personal Service Application\cite{sahu_vital_2022} &
        Sahu 2022 \\
        \hline
        35 &
        Internet-of-Things-Enabled Early Warning Score System for Patient Monitoring\cite{sahu_internet--things-enabled_2022} &
        Sahu 2022 \\
        \hline
        36 &
        Cloud-Based Remote Patient Monitoring System with Abnormality Detection and Alert Notification\cite{sahu_cloud-based_2022} &
        Sahu 2022 \\
    \end{tcolorbox}
    \caption{\label{tab:inclusion-table-2}List of reviewed articles \textit{(Part 2 of 3)}}
\end{table}

\begin{table}[!ht]
    \centering
    \begin{tcolorbox}[
            enhanced, width=\linewidth, boxrule=2pt, arc=4pt,
            tabularx={
                >{\footnotesize}r
                >{\footnotesize}X
                >{\footnotesize}l
            }
        ]
        \textbf{Number} & \textbf{Title} & \textbf{Author(s), Year} \\
        \specialrule{2pt}{0em}{0em}
        37 &
        Remote patient monitoring using artificial intelligence: Current state, applications, and challenges\cite{shaik_remote_2023} &
        Shaik 2023 \\
        \hline
        38 &
        Prototype development of continuous remote monitoring of ICU patients at home\cite{thippeswamy_prototype_2021} &
        Thippeswamy 2021 \\
        \hline
        39 &
        IoT based Smart Healthcare Monitoring Systems: A Review\cite{tiwari_iot_2021} &
        Tiwari 2021 \\
        \hline
        40 &
        Observational study on wearable biosensors and machine learning-based remote monitoring of COVID-19 patients\cite{un_observational_2021} &
        Un 2021 \\
        \hline
        41 &
        Adaptive threshold-based alarm strategies for continuous vital signs monitoring\cite{van_rossum_adaptive_2022} &
        van Rossum 2022 \\
        \hline
        42 &
        A retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach\cite{wu_predicting_2021} &
        Wu 2021 \\
        \hline
        43 &
        IoT based Real Time Health Monitoring\cite{yeri_iot_2020} &
        Yeri 2020 \\
        \hline
        44 &
        Vital Signs Prediction and Early Warning Score Calculation Based on Continuous Monitoring of Hospitalised Patients Using Wearable Technology\cite{youssef_ali_amer_vital_2020} &
        Youssef Ali Amer 2020 \\
        \hline
        45 &
        Features of electronic Early Warning systems which impact clinical decision making\cite{zarabzadeh_features_2012} &
        Zarabzadeh 2012 \\
    \end{tcolorbox}
    \caption{\label{tab:inclusion-table-3}List of reviewed articles \textit{(Part 3 of 3)}}
\end{table}

\subsubsection{Discussion}

While the application of \Glspl{ews} in ambulant care facilities and hospitals has been thoroughly investigated,
very little research has been done to assess their practicability for remote monitoring of at-risk patients at home.
Furthermore, it was observed that previous research on the use of \Gls{iot}-devices for this purpose was largely conducted in
laboratory settings, limiting the generalizability of the results.
Some studies have examined monitoring vital signs of at-home-patients for abnormalities,
however in most of them, no automated EWS calculations were made\cite{archip_iot_2016, azimi_medical_2016, chowdary_efficient_2018, yeri_iot_2020, lee_all-day_2020, athira_design_2020, phaltankar_curaband_2021, thippeswamy_prototype_2021}.
In 2015, Anzanpour et al. developed a monitoring system which collects vitals data and calculates an \Gls{ews}, however due to limited or nonexistent
availability of wireless sensors for all relevant vital signs, the work was limited to using a laboratory prototype
and required manual interaction in transferring vitals data\cite{anzanpour_internet_2015}.
Sahu et al. documented their development of an \Gls{ews}-supported digital early warning system using the PM6750\cite{sahu_internet--things-enabled_2022},
an experimental vitals data monitoring device capable of taking continuous measurements in a laboratory setting\cite{noauthor_pm6750_nodate}.
However, the methodology they used to calculate the \Gls{ews} in real-time with laboratory data is both inconsistent and weak.

Recent studies indicate a growing trend towards investigating automated \Gls{ews} calculations in real-world scenarios\cite{downey_strengths_2017, karvounis_hospital_2021, b_v_review_2022, dagan_use_2020}.
Notably, the availability of comprehensive, mobile vital signs monitoring equipment has seen a significant increase, especially in the wake of the SARS-CoV-2
pandemic\cite{paganelli_conceptual_2022, filho_iot-based_2021, otoom_iot-based_2020, gronbaek_continuous_2023}.
This surge in accessibility has paved the way for more extensive remote monitoring of outpatients.
Moreover, there is a growing interest in incorporating machine learning algorithms to enhance the predictive capabilities of
deterioration detection\cite{un_observational_2021, da_silva_deepsigns_2021, de_mello_dantas_data_2020}.
This demonstrates the evolving landscape of remote patient monitoring, aiming to improve clinical outcomes and alleviate the
burden on hospital wards.

Despite the wealth of literature reviewed, no existing empirical studies evaluating the use of early warning scores for
patients at home were identified.
This highlights a crucial research gap and prompts the need for further investigation in this area, potentially warranting the development
of an \Gls{ews} specialized for use outside of medical care facilities.

\subsubsection{Interpretation of Results}

Based on the findings, several key implications can be drawn.
Firstly, the improved availability of smart sensors and the demonstrated effectiveness of \Glspl{ews} in predicting deterioration in direct
medical care settings warrant research into their utilization at home.
By remotely monitoring patients, it may be possible to identify early signs of deterioration, allowing for earlier dismissal from
hospital care and thereby freeing up valuable resources.
Additionally, this approach holds the potential to reduce the frequency of adverse clinical outcomes, as well as mortality rates.

However, it is important to acknowledge the lack of research on the use of \Glspl{ews} at home, which calls for a feasibility study in this
specific context.
Such a study would need to address challenges such as the frequency of measurements required and the absence of immediate diagnosis
from qualified medical staff.
Overcoming these obstacles is essential to ensure the safety and efficacy of automated remote patient monitoring in home-based settings.

In conclusion, the literature review highlights the increasing interest in using smart medical devices and EWS for remote patient
monitoring, particularly in real-world scenarios.
The absence of studies evaluating the application of \Glspl{ews} for patients at home underscores the need for further investigation in this area.
Conducting a feasibility study to explore the practicality and challenges of implementing \Glspl{ews} in home-based care would contribute
significantly to the existing body of knowledge and help advance the field of automated early warning score monitoring in
non-medical care settings.

\subsection{Motivation}

Installing and operating traditional continuous monitoring systems, like the vital sign monitors used in medical facilities, demands
specialized equipment and technical expertise.
Furthermore, these systems are cumbersome for patients, as they involve connecting patient and sensor device with numerous electrodes
and cables, restricting patient mobility to the bed area, and physically tying the monitoring equipment
to a single location.
Conversely, battery-powered, wireless vitals monitoring devices, such as wearable armbands or smartwatches, can combine several
biometric sensors into one device, allowing for a much higher degree of patient mobility, faster deployment and better
scalability\cite{un_observational_2021}.
Therefore, utilizing such devices for \Gls{rpm} is a suitable approach.

With the current availability of wearable, networked biosensors and the validated effectiveness of \Glspl{ews} in medical facilities,
combining both aspects presents an important and interesting research opportunity which could help reduce mortality and improve clinical
outcomes for patients at risk of deterioration, both in their homes and on the go.

\subsection{State of the problem}

The rapid advancements in wearable, networked biosensors have expanded the horizons of \Gls{rpm}.
Still, their integration with \Glspl{ews} remains under-explored, especially for outpatients or those outside of traditional medical care settings.
While \Glspl{ews} have proven effective in hospitals and ambulant care facilities, the practicality of implementing them remotely, under real-life conditions,
leveraging state-of-the-art smart medical devices, remains uncertain.

Existing research on \Gls{rpm} predominantly focuses on the technology's capability for vitals monitoring, often sidelining the integration and automated calculation of \Glspl{ews}.
Consequently, there is a knowledge gap concerning the system's effectiveness, feasibility, and challenges when deployed in everyday environments,
particularly in patients' homes or during their daily routines.

There is a lack of available software which integrates \Gls{rpm} with \Glspl{ews} while being feasible for everyday use,
as well as a general knowledge gap in exploring the usefulness of this approach.

Existing research on \Gls{rpm} predominantly focuses on the technology's capability for vitals monitoring and often sidelines
the integration and automated calculation of \Glspl{ews}.
This results in a knowledge gap concerning the effectiveness, feasibility, and design challenges of a system which combines both concepts.
Moreover, there is a lack of available software which integrates \Gls{rpm} with \Glspl{ews} aimed at everyday use by patients.

\subsection{Research goals}

The objective of this research is to explore \gls{rwsm}: remote monitoring of patients dismissed from direct medical care using automated \Gls{ews}
assessments.

Specifically, the following questions are asked:

\begin{itemize}
    \item Can an \Gls{rwsm} system, feasible for everyday use, be implented using smart medical devices commercially available today?
    \item What are the technical and operational challenges of implementing such a system?
    \item Can existing, validated \Glspl{ews} be utilized in \Gls{rwsm}?
\end{itemize}

\subsection{Tasks}

The research questions stated above were pursued by designing, implementing and deploying an \Gls{rwsm} system
which utilizes a clinically validated \Gls{ews}, followed by a feasibility study examining its everyday use,
and a subsequent evaluation of the study.

A detailed outline of each step taken to carry out this investigation is shown here:

\begin{enumerate}
    \item \textbf{EWS selection:} identification of an \Gls{ews} which is:
        \begin{itemize}
            \item widely recognized
            \item clinically validated
            \item applicable for adult patient evaluation
            \item capable of assessing the overall risk of patient deterioration
            \item not limited to specific patient populations
        \end{itemize}
    \item \textbf{Device selection:} procurement of smart medical devices that:
        \begin{itemize}
            \item are commercially available and have clinical approval for medical use
            \item offer user-friendly, non-intrusive measurement procedures, suitable for the patient to use at home and on the go
            \item allow secure and automated retrieval of the vitals data needed to calculate the chosen \Gls{ews} via an \Gls{api}
        \end{itemize}
    \item \textbf{System design and implementation:} development of a software application facilitating \Gls{rwsm} using the selected medical devices, ensuring:
        \begin{itemize}
            \item regular vitals data capture from patients
            \item accurate calculation of the chosen \Gls{ews} based on captured data
        \end{itemize}
    \item \textbf{Real-world usability trial:} conduction of a trial, wherein a test subject utilizes the implemented system in real-world conditions,
        collecting data useful for system evaluation.
    \item \textbf{Evaluation:} methodical review of the implemented \Gls{rwsm} system:
        \begin{itemize}
            \item Analysis of the effectiveness of the system in regularly gathering vitals data and \Gls{ews} samples
            \item Investigation of failure points when sample retrieval or \Gls{ews} assessments were unsuccessful
            \item Collation of the subject's feedback on user experience
        \end{itemize}
    \item \textbf{Feasibility and challenge discussion:} reflection on the entire research process to:
        \begin{itemize}
            \item draw conclusions on the feasibility of \Gls{rwsm} using modern smart medical sensors
            \item discuss application of the chosen \Gls{ews} for \Gls{rwsm}
            \item highlight the identified technical and operational challenges
        \end{itemize}
\end{enumerate}

By completing these tasks, the research provided a comprehensive understanding of the practicality and potential pitfalls
of \Gls{rwsm} in everyday settings, using current technology.

%\subsection{Structure}

% TODO Describe the structure of the work
% TODO Describe used notations

\newpage
\section{Medwings}

The initial step in conceptualizing an \Gls{rwsm} system was to choose an appropriate \Gls{ews} to use for patient health evaluation.
Three widely used \Glspl{ews} emerged as potential candidates: the \Gls{pews}\cite{monaghan_detecting_2005}, \Gls{news2}\cite{noauthor_national_2017}
and \Gls{mews}\cite{subbe_validation_2001}.
All three are established as being effective in predicting clinical deterioration.
\Gls{pews} was excluded due to its application being limited to pediatric patients.
The choice between \Gls{mews} and \Gls{news2} was made by considering the input parameters to each score's calculation:
compared to \Gls{mews}, \Gls{news2} takes into account whether a patient is currently suffering from hypercapnic respiratory failure, and whether or not the
patient is currently being ventilated using supplemental oxygen.
These parameters are generally not applicable to patients dismissed from medical care, hence \Gls{mews}
was chosen as the early warning score for the envisioned \Gls{rwsm} system.

To calculate the \Gls{mews}, the following vital parameters must be recorded and processed:
\begin{itemize}
    \item Heart Rate
    \item Blood Pressure (systolic)
    \item Body Temperature
    \item Respiratory Rate
    \item \Gls{avpu}
\end{itemize}

To develop an \Gls{rwsm} system capable of gathering these vital signs and making them accessible remotely,
wirelessly networked vitals measurement devices were used.
The process of selecting the right smart sensors presented a series of challenges.
A significant portion of the available medical sensors on the market were either not accessible to the general public, or are not distributed
in the geographic area where the research was carried out.
While considering devices that met the procurement criteria outlined earlier, a large number of products had to be excluded, as they
would have constrained patient mobility to the confines of their home or bedside.
Several promising devices were identified, but dismissed due to still being in active development and having not yet received
clinical approval.

Among a few options on the final shortlist, Withings emerged as the most feasible choice for several reasons.
Notably, they were the only manufacturer who offers a publicly accessible \Gls{api}, allowing for automated retrieval of vitals data.
Consequently, three Withings devices were selected for the study.
The \textit{Withings Scanwatch}\cite{noauthor_worlds_nodate}, shown in Figure~\ref{fig:withings-scanwatch} is a smartwatch capable of
measuring a user's \Gls{ecg} and \Gls{spo2} among other things.
A \textit{Withings BPM Core}\cite{noauthor_bpm_nodate}, displayed in Figure~\ref{fig:withings-bpm-core}, was also procured.
It is a digital blood pressure meter capable of recording blood pressure and heart rate.
The third and last device used was a \textit{Withings Thermo}\cite{noauthor_smart_nodate}, a contactless digital thermometer
used to measure body temperature. A picture of a Whithings Thermo can be seen in Figure~\ref{fig:withings-thermo}.

\begin{figure}[!ht]
    \begin{center}
        \begin{subfigure}{.25\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/withings-scanwatch.png}
            \caption{\footnotesize Scanwatch}
            \label{fig:withings-scanwatch}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.25\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/withings-bpm-core.png}
            \caption{\footnotesize BPM Core}
            \label{fig:withings-bpm-core}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.25\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/withings-thermo.png}
            \caption{\footnotesize Thermo}
            \label{fig:withings-thermo}
        \end{subfigure}
        \caption{Withings smart medical devices \textit{(image sources: Withings Scanwatch\cite{noauthor_worlds_nodate}, Withings BPM Core\cite{noauthor_bpm_nodate}, Withings Thermo\cite{noauthor_smart_nodate})}}
        \label{fig:withings-devices}
    \end{center}
\end{figure}

All three devices are able synchronize vitals data with the Withings Cloud over the internet, as they connect to the user's phone
using a mobile application provided by Withings.

The chosen selection of devices allows measurements and programmatic access to the vital parameters required to determine a
\Gls{mews}, with some caveats: the AVPU score cannot be measured remotely, as it necessitates a clinical assessment from medical
staff.
Additionally, the inclusion of the Withings Scanwatch came with a notable limitation: although the device possesses the capability
to measure a patient's respiration rate, this functionality is restricted to nocturnal measurements, taken while the user is asleep.

After consulting with several medical professionals, a decision was made to forgo the traditional respiration rate measurement, as well as the AVPU parameter in the \Gls{mews} calculation.
Instead, a custom \textit{respiration score} was introduced, shown in Table~\ref{tab:respiration-score}, which represents any shortness of breath reported by the patient.

\begin{table}[!ht]
    \centering
    \begin{tcolorbox}[
            enhanced, width=0.9\linewidth, boxrule=2pt, arc=4pt,
            tabularx={l|>{\centering\arraybackslash}X}
        ]
        \textbf{Condition} & \textbf{Score} \\
        \specialrule{2pt}{0em}{0em}
        Patient is \textit{not} suffering from shortness of breath & 0 \\
        \hline
        Patient is experiencing \textit{some} shortness of breath & 1 \\
        \hline
        Patient is suffering from \textit{severe} shortness of breath & 2 \\
    \end{tcolorbox}
    \caption{\label{tab:respiration-score}Scoring table for Medwings' custom respiration score}
\end{table}

To replace the respiration rate in the \Gls{mews} calculation, we took into account the patient's \Gls{spo2}, as well as the described respiration score.

\subsection{Requirements}

Following the selection of an \Gls{ews} and suitable medical sensors, a comprehensive \Gls{rwsm} application was conceptualized,
dubbed as the \textit{Mobile Early Deterioration Warning System (Medwings)}.
Prior to its development, several key software requirements for the application were determined.

\subsubsection{Functional Requirements}

\begin{description}
    \item[User Authentication:] Medwings must provide a robust user authentication and authorization system to ensure confidentiality of sensitive patient data, while preventing
        unauthorized access to restricted information.
    \item[Portability:] Patients must be able to access Medwings from their mobile phone.
        Enabling access from other types of end user devices is also desirable.
    \item[Data Collection:] The application must be capable of collecting and storing vital sign readings from all three
        Withings smart medical sensors without the need to transfer data manually.
        Additionally, it must be able to determine the custom respiration score introduced earlier, by prompting patients
        on whether they are experiencing shortness of breath.
    \item[Measurement prompts:] To facilitate a regular \Gls{mews} assessment schedule, the application must regularly
        prompt patients to take vital sign readings.
    \item[MEWS Calculation:] When sufficient vital signs for a patient are collected, Medwings must automatically compute and store the \Gls{mews}.
    \item[Data Synchronization:] Vitals data, \Gls{mews} measurements and any associated metadata should be synchronized and accessible on
        each of the patient's end devices. Captured data should be stored by the system for later analysis.
    \item[Data Visualization] Patients should be able to view a history of their recorded vitals and \Gls{mews} values.
\end{description}

\subsubsection{Non-functional Requirements}

\begin{description}
    \item[Usability:] Medwings should be intuitive and user-friendly, requiring minimal technical expertise from end-users.
        The \Gls{ui} should be adaptive for display on both mobile displays and larger monitors.
        If measurements fail or errors occur, clear error messages should be displayed to the user.
    \item[Availability:] The system should be available for patients to use at all times, and from any location, with minimal downtime.
    \item[Data Validity:] Vitals records retrieved from the smart sensor devices must be converted and displayed correctly.
        Calculated \Gls{mews} values must be correct and take into account all relevant vital parameters:
        \begin{itemize}
            \item systolic blood pressure
            \item heart rate
            \item body temperature
            \item \Gls{spo2}
            \item respiration score
        \end{itemize}
        To ensure the medical validity of a \Gls{mews} assessment, when a set of vitals measurements is used to calculate a \Gls{mews},
        the time of measurement for any two measurements in the set must not be further apart than ten minutes.
    \item[Security:] All personal and medical data must be handled in a secure manner, both during storage and in transit.
        When transmitting data between Medwings and Withings, or between a user's end device and Medwings, communication confidentiality
        and integrity must be ensured. The identity of both communication partners must be cryptographically verifiable.
\end{description}

\subsection{Design and Implementation}

Medwings was designed as a web-based application, accessible through a web browser.
Opting for this format offers several advantages: the primary benefit is its inherent cross-platform compatibility, enabling usage
on a wide range of devices such as mobile phones and personal computers.

Adhering to the classic client-server paradigm, the Medwings design prioritizes centralized data storage and processing.
This architecture was found to be benificial for simplifying data synchronization, facilitating secure authentication,
and ensuring high system availability.

Django, a high-level Python web framework, was employed to develop both the frontend and backend components of the Medwings application.
Some of the primary motivations for selecting Django were its out-of-the-box user authentication and session management capabilities.
Such features substantially expedited the development process, freeing up time and resources to focus on the more unique functionalities
of the Medwings web application.
Furthermore, Django's integrated Object-Relational Mapping (ORM) system greatly simplified the creation, management, and querying of the application's database.
This was pivotal given the essential role of the database in storing vitals data.

While web applications offer many advantages, one limitation is their increased design complexity required to support push
notifications directly on the patient's mobile phone.
To simplify this aspect of the design, a separate push notification microservice was deployed on premise and integrated with Medwings.
Considering the time constraints under which the application was developed, this approach proved to be an effective compromise.

\subsubsection{Architecture}

\begin{figure}[!ht]
    \begin{center}
        \includegraphics[width=.75\textwidth]{./figures/components-macro.png}
        \caption{System diagram showing data flow and user interactions between components in the Medwings environment.}
        \label{fig:components-macro}
    \end{center}
\end{figure}

The overall system environment is shown in Figure~\ref{fig:components-macro}, depicting the following workflow:
\begin{enumerate}
    \item A patient receives a notification on their mobile phone, prompting them to take vitals measurements.
    \item Upon opening the notification, they are redirected to the Medwings website.
        Here, they are prompted to self-assess their respiration score by answering a short questionnaire,
        followed by a prompt to take one measurement on each Withings device.
    \item When a measurement is completed, each device transmits the data via Bluetooth to the Withings mobile app, installed on the user's phone.
        The mobile app now sends the data to the Withings Cloud for storage.
    \item A backend process on the Medwings server awaits the arrival of all recorded measurements from the Withings Cloud, storing them upon reception.
        Once all required vitals measurements have been retrieved, the MEWS is calculated, stored and displayed to the patient.
\end{enumerate}
Throughout the day, measurement prompt notifications are dispatched to the patient at regular intervals.

\subsubsection{Modules}\label{sec:modules}

To separate the different functional aspects of Medwings according to responsibility, its application code is split into the following five modules:

\begin{itemize}
    \item \code{core}
    \item \code{withings}
    \item \code{gotify}
    \item \code{authentication}
    \item \code{medwings}
\end{itemize}

Each module defines classes representing backend logic, database schemas and user interface elements  pertaining to its specific function.
Implementation details are encapsulated within these classes, while public interfaces are exposed to external program code to provide each module's core functionality.

The \code{core} module forms the backbone of the application.
It encompasses configuration settings, handling of secrets such as private encryption keys or API tokens, and functionalities
shared across multiple other modules.
This includes backend utility functions and shared \Gls{ui} components for the frontend.

Medwings interfaces with the Withings Cloud through the \code{withings} module.
Responsibilities include retrieving vitals data through authenticated requests to the Withings Cloud API, which implements the OAuth 2.0 Authorization Framework.
As per its specification, \enquote{In OAuth, the client requests access to resources controlled by the resource owner and hosted by the resource server\ldots~
    Instead of using the resource owner’s credentials to access protected resources, the client obtains an access token\ldots~
    The client uses the access token to access the protected resources hosted by the resource server.
}\cite{hardt_oauth_2012}
% TODO explain this in more detail. It is unclear what is happening here.
While this process is largely transparent for the resource owner --- the patient in this case --- the communication between
Medwings as the resource client and Withings as the resource server is complex, and is therefore abstracted by the module.
Aside from OAuth 2.0, \code{withings} also encapsulates fetching, parsing, and storing vitals data retrieved from Withings.

Medwings implements a standalone user authentication system, which is provided by the \code{authentication} module.
Patients must register with a username and password to be able to use the application.
The registration occurs in three stages:
\begin{enumerate}
    \item The patient grants Medwings the permission to retrieve their health data from Withings in an OAuth2 authorization flow. This process is shown in
        Figures \ref{fig:ui-register-init} and \ref{fig:ui-register-oauth2}.
    \item A registration form, displayed in Figure~\ref{fig:ui-register-continue}, is shown, prompting the user to choose a username and password, and to enter relevant personal information.
    \item The user is shown a confirmation that the account was created successfully, and is asked to download the Gotify app, described below, and log in using their Medwings credentials.
        Figure \ref{fig:ui-register-finalize} shows this final step.
\end{enumerate}
Following registration, the supplied information and numerous authentication tokens are saved in the Medwings database.
The patient can now log in to the Medwings website and begin using the system to take vitals and \Gls{mews} measurements.

\begin{figure}[!ht]
    \begin{center}
        \begin{subfigure}{.21\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-register-init.png}
            \caption{User signup initialization}
            \label{fig:ui-register-init}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.21\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-register-oauth2.png}
            \caption{Withings OAuth 2.0 grant}
            \label{fig:ui-register-oauth2}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.21\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-register-continue.png}
            \caption{User details prompt}
            \label{fig:ui-register-continue}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.21\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-register-finalize.png}
            \caption{Registration finalization}
            \label{fig:ui-register-finalize}
        \end{subfigure}
        \caption{Medwings user registration process}
        \label{fig:ui-registration}
    \end{center}
\end{figure}

The \code{medwings} module, pivotal to the core functionalty of Medwings, defines the data model used to represent and store the various vital signs handled by the application.
Furthermore, it provides interfaces to access the data, as well as the algorithm used to calculate the MEWS, which is listed in Algorithm~\ref{algo:mews}.
% TODO add some more text here

\begin{centering}
    \begin{tcolorbox}[
            enhanced, width=0.9\linewidth, boxrule=1pt, arc=4pt,
        ]
        \begin{algorithm}[H]
            \KwIn{$blood\_pressure\_systolic$, $body\_temperature$, $heart\_rate$, $spo2$, $respiration\_score$}
            \KwOut{$mews$}
            \DontPrintSemicolon
            \BlankLine

            $mews \leftarrow 0$ \;
            \BlankLine

            \If{$blood\_pressure\_systolic \leq 70$}{
                $mews \leftarrow mews + 3$ \;
            }\ElseIf{$blood\_pressure\_systolic \leq 80$}{
                $mews \leftarrow mews + 2$ \;
            }
            \ldots \;
            \tcp*[l]{systolic blood pressure, body temperature}
            \tcp*[l]{and heart rate as per Table~\ref{tab:mews-calculation}}
            \BlankLine
            \BlankLine
            \BlankLine

            \If{$respiration\_score = 1$}{
                $mews \leftarrow mews + 1$ \;
            }\ElseIf{$respiration\_score = 2$}{
                $mews \leftarrow mews + 2$ \;
            }
            \BlankLine

            \If{$spo2 \leq 90$}{
                $mews \leftarrow mews + 2$ \;
            }\ElseIf{$spo2 < 95$}{
                $mews \leftarrow mews + 1$ \;
            }
            \BlankLine

            \Return $mews$
            \BlankLine
            \BlankLine
            \BlankLine

            \caption{\label{algo:mews}Medwings \Gls{mews} calculation}
        \end{algorithm}
    \end{tcolorbox}
\end{centering}

In order to send push notifications to mobile devices, Medwings leverages \textit{Gotify} -- a dedicated notification microservice\cite{noauthor_gotify_nodate}.
Gotify is composed of a web server component, and a mobile app acting as the client software.
The server exposes its own \Gls{api}, which allows external applications like Medwings to dispatch push notifications programmatically.
It uses an independent database for client authentication. The \code{gotify} module ensures synchronization between the user databases of Gotify and Medwings.
In addition, the module provides interfaces to send customized push notifications to specific patients.

\subsubsection{User Interface}

The Medwings \Gls{ui} was developed with specific design goals in mind to ensure an efficient and intuitive user experience.
Figure \ref{fig:ui-screenshots} provides some impressions of what a user sees when using the application.

\begin{figure}[!ht]
    \begin{center}
        \begin{subfigure}{.3\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-home-page.png}
            \caption{Medwings home page}
            \label{fig:ui-screenshot-home-page}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.3\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-vitals-measurement.png}
            \caption{Starting a measurement}
            \label{fig:ui-screenshot-mews-init}
        \end{subfigure}
        \hfill
        \begin{subfigure}{.3\textwidth}
            \centering
            \includegraphics[width=\linewidth]{./figures/screenshot-recording.png}
            \caption{Waiting for synchronization}
            \label{fig:ui-screenshot-mews-continue}
        \end{subfigure}
        \caption{Medwings \Gls{ui} screenshots}
        \label{fig:ui-screenshots}
    \end{center}
\end{figure}

Aiming for a clutter-free and fast user experience, simplicity served as the guiding principle to enhance usability.
A focus was put on developing accessibility-friendly \Gls{ui} components, ensuring that the system is usable by visually impaired individuals.

Security was a top priority in the frontend implementation to protect users and the system from common vulnerabilities.
Overall, the use of JavaScript was kept to a minimum to reduce the attack surface.
Various security measures were carefully put in place to reduce the attack surface of the website:
\begin{itemize}
    \item User input and form field sanitization, alongside strong server-side validation to prevent cross-site scripting (XSS) and SQL injection attacks
    \item To counteract cross-site request forgery (CSRF) attacks, CSRF tokens were implemented on all forms
    \item To minimize the risk of supply chain and third-party XSS attacks, no external JavaScript dependencies are used by Medwings
\end{itemize}

Considering that mobile devices are the primary platform for Medwings, the \Gls{ui} was designed to be fully responsive.
It adapts seamlessly to different screen sizes, whether accessed through mobile phones, tablets, or devices with a larger form factor.
Fast load times were a crucial design goal to ensure usability under various network conditions.

Navigational elements and the overall layout follow conventional patterns.
Animations are sparingly used to visually indicate in-progress system states, such as when waiting for data retrieval from the Withings Cloud.

\subsubsection{Datamodel}

A relational database was used to store application data, whereby each Medwings module defines the database schema for the underlying data it is responsible for handling.
Module interdependencies correlate closely with the foreign key references in the data model.
A holistic representation of the Medwings data model is shown in Figure~\ref{fig:datamodel}.

\begin{figure}[!ht]
    \begin{center}
        \includegraphics[width=\textwidth]{./figures/datamodel.png}
        \caption{\label{fig:datamodel}Entity-Relationship diagram (Crow's Foot notation) showing the data model of the Medwings database.}
    \end{center}
\end{figure}

At its heart lies the \code{User} entity: it forms the nexus to which all vitals data and user information is linked.
Withings API tokens are stored in the \code{RefreshToken} and \code{AccessToken} entities, while the \code{GotifyUser} and \code{GotifyAccount} entities retain the Gotify API credentials.
The numerous vital signs, as well as the \Gls{mews} record which can potentially be calculated based on them, are also represented.
The \code{Profile} table stores additional medically relevant patient information as supplied during user registration.

\subsubsection{Deployment}

To use the smart devices to take measurements, patient users must first install the Withings mobile app on their phone, and use it to create a Withings user account.
Following registration, each device must be linked to the app and configured via Bluetooth.
Some basic configuration is required in order to enable specific device features, such as measurement of \Gls{spo2} on the Scanwatch.
Users are guided through the process by the app's \Gls{gui}.

Being a web application, no installation is necessary to access the Medwings interface, patients simply visit the website in a web browser.
Patients do need to create a Medwings account on the website however, followed by installation and configuration of the Gotify mobile app, as described in the registration
process in Section~\ref{sec:modules}.

The centralized server components, including the Gotify server, a task scheduler used to schedule sending notifications and the Medwings application code itself are deployed
on a publicly accessible web server using a Docker container environment.

\subsubsection{Design Challenges}\label{sec:design-challenges}

Since managing a user in Medwings requires the respective user's state to be mirrored by two other services, Withings and Gotify, keeping user accounts across
all three services in sync presents a challenge.
Particularly during user creation, user accounts must be linked to Withings, created on the Gotify server and finally saved to the Medwings database.
Various integrity checks, such as when the user aborts the registration process midway, were put in place to prevent diverging user states across the three services
and overcome this challenge.

Similarly, vitals records kept in the Medwings database must be synchronized with all records available on the Withings cloud.
Regularly recurring, as well as on-demand data synchronization hooks were implemented to keep the Medwings database up to date,
while database constraints ensure validity of imported data and prevent duplication of existing records.

The non-enterprise Withings API enforces a rate limit of 120 requests per minute.
Medwings polls the API regularly to retrieve the latest health data for patients.
At scale, with many patient users, the rate limit would quickly be reached.
The Withing API does provide functionality to notify client applications upon availability of new data, making it possible to avoid polling.
Given that Medwings was only used by a single patient user during the trial phase, falling back to polling was an acceptable compromise to lower complexity
while still operating within the rate limit.

A MEWS calculation should represent the patient's overall physiological state at -- ideally -- a discrete point in time.
Naturally, there is a delay from when a measurement is taken with a device until it can be retrieved from the API.
The transmission delays are mentioned in the Withings public API documentation:
\enquote{Delays are typically less than two minutes, but it can be longer.}\cite{noauthor_keep_nodate}.
However, in some cases, the measurements taken on a device do not get pushed to the Withings Cloud until much later.
While the cause for these longer than normal delays and missing data points is unknown and outside of the control of Medwings,
these edge cases had to be taken into account.
The time it takes a patient to take measurements using all three devices must also be accounted for.
Therefore, Medwings enforces a maximum allowed time difference of ten minutes between measurements of the different
vitals records used to calculate the \Gls{mews}.
If a set of vitals measurements is, across all records in the set, spaced further apart than ten minutes, no \Gls{mews} record
is calculated, and the user is shown an error message.

\section{System test and trial}

Following the development and deployment of the application, Medwings underwent a performance and usability study.
Over the course of one week, a test subject, impersonating a patient, used the application several times a day.

Each day, five notifications were dispatched.
Starting at 10 AM, one notification was sent every three hours.
When prompted by a notification, the subject was asked to visit the Medwings website and begin the measurement process.
Following a notification, the measurements were carried out in the following order:

\begin{enumerate}
    \item The subject responds to a Medwings prompt to assess their respiration score.
    \item Using the Scanwatch, the subject starts an \Gls{spo2} measurement and awaits its completion.
    \item A blood pressure and heart rate check is initiated on the BPM core, and the subject waits for the results.
    \item The subject takes a body temperature reading via the Thermo and waits for it to complete
    \item Finally, the subject allows time for Medwings to aggregate all the data and display the \Gls{mews}.
\end{enumerate}

Throughout this process, Medwings would continuously attempt to retrieve the vital sign readings from the Withings Cloud, and calculate the
\Gls{mews} once all required readings are available.
If not all readings could be retrieved within the ten minute timeout, Medwings displayed an error message and aborted the \Gls{mews}
measurement process.

\subsection{Methodology}

For each vital sign measurement, as well as for each MEWS calculation, Medwings stored the measurement results alongside the
time of measurement.

For each received notification, the test subject manually kept track of which environment the system was used in.
A distinction was made between the following environments:
\begin{itemize}
    \item At home: The subject was located at home, and their end device had access to a low latency broadband internet connection
    \item On the go: The subject was away from home, and their end device had access to a high latency mobile internet connection
\end{itemize}
The end device used by the subject to connect to Medwings was their phone throughout the trial.
In addition, the subject reflected on noteworthy experiences regarding use of the system after the trial was completed.

The occurrence of measurement failures was anticipated, and manually recorded.
Measurement failures were categorized into eight distinct classes, as listed in Table~\ref{tab:measurement-failures}.

\begin{table}[!ht]
    \centering
    \begin{tcolorbox}[
            enhanced, width=0.95\linewidth, boxrule=2pt, arc=4pt,
            tabularx={>{\centering\arraybackslash}c|>{\centering\arraybackslash}c|>{\arraybackslash}X}
        ]
        \textbf{Device} & \textbf{Failure Class} & \textbf{Description} \\
        \specialrule{2pt}{0em}{0em}
        \multirow{2}{*}{Scanwatch} & $S_1$ & Device aborted measurement \\
        \cline{2-3}
        & $S_2$ & Measurement synchronization failure \\
        \hline
        \multirow{2}{*}{BPM Core} & $B_1$ & Device aborted measurement \\
        \cline{2-3}
        & $B_2$ & Measurement synchronization failure \\
        \hline
        \multirow{2}{*}{Thermo} & $T_1$ & Device aborted measurement \\
        \cline{2-3}
        & $T_2$ & Measurement synchronization failure \\
        \hline
        \multirow{2}{*}{---} & $P_1$ & Patient did not take any measurements \\
        \cline{2-3}
        & $P_2$ & MEWS calculation timed out \\
    \end{tcolorbox}
    \caption{\label{tab:measurement-failures}Classification of measurement failures during the usability trial}
\end{table}

The Scanwatch and BPM Core are equipped with accellerometers\cite{noauthor_worlds_nodate, noauthor_bpm_nodate}.
If erratic movement is detected, the devices abort the measurement to avoid misinterpretation of sensor readings.
Similarly, upon failure to process captured sensor data into a plausible result, a measurement may be aborted by the device\cite{noauthor_scanwatch_nodate, noauthor_bpm_nodate-1, noauthor_guides_nodate}.
The measurement failure classes $S_1$, $B_1$ and $T_1$ were used to record these kinds of failure for the Scanwatch, BPM Core and Thermo respectively.
Following an $S_1$, $B_1$ or $T_1$ failure, the subject repeatedly carried out measurements using the affected device until a valid reading could be obtained.
Subsequent failures caused by the device aborting measurements were also recorded.
The count of \enquote{device aborted measurement}-failures of each device was compared to the total number of measurement attempts using that device.

As explained in Section~\ref{sec:design-challenges}, following a successful reading, a device may fail to push the measurement data to the Withings
Cloud within the ten minute validity range for a MEWS calculation imposed by Medwings.
Depending on whether the Scanwatch, BPM Core or Thermo failed to synchronize its data within the allowed time, an $S_2$, $B_2$ or $T_2$ failure was recorded respectively.
The number of measurement synchronization failures which occurred was compared to the number of successfully synchronized measurements for each device.
Following an $S_2$, $B_2$ or $T_2$ failure, the measurement process was not repeated until the next notification.

For each notification to which the subject responded, the duration between when the notification was dispatched and when the patient took the first vitals measurement was recorded.
Additionally, the average time taken to complete all three vitals measurements was noted.
If the subject did not visit the Medwings website or carry out any vitals measurements despite being prompted by a notification, a $P_1$ failure was noted.
Finally, if the patient failed to carry out all three required vitals measurements within the ten minute time limit, a $P_2$ failure was recorded.

Preceeding each MEWS measurement, metrics quantifying the quality of the connection between the subject's end device and other devices across the internet were measured.
This was done by sampling and averaging the data transmission rates, both uplink and downlink, as well as the connection round trip times from the end device to a distant reference server,
the location of which was kept constant throughout the trial.
The collected connection metrics were compared with the occurrence of measurement synchronization failures.

\subsection{Data Presentation}

The trial period encompassed seven days, on each of which five notifications were dispatched to the patient.
Thus, an overall of $N=35$ system interactions were recorded.
The patient was at home during $26$ of these, and on the go in all other cases.

Seven $P_1$ measurement failures occurred, wherein the patient did not react to a received notification by taking measurements.
Out of these, five occurred while the patient was on the go and, notably, four $P_1$ failures were consecutive, stemming from a period of 24 hours
during which the patient did not have access to the smart devices.
No $P_2$ measurement failures occured during the trial.
The patient reported feeling reluctant about taking measurements using the devices in public spaces, compared to the privacy of their home.

In total, vitals were measured using all three devices in $28$ cases.
However, in $11$ cases, at least one device failed to synchronize its measurements with the Withings Cloud within the ten minute timeout.
Throughout the trial, $17$ MEWS calculations were recorded successfully.
Figure~\ref{fig:measurement-stats} visualizes the overall measurement and failure counts.

\begin{figure}[!ht]
    \begin{center}
        \includegraphics[width=\textwidth]{./figures/chart-measurement-stats.png}
        \caption{\label{fig:measurement-stats}Measurement and measurement failure statistics at home and on the go.}
    \end{center}
\end{figure}

Out of $84$ successful individual vitals measurements across all devices throughout the trial, $18\%$ took longer than premitted by Medwings to synchronize with the Withings Cloud.
Particularly while on the go, synchronization was prone to taking too long: $25\%$ of measurements resulted in synchronization failure, compared
to $11\%$ at home.
Especially the BPM Core and Thermo devices suffered from slow synchronization times: in a total of $15$ synchronization timeouts, $n_{B_2} = 7$ were caused
by the blood pressure meter, and $n_{T_2} = 7$ by the thermometer.

The likelihood of each device aborting a measurement due to inconclusive sensor data was examined and is visualized in Figure~\ref{fig:measurement-repeats}.
For the BPM Core, $15\%$ of attempted measurements had to be repeated ($n_{B_1} = 5$).
For the Scanwatch, over $34\%$ of readings ($n_{S_1} = 15$) were inconclusive and had to be repeated.
The Withings Thermo did not abort any measurements ($n_{T_1} = 0$).

\begin{figure}[!ht]
    \begin{center}
        \includegraphics[width=0.8\textwidth]{./figures/chart-measurement-repeats.png}
        \caption{\label{fig:measurement-repeats}Number of measurement attempts and aborted measurements for each smart device.}
    \end{center}
\end{figure}

Figure~\ref{fig:connection-boxplot} illustrates the comparative boxplots for the \gls{downlink-datarate}, \gls{uplink-datarate}, and \Gls{rtt} connection metrics when
the patient was at home versus on the go.
While there are evident differences in the distributions of these metrics between the two environments, the points representing synchronization failures do not
predominantly cluster around areas of low datarate or high \Gls{rtt}.

\begin{figure}[!ht]
    \begin{center}
        \includegraphics[width=\textwidth]{./figures/chart-connection-boxplot.png}
        \caption{\label{fig:connection-boxplot}Connection quality and synchronization failures.}
    \end{center}
\end{figure}

A reaction delay $t_r$ existed from when a notification was dispatched until the subject visited the Medwings website to take measurements.
The average ($\overline{t_{r,\text{home}}} = 36\text{ min}$) and median ($M_{t_{r,\text{home}}} = 33\text{ min}$) delay was significantly lower
when the patient was at home, compared to when they were out of the house ($\overline{t_{r,\text{on the go}}} = 68\text{ min}$, $M_{t_{r,\text{on the go}}} = 70\text{ min}$).
The patient reported feeling fatigued by the regularity of the notifications from the second trial day onwards.

The average time it took the patient to carry out all three measurements was $4.5$ minutes, with no significant difference between the \enquote{at home} and
\enquote{on the go} environments.

In all cases where vitals measurements were taken using the devices, the vitals data was captured and stored by Medwings.
In cases where the measurement data was not uploaded to the Withings Cloud quickly enough for Medwings to calculate the \Gls{mews}, the recorded
vitals data was still pulled into the Medwings database.
Across all measurements, the subject's vital signs were within normal ranges, with the exception of
two outliers where a slightly increased heart rate was measured.
Both outliers were detected by Medwings in the form of an elevated \Gls{mews}.

The complete trial data is listed in Appendix~\ref{apdx:trial-data}.

\subsection{Interpretation}

The system was successful in retrieving a wide range of vital signs from the patient at regular intervals, and detected abnormal readings
using the \Gls{mews}.
All \Gls{mews} calculations carried out by Medwings resulted in the correct value based on the recorded vitals data.
Both at home and on the go, the patient's vital parameters could be monitored using the system.

While at home, the test subject was able to adhere well to the measurement schedule, missing measurements only twice throughout the week.
With a three-hour window to respond to each notification, the average response time of 36 minutes and the minimal number of $P_1$ failures show that the
4.5-minute measurements were manageable within the home setting.

On the go however, a significant portion of the received notifications were not followed up by vitals measurements.
When leaving the house for extended periods of time, the subject was not always able to bring the whole array of medical devices with them.
Additionally, taking measurements could not always be done discreetly in public spaces, and finding a private area to take measurements
was not always possible.
This lead to a high rate of $P_1$ failures on the go, coupled with a comparatively long average reaction time of $\overline{t_{r,\text{on the go}}} = 68\text{ minutes}$.

The rate at which device synchronization failures occured was high, with $39\%$ of successful measurements not being pushed to the Withings cloud in time
for a \Gls{mews} calculation to be valid.
A combination of three factors was determined to be the cause for this.

Firstly, while the Scanwatch is constantly connected to the patient's phone, the BPM Core and Thermo devices only establish
their bluetooth connection intermittently.
Presumably, measurement data updates from these devices are sent to the phone less frequently than for the Scanwatch.
This is strongly underlined by the fact that $93\%$ of all synchronization timeouts were caused by the BPM Core or Thermo.

The second factor becomes apparent after examining the likelihood of each device aborting its measurement due to inconclusive sensor data,
as displayed in Figure~\ref{fig:measurement-repeats}.
Although the aborted measurements did not cause synchronization failures directly, the time taken to repeat measurements impacted
the likelihood of the MEWS calculation timing out before all vitals data was synchronized.
The Scanwatch was particularly prone to prolonging the overall time it took the patient to complete all readings.

The third factor was the time it took the Withings app to push vitals data to the Cloud, and data being available for retrieval through the Withings API.
While no concrete data to quantify this duration could be gathered, it is clear that the two minute delay estimate provided by the manufacturer\cite{noauthor_keep_nodate} was exceeded
substantially in many cases.
The relationship between the connection quality and the likelihood of synchronization failures was examined, as shown in Figure~\ref{fig:connection-boxplot},
but no correlation was found, suggesting that synchronization failures were not majorly influenced by the current connection quality of the subject's end device.

Ultimately, while $61\%$ of notifications to which the patient responded did result in a successful \Gls{mews} calculation, the high rate of device synchronization timeouts
suggests that routing data from the device, to the mobile app, to the Withings cloud, and finally to the Medwings server introduces substantial delays,
thereby affecting the system's usability for its near-real-time calculation requirement.

\newpage
\section{Evaluation}

The Medwings application successfully met its predetermined software requirements.
The system demonstrated robust user authentication mechanisms and offered portability by being accessible on mobile phones.
Additionally, it provided an intuitive interface for data visualization.
Vital signs were collected and stored automatically from all three Withings smart medical sensors, and the respiration score was able to be determined interactively.
The system was consistently available and ensured data validity, while maintaining stringent security protocols.
While certain challenges were encountered during the usability trial, the software's overarching objectives in terms of functional and non-functional
requirements were successfully achieved.

The usability study of the Medwings system provided valuable insights into the system's performance, reliability, and user interaction experience.
Classifying measurement failures into types helped to identify bottlenecks and other areas for improvement.
Overall, the system performed well in obtaining vitals readings regularly from a patient test subject.
The system successfully aggregated data from multiple sources to compute the \Gls{mews}, with all calculations being accurate.
At home, the patient was able to take all vitals measurements quickly and accommodate the measurement process into their daily routine, leading to a
high rate of interaction with the system.
Within the limited trial period, Medwings was able to detect abnormal readings effectively.

A significant portion of recorded vitals data, however, could not successfully be converted into \Gls{mews} records.
These calculation failures pertained to device synchronization delay with the Withings Cloud, highlighting a critical issue.
The BPM Core and Thermo devices synchronize only when turned on through manual interaction.
This leads to longer sync times for these devices, inhibiting Medwings from accessing new vitals data swiftly and performing a timely \Gls{mews} calculation.
A timeout period more lenient than the ten minute window imposed by Medwings may reduce the rate of synchronization failures, but it may also negatively
impact the validity of a \Gls{mews} record if implemented.
The study found that connection metrics such as data transmission rates and round-trip times did not show a significant correlation with synchronization failures.
This suggests that other factors, likely server-side processing delays on the Withings Cloud, contribute to these failures.

The study also revealed that the system is generally more reliable when the patient is at home, as indicated by the low interaction rate while on the go.
One reason was that carrying a a range of vitals sensors while out of the house is not always feasible.
Another was the subject's reluctance to take measurements in public spaces, suggesting a need for more discreet solutions for on the go vitals monitoring.
However, when measurements were made in the mobile environment, Medwings was successful at aggregating the measured data and, aside from the aforementioned
synchronization issues, was able to calculate the subject's \Gls{mews}.

%Patients appreciate the face-to-face aspect of early warning score monitoring as it allows for reassurance, social interaction, and gives them further opportunity to ask questions about their medical care\cite{downey_patient_2018}.

\subsection{Limitations}

Several limitations of the current study and the Medwings system must be noted.

Firstly, the original \Gls{mews} algorithm could not be used in its unaltered form to facilitate automated deterioration scoring.
Measuring the respiration rate and \Gls{avpu} of a mobile patient presented a challenge, which had to be overcome by disregarding
the \Gls{avpu} from the calculation and replacing the respiration rate in favor of a custom respiration score and \Gls{spo2} measurements.

Furthermore, the usability study had only a single test subject, which does not capture the diversity of potential user experiences.
Due to time constraints, the trial period was limited to lasting only one week, reducing the sample size and potentially introducing
bias in the gathered data.

Measurement prompts were dispatched every 3 hours during the day. This allows for just a restricted, intermittent view of a patient's
physiological state, and the chosen sampling period may require adequate ajdustment.

Moreover, although designed and implemented as a multi-user system, Medwings was only tested with one active user.
When used by many more users, the proof-of-concept system may encounter scalability issues, such as exceeding the API rate limits
imposed by Withings.

Lastly, using Medwings and the smart medical devices requires the patient to have internet access, which will negatively impact the
system's effectiveness in some circumstances where a stable connection is not available.

\newpage
\subsection{Future Work and Improvements}

The Medwings system, while successful in its current iteration, offers multiple avenues for improvement.

Its dependence on web-based access to the Withings API to retrieve vitals readings results in significant delays and mandates an internet connection.
A native mobile application offering direct Bluetooth access to sensor measurements would enhance real-time responsiveness and negate the necessity for a stable internet connection.
Unfortunately, Withings do not offer such an option to developers.
Vendor dependence in general introduces a risk to Medwings' continued operation.
Service outages, discontinued device updates, or the vendor ceasing business operations all render the smart devices, and consequently Medwings, unusable.
Vendor-independent access and open-source firmware for the medical devices would mitigate these risks and open up exciting possibilities for further system integrations.

Additionally, expanding the range of monitored vitals could allow implementation of more comprehensive \Gls{rwsm} techniques.
The integration of a smart device capable of accurately measuring the respiration rate of a mobile patient would refine the \Gls{mews} calculations,
leading to a more accurate deterioration monitoring system.
The system could also benefit from real-time alerting for emergency situations, rather than relying solely on periodic \Gls{mews} calculations.
This would not only make the system more robust but would also be crucial for immediate medical intervention.
For healthcare providers, a monitoring platform could be developed to allow medical staff to have direct visibility into their patient's vitals and recent developments. 

Intermittent vitals monitoring presents an incomplete picture of the patient's health status.
A continuous monitoring system would not only improve the system's efficacy\cite{gronbaek_continuous_2023, shaik_remote_2023}, but could greatly enhance the sampling
frequency and obviate the need for manual patient interaction for taking measurements.

Given more development time, a range of auxiliary features could be integrated into Medwings.
Native mobile notifications, more detailed vitals analysis, and utilization of additional functionalities available in Withings devices
would elevate the system's overall capabilities and usefulness.
As technology advances, future work could explore machine learning models to predict potential health anomalies based on historical data.

\newpage
\subsection{Implications and Conclusions}

The objective of this research was to explore the feasibility and of an \gls{rwsm} system for use by outpatients, using
commercially available smart medical devices.
The Medwings system was successful in demonstrating the feasibility of such an approach.
Key operational challenges and successes were identified, providing insights for the future development
and refinement of systems in this domain.

The research found that \gls{rwsm} is feasible.
While the \Gls{mews} could not be directly applied in its unaltered form,
the use of consumer-grade smart devices in the Medwings system
successfully facilitated remote patient monitoring in combination with \Gls{ews} assessments.

Given the rapidly evolving market for advanced smart medical devices, the implications for healthcare providers are significant.
Potentially freeing up medical resources and improving patient mobility and autonomy by allowing for earlier dismissal of patients
from care facilities, systems such as Medwings may hold considerable value.

While the Medwings system has demonstrated the feasibility of \gls{rwsm},
it remains somewhat rough around the edges.
Continued research in this area is essential to enhance the robustness and effectiveness of \Gls{rwsm} systems.
The potential for life-saving interventions via automated alerts makes the case for a more robust system compelling.
Although the current system uses a web-based client-server architecture, alternative approaches should be explored.
Research into the development of a local, body-area-network system could offer improved reliability and responsiveness.

In conclusion, the research has successfully filled an important knowledge gap in the field of remote patient monitoring
with early warning scores, and outlines the scope for future advancements.
Given the continuing technological advancements in smart medical devices, the future appears promising for the adoption and refinement
of systems like Medwings for better patient care and resource optimization in healthcare.

\newpage
% TODO adjust: carry on from TOC
\pagenumbering{Roman}
\setcounter{page}{4}
\printnoidxglossary[title=Glossary, toctitle=Glossary]\label{sec:glossary}

\newpage
\printnoidxglossary[type=\acronymtype, title=Acronyms, toctitle=Acronyms]\label{sec:acronyms}

\newpage
\listoffigures
\addcontentsline{toc}{section}{List of figures}

\newpage
\listoftables
\addcontentsline{toc}{section}{List of tables}

\newpage
\listofalgorithms
\addcontentsline{toc}{section}{List of algorithms}

\newpage
\printbibliography
\addcontentsline{toc}{section}{References}

\newpage
\appendix
\includepdf[pages=1, scale=0.95, pagecommand={\vspace*{-2.75cm}\section{Trial Data}\label{apdx:trial-data}}, angle=-90]{./attachments/trial-data.pdf}
\includepdf[pages=2-, scale=0.95, pagecommand={}, angle=-90]{./attachments/trial-data.pdf}

\newpage
\section*{Ehrenwörtliche Erklärung}
\addcontentsline{toc}{section}{Ehrenwörtliche Erklärung}

Ich versichere, dass ich die beiliegende Bachelorarbeit ohne Hilfe Dritter und ohne Benutzung anderer als der angegebenen
Quellen und Hilfsmittel angefertigt und die den benutzten Quellen wörtlich oder inhaltlich entnommenen Stellen als solche
kenntlich gemacht habe.
Diese Arbeit hat in gleicher Form noch keiner Prüfungsbehörde vorgelegen.

Ich bin mir bewusst, dass eine falsche Erklärung rechtliche Folgen haben wird.


Braunschweig, 12.09.2023
\vspace{3cm}

\end{document}