Projects | Patrick Langer

TSXAI - Explainable Deep Learning Time-Series Extrinsic Regression Methods in Health

Mon, 01 Jul 2024 00:00:00 +0000

Coming soon! Publication in preparation.

Alex - Design, Development and Evaluation of a Digital Health Assistant for Paediatric Asthma

Sat, 01 Jun 2024 00:00:00 +0000

Abstract

Poor adherence to medication and insufficient monitoring are key factors that contribute to inadequate asthma control in children and adolescents. This project aims to improve asthma control in adolescents using a smartphone-based digital health assistant (DHA) designed for remote disease monitoring and patient coaching. This novel, age‐specific longitudinal tele-monitoring system will measure lung function and inflammation, environmental exposures, and medication adherence. Through digital and AI-based approaches to passively assess the patient’s health status, this technology will minimise the patient’s burden during the process of disease monitoring. Finally, the project will assess the feasibility and scalability of the DHA “Alex” for paediatric asthma monitoring in the socio-economic setting characteristic of low- and middle-income countries.

CROCO - Using Commercial Wearables for Circadian Rhythm Home Monitoring

Sat, 01 Jun 2024 00:00:00 +0000

Publications

Fan Wu, Patrick Langer, Jinjoo Shim, Elgar Fleisch, Filipe Barata (2024). Comparative Efficacy of Commercial Wearables for Circadian Rhythm Home Monitoring from Activity, Heart Rate, and Core Body Temperature. (under review)

Summary

We conducted a study with 36 participants during which we collected circadian rhythm related data (physical activity and core body temperature) over 14 days. We used commercially available devices like Samsung Smartwatches and CORE body temperature sensors and compared collected data to the clinical grade activity trackers (actigraph). Data collection was done using our CLAID framework.

Abstract

Circadian rhythms govern biological patterns that follow a 24-hour cycle. Dysfunctions in circadian rhythms can contribute to various health problems, such as sleep disorders. Current circadian rhythm assessment methods, often invasive or subjective, limit circadian rhythm monitoring to laboratories. Hence, this study aims to investigate scalable consumer-centric wearables for circadian rhythm monitoring outside traditional laboratories. In a two-week longitudinal study conducted in real-world settings, 36 participants wore an Actigraph, a smartwatch, and a core body temperature sensor to collect activity, temperature, and heart rate data. We evaluated circadian rhythms calculated from commercial wearables by comparing them with circadian rhythm reference measures, i.e., Actigraph activities and chronotype questionnaire scores. The circadian rhythm metric acrophases, determined from commercial wearables using activity, heart rate, and temperature data, significantly correlated with the acrophase derived from Actigraph activities (r=0.96, r=0.87, r=0.79; all p<0.001) and chronotype questionnaire (r=-0.66, r=-0.73, r=-0.61; all p<0.001). The acrophases obtained concurrently from consumer sensors significantly predicted the chronotype (R2=0.64; p<0.001). Our study validates commercial sensors for circadian rhythm assessment, highlighting their potential to support maintaining healthy rhythms and provide scalable and timely health monitoring in real-life scenarios.

CLAID - Closing the Loop on AI & Data Collection

Wed, 15 May 2024 00:00:00 +0000

Publications

Patrick Langer, Stephan Altmüller, Elgar Fleisch, Filipe Barata, CLAID (2024). Closing the loop on AI & data collection—A cross-platform transparent computing middleware framework for smart edge-cloud and digital biomarker applications. Future Generation Computer Systems, 2024, ISSN 0167-739X, https://doi.org/10.1016/j.future.2024.05.026
Fan Wu, Patrick Langer, Jinjoo Shim, Elgar Fleisch, Filipe Barata (2024). Comparative Efficacy of Commercial Wearables for Circadian Rhythm Home Monitoring from Activity, Heart Rate, and Core Body Temperature. (under review)

Summary

CLAID is an open-source framework for deploying machine learning an data collection applications across cloud, edge, and mobile devices. The framework enables to build applications from simple configuration files and comes with various ready-to-use Modules. It provides bindings for many different programming languages including C++, Dart, Java, Python and Objective-C and supports Linux, macOS, Windows, Android, WearOS and iOS.

More information: CLAID Website

Abstract

The increasing number of edge devices with enhanced sensing capabilities, such as smartphones, wearables, and IoT devices equipped with sensors, holds the potential for innovative smart-edge applications in healthcare. These devices generate vast amounts of multimodal data, enabling the implementation of digital biomarkers which can be leveraged by machine learning solutions to derive insights, predict health risks, and allow personalized interventions. Training these models requires collecting data from edge devices and aggregating it in the cloud. To validate and verify those models, it is essential to utilize them in real-world scenarios and subject them to testing using data from diverse cohorts. Since some models are too computationally expensive to be run on edge devices directly, a collaborative framework between the edge and cloud becomes necessary. In this paper, we present CLAID, an open-source cross-platform middleware framework based on transparent computing compatible with Android, iOS, WearOS, Linux, macOS, and Windows. CLAID enables logical integration of devices running different operating systems into an edge-cloud system, facilitating communication and offloading between them, with bindings available in different programming languages. We provide Modules for data collection from various sensors as well as for the deployment of machine-learning models. Furthermore, we propose a novel methodology, **ML-Model in the Loop** for verifying deployed machine learning models, which helps to analyze problems that may occur during the migration of models from cloud to edge devices. We verify our framework in three different experiments and achieve 100% sampling coverage for data collection across different sensors as well as an equal performance of a cough detection model deployed on both Android and iOS devices. Additionally, we compare the memory and battery consumption of our framework across the two mobile operating systems.

Open Source AI-enabled Smart Inhaler for Asthmatic Patients

Fri, 15 Mar 2024 00:00:00 +0000

More details, source code and publication coming soon!

Visual Scene Change Detection from RGB Images

Thu, 01 Jun 2023 00:00:00 +0000

Implemented Outdoor Visual Scene Change Detection (VSCD) using state of the art approaches for change detection (e.g., DR-TANet). A major challenge here is the lack of extensive datasets for outdoor change detection. Available datasets like VL-CMU-CD and PSCD only offer a few thousand samples (most of them are duplicates or generated from other samples via data augmentation strategies). Indoor datasets like ChangeSim, however, offer several 100k images. Major improvements to outdoor scene change detection were achieved by pretraining on ChangeSim and then testing on outdoor datasets. Concrete project details under NDA.

Realtime-capable Human Activity Recognition

Thu, 16 Mar 2023 00:00:00 +0000

Implemented a realtime video analysis pipeline for Human Activity Recognition (HAR) using deep learning models. A major challenge was the complexity of modern deep learning approaches for HAR, which were not realtime capable. Several improvements have been made to SlowFast and PoseC3D (two state of the art models at that time) to optimize computations and speed up inference time. Concrete project details under NDA.

Grand Theft Data Five

Sun, 01 Jan 2023 00:00:00 +0000

Source code, documentation and datasets to be published at a later point!

FPGA-Based Realtime Detection of Freezing of Gait in Parkinson Patients Using Neural Networks

Tue, 28 Sep 2021 00:00:00 +0000

Publications

Patrick Langer, Ali Haddadi Esfahani, Zoya Dyka, Peter Langendorfer (2022). FPGA-Based Realtime Detection of Freezing of Gait of Parkinson Patients. EAI Bodynets

Summary

This work was done as part of my master's thesis, and evantually published in EAI bodynets. Freezing of Gait is a condition in which patients freeze during their movement, potentially leading to falls. During my thesis, I investigated and trained various neural networks for time-series classification tasks, such as Temporal Convulational Networks (TCNs), Time-Delay Neural Networks (TDNNs) and Long-Short Term Memory Networks (LSTMs). The goal is to release a cueing signal based on audio or electrical feedback to release the freezing upon it's detection. Due to realtime requirements (cueing needs to be issued within 20ms), the neural network was implemented on an FPGA using VitisAI.

Abstract

In our paper we report on our implementation of a temporal convolutional network trained to detect Freezing of Gait on an FPGA. In order to be able to compare our results with state of the art solutions we used the well-known open dataset Daphnet. Our most important findings are even though we used a tool to map the trained model to the FPGA we can detect FoG in less than a millisecond which will give us sufficient time to trigger cueing and by that prevent the patient from falling. In addition, the average sensitivity achieved by our implementation is comparable to solutions running on high end devices.

Acceleration of CNNs for Person Detection in Autonomous Driving using FPGA's and SIMD

Fri, 01 Jan 2021 00:00:00 +0000

Optimized a proprietary FPGA hardware accelerator implementation for Convolutional Neural Networks and computer vision algorithms on a Xilinx UltraZED SoC platform for Person Detection and Segmentation from RGB videos in the context of autonomous driving.

Reduced combined CPU pre- and postprocessing times from 800ms per frame to 55ms per frame by speeding up calculations using SIMD (Arm Neon)
Reduced inference time of the FPGA hardware accelerator (written in HLS) from 240ms to 140ms per frame by redesigning the accelerator from the ground up in order to use an optimized memory layout and achieve better resource utilization (increased utilization of DSP-Units from 70% to 95%)
Reduced total processing time per frame from 1040ms to 195ms.

Efficient Semantic 3D Mapping for Indoor Environments

Sat, 26 Sep 2020 00:00:00 +0000

Publications

Daniel Seichter, Patrick Langer, Tim Wengefeld, Benjamin Lewandowski, Dominik Höchemer and H. -M. Gross (2022). Efficient and Robust Semantic Mapping for Indoor Environments. 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022, pp. 9221-9227, doi: 10.1109/ICRA46639.2022.9812205

Summary

This project, conducted as part of my bachelor's thesis and later published at ICRA, involved implementing state-of-the-art semantic 3D maps using Normal Distribution Transform (NDT). We projected class labels, predicted by a semantic segmentation model, onto individual 3D points to create class histograms. Alongside the semantic 3D NDT maps, we also implemented 3D semantic voxel maps for comparison. We evaluated the accuracy of these semantic mapping approaches by projecting the 3D map back to a 2D image and comparing it with the ground truth segmentation.

Semantic NDT Map

Abstract

A key proficiency an autonomous mobile robot must have to perform high-level tasks is a strong understanding of its environment. This involves information about what types of objects are present, where they are, what their spatial extend is, and how they can be reached, i.e., information about free space is also crucial. Semantic maps are a powerful instrument providing such information. However, applying semantic segmentation and building 3D maps with high spatial resolution is challenging given limited resources on mobile robots. In this paper, we incorporate semantic information into efficient occupancy normal distribution transform (NDT) maps to enable real-time semantic mapping on mobile robots. On the publicly available dataset Hypersim, we show that, due to their sub-voxel accuracy, semantic NDT maps are superior to other approaches. We compare them to the recent state-of-the-art approach based on voxels and semantic Bayesian spatial kernel inference (S-BKI) and to an optimized version of it derived in this paper. The proposed semantic NDT maps can represent semantics to the same level of detail, while mapping is 2.7 to 17.5 times faster. For the same grid resolution, they perform significantly better, while mapping is up to more than 5 times faster. Finally, we prove the real-world applicability of semantic NDT maps with qualitative results in a domestic application.

Signapse - Realtime Road Sign Recognition on Smartphones

Sun, 01 Sep 2019 00:00:00 +0000

Detailed description and source code to follow soon!

Biopotential based Human-Machine-Interaction

Sun, 01 May 2016 00:00:00 +0000

For the prestigious, nation-wide youth science competition “Jugend forscht” in Germany, I designed and built from the ground up a device that was able to record and analyze electrical muscle activity using electromyography. The device can be attached via 3 electrodes (bipolar ablation) to a person’s arm, enabling to write texts or play games on a computer. The design involved designing a signal amplifier, low-pass filter and 3rd order Tschebyscheff low-pass filter. The conversion of the analog signal for the PC was done via an arduino. The analysis of the signal involved basic machine learning to translate muscle activity to control commands. Check official website for more details