Fusing ICU Time Series with TabPFN-Based Clinical Context

Co-Supervised by: Nico Bigler

If you are interested in this topic or have further questions, do not hesitate to contact daniel.bigler@students.unibe.ch.

Background / Context

ICU risk prediction can combine evolving physiology with tabular clinical context. This project investigates whether a TabPFN-derived representation adds predictive value beyond a conventional context branch. HiRID contains more than 33,000 ICU admissions and longitudinal physiological, laboratory and treatment data. Its static context is limited: age and sex can be supplemented with explicitly defined early measurements, but a comprehensive comorbidity profile should not be assumed. The project builds on the HiRID-ICU-Benchmark and requires credentialed PhysioNet access.

Research Question(s) / Goals

The research aims to evaluate the incremental value of TabPFN-based context by:

Comparing temporal-only, tabular-only and combined prediction models
Testing whether TabPFN improves fusion relative to an MLP context branch
Examining the value of temporal history while keeping the prediction landmark fixed
Evaluating discrimination, calibration and computational feasibility

Approach / Methods

The student will:

Use HiRID v1.1.1 and a pinned benchmark pipeline; audit eligible admissions, events, missingness and compute requirements before modelling
Initially predict circulatory failure in the next 12 hours at a 6-hour ICU landmark, excluding current failure and applying explicit rules for incomplete outcome observation
Combine age, sex and selected first-2-hour laboratory/treatment summaries with an LSTM physiological branch; exclude APACHE group unless its timely availability is established
Compare logistic regression, LightGBM and TabPFN baselines with LSTM-only, LSTM + MLP and LSTM + TabPFN; include LightGBM on temporal summaries
Generate local TabPFN training embeddings out of fold and restrict context to training data; keep all observations from each admission together and prevent future-information leakage
Evaluate AUPRC, AUROC, Brier score and calibration with admission-level uncertainty estimates; document that repeated stays of one person cannot be linked in HiRID
Use out-of-fold probability fusion as a stated alternative if embedding extraction exceeds the available compute budget

Expected Contributions / Outcomes

A reproducible comparison of tabular foundation-model representations and conventional clinical-context branches
Evidence on whether fusion adds useful information, including a valid negative result
A documented cohort, leakage controls and an analysis of performance versus computational cost

Required Skills / Prerequisites

Python and machine learning; PyTorch and basic sequence modelling for the full Master project
Interest in clinical time series and careful evaluation; clinical supervision for label review
For a Bachelor thesis, a prepared authorized landmark dataset and a reduced comparison of tabular models; neural fusion is outside the core scope

Possible Extensions

A Transformer temporal encoder or an additional prediction landmark
Respiratory-failure prediction after completing the circulatory task
External validation after confirming compatible variables and outcome definitions