Multimodal Alignment for Synthetic Clinical Time Series.

Key findings

1. Conditional MMD improves with complexity. Conditioned on gender, distance to real data drops from 0.351 (AR-only) to 0.328 (ridge) to 0.304 (GBM). Statistical similarity follows model complexity in the expected direction.
2. Clinical rules move the wrong way. Table 3 reports rule-violation rates: the share of fever hours (temperature above 38.3 °C) without tachycardia (heart rate above 90). Real ICU data breaks the rule 33% of the time, so fever comes with tachycardia in about two fever hours in three. The models break it more often, AR-only 57%, ridge 69%, GBM 69%, so they under-produce the pair; in the paper’s words the relationship “remained mostly unlearned”. For the stricter high-fever rule the violation rates are 37% in the real data against 60%, 89% and 96%. The training data carries a strong fever-to-HR signal (mean heart rate 99.1 bpm with fever against 84.3 bpm without, Cohen’s d = 0.84), so the failure is not from missing signal.
3. Reordering helps per-feature metrics, not cross-feature ones. Moving temperature ahead of heart rate substantially improves per-feature trend consistency for temperature (ridge: +0.45 to +0.75), but leaves the fever-to-tachycardia rule essentially unchanged (ridge: 0.689 to 0.697). Conditional MMD is also nearly invariant to ordering.
4. The pattern points to architecture. The chosen autoregressive conditional-mean class, with its AR(1) residual, captures marginals well but consistently misses joint conditional structure across modalities. This is suggestive, not proof. Three model variants and three orderings were tested on a single dataset.
5. Future work, not yet tested. Causal-discovery-driven orderings, and joint per-timestep generation, are the directions we point to. Neither is evaluated here.

Why it matters

Synthetic clinical time series are increasingly used to substitute for real records that cannot be shared. To be useful for any downstream clinical task they need to look real in the right ways. Statistical resemblance at the feature level is not enough. The relationships between features are where clinical meaning lives.

Three autoregressive conditional-mean models, each at a different complexity, all break the fever-to-tachycardia rule more often than the real data does. The two more complex models break it more often than the AR-only baseline (69% against 57%), so added capacity does not narrow the gap. The signal is there in the training data. The class of models tested cannot use it to predict the joint. We do not claim this is true of every autoregressive model. We do suggest that without changing the inductive bias, more capacity will not help.

Scope and limitations

A single dataset (PhysioNet/CinC Challenge 2019). One conditioning variable for MMD (gender). Ten manually defined clinical rules. Three feature orderings. No comparison to deep generative baselines such as GANs, VAEs or diffusion. No downstream prediction task is evaluated. Findings are framed as patterns and suggestions, not proofs.