Foundation-style Methods for Real-Time Statistical Dependency Measurement and Its Applications
Date:
Invited departmental seminar at HKU ECE (20 May 2026, 11:00 AM – 12:00 PM).
The talk introduces a foundation-style approach to statistical dependency measurement — using large-scale synthetic pre-training to turn classical questions such as mutual information estimation and arbitrary-dimensional dependency measurement into fast, zero-shot inferences. I cover InfoNet (ICML 2024, Oral) and its full upgrade InfoAtlas (ICML 2026), and then discuss real-time applications: latent-space anisotropy and statistical correlation as online diagnostics for Transformer training, validated on NanoGPT, ViT, and Pythia-2.8B.