InfoNet: Neural Estimation of Mutual Information without Test-time Optimization

We propose InfoNet, a Transformer-based model that, after a single round of supervised pre-training on large-scale synthetic distributions, estimates the mutual information between arbitrary 1-D variable pairs (X, Y) in a zero-shot manner — no per-task optimization is required at inference time. InfoNet runs roughly 100x faster than prior neural MI estimators while matching or slightly exceeding their accuracy, and serves as a general-purpose building block for tasks that depend on dependency measurement.

Authors: Zhengyang Hu, Song Kang, Qunsong Zeng, Kaibin Huang, Yanchao Yang.