1

We study policy learning with the ability to abstain, and demonstrate its usefulness in ensuring safety.

We consider structure-agnostic causal inference that estimates treatment effect estimation using black-box ML estimates of nuisance functions, and show that the celebrated DML is optimal when the treatment noise is Gaussian. When the noise is non-Gaussian, we propose ACE, a novel class of higher-order structure-agnostic estimators.

We build a novel benchmark for evaluating how well can state-of-the-art LLMs solve inequality proving problems. Our evaluation studies provide insights on their advanced math reasoning capabilities and highlight their common reasoning flaws.

We show that first-order debiasing of black-box ML estimators is optimal for estimating average treatment effect.

We study the best-achievable identification guarantees and provable identification algorithms for causal representation learning when hard interventions are not available.

We investigate the “grokking” phenomenon in deep learning on some simple setups, and show that it is caused by a dichotomy of the implicit biases between the early phase and late phase during training.

We prove that GD applied to the matrix sensing problem has intriguing properties – with small initialization and early stopping, it follows an incremental/greedy low-rank learning procedure. This form of simplicity bias allows GD to recover the ground-truth, despite over-parameterization and non-convexity.

We consider the problem of learning a linear operator between Sobolev RKHSs from noisy data. Different from its finite-dimensional counterpart where regularized least squares is optimal, we prove that estimators with a certain multilevel structure is necessary (and sufficient) to achieve optimality.

We provide theoretical evidence that the hardness of robust generalization may stem from the expressive power of deep neural networks. Even when standard generalization is easy, robust generalization provably requires the size of DNNs to be exponentially large.

We provide an improved analysis of the convergence rates of clipping algorithms, theoretically justifying their superior performance in deep learning.