(09/26/20) NeurIPS paper: High-dimensional perceptrons: Approaching Bayes error with convex optimization

September 26, 2020
In our paper to appear at this year's NeurIPS, we consider a supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer neural network with random i.i.d inputs. We prove a formula for the generalization error achieved by l2 regularized classifiers that minimize a convex loss. We also design an optimal loss and regularizer that provably leads to Bayes-optimal generalization error.