Skip to content

Formalize the Leshno universal approximation theorem - #246

Open
yuanyi-350 wants to merge 9 commits into
LeanMachineLearning:mainfrom
yuanyi-350:formalize-llps-universal-approximation
Open

yuanyi-350 wants to merge 9 commits into
LeanMachineLearning:mainfrom
yuanyi-350:formalize-llps-universal-approximation

Conversation

@yuanyi-350

@yuanyi-350 yuanyi-350 commented Sep 12, 2026 •

Copy link
Copy Markdown

Formalizes the continuous Leshno–Lin–Pinkus–Schocken theorem: for every positive input dimension and continuous activation σ, biased single-hidden-layer networks are uniformly dense on every compact set if and only if σ is not a polynomial.

Roadmap

All files below are in LeanMachineLearning/NeuralNetwork/UniversalApproximation/ , Here, universal means uniformly dense on every compact set.

  • Leshno.lean — main theorem: on a nontrivial real inner-product space, a continuous activation is universal if and
    only if it is nonpolynomial.
  • Nonpolynomial.lean: every continuous nonpolynomial activation is universal.
  • PolynomialObstruction.lean: for every polynomial activation on a nontrivial real inner-product space, there exists a
    finite compact set on which its networks are not dense.
  • Discriminatory.lean: universality is equivalent to the following dual criterion: on every compact K, any continuous
    linear functional on C(K, ℝ) vanishing on all neurons is zero. In particular, a smooth activation is universal if
    none of its derivatives is identically zero.
  • Convolution.lean: if φ ∗ σ is universal for some compactly supported continuous kernel φ, then σ is universal.

The proof also establishes the equivalence for every nontrivial real inner-product space.

AI usage: I planned the blueprint in advance and used GPT-5.6 Sol to help implement the code according to that blueprint.

Closes #244.

@yuanyi-350
yuanyi-350 force-pushed the formalize-llps-universal-approximation branch 2 times, most recently from e31a1c9 to 182353e Compare September 12, 2026 03:33
@yuanyi-350
yuanyi-350 force-pushed the formalize-llps-universal-approximation branch from 182353e to 51d6faa Compare September 12, 2026 03:41
mathlib-bors Bot pushed a commit to leanprover-community/mathlib4 that referenced this pull request Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: Formalize the Leshno–Lin–Pinkus–Schocken universal approximation theorem

1 participant