New Equivalences Between Interpolation and SVMs: Kernels and Structured Features

[2305.02304] New Equivalences Between Interpolation and SVMs: Kernels and Structured Features Skip to main content Search Submit Donate Log in Search arXiv Press Enter to search · Advanced search Statistics > Machine Learning arXiv:2305.02304 (stat) [Submitted on 3 May 2023] Title: New Equivalences Between Interpolation and SVMs: Kernels and Structured Features Authors: Chiraag Kaushik , Andrew D. McRae , Mark A. Davenport , Vidya Muthukumar View a PDF of the paper titled New Equivalences Between Interpolation and SVMs: Kernels and Structured Features, by Chiraag Kaushik and 3 other authors View PDF HTML (experimental) Abstract: The support vector machine (SVM) is a supervised learning algorithm that finds a maximum-margin linear classifier, often after mapping the data to a high-dimensional feature space via the kernel trick. Recent work has demonstrated that in certain sufficiently overparameterized settings, the SVM decision function coincides exactly with the minimum-norm label interpolant. This phenomenon of support vector proliferation (SVP) is especially interesting because it allows us to understand SVM performance by leveraging recent analyses of harmless interpolation in linear and kernel models. However, previous work on SVP has made restrictive assumptions on the data/feature distribution and spectrum. In this paper, we present a new and flexible analysis framework for proving SVP in an arbitrary reproducing kernel Hilbert space with a flexible class of generative models for the labels. We present conditions for SVP for features in the families of general bounded orthonormal systems (e.g. Fourier features) and independent sub-Gaussian features. In both cases, we show that SVP occurs in many interesting settings not covered by prior work, and we leverage these results to prove novel generalization results for kernel SVM classification. Comments: 23 pages, 2 figures Subjects: Machine Learning (stat.ML) ; Machine Learning (cs.LG) MSC classes: 62H30, 62J07, 68Q32, 46E22 Cite as: arXiv:2305.02304 [stat.ML] (or arXiv:2305.02304v1 [stat.ML] for this version) https://doi.org/10.48550/arXiv.2305.02304 Focus to learn more arXiv-issued DOI via DataCite Related DOI : https://doi.org/10.1137/23m1568764 Focus to learn more DOI(s) linking to related resources Submission history From: Chiraag Kaushik [ view email ] [v1] Wed, 3 May 2023 17:52:40 UTC (324 KB) Full-text links: Access Paper: View a PDF of the paper titled New Equivalences Between Interpolation and SVMs: Kernels and Structured Features, by Chiraag Kaushik and 3 other authors View PDF HTML (experimental) TeX Source view license Current browse context: stat.ML < prev | next > new | recent | 2023-05 Change to browse by: cs cs.LG stat References & Citations NASA ADS Google Scholar Semantic Scholar export BibTeX citation Loading... BibTeX formatted citation × loading... Data provided by: Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer ( What is the Explorer? ) Connected Papers Toggle Connected Papers ( What is Connected Papers? ) Litmaps Toggle Litmaps ( What is Litmaps? ) scite.ai Toggle scite Smart Citations ( What are Smart Citations? ) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv ( What is alphaXiv? ) Links to Code Toggle CatalyzeX Code Finder for Papers ( What is CatalyzeX? ) DagsHub Toggle DagsHub ( What is DagsHub? ) GotitPub Toggle Gotit.pub ( What is GotitPub? ) Huggingface Toggle Hugging Face ( What is Huggingface? ) ScienceCast Toggle ScienceCast ( What is ScienceCast? ) Demos Demos Replicate Toggle Replicate ( What is Replicate? ) Spaces Toggle Hugging Face Spaces ( What is Spaces? ) Spaces Toggle TXYZ.AI ( What is TXYZ.AI? ) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower ( What are Influence Flowers? ) Core recommender toggle CORE Recommender ( What is CORE? ) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs . Which authors of this paper are endorsers? | Disable MathJax ( What is MathJax? ) We gratefully acknowledge support from our major funders , member institutions , , and all contributors. About · Help · Contact · Subscribe · Copyright · Privacy · Accessibility · Operational Status (opens in new tab) Major funding support from

Related documents

• Rotational Equivariance in Machine Learning: A Comprehensive Tutorial#622518
• Gaussian Approximation for High-Dimensional Weighted and Two-Sample $U$-statistics with Size-Dependent Kernels#1105935
• Grokking through the Lens of Minimum-Norm Interpolation#1143054
• Riemannian Simultaneous Inference for Tangent Vector Field Regression#1036299
• Riemannian Simultaneous Inference for Tangent Vector Field Regression#1006848