Lecture 20: CS217 | SVM: Hard/Soft Margins, Slack Variables & Outlier Handling | AI-ML | IITB 2025

Welcome to Lecture 20 of the CS217: AI-ML Course by IIT Bombay. This lecture, delivered by Nihar Ranjan Sahoo (a final-year PhD student), continues our exploration of Support Vector Machines (SVMs) introduced in the previous class, focusing on the mathematical foundations, optimization techniques, and practical considerations for handling real-world data challenges.

Topics Covered:

Hard Margin SVM: Mathematical formulation for perfectly separable data, maximizing the margin by minimizing ‖w‖² under strict constraints.
Soft Margin & Slack Variables: Introduction to "slack" terms (ξ) that allow margin violations; how penalizing these violations helps accommodate noisy or non-separable data.
Outlier Management: Discussion on how outliers can shift the decision boundary and why "soft margin" approaches offer more robustness.
Geometry of SVM: Understanding perpendicular distance to the hyperplane, the role of the normal vector (w), and why maximizing the margin reduces overfitting.
Constraint Optimization: Translating the classification objective into an optimization problem that balances margin maximization with controlled margin violations.
Parameter Selection: Understanding the role of the C parameter in controlling the trade-off between margin width and misclassification error.
Optimization Techniques: Introduction to Lagrangian methods and the conversion from primal to dual formulations for solving SVM optimization problems.



This lecture is part of the CS217 course taught by Prof. Pushpak Bhattacharya and builds on fundamental machine learning principles by providing rigorous mathematical formulations for SVMs, preparing students for both theoretical understanding and practical implementation. The soft margin SVM discussion is particularly relevant for handling real-world, noisy datasets where perfect linear separation is rarely possible.

#artificialintelligence #machinelearning #iitbombay #supportvectormachines #svm #cs217 #aicourse #maximummargin #hyperplanes #optimizationtechniques #softmargin #hardmargin #lagrangian #dualformulation #slackvariables #machinelearningtheory #aiml #iitb #computerscience #classifiers #computerscience #hyperplane #marginoptimization #outliers #iitlecture #chatgpt #deepseek #anthropic