This repository contains solutions and reports for the Statistical Machine Learning (SML) assignments completed as part of the course.
- Maximum Likelihood Estimation (MLE)
- Linear Discriminant Analysis (LDA)
- Quadratic Discriminant Analysis (QDA)
- Covariance estimation
- t-SNE visualization
- Gaussian discriminant classifiers
- Implemented LDA and QDA from scratch
- Compared classification performance
- Visualized class separability using t-SNE
- Analysed covariance structures of MNIST digits
- Principal Component Analysis (PCA)
- Fisher Discriminant Analysis (FDA)
- Eigenvalue decomposition
- Projection & reconstruction
- Variance preservation
- Classification in transformed feature spaces
- Implemented PCA from scratch
- Implemented FDA using generalized eigenvalue problems
- Compared FDA and PCA based classification accuracy
- Visualized transformed feature spaces
- Analysed reconstruction error
- Ridge Regression
- Lasso Regression
- Regularization paths
- PCA with regression
- MSE analysis
- Implemented Ridge Regression using closed form solution
- Used Lasso Regression for sparse feature selection
- Analysed impact of lambda on MSE
- Compared train and test performance
- Decision Trees
- Gini Index
- Bagging
- Random Forests
- Bootstrap sampling
- Out-of-bag (OOB) error
- Built decision trees from scratch
- Implemented bagging and random forests
- Compared single tree vs ensemble performance
- Analysed variance reduction using decorrelation
- Regression Trees
- Decision Stumps
- Bagging for regression
- Sum of Squared Residuals (SSR)
- Implemented regression stumps
- Compared single stump vs bagging models
- Analysed underfitting and variance reduction
- Visualized regression predictions
- AdaBoost
- Decision Stumps
- Weighted classification
- Ensemble learning
- Implemented AdaBoost from scratch
- Used weighted error minimization
- Trained ensembles of decision stumps
- Analysed validation accuracy trends
- Gradient Boosting
- Absolute loss minimization
- Pseudo residuals
- Learning rate analysis
- Implemented gradient boosting using decision stumps
- Compared different learning rates
- Analysed underfitting and overfitting behavior
- Studied convergence trends
- Perceptron Learning Algorithm
- Linear separability
- Stochastic Gradient Descent (SGD)
- Binary classification
- Implemented perceptron learning from scratch
- Studied convergence behavior on synthetic datasets
- Analysed effect of covariance on convergence
- Compared dataset separability
- Python
- NumPy
- Matplotlib
- scikit-learn
- PCA / FDA implementations from scratch
- Ensemble learning techniques
- Statistical classification models
- Most algorithms were implemented from scratch for conceptual understanding.
- PCA preprocessing was reused across multiple assignments.
- Extensive plots and analysis are included in the assignment reports.
- All code is documented with comments and explanation of methodology.