Machine Learning

Supervised learns from labeled examples, unsupervised finds structure without labels, and RL learns from trial-and-error rewards.

Regression predicts a number from features — a line, a curve, or a tree — scored by how far predictions miss, usually with MSE.

Classification assigns a discrete label — spam or not, cat or dog — by learning a decision boundary from labeled examples.

Clustering groups unlabeled examples by similarity. k-means is the starter method; k is a choice, and clusters are not classes.

Overfitting memorizes training noise; underfitting is too simple. Watch train vs val curves — that is how later evaluation makes sense.

A decision tree asks yes/no questions to split tabular data. Depth buys fit and interpretability — and quickly overfits.

A random forest averages many trees trained on bootstrap samples and random features, so the ensemble beats one overfit tree.

Gradient boosting adds trees in sequence to fix leftover errors. XGBoost and LightGBM still default for tabular; use deep learning for images and language.

Accuracy lies under imbalance. Precision, recall, F1, and ROC-AUC measure different mistakes — pick the one that matches the cost.

Cross-validation rotates which slice is the test set so one lucky split cannot lie. Watch leakage; time series needs a forward split.