Learn how gradient boosting works: residuals, the loss gradient, learning rate and early stopping, with a hand-worked example, scikit-learn code for classification and regression, diagrams and real results.
Learn how cross-validation and K-fold work in machine learning: choosing K, stratified, repeated, group and time-series splits, data leakage, and complete scikit-learn examples with real outputs.
Learn how Support Vector Machines (SVM) work: hyperplanes, margins, support vectors, the C parameter, and the kernel trick, with a complete scikit-learn tutorial, diagrams and measured results.
Understand how the K-Nearest Neighbors (KNN) algorithm works, how to choose k and a distance metric, and how to build and tune a KNN classifier in Python with scikit-learn, with diagrams and real results.
Learn how feature scaling works, when to use normalization (Min-Max) or standardization (Z-score), and how to apply both correctly in Python with scikit-learn, with worked examples, charts, and real results on the Wine dataset.
Learn anomaly detection in machine learning: Isolation Forest, Local Outlier Factor, Python examples, evaluation challenges, and false positives.
Understand principal component analysis: variance, covariance, eigenvectors, explained variance, standardization, and PCA in Python with scikit-learn.
Learn DBSCAN clustering: epsilon, min_samples, core, border and noise points, choosing parameters with a k-distance plot, Python code, and limits.
Understand K-means clustering: centroids, assignment, initialization, the elbow method, silhouette score, limits, and full Python code from scratch.
Supervised machine learning with Python is easier to start than most beginners expect. In about 40 lines of code, you can load a dataset, explore it, split it fairly, compare two models, train the better one, and measure how well it predicts on unseen data. This tutorial walks through that entire workflow with scikit-learn, one step at a time. You will build a classifier that identifies the grape variety (cultivar) of a wine from 13 chemical measurements. Along the way, you will learn good habits that professionals use, such as keeping a hidden test set and putting preprocessing inside a pipeline so that no test information leaks into training. If the vocabulary is new, skim What Is Supervised Learning? first. Table of contents What you will build Step 1: Set up your environment Step 2: Know your Python libraries Step 3: Load the dataset Step 4: Explore the data Step 5: Split into training and test sets Step 6: Choose a model with cross-validation Step 7: Train the model and make predictions Step 8: Evaluate the results Troubleshooting common problems What to try next Frequently asked questions Conclusion What you will build The project is a classification task. The target has three classes, and each wine sample belongs to exactly one. To review the task types, see Classification vs Regression in Machine Learning. The plan has eight steps: Set up Python and install the libraries. Load the wine dataset. Explore the data. Split it into training and test sets. Compare two candidate models using cross-validation on the training data. Train the chosen model. Predict on the test set. Evaluate with accuracy, a confusion matrix, and a classification report. The dataset is small on purpose. It contains 178 samples, it ships with scikit-learn, and the code runs in seconds on any laptop. Step 1: Set up your environment Install a recent version of Python 3 from python.org or through your system’s package manager. Then create a virtual environment, which is an isolated folder of packages for one project. This keeps your tutorial libraries from clashing with other work. python -m venv .venv # Activate it on macOS or Linux: source .venv/bin/activate # Activate it on Windows (Command Prompt): .venv\Scripts\activate pip install scikit-learn pandas Check that the install worked: python -c “import sklearn, pandas; print(sklearn.__version__, pandas.__version__)” If this prints two version numbers, you are ready. You can also work inside Jupyter Notebook or Google Colab, where scikit-learn and pandas are usually available already. The scikit-learn project documents installation options on its install page, linked in the references. Step 2: Know your Python libraries This tutorial uses two main libraries: scikit-learn (imported as sklearn) provides datasets, preprocessing tools, models, and evaluation metrics. Its models share the same interface: fit() to train and predict() to make predictions. pandas provides the DataFrame, a table that makes exploration easy. Scikit-learn depends on NumPy, which pip installs automatically. If you want charts later, add matplotlib as well. The code in this tutorial does not need it. Step 3: Load the dataset The wine dataset contains chemical analysis results for wines grown in the same region of Italy by three different cultivars. Each row is one wine. The 13 feature columns include alcohol, flavanoids, and color intensity. The target column is the cultivar, labeled 0, 1, or 2. Setting as_frame=True returns the data as a pandas DataFrame. import pandas as pd from sklearn.datasets import load_wine wine = load_wine(as_frame=True) df = wine.frame print(df.shape) The next block runs the first exploration steps, so keep reading before running them separately. Step 4: Explore the data Exploratory data analysis (EDA) means looking at your data before modeling. You want to learn how big it is, whether classes are balanced, whether values are missing, and what ranges the features cover. Add these lines to the same script: print(df[“target”].value_counts().sort_index().to_dict()) print(“missing values:”, int(df.isna().sum().sum())) print(df[[“alcohol”, “flavanoids”, “color_intensity”]].describe().loc[[“mean”, “min”, “max”]].round(2)) When I ran these lines with Python 3.12, pandas 3.0.2, and scikit-learn 1.8.0, the output was: (178, 14) {0: 59, 1: 71, 2: 48} missing values: 0 alcohol flavanoids color_intensity mean 13.00 2.03 5.06 min 11.03 0.34 1.28 max 14.83 5.08 13.00 This tells you four useful things: The table has 178 rows and 14 columns: 13 features plus the target. The three classes hold 59, 71, and 48 samples. They are not perfectly equal, but none is rare, so plain accuracy is a reasonable starting metric. We will still check per-class results. There are no missing values, so we do not need an imputer. Real datasets often do need one. The features live on very different scales. Color intensity reaches 13, while flavanoids stay mostly below 5. This matters for scale-sensitive models such as logistic regression, so we will scale the features. Step 5: Split into training and test sets Now separate the features from the target and hold out a test set. The model will never see the test set until the very end. This step must come before any scaling or tuning, so no test information leaks into training. The article on training data vs testing data explains the reasoning. from sklearn.model_selection import train_test_split X = df.drop(columns=”target”) y = df[“target”] X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, stratify=y, random_state=42) Three details matter here. test_size=0.2 reserves 20% of the rows for testing. stratify=y keeps the class proportions the same in both parts. random_state=42 makes the split repeatable, so you get the same result each time. Step 6: Choose a model with cross-validation You rarely know in advance which algorithm works best, so compare a few. We will test two: logistic regression, a simple and fast linear classifier, and a random forest, a flexible ensemble described in Random Forest vs Decision Tree. The logistic regression model goes inside a pipeline with a StandardScaler. A pipeline chains steps together, and when you cross-validate it, scikit-learn refits the scaler on the training folds only. That prevents scaling information from leaking from the validation fold. The random forest does not need scaling. from sklearn.pipeline import make_pipeline
Linear regression is a supervised learning method that predicts a number by fitting a straight line (or, with several features, a flat plane) through your data. It finds the line that keeps the prediction errors as small as possible. Because the method is simple, fast, and easy to interpret, it is the standard first algorithm for anyone learning machine learning. This guide explains linear regression with examples. You will see the equation, learn what coefficients and the intercept mean, work through a tiny least squares calculation by hand, understand gradient descent, and run a complete Python example. If you want background on the task type first, read Classification vs Regression in Machine Learning. Table of contents The core idea The linear regression equation Loss function and the best-fit line Least squares: a worked example Gradient descent Assumptions of linear regression Evaluation metrics Working Python example Limitations Frequently asked questions Conclusion The core idea Suppose you want to predict a house’s price from its size. You plot many past sales, with size on the horizontal axis and price on the vertical axis. The points rise from left to right but do not sit on a perfect line. Linear regression draws the one straight line that passes as close as possible to all the points. To predict a new house, you find its size on the line and read off the price. With one feature, this is simple linear regression. With several features, such as size, bedrooms, and age, it is multiple linear regression. The idea stays the same: the prediction is a weighted sum of the features plus a starting value. The linear regression equation For one feature, the model is: ŷ = β₀ + β₁x Here is what each symbol means: ŷ (read “y-hat”) is the predicted value of the target. x is the feature, for example the house size. β₁ is the coefficient (also called the slope or weight). It tells you how much the prediction changes when x increases by one unit. β₀ is the intercept. It is the prediction when x equals zero. For several features, you add one coefficient per feature: ŷ = β₀ + β₁x₁ + β₂x₂ + … + βₚxₚ Each coefficient describes the effect of its feature while the other features stay fixed. If β₁ equals 1.5 for house size in square meters, then each extra square meter adds 1.5 units to the predicted price, assuming the other features do not change. Be careful with the word “effect.” The model describes an association in the data. It does not prove that changing the feature would cause the target to change. The intercept sometimes has no practical meaning. A house of zero square meters does not exist, so the intercept simply positions the line. Loss function and the best-fit line Infinitely many lines pass through a cloud of points. To choose the best one, we need a way to score each line. That score is the loss function, and linear regression usually uses the sum or mean of squared errors. For each training example, the residual is the difference between the true value and the predicted value: residual = y − ŷ. Squaring the residuals makes every error positive, so errors above and below the line cannot cancel each other out. Squaring also punishes large errors more than small ones. The mean squared error is: MSE = (1/n) × Σ (yᵢ − ŷᵢ)² Here n is the number of examples, and Σ means “add up over all examples.” The best-fit line is the one with the smallest MSE. This approach is called ordinary least squares (OLS). Least squares: a worked example You can find the best line with a short calculation. Take three points: (1, 2), (2, 4), and (3, 5), where the first number is x and the second is y. Compute the means: x̄ = 2 and ȳ = 11/3 ≈ 3.667. Compute the slope with β₁ = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)². The numerator is (−1)(−1.667) + (0)(0.333) + (1)(1.333) = 3. The denominator is 1 + 0 + 1 = 2. So β₁ = 1.5. Compute the intercept with β₀ = ȳ − β₁x̄ = 3.667 − 1.5 × 2 ≈ 0.667. The fitted line is ŷ = 0.667 + 1.5x. Its predictions for the three points are about 2.167, 3.667, and 5.167. The residuals are about −0.167, 0.333, and −0.167, and the sum of their squares is about 0.167. No other straight line gives a smaller sum for these three points. For large datasets and many features, software solves the same problem with matrix algebra. Scikit-learn’s LinearRegression does exactly that, so you rarely calculate it yourself. Gradient descent The least squares formula gives the answer in one step for linear regression. However, many other models have no such formula, so machine learning often uses gradient descent, an iterative method. Linear regression is the classic way to learn it. Picture the loss as a hilly landscape, where each position is a pair of values (β₀, β₁) and the height is the MSE. You start at a random position and take small steps downhill until you reach the bottom. At each step, you: Calculate the gradient, which points in the direction where the loss grows fastest. Move each parameter a small step in the opposite direction. The update rule is β ← β − α × (gradient). The symbol α is the learning rate, which controls the step size. If α is too small, training is slow. If α is too large, the steps overshoot and the loss may grow instead of shrink. For linear regression with squared error, the loss surface is bowl-shaped, so gradient descent heads toward a single global minimum rather than getting stuck in local dips. Gradient descent matters most for huge datasets, where a direct solution is slow, and for models with no direct solution. In scikit-learn, the SGDRegressor class uses a stochastic version. Features should be scaled before using it
