Build Your First Supervised Machine Learning Model with Python

Supervised machine learning with Python is easier to start than most beginners expect. In about 40 lines of code, you can load a dataset, explore it, split it fairly, compare two models, train the better one, and measure how well it predicts on unseen data. This tutorial walks through that entire workflow with scikit-learn, one step at a time.

You will build a classifier that identifies the grape variety (cultivar) of a wine from 13 chemical measurements. Along the way, you will learn good habits that professionals use, such as keeping a hidden test set and putting preprocessing inside a pipeline so that no test information leaks into training. If the vocabulary is new, skim What Is Supervised Learning? first.

What you will build

The project is a classification task. The target has three classes, and each wine sample belongs to exactly one. To review the task types, see Classification vs Regression in Machine Learning. The plan has eight steps:

1. Set up Python and install the libraries.
2. Load the wine dataset.
3. Explore the data.
4. Split it into training and test sets.
5. Compare two candidate models using cross-validation on the training data.
6. Train the chosen model.
7. Predict on the test set.
8. Evaluate with accuracy, a confusion matrix, and a classification report.

The dataset is small on purpose. It contains 178 samples, it ships with scikit-learn, and the code runs in seconds on any laptop.

Step 1: Set up your environment

Install a recent version of Python 3 from python.org or through your system’s package manager. Then create a virtual environment, which is an isolated folder of packages for one project. This keeps your tutorial libraries from clashing with other work.

python -m venv .venv

# Activate it on macOS or Linux:
source .venv/bin/activate
# Activate it on Windows (Command Prompt):
.venv\Scripts\activate

pip install scikit-learn pandas

Check that the install worked:

python -c "import sklearn, pandas; print(sklearn.__version__, pandas.__version__)"

If this prints two version numbers, you are ready. You can also work inside Jupyter Notebook or Google Colab, where scikit-learn and pandas are usually available already. The scikit-learn project documents installation options on its install page, linked in the references.

Step 2: Know your Python libraries

This tutorial uses two main libraries:

• scikit-learn (imported as sklearn) provides datasets, preprocessing tools, models, and evaluation metrics. Its models share the same interface: fit() to train and predict() to make predictions.
• pandas provides the DataFrame, a table that makes exploration easy.

Scikit-learn depends on NumPy, which pip installs automatically. If you want charts later, add matplotlib as well. The code in this tutorial does not need it.

Step 3: Load the dataset

The wine dataset contains chemical analysis results for wines grown in the same region of Italy by three different cultivars. Each row is one wine. The 13 feature columns include alcohol, flavanoids, and color intensity. The target column is the cultivar, labeled 0, 1, or 2. Setting as_frame=True returns the data as a pandas DataFrame.

import pandas as pd
from sklearn.datasets import load_wine

wine = load_wine(as_frame=True)
df = wine.frame
print(df.shape)

The next block runs the first exploration steps, so keep reading before running them separately.

Step 4: Explore the data

Exploratory data analysis (EDA) means looking at your data before modeling. You want to learn how big it is, whether classes are balanced, whether values are missing, and what ranges the features cover. Add these lines to the same script:

print(df["target"].value_counts().sort_index().to_dict())
print("missing values:", int(df.isna().sum().sum()))
print(df[["alcohol", "flavanoids", "color_intensity"]].describe().loc[["mean", "min", "max"]].round(2))

When I ran these lines with Python 3.12, pandas 3.0.2, and scikit-learn 1.8.0, the output was:

(178, 14)
{0: 59, 1: 71, 2: 48}
missing values: 0
      alcohol  flavanoids  color_intensity
mean    13.00        2.03             5.06
min     11.03        0.34             1.28
max     14.83        5.08            13.00

This tells you four useful things:

• The table has 178 rows and 14 columns: 13 features plus the target.
• The three classes hold 59, 71, and 48 samples. They are not perfectly equal, but none is rare, so plain accuracy is a reasonable starting metric. We will still check per-class results.
• There are no missing values, so we do not need an imputer. Real datasets often do need one.
• The features live on very different scales. Color intensity reaches 13, while flavanoids stay mostly below 5. This matters for scale-sensitive models such as logistic regression, so we will scale the features.

Step 5: Split into training and test sets

Now separate the features from the target and hold out a test set. The model will never see the test set until the very end. This step must come before any scaling or tuning, so no test information leaks into training. The article on training data vs testing data explains the reasoning.

from sklearn.model_selection import train_test_split

X = df.drop(columns="target")
y = df["target"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42)

Three details matter here. test_size=0.2 reserves 20% of the rows for testing. stratify=y keeps the class proportions the same in both parts. random_state=42 makes the split repeatable, so you get the same result each time.

Step 6: Choose a model with cross-validation

You rarely know in advance which algorithm works best, so compare a few. We will test two: logistic regression, a simple and fast linear classifier, and a random forest, a flexible ensemble described in Random Forest vs Decision Tree.

The logistic regression model goes inside a pipeline with a StandardScaler. A pipeline chains steps together, and when you cross-validate it, scikit-learn refits the scaler on the training folds only. That prevents scaling information from leaking from the validation fold. The random forest does not need scaling.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

candidates = {
    "Logistic regression": make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
    "Random forest": RandomForestClassifier(n_estimators=200, random_state=42),
}
for name, m in candidates.items():
    s = cross_val_score(m, X_train, y_train, cv=5)
    print(name, "CV mean", round(s.mean(), 3))

The output from my run was:

Logistic regression CV mean 0.993
Random forest CV mean 0.986

Both models score very well, and the difference of 0.007 is tiny. On a training set of 142 rows, it corresponds to roughly one sample. Neither model is clearly better, so we pick the simpler, faster, and more interpretable one: logistic regression. Notice that we used only the training data for this comparison. The test set remains untouched.

Step 7: Train the model and make predictions

Now fit the chosen pipeline on the full training set and predict on the test set. This is the first and only time the model sees the test features, and it never sees the test labels during training.

final = candidates["Logistic regression"].fit(X_train, y_train)
pred = final.predict(X_test)

The variable pred now holds one predicted cultivar for each of the 36 test wines.

Step 8: Evaluate the results

Compare the predictions with the true labels. Accuracy gives a quick summary. The confusion matrix shows which classes get mixed up, and the classification report gives precision, recall, and F1-score for each class. The article on how to evaluate a supervised learning model explains each of these in detail.

from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

print("Test accuracy:", round(accuracy_score(y_test, pred), 3))
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred, target_names=wine.target_names))

The output from my run was:

Test accuracy: 0.972
[[12  0  0]
 [ 0 14  0]
 [ 0  1  9]]
              precision    recall  f1-score   support

     class_0       1.00      1.00      1.00        12
     class_1       0.93      1.00      0.97        14
     class_2       1.00      0.90      0.95        10

    accuracy                           0.97        36
   macro avg       0.98      0.97      0.97        36
weighted avg       0.97      0.97      0.97        36

The model classified 35 of the 36 test wines correctly, for an accuracy of 0.972. The confusion matrix shows the only mistake: one wine from class 2 was predicted as class 1. That is why class 1 has a precision of 0.93 and class 2 has a recall of 0.90.

Keep this result in perspective. The test set has only 36 rows, so a single wine changes the accuracy by about 0.028. The dataset is also clean and well separated, so you should not expect such scores on messier real-world data. The cross-validation score (0.993) and the test score (0.972) agree closely enough that nothing here suggests a leak or serious overfitting.

Troubleshooting common problems

ModuleNotFoundError: No module named ‘sklearn’

Python cannot find scikit-learn. The package name for pip is scikit-learn, not sklearn. Make sure that your virtual environment is active and that you ran pip install scikit-learn inside it. In a notebook, restart the kernel after installing.

ConvergenceWarning from LogisticRegression

The optimizer ran out of iterations. Scale your features with StandardScaler, and raise max_iter, for example to 1000 or more.

ValueError: Expected 2D array, got 1D array

Scikit-learn expects features in a two-dimensional shape: rows by columns. If you have a single feature in a one-dimensional array, reshape it with X.reshape(-1, 1), or select it with double brackets in pandas, such as df[["alcohol"]].

ValueError: Input contains NaN

Some models cannot handle missing values. Fill them with SimpleImputer inside your pipeline so that the fill values come from the training data only.

ValueError about inconsistent numbers of samples

Your feature table and target have different lengths. Check X.shape and y.shape, and make sure you did not drop rows from one but not the other.

Results change every time I run the code

Random choices, such as the data split and the random forest, change from run to run. Set random_state to a fixed number in train_test_split and in every model that has the parameter.

My accuracy is suspiciously close to 100%

Be skeptical. Common causes are data leakage, such as scaling or feature selection before the split, a feature that contains the answer, or duplicate rows across the train and test sets. Review the order of your steps, and use a pipeline.

Different numbers from the ones in this article

Small differences are normal when you use other library versions. Algorithms and defaults can change between releases. If the numbers differ wildly, check that you used the same random_state, split, and settings.

What to try next

• Change the classifier to a decision tree and watch how overfitting appears when you remove the depth limit. The article on overfitting vs underfitting explains why.
• Use GridSearchCV to tune settings such as C for logistic regression, always inside the training data.
• Apply the same workflow to a regression problem, following Linear Regression Explained with Examples.
• Load your own CSV file with pd.read_csv() and repeat these eight steps on it.

Frequently asked questions

Do I need to know advanced math to build a supervised learning model?

No. You can build and evaluate useful models by understanding the workflow and the main ideas. Math becomes more important when you want to tune models deeply or invent new methods, but it is not a barrier to starting.

Which Python library is best for beginners in machine learning?

Scikit-learn is a popular first choice. It offers consistent tools for preprocessing, modeling, and evaluation, and its documentation is thorough. Deep learning libraries such as PyTorch or TensorFlow come later.

Why did we scale the features for logistic regression but not for the random forest?

Logistic regression is sensitive to feature scale, especially with regularization and iterative optimization. Tree-based models compare features with thresholds, so scaling does not change their splits.

Why did we use cross-validation if we already have a test set?

Cross-validation on the training data lets us compare models without touching the test set. The test set is reserved for one final, unbiased check of the chosen model.

How much data do I need?

It depends on the problem. This tutorial uses 178 rows because the task is simple and the classes separate well. Harder problems need more examples. More important than size is whether your data represents the situations the model will face.

Can I use this workflow for my own project?

Yes. Replace the dataset, check the target type, choose suitable metrics, and keep the same safeguards: a hidden test set, preprocessing inside a pipeline, and a fixed random seed.

Conclusion

You have now completed a full supervised learning project in Python. You loaded and explored data, held out a test set, compared two models with cross-validation, trained the winner, and evaluated it honestly. The same pattern scales to larger datasets and more complex models, and the safeguards you practiced here protect your results from leakage.

The best way to learn next is to repeat the workflow on a new dataset and read the related articles in this series whenever a concept feels unclear. Start from What Is Supervised Learning? to review the foundations, or dive into any algorithm article that interests you.

This article is for educational and informational purposes only. Examples and code are provided to help readers understand how to build a supervised machine learning model with Python and may require adaptation for specific projects.

Related guides

Before training, understand Training Data vs Testing Data, then learn how to assess your model in How to Evaluate a Supervised Learning Model.

References and further reading

• Installing scikit-learn for installation options and supported Python versions.
• Toy datasets (scikit-learn) for details on the wine dataset.
• Common pitfalls and recommended practices (scikit-learn) for pipelines and data leakage.