Random Forest vs Decision Tree: Key Differences

The main difference in random forest vs decision tree is simple: a decision tree is one model that asks a chain of questions, while a random forest is a team of many decision trees whose predictions are combined. A single tree is easy to read but unstable. A forest is usually more accurate and more stable, but harder to interpret and more expensive to run.

This guide explains both models, shows how bootstrap sampling, feature randomness, and bagging create a forest, and compares them on accuracy, interpretability, cost, and feature importance. A Python comparison with real output follows. If you have not met single trees yet, start with Decision Trees in Machine Learning: How They Work.

Random forest vs decision tree at a glance

Aspect	Decision tree	Random forest
What it is	A single tree	An ensemble of many trees
Prediction	The value in one leaf	The combined result of all trees
Overfitting	High risk when the tree is deep	Lower risk, because averaging reduces variance
Accuracy	Often lower	Often higher
Stability	Small data changes can alter the tree	More stable across data changes
Interpretability	High: you can read the whole tree	Lower: hundreds of trees are hard to inspect
Training and prediction cost	Low	Higher, roughly proportional to the number of trees
Main settings	Depth, leaf size, pruning	Number of trees, features per split, plus tree settings

A quick look at the decision tree

A decision tree splits the data again and again with simple questions until it reaches leaves that hold predictions. It works for classification and regression, it needs no feature scaling, and you can display it as a flowchart. Its weakness is high variance. If you change a few training rows, the best first question may change, and the whole tree can look different. A deep tree also tends to memorize noise. You can read about this in Overfitting vs Underfitting in Machine Learning.

Ensemble learning in plain English

Ask one person to guess the number of candies in a jar, and the answer may be far off. Ask 200 people, average the guesses, and the result is usually much closer. Individual mistakes point in different directions, so they partly cancel out. This is the idea behind ensemble learning: combine several models so that the group performs better than a typical member.

The trick only works if the members make different mistakes. A hundred identical trees would just repeat the same errors. A random forest therefore uses two sources of randomness to make the trees differ.

How a random forest works

Bootstrap sampling

For each tree, the algorithm draws a random sample of the training rows with replacement. That means a row can appear several times in one sample, and other rows can be missing. If the original dataset has n rows, each bootstrap sample also has n rows, and on average about 63% of the distinct original rows appear in it. Every tree therefore trains on a slightly different dataset.

Bagging

Bagging is short for bootstrap aggregating. It means training each model on its own bootstrap sample and then aggregating the predictions. Bagging alone, applied to decision trees, already reduces variance.

Feature randomness

A random forest adds a second source of randomness. At every split inside every tree, the algorithm considers only a random subset of the features instead of all of them. This stops one strong feature from dominating every tree. As a result, the trees look less alike, and their errors are less correlated. In scikit-learn, the max_features parameter controls the subset size. For classifiers the default is the square root of the number of features. For regressors the default uses all features, so you may want to tune it.

Prediction aggregation

To predict, the forest sends the example through every tree and combines the results. For regression, it averages the predicted numbers. For classification, the classic description is majority voting. Scikit-learn’s implementation averages the class probabilities from the trees and picks the class with the highest average, which is a softer form of voting.

Out-of-bag evaluation

Each tree never sees the rows left out of its bootstrap sample. Those rows are called out-of-bag samples, and you can use them to estimate performance without a separate validation set. In scikit-learn, set oob_score=True. Still keep a real test set for the final check, as described in Training Data vs Testing Data.

Key differences explained

Accuracy and generalization

A random forest usually generalizes better than a single tree. Averaging many different trees cancels much of the noise that a single deep tree memorizes. The improvement is not guaranteed, so compare both models on your own data with cross-validation. On very small or very clean datasets, a well-tuned single tree can come close.

Interpretability

A single tree wins here. You can print its rules and explain a prediction by naming one path. A forest of 200 trees has no single path to explain. You can still summarize it with feature importance scores and other tools, but you lose the simple flowchart.

Computational cost

A forest trains many trees, so it takes longer and uses more memory than a single tree. Prediction time also grows with the number of trees. The trees are independent, so scikit-learn can train them in parallel when you set n_jobs. For most beginner datasets the cost is small, but it can matter on large data or in systems that must answer in milliseconds.

Tuning effort

A forest often gives good results with default settings, which is a practical advantage. Important settings include n_estimators (the number of trees), max_features, max_depth, and min_samples_leaf. More trees rarely cause overfitting, but they stop helping after a point.

Feature importance

A forest can report how useful each feature was. Scikit-learn’s default feature_importances_ attribute measures how much each feature reduced impurity across all the splits where the trees used it, averaged over the forest. The scores add up to 1.

Use these scores with care:

• Impurity-based importance can favor features with many distinct values.
• When two features carry the same information, the forest may split the credit between them, which makes each look less important than it is.
• The scores describe the model, not the real world. A high score does not mean that the feature causes the outcome.

Permutation importance, which measures how much the score drops when you shuffle one feature on held-out data, is a useful cross-check.

Practical comparison in Python

The code below compares a single decision tree and a random forest with 200 trees on the breast cancer dataset. It uses 5-fold cross-validation on the training portion, then tests each model once on the held-out test set. It also prints the five most important features of the forest.

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier

data = load_breast_cancer()
X, y = data.data, data.target
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42)

tree = DecisionTreeClassifier(random_state=42)
forest = RandomForestClassifier(n_estimators=200, random_state=42)

for name, m in [("Decision tree", tree), ("Random forest", forest)]:
    cv = cross_val_score(m, X_train, y_train, cv=5)
    m.fit(X_train, y_train)
    print(name, "| CV mean", round(cv.mean(), 3),
          "| test acc", round(m.score(X_test, y_test), 3))

top = sorted(zip(forest.feature_importances_, data.feature_names), reverse=True)[:5]
for score, feature in top:
    print(round(score, 3), feature)

When I ran this code with Python 3.12 and scikit-learn 1.8.0, it printed:

Decision tree | CV mean 0.91 | test acc 0.912
Random forest | CV mean 0.96 | test acc 0.956
0.133 worst perimeter
0.128 worst area
0.108 worst concave points
0.094 mean concave points
0.091 worst radius

On this dataset, the forest improves cross-validated accuracy from 0.91 to 0.96 and test accuracy from 0.912 to 0.956. The consistent gain in both measurements supports the general pattern that a forest generalizes better than one unrestricted tree. The test set has only 114 rows, so a difference of a few rows can move the number, and the result applies to this dataset and these settings only.

The importance list also shows the caveat about correlated features. “Worst perimeter,” “worst area,” and “worst radius” all describe tumor size, so they carry overlapping information, and the forest shares credit among them.

When to use each model

Choose a single decision tree when:

• You must explain predictions as simple rules to non-technical people.
• The dataset is small and you want a quick baseline.
• You want to learn how splitting works by looking at a real tree.

Choose a random forest when:

• You want strong accuracy with little tuning.
• Your data is tabular and has non-linear patterns or feature interactions.
• You can accept a less transparent model and a higher computing cost.

In many projects, you train both and compare them using the metrics in How to Evaluate a Supervised Learning Model. Also compare against simpler models such as logistic regression, because a basic model sometimes performs just as well.

Limitations of random forests

• Less interpretable. You cannot read a forest like a flowchart.
• Heavier models. Large forests take space and time to store and run.
• No extrapolation. Like a single tree, a forest regressor cannot predict beyond the range of target values in its training data.
• Still needs good data. Leakage, label noise, and biased samples harm a forest as much as any other model.
• Not always the best. On some problems, boosted trees or other methods can outperform a random forest.

Leo Breiman introduced the random forest method in a 2001 paper, which remains the standard reference.

Frequently asked questions

Is a random forest always better than a decision tree?

No. A forest usually gives better predictive performance, but a single tree is faster, smaller, and easier to explain. The better choice depends on whether you value accuracy or transparency more.

What is the difference between bagging and a random forest?

Bagging trains each tree on a bootstrap sample and combines the results. A random forest adds feature randomness: each split considers only a random subset of features. That extra step makes the trees less alike.

How many trees should a random forest have?

Start with 100 to 300 trees and check whether the cross-validated score still improves. After a point, more trees give little benefit and only add computing cost.

Can a random forest overfit?

Yes, although it overfits less than a single deep tree. Very noisy data, leakage, or unrestricted tree depth can still cause problems. Cross-validation helps you detect it.

Do random forests need feature scaling?

No. Like single trees, they split on thresholds, so the scale of a feature does not change the result.

Can random forests do regression?

Yes. A random forest regressor averages the numeric predictions of its trees. See Classification vs Regression in Machine Learning to review the two task types.

Conclusion

A decision tree is a single, transparent but unstable model. A random forest combines many trees built from bootstrap samples and random feature subsets, and it usually delivers better and steadier predictions in exchange for lower interpretability and higher cost. Neither choice is universally right, so test both with cross-validation on your data.

Both models can overfit or underfit, and understanding those failures helps you tune them. Continue with Overfitting vs Underfitting in Machine Learning.

This article is for educational and informational purposes only. Examples and code are provided to help readers understand random forests and decision trees and may require adaptation for specific projects.

Related guides

Review the underlying Decision Tree Algorithm and learn how to compare model performance in How to Evaluate a Supervised Learning Model.

References and further reading

• Ensembles: Gradient boosting, random forests, bagging (scikit-learn user guide) for parameters and out-of-bag estimates.
• Breiman, L. (2001). Random Forests. Machine Learning, 45, 5 to 32.
• Permutation feature importance (scikit-learn user guide) as an alternative to impurity-based importance.