Blend Models
Blending models is a method of ensembling which uses consensus among estimators to generate final predictions. The idea behind blending is to combine different machine learning algorithms and use a majority vote or the average predicted probabilities in case of classification to predict the final outcome. Blending models in PyCaret is as simple as writing blend_models. This function can be used to blend specific trained models that can be passed using estimator_list parameter within blend_models or if no list is passed, it will use all the models in model library. In case of Classification, method parameter can be used to define ‘soft‘ or ‘hard‘ where soft uses predicted probabilities for voting and hard uses predicted labels. This functions returns a table with k-fold cross validated scores of common evaluation metrics along with trained model object. The evaluation metrics used are:
- Classification: Accuracy, AUC, Recall, Precision, F1, Kappa
- Regression: MAE, MSE, RMSE, R2, RMSLE, MAPE
The number of folds can be defined using fold parameter within blend_models function. By default, the fold is set to 10. All the metrics are rounded to 4 decimals by default by can be changed using round parameter within blend_models.
This function is only available in pycaret.classification and pycaret.regression modules.
Classification Example
Code
# Importing dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# Importing module and initializing setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# blending all models
blend_all = blend_models()
Output
VotingClassifier(estimators=[('Logistic Regression_0',
LogisticRegression(C=1.0, class_weight=None,
dual=False, fit_intercept=True,
intercept_scaling=1,
l1_ratio=None, max_iter=100,
multi_class='auto',
n_jobs=None, penalty='l2',
random_state=123,
solver='lbfgs', tol=0.0001,
verbose=0, warm_start=False)),
('K Neighbors Classifier_1',
KNeighborsClassifier(algorithm='aut...
importance_type='split',
learning_rate=0.1, max_depth=-1,
min_child_samples=20,
min_child_weight=0.001,
min_split_gain=0.0,
n_estimators=100, n_jobs=-1,
num_leaves=31, objective=None,
random_state=123, reg_alpha=0.0,
reg_lambda=0.0, silent=True,
subsample=1.0,
subsample_for_bin=200000,
subsample_freq=0))],
flatten_transform=True, n_jobs=None, voting='hard',
weights=None)Regression Example
Code
# Importing dataset
from pycaret.datasets import get_data
boston = get_data('boston')
# Importing module and initializing setup
from pycaret.regression import *
reg1 = setup(data = boston, target = 'medv')
# creating multiple models for blending
dt = create_model('dt')
catboost = create_model('catboost')
lightgbm = create_model('lightgbm')
# blending specific models
blender = blend_models(estimator_list = [dt, catboost, lightgbm])
Output
VotingRegressor(estimators=[('Decision Tree Regressor_0',
DecisionTreeRegressor(ccp_alpha=0.0,
criterion='mse',
max_depth=None,
max_features=None,
max_leaf_nodes=None,
min_impurity_decrease=0.0,
min_impurity_split=None,
min_samples_leaf=1,
min_samples_split=2,
min_weight_fraction_leaf=0.0,
presort='deprecated',
random_state=123,
splitter='best')),
('CatBoost Regres...
class_weight=None,
colsample_bytree=1.0,
importance_type='split',
learning_rate=0.1, max_depth=-1,
min_child_samples=20,
min_child_weight=0.001,
min_split_gain=0.0, n_estimators=100,
n_jobs=-1, num_leaves=31,
objective=None, random_state=123,
reg_alpha=0.0, reg_lambda=0.0,
silent=True, subsample=1.0,
subsample_for_bin=200000,
subsample_freq=0))],
n_jobs=None, weights=None)Try this next
Was this page helpful?
GitHub