CatBoost

Language: Python

ML/AI

CatBoost was developed by Yandex in 2017 to provide an easy-to-use, fast, and accurate gradient boosting implementation that natively handles categorical variables. It reduces the need for extensive data preprocessing and is widely used in machine learning competitions and production systems.

CatBoost is a high-performance gradient boosting library from Yandex that handles categorical features automatically and efficiently. It is designed for classification, regression, and ranking tasks with minimal preprocessing.

Installation

pip: pip install catboost

conda: conda install -c conda-forge catboost

Usage

CatBoost can handle categorical and numerical features directly, supports GPU acceleration, and provides Python, R, and CLI interfaces. Models can be trained using `CatBoostClassifier` or `CatBoostRegressor` and can be evaluated with built-in metrics.

Training a classifier

from catboost import CatBoostClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = CatBoostClassifier(verbose=0)
model.fit(X_train, y_train)
print(model.score(X_test, y_test))

Trains a CatBoost classifier on the Iris dataset and evaluates accuracy.

Training a regressor

from catboost import CatBoostRegressor
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_boston

X, y = load_boston(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = CatBoostRegressor(verbose=0)
model.fit(X_train, y_train)
print(model.score(X_test, y_test))

Trains a CatBoost regressor on the Boston housing dataset and evaluates performance.

Handling categorical features

from catboost import CatBoostClassifier
model = CatBoostClassifier(cat_features=[0,2,5], verbose=0)

Specifies categorical feature indices so CatBoost can handle them natively.

Using Pool for efficient training

from catboost import Pool
train_pool = Pool(X_train, y_train, cat_features=[0,2,5])
model.fit(train_pool)

Uses CatBoost Pool object to efficiently store data and categorical features for training.

Early stopping

model.fit(X_train, y_train, eval_set=(X_test, y_test), early_stopping_rounds=10)

Stops training early if validation metric does not improve for a specified number of rounds.

Feature importance

import matplotlib.pyplot as plt
feature_importances = model.get_feature_importance()
plt.bar(range(len(feature_importances)), feature_importances)
plt.show()

Visualizes the importance of each feature after training the model.

Error Handling

CatBoostError: Invalid feature index: Ensure the specified categorical feature indices exist in the dataset.

CatBoostError: GPU not available: Install CatBoost with GPU support and ensure a compatible GPU is available.

ModuleNotFoundError: No module named 'catboost': Install CatBoost using pip or conda in your current Python environment.

Best Practices

Use CatBoost Pool to handle categorical features efficiently.

Enable early stopping to prevent overfitting.

Leverage GPU acceleration for large datasets.

Use built-in evaluation metrics to monitor model performance.

Tune hyperparameters such as `depth`, `learning_rate`, and `iterations` for optimal results.

Official Docs Github