What it is
CatBoost is a high-performance gradient boosting library from Yandex that handles categorical features automatically and efficiently. It is designed for classification, regression, and ranking tasks with minimal preprocessing.
CatBoost can handle categorical and numerical features directly, supports GPU acceleration, and provides Python, R, and CLI interfaces. Models can be trained using `CatBoostClassifier` or `CatBoostRegressor` and can be evaluated with built-in metrics.
Installation
pip install catboostGetting started
The smallest useful thing you can do with it, and what each part means.
from catboost import CatBoostClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = CatBoostClassifier(verbose=0)
model.fit(X_train, y_train)
print(model.score(X_test, y_test))from catboost import CatBoostRegressor
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = CatBoostRegressor(verbose=0)
model.fit(X_train, y_train)
print(model.score(X_test, y_test))Advanced usage
Where the library earns its place over a simpler alternative.
from catboost import CatBoostClassifier
model = CatBoostClassifier(cat_features=[0,2,5], verbose=0)from catboost import Pool
train_pool = Pool(X_train, y_train, cat_features=[0,2,5])
model.fit(train_pool)model.fit(X_train, y_train, eval_set=(X_test, y_test), early_stopping_rounds=10)import matplotlib.pyplot as plt
feature_importances = model.get_feature_importance()
plt.bar(range(len(feature_importances)), feature_importances)
plt.show()Errors and fixes
The failures you are most likely to hit, and what actually resolves them.
- CatBoostError: Invalid feature index
- Ensure the specified categorical feature indices exist in the dataset.
- CatBoostError: GPU not available
- Install CatBoost with GPU support and ensure a compatible GPU is available.
- ModuleNotFoundError: No module named 'catboost'
- Install CatBoost using pip or conda in your current Python environment.
Best practices
- Use CatBoost Pool to handle categorical features efficiently.
- Enable early stopping to prevent overfitting.
- Leverage GPU acceleration for large datasets.
- Use built-in evaluation metrics to monitor model performance.
- Tune hyperparameters such as `depth`, `learning_rate`, and `iterations` for optimal results.
Background
Why it exists, and what it was reacting to.
CatBoost was developed by Yandex in 2017 to provide an easy-to-use, fast, and accurate gradient boosting implementation that natively handles categorical variables. It reduces the need for extensive data preprocessing and is widely used in machine learning competitions and production systems.