Skip to content

OpenAI Gym

AI & Machine LearningReinforcement LearningPython
Official docs ↗GitHub ↗

What it is

OpenAI Gym is a toolkit for developing and comparing reinforcement learning (RL) algorithms. It provides a wide variety of environments, from classic control problems and Atari games to robotics simulations, to test and benchmark RL agents.

OpenAI Gym provides a unified interface for all environments. Agents interact with environments by taking actions and receiving observations and rewards. The library supports vectorized environments, wrappers, and custom environment creation.

Installation

pip install gym

Getting started

The smallest useful thing you can do with it, and what each part means.

Basic environment interaction
import gym
env = gym.make('CartPole-v1')
obs = env.reset()
for _ in range(100):
    env.render()
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)
    if done:
        obs = env.reset()
env.close()
Creates the CartPole-v1 environment, samples random actions, and renders the environment while interacting with it.
Inspecting action and observation spaces
import gym
env = gym.make('MountainCar-v0')
print(env.action_space)
print(env.observation_space)
Shows the possible actions the agent can take and the shape/range of observations.

Advanced usage

Where the library earns its place over a simpler alternative.

Custom environment wrapper
from gym import Wrapper

class NormalizeWrapper(Wrapper):
    def step(self, action):
        obs, reward, done, info = self.env.step(action)
        obs = obs / 10.0  # simple normalization example
        return obs, reward, done, info

env = gym.make('CartPole-v1')
env = NormalizeWrapper(env)
Demonstrates creating a custom wrapper to preprocess observations or modify rewards.
Vectorized environments
from gym.vector import SyncVectorEnv
import gym

def make_env():
    return gym.make('CartPole-v1')

env = SyncVectorEnv([make_env for _ in range(4)])
obs = env.reset()
print(obs.shape)
Runs multiple environments in parallel to speed up training of RL agents.
Seeding environments for reproducibility
env = gym.make('CartPole-v1')
env.seed(42)
import numpy as np
np.random.seed(42)
Sets seeds for the environment and NumPy to ensure reproducible results.

Errors and fixes

The failures you are most likely to hit, and what actually resolves them.

Error: Environment ID not found
Ensure the environment name passed to `gym.make()` is correct and installed.
Action out of bounds
Make sure the action is within the range defined by `env.action_space`.
Observation dimension mismatch
Check that your agent handles the correct observation shape from `env.observation_space`.

Best practices

  • Always call `env.close()` after finishing to release resources.
  • Use environment wrappers to preprocess observations and rewards.
  • Vectorize environments to improve training efficiency for RL agents.
  • Monitor environment performance using `gym.wrappers.Monitor`.
  • Set seeds for reproducibility when experimenting with algorithms.

Background

Why it exists, and what it was reacting to.

OpenAI Gym was released by OpenAI in 2016 to standardize the process of developing and comparing reinforcement learning algorithms. Its modular design and extensive suite of environments have made it a key tool for researchers and practitioners in the RL community.

Related libraries

Others in AI & Machine Learning for Python.

← All Python librariesPython guide