openenv-cli

OpenEnv: Agentic Execution Environments

An e2e framework for creating, deploying and using isolated execution environments for agentic RL training, built using Gymnasium style simple APIs.

---

Featured Example: Train LLMs to play BlackJack using torchforge (PyTorch's agentic RL framework): examples/grpo_blackjack/

Zero to Hero Tutorial: End to end tutorial from our GPU Mode lecture and other hackathons.

Quick Start

Install the OpenEnv package:

pip install openenv

Install an environment client (e.g., Echo):

pip install git+https://huggingface.co/spaces/openenv/echo_env

Then use the environment:

import asyncio
from echo_env import CallToolAction, EchoEnv

async def main():
    # Connect to a running Space (async context manager)
    async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as client:
        # Reset the environment
        result = await client.reset()
        print(result.observation.metadata["message"])  # "Echo environment ready!"

        # Send messages
        result = await client.step(
            CallToolAction(
                tool_name="echo_message",
                arguments={"message": "Hello, World!"},
            )
        )
        print(result.observation.result)  # "Hello, World!"
        print(result.reward)

asyncio.run(main())

Synchronous usage is also supported via the .sync() wrapper:

from echo_env import CallToolAction, EchoEnv

# Use .sync() for synchronous context manager
with EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as client:
    result = client.reset()
    result = client.step(
        CallToolAction(
            tool_name="echo_message",
            arguments={"message": "Hello, World!"},
        )
    )
    print(result.observation.result)

For a detailed quick start, check out the docs page.

Overview

OpenEnv provides a standard for interacting with agentic execution environments via simple Gymnasium style APIs - step(), reset(), state(). Users of agentic execution environments can interact with the environment during RL training loops using these simple APIs.

In addition to making it easier for researchers and RL framework writers, we also provide tools for environment creators making it easier for them to create richer environments and make them available over familiar protocols like HTTP and packaged using canonical technologies like docker. Environment creators can use the OpenEnv framework to create environments that are isolated, secure, and easy to deploy and use.

The OpenEnv CLI (openenv) provides commands to initialize new environments and deploy them to Hugging Face Spaces.

⚠️ Early Development Warning OpenEnv is currently in an experimental stage. You should expect bugs, incomplete features, and APIs that may change in future versions. The project welcomes bugfixes, but significant changes should be discussed before implementation so the technical committee and community can coordinate scope, compatibility, and release timing. It's recommended that you signal your intention to contribute in the issue tracker, either by filing a new issue or by claiming an existing one.

RFCs

Below is a list of active and historical RFCs for OpenEnv. RFCs are proposals for major changes or features. Please review and contribute!

• RFC 000: Project Phases and Design Principles
• RFC 001: Baseline API and Interface Specifications
• RFC 002: Discoverability of environment tools by agents
• RFC 003: Add MCP (Model Context Protocol) support
• RFC 004: Add delayed rewards support for trajectory-based scoring
• RFC 005: Agentic Harness Integration
• RFC 010: Env-token World Modeling (ECHO)

Architecture

Component Overview

┌─────────────────────────────────────────────────────────┐
│                    Client Application                   │
│  ┌────────────────┐              ┌──────────────────┐   │
│  │  EchoEnv       │              │  CodingEnv       │   │
│  │  (EnvClient)   │              │   (EnvClient)    │   │
│  └────────┬───────┘              └────────┬─────────┘   │
└───────────┼───────────────────────────────┼─────────────┘
            │ WebSocket                     │ WebSocket
            │ (reset, step, state)          │
┌───────────▼───────────────────────────────▼─────────────┐
│              Docker Containers (Isolated)               │
│  ┌──────────────────────┐    ┌──────────────────────┐   │
│  │ FastAPI Server       │    │ FastAPI Server       │   │
│  │   EchoEnvironment    │    │ PythonCodeActEnv     │   │
│  │ (Environment base)   │    │ (Environment base)   │   │
│  └──────────────────────┘    └──────────────────────┘   │
└─────────────────────────────────────────────────────────┘

Core Components

1. Web Interface

OpenEnv includes a built-in web interface for interactive environment exploration and debugging. The web interface provides:

• Two-Pane Layout: HumanAgent interaction on the left, state observation on the right
• Real-time Updates: WebSocket-based live updates without page refresh
• Dynamic Forms: Automatically generated action forms based on environment Action types
• Action History: Complete log of all actions taken and their results

The web interface is conditionally enabled based on environment variables:

• Local Development: Disabled by default for lightweight development
• Manual Override: Enable with ENABLE_WEB_INTERFACE=true

To use the web interface:

from openenv.core.env_server import create_web_interface_app
from your_env.models import YourAction, YourObservation
from your_env.server.your_environment import YourEnvironment

env = YourEnvironment()
app = create_web_interface_app(env, YourAction, YourObservation)

When enabled, open http://localhost:8000/web in your browser to interact with the environment.

2. Environment (Server-Side)

Base class for implementing environment logic:

• reset(): Initialize a new episode, returns initial Observation
• step(action): Execute an Action, returns resulting Observation
• state(): Access episode metadata (State with episode_id, step_count, etc.)

3. EnvClient (Client-Side)

Base class for environment communication:

• Async by default: Use async with and await for all operations
• Sync wrapper: Call .sync() to get a SyncEnvClient for synchronous usage
• Handles WebSocket connections to environm