Rust 기반 강화 학습, Python보다 140배 빠름

Rust-accelerated reinforcement learning — the Polars architecture pattern applied to RL. RL frameworks like Stable-Baselines3 and TorchRL do everything in Python — environment stepping, buffer storage, advantage computation. This works, but Python interpreter overhead becomes the bottleneck long before your GPU does. rlox applies the Polars architecture pattern to RL: a Rust data plane handles the compute-heavy, latency-sensitive work (env stepping, buffers, GAE) while a Python control plane stays in charge of training logic, configs, and neural networks via PyTorch. The two connect through PyO3 with zero-copy where possible. The result: 3-50x faster than SB3/TorchRL on data-plane operations, with the same Python training API you're used to. pip install rlox Or build from source (requires Rust 1.75+): python3 -m venv .venv && source .venv/bin/activate pip install maturin numpy gymnasium torch maturin develop --release Train PPO on CartPole in 3 lines: from rlox.trainers import PPOTrainer trainer = PPOTrainer(env="CartPole-v1", seed=42) metrics = trainer.train(total_timesteps=50_000) print(f"Mean reward: {metrics['mean_reward']:.1f}") Train SAC on Pendulum: from rlox.trainers import SACTrainer trainer = SACTrainer(env="Pendulum-v1", config={"learning_starts": 500}) metrics = trainer.train(total_timesteps=20_000) Config-driven training (YAML): python -m rlox train --config config.yaml from rlox import TrainingConfig, train_from_config config = TrainingConfig.from_yaml("config.yaml") metrics = train_from_config(config) Use Rust primitives directly: import rlox # 140x faster GAE than Python loops advantages, returns = rlox.compute_gae(rewards, values, dones, last_value, gamma=0.99, lam=0.95) # 35x faster GRPO advantages advantages = rlox.compute_batch_group_advantages(rewards, group_size=4) # Parallel env stepping (2.7M steps/s at 512 envs) env = rlox.VecEnv(n=256, seed=42, env_id="CartPole-v1") result = env.step_all(actions) More examples in examples/ — PPO, SAC, GRPO custom rewards, fast GAE, VecEnv throughput. | Resource | Link | |---|---| | Full Documentation | riserally.github.io/rlox | | Getting Started | Tutorial | | Python API Guide | User Guide | | Examples | Code Examples | | Rust API | cargo doc | | Migrating from SB3 | Migration Guide | | API Reference | Autodoc | ┌──────────────────────────────────────────────────┐ │ Python (control plane) │ │ PPO, SAC, DQN, TD3, A2C, MAPPO, DreamerV3, │ │ IMPALA, GRPO, DPO │ │ GymVecEnv, VecNormalize, callbacks, │ │ YAML/TOML configs, trainers, checkpointing, │ │ diagnostics dashboard │ │ vLLM/TGI/SGLang backends, multi-GPU (DDP) │ ├────────────── PyO3 boundary ─────────────────────┤ │ Rust (data plane) │ │ rlox-core: envs (CartPole, Pendulum), │ │ Rayon parallel stepping, │ │ buffers (ring, mmap, priority), │ │ GAE, V-trace, GRPO, pipeline │ │ rlox-nn: RL algorithm traits │ │ rlox-burn: Burn backend (NdArray) │ │ rlox-candle: Candle backend (CPU) │ │ rlox-python: PyO3 bindings │ └──────────────────────────────────────────────────┘ Multi-crate workspace (crates.io): - rlox-core — pure Rust: environments, buffers (ring, mmap, priority), GAE, V-trace, GRPO, pipeline - rlox-nn — RL algorithm traits ( ActorCritic ,QFunction ,StochasticPolicy , etc.) - rlox-burn — Burn Autodiff implementations - rlox-candle — Candle CPU implementations - rlox-python — PyO3 bindings exposing rlox-core to Python For a deep-dive into the architecture, module relationships, and API reference, see the DeepWiki. All benchmarks on Apple M4 with bootstrap 95% CI (10,000 resamples). All results statistically significant (CI lower bound > 1.0). | Component | vs SB3 | vs TorchRL | Details | |---|---|---|---| | GAE (32K steps) | 147x vs NumPy | 1,700x | docs/benchmark/gae.md | | Buffer push (10K) | 9.7x | 148x | docs/benchmark/buffer-ops.md | | Buffer sample (1024) | 8.1x | 10x | docs/benchmark/buffer-ops.md | | E2E rollout (256×2048) | 3.9x | 53x | docs/benchmark/e2e-rollout.md | | GRPO advantages | 35x vs NumPy | 34x vs PyTorch | docs/benchmark/llm-ops.md | | Env stepping (512 envs) | — | — | 2.7M steps/s | Full methodology, raw data, and reproducibility instructions: docs/benchmark/ Same hyperparameters (rl-zoo3 defaults), 5 seeds per experiment. On-policy algorithms (PPO, A2C) show 1.4-3.3x faster wall-clock convergence with matching reward thresholds. | Algorithm | Environment | rlox Wall-clock | SB3 Wall-clock | rlox SPS | SB3 SPS | |---|---|---|---|---|---| | PPO | CartPole-v1 | 1.6s | 5.2s | 9,121 | 4,026 | | A2C | CartPole-v1 | 1.8s | 2.1s | 10,445 | 4,206 | | PPO | Acrobot-v1 | 6.4s | 9.1s | 12,030 | 7,727 | Full convergence results, learning curves, and performance profiles: benchmarks/convergence/ - 8 Algorithms: PPO, SAC, DQN, TD3, A2C, MAPPO, DreamerV3, IMPALA (+ GRPO, DPO for LLM) - 8 Trainers: Each algorithm has a high-level Trainer withtrain() ,save() ,from_checkpoint() - Environments: Gymnasium-compatible, Rayon-parallel VecEnv, CartPole and Pendulum-v1 built-in - VecNormalize: O