Skip to contentDeveloper
Changelog#
Version 1.4.3#
- Support MAPL algorithms, including MARLHF and MADPO (PR #72).
- Support centralized multi-agent system training (PR #72).
- Refine the documentation.
Version 1.4.2#
- Support more comparator models (commit e1b3d80).
- Fix the homogeneous generation issue (commit 11d89a9).
Version 1.4.1#
- Support using the comparator as a centralized coordinator (PR #69).
- Allow deploying LLM agents, the reward model, and the comparator on different devices (commit 7cbb792).
- Allow copying current models as comparators on new devices (commit d8d493a).
- Optimize GPU memory usage by promptly releasing unused models (commit d0c6376).
Version 1.4.0#
- Add online PbRL methods (PR #68).
- Add dataset drift logger.
Version 1.3.9#
- Add offline PbRL methods such as MADPO and MARLHF (PR #67).
Version 1.3.8#
- Allow using a reference model for all algorithms (PR #65).
- Unify the logger between different trainers (PR #66).
Version 1.3.7#
- Remove the redundant sampling hyperparameters in algorithms.
- Allow multi-GPU training with MP (see results in PR #62).
Version 1.3.6#
- Fix a critical bug in loading heterogeneous models and refactor the model-loading logic (see results in PR #60).
- Polish the docs.
Version 1.3.5#
- Add unit tests for hyperparameter constraints.
- Clean legacy interfaces.
Version 1.3.4#
- Fix the bug in loading heterogeneous models and refactor the loading logic.
- Enable MBGD in MAGRPO to align with MAAC and IAC.
- Remove redundant and legacy hyperparameters (e.g., model kwargs, patching hyperparameters).
- Clean up multi-device legacy code, such as
drop_last and num_workers. - Add unit tests for model loading and separate them from CI as a badge.
- Clean short functions.
- Reorganize the docs and align the parameters.
Version 1.3.3#
- Compact the MAREINFORCETrainer derivation and move it to the new folder.
- Unify the interface for different trainers.
- Remove redundant patches and wrappers.
- Reorganize the variables in the config YAML files.
Version 1.3.2#
- Fix wandb logging issue in MAGRPOTrainer.
Version 1.3.1#
- Allow batch training in MAGRPOTrainer, IACTrainer, and MAACTrainer.
- Allow multi-turn training in IACTrainer and MAACTrainer.
- Change the x-axis from
data_step to env_step.
Version 1.3.0#
- Use TD error as the critic update target in IACTrainer and MAACTrainer.
Version 1.2.9#
- Add MAACTrainer (separate centralized critic); both IACTrainer and MAACTrainer now support single-turn training.
Version 1.2.8#
- The critic in IACTrainer now estimates V rather than Q.
Version 1.2.7#
- Change IPPOTrainer to IACTrainer.
Version 1.2.6#
- Include MAGRPO, MAREINFORCE, MARLOO, MAREMAX, and IPPO trainers for multi-agent reinforcement learning with LLMs.
- Support for multi-turn training with custom external feedback mechanisms.
- LLM collaboration environments for various tasks.
- Comprehensive documentation and examples for getting started.