MASA: Model-Aware Skill Alignment for LLM Agents

Aligning skill formulations with each target backbone's capacity through hierarchical model-conditioned evolution.

🔥 News

• [06/2026] Code for MASA is released!
• [05/2026] MASA paper is released on arXiv.

📖 Overview

Existing skill-library systems construct a single shared library and reuse it across different LLM backbones. However, we empirically show that a skill that boosts one backbone can actively degrade another — the optimal skill formulation depends on the target model's capacity.

MASA (Model-Aware Skill Alignment) addresses this by aligning skill formulations with each target backbone without modifying agent weights. It treats skill alignment as a hierarchical search problem driven by environment feedback:

1. Hierarchical Model-Conditioned Skill Evolution: A teacher LLM iteratively rewrites skills conditioned on a structured model card, using hill-climbing for general skills and UCB-driven tree search for task-specific skills.

2. Model-Conditioned Skill Rewriter (MASA-Rewriter): A lightweight Qwen3-4B model trained to replicate the evolution pipeline's rewriting policy, enabling skill adaptation in a single forward pass without environment interaction.

Key Results

• Achieves the highest success rate across 3 environments (ALFWorld, WebShop, Search) and 4 Qwen3 backbones (4B–32B)
• Up to +25.8 points improvement over the strongest baseline (Qwen3-8B on ALFWorld)
• MASA-Rewriter (4B) outperforms DeepSeek-V4-Pro teacher at negligible cost
• Generalizes to unseen tasks and environments in a single forward pass

---

📥 Supported Models

Backbone	Parameters	Model Card
Qwen3-4B	4B	model_cards/qwen3_4b.yaml
Qwen3-8B	8B	model_cards/qwen3_8b.yaml
Qwen3-14B	14B	model_cards/qwen3_14b.yaml
Qwen3-32B	32B	model_cards/qwen3_32b.yaml

Embedding model for skill retrieval: Qwen3-Embedding-0.6B

---

🚀 Getting Started

Installation

git clone https://github.com/jianxiangyu/MASA_.git
cd MASA_

pip install -e .
pip install vllm==0.11.0
pip install flash-attn==2.7.4.post1 --no-build-isolation --no-cache-dir
pip install openai

Environment Setup

ALFWorld

pip install alfworld
pip install gymnasium==0.29.1
pip install stable-baselines3==2.6.0

# Download PDDL & Game files and pre-trained MaskRCNN detector
alfworld-download -f

WebShop

cd agent_system/environments/env_package/webshop/webshop
./setup.sh -d all

Search

cd agent_system/environments/env_package/search/third_party
pip install -e .
pip install gym==0.26.2

Download Models

Place model weights under a ./model/ directory (or modify paths in scripts):

model/
├── Qwen3-4B/
├── Qwen3-8B/
├── Qwen3-14B/
├── Qwen3-32B/
└── Qwen3-Embedding-0.6B/

Download Search Data

For the Search environment, download the retrieval index and corpus:

python examples/search/searchr1_download.py

This downloads the wiki-18 index and corpus from HuggingFace to ./data/searchR1/.

---

🏃 Evaluation with MASA Skills

ALFWorld

# Edit MODEL_SIZE (4b/8b/14b/32b) and SKILL_JSON as needed
bash scripts/run_alfworld.sh

WebShop

bash scripts/run_webshop.sh

Search

First, start the retrieval server:

bash scripts/start_retrieval_server.sh

Then run evaluation:

bash scripts/run_search.sh

Configuring Model Size

Each script supports switching model sizes:

MODEL_SIZE="8b"   # Options: 4b, 8b, 14b, 32b

The corresponding MASA skill file is selected via:

SKILL_JSON="./memory_data/<env>/<size>_masa_skills.json"

---

📋 Skill Bank Structure

Available Skill Variants

Each environment provides multiple skill formulations for comparison:

File Pattern	Description
<size>_masa_skills.json	MASA — Model-aware evolved skills (our method)
<size>_adapter_skills.json	DS-Adapter — One-shot teacher rewrite baseline
base_skills.json	Base Skill — Shared model-agnostic skills (from SkillRL)
noskill.json	No-skill control

---

📚 Citation

If you find our work helpful, please consider citing:

@article{yu2026skill,
  title={Skill is Not One-Size-Fits-All: Model-Aware Skill Alignment for LLM Agents},
  author={Yu, Jianxiang and Zhu, Jiapeng and Lin, Bochen and Cui, Qier and Ding, Zichen and Li, Xiang},
  journal={arXiv preprint arXiv:2605.30723},
  year={2026}
}

🙏 Acknowledgement

We thank the open-source community and the following projects for making this work possible: verl, verl-agent, SkillRL, Qwen, ALFWorld, WebShop.