AutoGuide

This repository is the implementation of AutoGuide (https://arxiv.org/pdf/2403.08978).

Environment Setup

The webarena environment is built on top of WebArena. Follow WebArena to set up the environment, or reproduce via:

conda env create -f environment.yaml
conda activate autoguide
playwright install

Demonstrations

Human demonstrations are stored as pickle files under extracted_data/reddit/<task_id>/:

• human.pkl: Demonstrations with action sequences (no reasoning, used in original experiments).
• human_reason.pkl: Demonstrations with action sequences and reasoning.

Both formats are supported.

Running Experiments

To run both guideline extraction and evaluation at the same time:

python main.py --model gpt-4-1106-preview --result_dir ./exp_logs --apply_guidelines --extract_guidelines

Using --apply_guidelines together with --extract_guidelines runs both extraction and evaluation simultaneously.