Skip to content
BranchesTags

Repository files navigation

AssembleX

AssembleX takes a multi-part CAD assembly (for example one generated by an AI CAD tool) and finds out how it can be put together. If it can, AssembleX writes an illustrated assembly manual; if it cannot, it explains which part causes the problem and what to change.

For each assembly it:

  1. Checks the parts. Every part must be a closed solid; small gaps are repaired automatically, and overlapping parts are reported.
  2. Plans the assembly sequence. It searches for an order in which the parts can be taken apart (and therefore put together), checking every step in a physics simulator for collisions, stability under gravity and the number of parts that must be held.
  3. Writes the results. A PDF manual with one illustrated page per step and an instruction per step written by a vision-language model, an animation of the assembly, and, when the design cannot be assembled, a feedback PDF with one page per problem.

Planning runs on the RedMax physics simulator through ASAPx, a fork of the ASAP sequence planner.

Platform. RedMax builds on Linux only. On Windows, use WSL.

Table of Contents

Quick start

After installing:

export OPENAI_API_KEY="sk-..."
python main.py run_pipeline --input path/to/my_assembly

my_assembly is a folder with one mesh file per part, or a single file holding the whole assembly (see Input). The results land in output/my_assembly/; start with summary.json, then manual.pdf or feedback.pdf. The run ends with an overview of the same.

To try it without your own model, run the seven-part example that ships with the repository:

python main.py run_pipeline                # results in output/04489/
python main.py run_pipeline --plan-only    # the same without an API key

Installation

The four numbered steps below are the supported install path. setup.sh runs steps 1 through 3 in one go.

Before starting, make sure you have:

  • Linux, or Windows with WSL.
  • Conda (Miniconda or Miniforge).
  • cmake and a C++ toolchain (build-essential or g++) to build RedMax.
  • An OpenAI API key, needed only for naming, instructions and feedback text.

1. Clone the repository and fetch the planner backend

git clone https://github.com/IDEALLab/AssembleX.git
cd AssembleX
git submodule update --init --recursive ASAPx

2. Create the conda environment

conda env create -f environment.yml
conda activate assemblex

3. Build the RedMax physics binding

cd ASAPx/simulation
python setup.py install
cd ../..

If g++ fails, run sudo apt update && sudo apt install build-essential and retry. To confirm the simulator built correctly:

cd ASAPx && python test_sim/test_simple_sim.py --model box/box_stack --steps 2000 && cd ..

4. Provide your OpenAI API key

The key is read from the OPENAI_API_KEY environment variable or from a .env file in the repository root (OPENAI_API_KEY=sk-...).

Usage

python main.py run_pipeline --input <file or folder> [options]
python main.py run_pipeline --help        # every option

Input

  • One file per part: a folder of .obj, .stl, .ply or .off files. The file names are passed to the part naming as hints.
  • One file for the whole assembly: a .glb, .gltf, .step or .stp file; it is split into its bodies.

All parts must be in one coordinate frame, in their assembled positions, and should be closed solids (watertight). The units do not matter: the assembly is rescaled internally. Your files are never modified.

Options

Every stage runs by default. These switch stages off or add optional ones:

Option Effect
--plan-only plan and render only: no LLM/VLM calls, manual, feedback or animation
--no-manual skip step instructions, manual pages and manual.pdf
--no-feedback skip the feedback for designs that cannot be assembled
--no-video skip the assembly animation
--subassemblies also split the assembly into subassemblies that are put together separately, used wherever the predicted assembly time is shorter
--plan-arm predict the assembly time (on by default with --subassemblies)
--tool-check ask the VLM which tool (if any) each part needs; the manual page shows it
--seq-tool-check require during planning that the tool fits on each part without colliding (implies --tool-check)
--interactive show overlapping parts in a viewer and ask before planning
--replan start over instead of reusing the previous plan
--out DIR results go to DIR/<name>/ instead of output/<name>/

Planner options (--planner, --budget, --max-grippers, ...) are listed by --help; the defaults are the evaluated setup.

Running again

A second run on the same input reuses the plan, the renders and the part names, and only rewrites the documents. When you edit any part file, the next run notices and starts over by itself; --replan forces that.

Datasets

Assemblies of a dataset under assets/<dir>/<id>/ are selected with --id (a single id, a range 00010-00050 or a list 00042,00107) and --dir (default data). run_pipeline_batch --id 00000-00100 runs many of them in parallel. For datasets, see the ASAP download links in ASAPx/README.md.

Outputs

Each assembly gets a folder output/<name>/:

output/<name>/
├── summary.json                # outcome, parts (file -> name), steps, overlaps, links
├── manual.pdf                  # the assembly manual (when the design can be assembled)
├── manual/page_01.png, ...     # its pages as images
├── assembly_instructions.txt   # the step instructions as text
├── assembly.gif                # the assembly, animated
├── feedback.pdf                # one page per problem (when there are problems)
├── feedback/                   # the feedback pages as images + feedback.json
└── internal/                   # plan, renders, preprocessed meshes, caches

summary.json status is one of:

Status Meaning
assemblable a sequence was found; see manual.pdf
not_assemblable no sequence was found; feedback.pdf explains why
invalid_meshes some parts are not closed solids even after repair; see feedback.pdf
invalid_input the input could not be read (message says why)

Overlapping parts are listed under overlapping_parts and get a feedback page even when the design can be assembled. internal/log/ holds the planning graph (tree.pkl) and stats.json; internal/ also keeps the per-step animations from four camera angles.

Changing the behaviour

What Where
Model, part naming, which heuristic weights, subassemblies, rendering, initial pose the top section of settings.py
Manual page look (path trail, subassembly frames, colours) "Manual page rendering" in settings.py; core/manual_generator.py
Wording of every LLM/VLM prompt core/prompts.py
What the prompts are sent with (facts, images) core/feedback_generator.py (naming, instructions, feedback), core/tool_analyzer.py (tools)
Planner internals, timing model "Advanced" in settings.py
Interactive initial-pose choice, debug renders "Debug" in settings.py
The order of the pipeline stages _run_assembly in run_pipeline.py

Tools. The tool catalog is assets/tools/, one folder per tool: <Name>/<Name>.obj (the tool, modelled in the same units as your assemblies), normalization.json ({"scale": s}, the factor its mesh was scaled by) and, optionally, <name>_axes.json (direction: the tool's axis; contact: its tip). A tool without an axes file gets one picked by the VLM the first time it is needed, or by you with --user. The catalog ships with a Phillips-head screwdriver, a hex torque screwdriver and a hex allen key.

How it works

your CAD export
   │
   ▼
core/preprocess.py ─────► closed, normalised part meshes (output/<name>/internal/meshes)
   │
   ▼
core/assembly.py (Eval, Assembly) ── run_pipeline.py drives the stages
   │
   ├─ core/collision_checker.py ─► overlapping parts
   ├─ core/feedback_generator.py ► part names
   ├─ core/sequence_planner.py ──► ASAPx: sequence search in the RedMax simulator,
   │                               then rendering ── tree.pkl + stats.json
   ├─ core/manual_generator.py ──► manual pages, manual.pdf
   └─ core/feedback_generator.py ► step instructions, feedback pages, feedback.pdf

docs/ARCHITECTURE.md describes the modules, the data that passes between them and the planner backend in more detail.

Project Structure

AssembleX/
├── main.py                 # command line: run_pipeline, run_pipeline_batch, rerender
├── run_pipeline.py         # the pipeline stages, summary and batch runner
├── run_common.py           # assembly-id selection helpers
├── settings.py             # configuration (main / advanced / debug)
├── setup.sh                # convenience installer (install steps 1-3)
├── core/                   # the pipeline
│   ├── preprocess.py       # CAD export -> closed, normalised part meshes
│   ├── assembly.py         # Eval / Assembly, workspace handling
│   ├── models.py           # Object, Step, Tool, LLM response schemas
│   ├── sequence_planner.py # SequencePlanner (wraps ASAPx)
│   ├── collision_checker.py   # static overlap check
│   ├── renderer.py         # part images, overlap views, assembly animation
│   ├── feedback_generator.py  # part naming, step instructions, feedback
│   ├── manual_generator.py    # manual pages and PDF
│   ├── tool_analyzer.py    # tool decisions and placement
│   ├── prompts.py          # every LLM/VLM system prompt
│   └── llm.py              # OpenAI client helpers
├── ASAPx/                  # planner backend (fork of ASAP) and RedMax simulator
├── research/               # benchmarks, weight training, diagnostics (python -m research)
├── tests/                  # CLI, preprocessing and end-to-end tests
├── docs/ARCHITECTURE.md    # how the code fits together
├── assets/
│   ├── data/04489/         # the example assembly
│   └── tools/              # tool catalog
├── environment.yml         # conda environment (name: assemblex, python 3.11)
└── pyproject.toml          # packaging, ruff and mypy configuration

Research tooling

The code behind the planner evaluation lives in research/ and is not needed to use the pipeline: planner benchmarks, heuristic-weight training and evaluation, manual validation and diagnostics. It runs as python -m research <command>; python -m research --help lists the commands. These commands work on datasets under assets/ that are already preprocessed, and cache their plans under assets/assembly_cache/.

Contributing

Contributions are welcome through issues and pull requests. The code follows ruff with line-length = 88 and the rule sets configured in pyproject.toml; the ASAPx/ submodule is excluded. Run ruff check . before submitting, and keep emojis and decorative formatting out of code and markdown. The development extras (ruff, mypy, pytest, pre-commit) install with pip install -e ".[dev]". Run the tests with python -m pytest tests/; tests/test_smoke.py runs the pipeline end to end (a few minutes, no API key needed).

License

This project is released under the MIT License. The bundled ASAPx/ backend derives from prior work (see below) and carries its own upstream license, which the MIT license here does not override.

Credits and Acknowledgments

The ASAPx backend is a fork of ASAP (Tian et al., Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility, ICRA 2024), which builds on Assemble-Them-All (Tian et al., Assemble Them All: Physics-Based Planning for Generalizable Assembly by Disassembly, SIGGRAPH Asia 2022). Both rely on the RedMax (REDMAX: Efficient & Flexible Approach for Articulated Dynamics, Wang et al., SIGGRAPH 2019) rigid-body simulator for the feasibility and stability checks. The example assembly 04489 comes from the ASAP multi-part dataset. Naming, instructions and feedback text use the OpenAI API.

I developed this work in the IDEAL Lab at ETH Zürich.

Contact

Maintained by Faustin von Arx (@FaustinVonArx). Reach me at fvonarx@ethz.ch, or open a GitHub issue for questions and bugs.

About

No description, website, or topics provided.

Resources

Readme
MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages