graphrag-drawings

Demo

Demo 1

Demo 2

If the videos do not render in your viewer, open demo1.mp4 or demo2.mp4.

Mini Hybrid GraphRAG system for construction drawings. It uses a vision-language model to extract structured facts from drawing images, links the same real-world entities across sheets, imports them into Neo4j, and answers natural-language questions with source-sheet evidence.

The key idea is simple: drawings are not isolated pages. A room may appear in a floor plan, a door schedule, and a finish material list under slightly different names. This project builds a graph so the system can follow those links instead of relying on page-level vector search alone.

System Flow

What It Can Answer (my demo)

1. "What material is the master bedroom door made of?"

Essential information

• Floor finish: CARPET TILE
• Matched room: BADROOM
• Floor source: G-101

Essential information

• Door tag: 10X23
• Door type: WD
• Door material/finish: THK. 6mm MDF 1PLY / APP' (SPECIAL) DECO SHEET FIN.
• Frame finish: SOLID WOOD / APP' (SPECIAL) DECO SHEET FIN.
• Source: A-003

2. "Tell me the frame finish for the door installed at the entry."

Essential information

• Door tag: 11X22
• Door type: FSD
• Frame finish: THK. 1.6mm STEEL PLATE / APP' (SPECIAL) PAINT FIN.
• Source: A-003

3. "What is the floor finish material for the master bedroom, and what type of door is installed in that room?"

Essential information

• Floor finish: CARPET TILE
• Matched room: BADROOM
• Floor source: G-101

Essential information

• Door tag: 10X23
• Door type: WD
• Door material/finish: THK. 6mm MDF 1PLY / APP' (SPECIAL) DECO SHEET FIN.
• Frame finish: SOLID WOOD / APP' (SPECIAL) DECO SHEET FIN.
• Source: A-003

4. "How does the top reinforcement arrangement of the G1 beam differ by location?"

Essential information

• CENTER: 4-UHD22 (source: S-222)
• INT.(BOTH): 15-UHD22 (source: S-222)

Graph View

The six drawings are imported as one connected graph. Drawing and Evidence nodes keep the provenance for each extracted fact.

Quickstart

The repository includes extracted JSON under data/outputs/, so you can try the graph query demo without calling a vision model.

Start Neo4j:

docker run --rm --name graphrag-neo4j \
  -p 7474:7474 -p 7687:7687 \
  -e NEO4J_AUTH=neo4j/password \
  neo4j:5

Install the package, import the committed extraction, and ask a question:

pip install -e .

export NEO4J_URI=bolt://localhost:7687
export NEO4J_USER=neo4j
export NEO4J_PASSWORD=password

python -m graphrag_drawings.import_to_neo4j --input-dir data/outputs

bash scripts/answer_queries.sh "what is the master bedroom floor finish and door type"

The answer script works without an API key by using keyword routing. If LLM_API_KEY is set, it can use the configured LLM endpoint for query planning.

Full Pipeline

To re-extract the graph from the source drawing images, point the project at an OpenAI-compatible vision endpoint and run the full pipeline:

export LLM_API_KEY=...
# defaults: LLM_BASE_URL=http://localhost:20128/v1  LLM_MODEL=ocg/minimax-m3
export NEO4J_PASSWORD=password

bash scripts/run_full_pipeline.sh
bash scripts/answer_queries.sh "what is the master bedroom floor finish and door type"

The full pipeline extracts all six drawings, combines the per-sheet JSON, links entities across drawings, imports the graph into Neo4j, and then runs question answering over the graph.

Repository Layout

src/graphrag_drawings/
  extract.py              drawing image -> structured JSON
  schema.py               Pydantic extraction schema
  prompts.py              VLM extraction prompts
  combine_extractions.py  merge per-drawing JSON
  link_entities.py        cross-sheet entity linking
  import_to_neo4j.py      load JSON into Neo4j

scripts/
  run_full_pipeline.sh    extract, combine, link, and import all drawings
  answer_queries.sh       answer natural-language questions from Neo4j

data/
  drawings/               six source drawing images
  outputs/                committed extraction results for the demo

docs/
  design/                 architecture, retrieval, and evaluation notes
  assignment/             original assignment brief

Provenance

This project started as a take-home AI engineering assignment about Hybrid GraphRAG for construction drawings. The original brief is preserved in docs/assignment/. The implementation and design notes in this repository are my own work.

License

MIT