AutoValid — AI-Powered Web Testing Automation

Microservices platform that automates web application testing end-to-end: crawl a target site, extract user requirements from uploaded documents with ML, generate test cases, synthesize test data, execute tests, and analyze failures — all from a single modern React dashboard.

🏗️ Architecture Overview

┌──────────────────────────────────────────────────────────────┐
│                  Browser / React Dashboard                   │
│                   http://localhost:3000                      │
│  Crawler • User Stories • Context • Test Cases • Data Gen   │
│  Execution • Bugs • History • Reports • Settings            │
└───────────────────────┬──────────────────────────────────────┘
                        │ HTTP REST + WebSocket (live events)
┌───────────────────────▼──────────────────────────────────────┐
│                   Orchestrator  :3001                        │
│  pipeline controller • session lifecycle • WebSocket relay  │
│  pause / stop / resume  •  heartbeat monitoring             │
└──┬──────────┬────────────┬──────────┬───────────┬───────────┘
   │          │            │          │           │
┌──▼───┐ ┌───▼──┐ ┌───────▼──┐ ┌────▼──┐ ┌─────▼──────┐
│Explor│ │Contxt│ │Test Cases│ │DataGen│ │ Execution  │
│:5000 │ │:3003 │ │   :3007  │ │ :3004 │ │   :3005    │
└──┬───┘ └──────┘ └──────────┘ └───────┘ └─────┬──────┘
   │  State graph → Neo4j                       │ 3-strategy click fallback
   │                                            │ live WS streaming
   │  ┌─────────────────────┐           ┌───────▼──────┐
   └──│  UserStoriesExtract │           │ Bug Analyzer │
      │  (FLAN-T5 + LoRA)   │           │   :3006      │
      │       :8000          │           └──────────────┘
      └─────────────────────┘
                                   (All services share)
  ┌──────────────────────────────────────────────────────┐
  │  MongoDB :27017  │  Redis :6379  │  Neo4j :7687      │
  └──────────────────────────────────────────────────────┘

Pipeline Flow: Upload user stories document + start crawl → Exploration crawls site & ML extracts stories in parallel → Context Detection → Test Case Generation → Data Generation → Execution → Bug Analysis → Reports

New in this release: Full pipeline pause / stop / resume controls, improved execution engine with three-strategy click fallback, professional StateFlow graph visualization (Cytoscape.js), redesigned History and Settings pages, and real-time WebSocket execution streaming.

📦 Services

Service	Port	Language	Role
Frontend	3000	React + Vite + Tailwind	Dashboard UI — Crawler, Execution, Reports, History, Settings
Orchestrator	3001	Node.js	Pipeline controller, sessions, WebSocket relay, pause/resume
Exploration	5000	Node.js + Playwright	Web crawler, state-graph extraction, user stories API
Context Detection	3003	Node.js	Form/element classification with AI
Test Case Generation	3007	Node.js	AI-driven test scenario synthesis
Data Generator	3004	Node.js	Synthetic test data via Faker.js
Execution	3005	Node.js + Playwright	Test runner with live WS streaming, pass/fail recording
Bug Analyzer	3006	Node.js	Failure detection and AI root-cause analysis
UserStoriesExtractor	8000	Python / FastAPI	FLAN-T5 + LoRA for structured user story extraction

Note: The UserStoriesExtractor ML service is optional in local mode. When unavailable, the Exploration service automatically falls back to a rule-based "As a… I want… so that…" parser — the rest of the pipeline continues normally.

Infrastructure

Component	Port	Notes
MongoDB	27017	Local install or Docker (full mode)
Redis	6379	Docker (`docker compose -f docker-compose.full.yml up -d redis`)
Neo4j	7687	Docker (full mode) — stores StateFlow graph data

Service Details

Frontend — React + Vite + Tailwind dashboard. Sections:

Crawler — start/configure crawl sessions, upload user story documents
User Stories — view ML-extracted story cards with confidence rings
Context — classified form fields and navigation elements
Test Cases — AI-generated test scenarios
Data Generator — synthetic data sets per test case
Execution — live test run with pass/fail per case, skip controls, real-time progress
Bugs — deduplicated failure list with AI root-cause summaries
History — all past sessions with metadata, filterable table
Reports — Application Context summary + interactive StateFlow Graph (Cytoscape.js)
Settings — 5-tab sidebar: General, AI Providers, Crawler, Execution, Appearance

Orchestrator — Central pipeline controller. Manages crawl sessions in MongoDB, routes pipeline-stage jobs to downstream services, relays real-time events over WebSocket. Supports pause / stop / resume at any pipeline stage with heartbeat monitoring to detect stalled services. Uses ORCHESTRATOR_INTERNAL_URL for Docker-internal callback routing.

Exploration — Playwright-based crawler. Navigates the target site, extracts page elements (forms, links, screenshots), and builds a state graph stored in Neo4j and local file storage. Also exposes the User Stories REST API (POST /api/user-stories/upload, POST /api/user-stories/process). Falls back to local regex extraction when the ML service is unreachable.

UserStoriesExtractor (ML) — Python FastAPI service in services/exploration/src/UserStoriesExtractor/. Uses a fine-tuned FLAN-T5 model with LoRA adapters to parse uploaded PDF/DOCX/TXT documents into structured user stories (actor · action · benefit · acceptance criteria). Required only for Docker full mode or if you install PyTorch locally.

Context Detection — Classifies form fields and UI elements using AI (Gemini / Ollama / OpenRouter).

Test Case Generation — Generates test cases from the classified context using configurable AI models.

Data Generator — Produces realistic synthetic values for each test-case field using Faker.js, with field-type awareness (email, phone, name, date…).

Execution — Replays test cases against the live target site with Playwright. Features:

Three-strategy click fallback: standard → force-click → keyboard Enter (handles custom button implementations)
Live WebSocket streaming of each step result to the dashboard
Skip-test support (skips remaining steps in a case, continues to next)
Pipeline pause / stop hooks — executor checks flags between steps
Artifacts saved per crawl session (screenshots, logs)

Bug Analyzer — Correlates execution failures, infers root causes with AI, and surfaces a deduplicated bug report with severity ratings.

StateFlow Graph — Interactive state-graph visualization in the Reports tab, powered by Cytoscape.js:

Node types: Entry (diamond, blue), Form (amber), Content-Heavy (emerald), Dead-End (red dashed), Standard (violet)
Typed edges: Navigation, Form Submit (dashed amber), Button Click (indigo)
Five layout modes: Grid (default), Tree Top-Down, Concentric (depth rings), Force-Directed, Radial
Glass-morphism floating UI: stats badge, collapsible legend, zoom/fit/export toolbar, refresh + fullscreen toggle
Neighborhood dimming on hover, rich tooltips, click-to-open detail modals

🚀 Quick Start (Local — no Docker for app services)

Prerequisites

Node.js 18+ and npm 9+
Docker Desktop (for MongoDB / Redis / Neo4j infrastructure)
PowerShell 5.1+ (Windows)
An AI API key — OpenRouter is recommended (free tier at https://openrouter.ai); Gemini or Ollama also work
(Optional) Python 3.9+ with PyTorch — only needed for ML-based user story extraction

1. Clone and Configure

git clone https://github.com/Hassan-301/FYP-AutoValid.git
cd autovalid-microservices

copy .env.example .env
notepad .env

At minimum set OPENROUTER_API_KEY (or GEMINI_API_KEY) in .env.

2. Install Dependencies

# Installs npm packages for all Node services
.\install-all.ps1

(Optional) To also install Python deps for ML extraction:

cd services\exploration\src\UserStoriesExtractor
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
cd ..\..\..\..

3. Start Infrastructure

docker compose -f docker-compose.full.yml up -d mongodb redis neo4j

4. Start All Services

.\start-all.ps1

Starts all Node services in separate terminal windows and attempts to start the Python ML service. If Python/PyTorch is not installed, a warning is shown and the app continues with the rule-based fallback — no error.

Optional flags:

.\start-all.ps1 -ForceRestart      # Kill existing listeners first
.\start-all.ps1 -InstallDeps       # Run npm install before starting
.\start-all.ps1 -InstallPythonDeps # Run pip install before starting Python service

5. Open the Dashboard

http://localhost:3000

Sign up for an account, then follow the pipeline walkthrough below.

6. Stop All Services

.\stop-all.ps1

🐳 Docker Mode (Fully Containerized)

No Node.js, no MongoDB, no Python required on your machine.
Everything runs inside Docker containers — including the ML service.

Prerequisites

Docker Desktop installed and running
An OpenRouter API key (free at https://openrouter.ai)

Setup

git clone https://github.com/Hassan-301/FYP-AutoValid.git
cd autovalid-microservices

copy .env.example .env
notepad .env           # Set OPENROUTER_API_KEY and change passwords if desired

Start

.\docker-up.ps1

Builds all images and starts every service. First run downloads base images and installs Playwright browser binaries — allow ~5–10 minutes.

Access

URL	Service
http://localhost	Main app (via API gateway)
http://localhost:3000	Frontend direct
http://localhost:3001	Orchestrator API
http://localhost:7474	Neo4j Browser

Stop

.\docker-up.ps1 -Down

View Logs

.\docker-up.ps1 -Logs

Wipe Everything (fresh DB)

docker compose -f docker-compose.full.yml down -v

📋 Environment Configuration

Copy .env.example to .env and edit the values. Key variables:

# ===== AI Provider (choose one) =====
# openrouter — multi-model via API (recommended for Docker mode)
# gemini     — cloud, fast (recommended for local mode)
# ollama     — local, no API key required, slower

CONTEXT_AI_PROVIDER=openrouter
OPENROUTER_API_KEY=your_openrouter_key_here
CONTEXT_AI_MODEL=meta-llama/llama-3.1-8b-instruct

TEST_CASE_GEN_AI_PROVIDER=openrouter
TEST_CASE_GEN_AI_MODEL=meta-llama/llama-3.1-8b-instruct

# Optional: Ollama for local models
OLLAMA_BASE_URL=http://127.0.0.1:11434

# ===== Service Ports =====
FRONTEND_PORT=3000
ORCHESTRATOR_PORT=3001
CONTEXT_PORT=3003
DATA_GEN_PORT=3004
EXECUTION_PORT=3005
BUG_ANALYZER_PORT=3006
TEST_CASE_GEN_PORT=3007

# ===== Databases =====
MONGODB_URL=mongodb://127.0.0.1:27017/autovalid
REDIS_URL=redis://127.0.0.1:6379

# ===== Crawler Defaults =====
MAX_DEPTH=10
MAX_PAGES=100
CRAWLER_TIMEOUT=300000

# ===== Execution Tuning =====
EXECUTION_STEP_TIMEOUT_MS=20000
EXECUTION_NAV_TIMEOUT_MS=60000

# ===== UserStoriesExtractor ML Tuning (Docker full mode only) =====
USER_STORIES_NUM_BEAMS=2        # beam search width (quality vs speed)
USER_STORIES_MAX_TOKENS=384     # max output tokens per story

✅ Using the Pipeline

1. Open the Dashboard

http://localhost:3000

Sign up for an account and log in.

2. Start a Crawl with User Stories

Open the Crawler section:

Target — enter the URL to crawl (e.g. https://example.com)
Authentication — optional: provide credentials if the site requires login
User Stories — drag-and-drop or select a PDF, DOCX, or TXT document containing your requirements; a progress bar confirms the upload

Click Start Crawl. The service simultaneously crawls the site with Playwright and sends the uploaded document to the ML service for extraction. Both complete asynchronously and results appear in their respective dashboard sections.

3. View Extracted User Stories

Navigate to User Stories. Each story card shows:

Actor · Action · Benefit breakdown
Acceptance criteria list
Confidence score ring (hybrid: 25 % model probability + 50 % structural quality + 25 % JSON completeness)
Summary statistics: total stories, average confidence, actor distribution

4. Run the Automated Pipeline

From the crawl session card, trigger each stage in order. Each stage shows a live status badge (Pending → Running → Completed / Failed) and a result count.

Pipeline Controls — once a pipeline stage is running you can:

Pause — suspends execution between steps (resumes from where it left off)
Stop — gracefully halts the current stage and marks it stopped
Resume — continues a paused pipeline from the current step

Stage	Service	What it does
Generate App Context	Context Detection :3003	Classifies forms and navigation with AI
Generate Test Cases	Test Case Generation :3007	Creates natural-language test scenarios
Generate Test Data	Data Generator :3004	Fills each test case with synthetic data
Execute Test Cases	Execution :3005	Runs tests with Playwright, streams results live
Analyze Bugs	Bug Analyzer :3006	Infers root causes for failures

5. Review Results

Execution — per-test pass/fail with logs, screenshots, and live step streaming
Bugs — deduplicated list of failures with AI root-cause summaries and severity
History — all past crawl sessions in a searchable table (MongoDB-backed)
Reports — Application Context summary + interactive StateFlow Graph
- Five layout modes: Grid, Tree, Concentric, Force-Directed, Radial
- Click nodes/edges to open detail modals; hover to highlight neighborhood
- Collapsible legend; fullscreen toggle; export PNG

🛑 Stopping Services

.\stop-all.ps1

Kills all Node and Python service processes by port (3000, 3001, 3003–3007, 5000, 8000) and stops local infrastructure containers.

To stop only the Docker infrastructure:

docker compose -f docker-compose.full.yml down

📊 Monitoring & Logs

Service Windows

start-all.ps1 opens each service in its own PowerShell window titled with the service name and port. Scroll the window or close it individually to stop that service.

Database Access

# MongoDB — inspect sessions, stories, results
mongosh mongodb://127.0.0.1:27017/autovalid

# Redis — inspect job queues
redis-cli
> INFO
> KEYS *

🔧 Advanced Configuration

AI Model Selection

OpenRouter (Recommended — multi-model, free tier available)

CONTEXT_AI_PROVIDER=openrouter
OPENROUTER_API_KEY=your_key
CONTEXT_AI_MODEL=meta-llama/llama-3.1-8b-instruct
TEST_CASE_GEN_AI_MODEL=meta-llama/llama-3.1-8b-instruct

Gemini (Cloud, fast)

CONTEXT_AI_PROVIDER=gemini
GEMINI_API_KEY=your_key
GEMINI_MODEL=gemini-2.0-flash
TEST_CASE_GEN_AI_MODEL=gemini-2.0-flash

Ollama (Local — no API key, slower)

CONTEXT_AI_PROVIDER=ollama
OLLAMA_BASE_URL=http://127.0.0.1:11434

# Pull model first
ollama pull llama3.1:8b
ollama serve

User Story Extraction

In local mode (without Python/PyTorch), user stories are extracted using a rule-based parser:

Parses "As a <actor>, I want to <action>, so that <benefit>" templates
Extracts acceptance criteria from "verified by…" and "- Verify…" bullet patterns

In Docker full mode, the FLAN-T5 + LoRA ML model is used instead, giving richer structured output with confidence scores.

Tune ML behaviour via .env (Docker only):

USER_STORIES_NUM_BEAMS=2   # more beams → better quality, slower
USER_STORIES_MAX_TOKENS=384

🐛 Troubleshooting

A service fails to start

# Re-run with install flags to refresh dependencies
.\start-all.ps1 -ForceRestart -InstallDeps -InstallPythonDeps

Check the service window for the stack trace.

MongoDB connection errors

Ensure mongod is running locally before calling start-all.ps1:

# Check if MongoDB is listening
Test-NetConnection -ComputerName 127.0.0.1 -Port 27017

Start it via MongoDB Compass, the Windows service, or:

mongod --dbpath C:\data\db

Redis unavailable

docker compose up -d redis
# verify
docker compose ps

UserStoriesExtractor ML model errors

The model is downloaded on first run (~1 GB). Ensure you have internet access and enough disk space. On CPU the first inference request may take 30–60 seconds.

# Run the service manually to see errors
cd services\exploration\src\UserStoriesExtractor
.\.venv\Scripts\python.exe -m uvicorn src.serve:app --host 0.0.0.0 --port 8000

Port already in use

# Find and kill the occupying process
.\stop-all.ps1

# Or target a single port
Get-NetTCPConnection -LocalPort 5000 -State Listen | Select-Object OwningProcess
Stop-Process -Id <PID> -Force

Gemini API key errors

Generate a key: https://makersuite.google.com/app/apikey
Set GEMINI_API_KEY=your_key in .env
Restart affected services with .\stop-all.ps1 ; .\start-all.ps1

Pull required model

ollama pull llama3.1:8b

Verify connectivity

curl http://127.0.0.1:11434/api/tags


### Services keep restarting

```bash
# View detailed service logs
docker-compose logs -f --tail=50

# Check memory/CPU
docker stats

# Increase Docker memory limit in Docker Desktop settings

📁 Project Structure

autovalid-microservices/
├── frontend/                    # React Dashboard UI (Port 3000)
│   ├── src/
│   │   ├── pages/dashboard/
│   │   │   ├── CrawlerPage.jsx         # Automation Pipeline UI
│   │   │   ├── ExecutionPage.jsx       # Test Execution Results
│   │   │   ├── BugsPage.jsx            # Bug Reports & Analysis
│   │   │   ├── HistoryPage.jsx         # Session History
│   │   │   ├── SettingsPage.jsx        # Configuration UI
│   │   │   └── ...
│   │   ├── components/
│   │   │   ├── DashboardSidebar.jsx    # Navigation (incl. User Stories)
│   │   │   ├── DashboardHeader.jsx     # Page Headers
│   │   │   └── DashboardSections.jsx   # Page Router
│   │   ├── pages/dashboard/
│   │   │   ├── UserStoriesPage.jsx      # User Stories viewer
│   │   │   └── ...                     # Other dashboard pages
│   │   └── legacy/                     # Legacy Interop Layer
│   └── package.json
│
├── services/
│   ├── exploration/                    # Web Crawler (Port 5000)
│   │   ├── src/crawler/                # Playwright automation
│   │   ├── src/models/                 # Data models (incl. UserStory)
│   │   ├── src/api/                    # API routes (incl. user-stories-routes)
│   │   └── src/routes/                 # Legacy API endpoints
│   │
│   ├── orchestrator/                   # Job Queue Manager (Port 3001)
│   │   ├── src/middleware/             # Auth, logging
│   │   ├── src/routes/                 # Session management
│   │   └── src/services/               # BullMQ queue
│   │
│   ├── context-detection/              # AI Form Analysis (Port 3003)
│   │   ├── src/services/               # AI client, analysis
│   │   └── src/routes/                 # Analysis endpoints
│   │
│   ├── data-generator/                 # Synthetic Test Data (Port 3004)
│   │   ├── src/services/               # Faker.js generation
│   │   └── src/routes/                 # Generation endpoints
│   │
│   ├── execution/                      # Playwright Executor (Port 3005)
│   │   ├── src/services/               # Script generator, runner
│   │   └── src/routes/                 # Execution endpoints
│   │
│   ├── bug-analyzer/                   # Bug Detection (Port 3006)
│   │   ├── src/services/               # AI analysis service
│   │   └── src/routes/                 # Analysis routes
│   │
│   └── test-case-generation/           # AI Test Creation (Port 3007)
│       ├── src/services/               # AI client, generator
│       └── src/routes/                 # Generation endpoints
│
├── docker-compose.yml                  # Service definitions
├── .env.example                        # Configuration template
├── .env                                # Configuration (gitignored)
├── package.json                        # Root scripts
├── README.md                           # This file
└── install-all.ps1, start-all.ps1      # Helper scripts

# Service URLs
- Frontend: http://localhost:3000
- Exploration API: http://localhost:5000/api
- Orchestrator API: http://localhost:3001/api
- Neo4j Browser: http://localhost:7474
- MongoDB: mongodb://127.0.0.1:27017/autovalid

📝 Available NPM Scripts

# Root directory
npm run install-all              # Install all service dependencies
npm run start-all               # Start all services (background)
npm run stop-all                # Stop all services

# Docker operations
docker-compose up -d            # Start all with Docker
docker-compose down             # Stop all
docker-compose logs -f          # View live logs
docker-compose build            # Rebuild images

🚀 Deployment Guide

Production Deployment Checklist

Docker Production Build

# Build optimized images
docker-compose build --no-cache

# Verify image sizes
docker images | grep autovalid

# Test production environment
docker-compose -f docker-compose.yml up -d

📞 Support & Contributing

Reporting Issues

GitHub Issues: https://github.com/Hassan-301/FYP-AutoValid/issues
Include logs from docker-compose logs -f
Provide .env configuration (sanitized)

Contributing

Fork the repository
Create a feature branch: git checkout -b feature/your-feature
Commit changes: git commit -m "Add your feature"
Push to branch: git push origin feature/your-feature
Submit a Pull Request

📄 License & Attribution

AutoValid - Automated Test Case Generation and Bug Detection Platform FYP (Final Year Project)

Repository: https://github.com/Hassan-301/FYP-AutoValid
Architecture: Microservices with AI/ML pipeline
Technologies: Node.js, React, MongoDB, Docker, Playwright, Gemini AI

📋 API Endpoints

Orchestrator (Main API)

POST   /api/sessions              # Create crawl session
GET    /api/sessions              # List sessions
GET    /api/sessions/:sessionId   # Get session status
WS     /ws?sessionId=xxx          # WebSocket for real-time updates

Exploration Service

POST   /explore                   # Start crawling
GET    /session/:sessionId        # Get crawl data

User Stories (Exploration Sub-module)

POST   /api/user-stories/upload              # Upload document (PDF/DOCX/TXT)
POST   /api/user-stories/process             # Extract & store user stories
GET    /api/user-stories/:sessionId          # Get processed stories + stats
DELETE /api/user-stories/:sessionId          # Delete session stories

Context Detection

POST   /analyze                   # Analyze elements

Data Generator

POST   /generate/:fieldId         # Generate test data

Execution

POST   /execute/:testId           # Execute test

Bug Analyzer

POST   /analyze                   # Analyze results

🔄 Migration Timeline (4 Weeks)

Week 1: Frontend + Orchestrator ✅

✅ Extract frontend service
✅ Create orchestrator with job queue
✅ Setup API Gateway
✅ MongoDB + Redis infrastructure

Week 2: Crawler → 3 Services

Extract exploration service (from backend/crawler)
Create context-detection service
Map element relationships
Implement Neo4j graph storage

Week 3: Execution + Bug Analyzer

Create execution service
Create bug-analyzer service
Implement defect detection
Create comprehensive reporting

Week 4: Docker Optimization + Testing

Optimize Docker images
Setup CI/CD pipeline
Load testing & scaling
Production deployment

🐳 Docker Commands

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f [service-name]

# Rebuild service
docker-compose build [service-name]

# Restart service
docker-compose restart [service-name]

# Stop all services
docker-compose down

# Remove volumes (reset data)
docker-compose down -v

🔧 Development

Running services locally (without Docker)

# Terminal 1: Start orchestrator
cd orchestrator
npm install && npm run dev

# Terminal 2: Start exploration
cd services/exploration
npm install && npm run dev

# Terminal 3: Start frontend
cd frontend
npm install && npm run dev

Environment Variables

Create .env in project root:

NODE_ENV=development
LOG_LEVEL=debug
DB_PASSWORD=your_secure_password
NEO4J_PASSWORD=neo4j_password
REDIS_URL=redis://redis:6379
MONGODB_URL=mongodb://autovalid:password@mongodb:27017/autovalid?authSource=admin

📊 Monitoring

Health Checks

All services expose /health endpoint:

curl http://localhost:3001/health    # Orchestrator
curl http://localhost:3002/health    # Exploration
# etc...

Redis Queue Monitor

docker-compose exec redis redis-cli
> INFO
> KEYS crawls:*

MongoDB

docker-compose exec mongodb mongosh -u autovalid -p <password> --authenticationDatabase admin
> use autovalid
> db.crawl_sessions.find()
> db.job_queue.find()

Neo4j

Access Neo4j Browser at http://localhost:7474

User: neo4j
Password: (from NEO4J_PASSWORD)

📚 Documentation

🔐 Security

API Gateway validates all requests
Service-to-service communication on internal network only
JWT authentication (to implement in Week 2)
Rate limiting per endpoint
CORS configured for frontend domain

🔌 Integration Points

Frontend ↔ Orchestrator

REST API for session management
WebSocket for real-time updates
Polling for status updates

Orchestrator ↔ Services

HTTP REST endpoints
BullMQ job queue
Redis pub/sub for events

Services ↔ Databases

MongoDB: persistent session/test data
Redis: caching and job queue
Neo4j: graph data and relationships

🐛 Troubleshooting

Service won't start

# Check logs
docker-compose logs [service-name]

# Verify dependencies
docker-compose exec mongodb mongosh -u autovalid -p <password> --authenticationDatabase admin
docker-compose exec redis redis-cli ping

Database connection issues

# Verify DATABASE_URL in .env
# Check MongoDB is healthy
docker-compose exec mongodb mongosh -eval 'db.adminCommand("ping")'

Redis connection issues

# Check Redis is running
docker-compose exec redis redis-cli ping
# Should return PONG

📈 Scaling

Horizontal Scaling

Each service can be replicated:

# docker-compose.yml
exploration:
  deploy:
    replicas: 3

Load Distribution

Nginx routes requests to service replicas
Redis pub/sub for distributed events
MongoDB for shared state

Performance Optimization

Redis cache layer for frequent queries
Database connection pooling
Service-level caching
CDN for static frontend assets

🤝 Contributing

Branch naming: feature/service-name or fix/issue-number
Commit messages: [service-name] description
Create pull request with test results
Update service README with changes

📝 License

MIT - See LICENSE file

📞 Support

For issues or questions:

Check documentation
Review service logs
Open GitHub issue with error details and logs

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
api-gateway		api-gateway
databases		databases
frontend		frontend
orchestrator		orchestrator
services		services
shared/libs		shared/libs
.env		.env
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
README.md		README.md
docker-compose.full.yml		docker-compose.full.yml
docker-up.ps1		docker-up.ps1
install-all.ps1		install-all.ps1
install-log.txt		install-log.txt
package-lock.json		package-lock.json
package.json		package.json
patch_generator.py		patch_generator.py
start-all.ps1		start-all.ps1
stop-all.ps1		stop-all.ps1
test-ml-batch.mjs		test-ml-batch.mjs
test-userstories-e2e.mjs		test-userstories-e2e.mjs

Latest commit

History

Folders and files

Repository files navigation

AutoValid — AI-Powered Web Testing Automation

🏗️ Architecture Overview

📦 Services

Infrastructure

Service Details

🚀 Quick Start (Local — no Docker for app services)

Prerequisites

1. Clone and Configure

2. Install Dependencies

3. Start Infrastructure

4. Start All Services

5. Open the Dashboard

6. Stop All Services

🐳 Docker Mode (Fully Containerized)

Prerequisites

Setup

Start

Access

Stop

View Logs

Wipe Everything (fresh DB)

📋 Environment Configuration

✅ Using the Pipeline

1. Open the Dashboard

2. Start a Crawl with User Stories

3. View Extracted User Stories

4. Run the Automated Pipeline

5. Review Results

🛑 Stopping Services

📊 Monitoring & Logs

Service Windows

Database Access

🔧 Advanced Configuration

AI Model Selection

User Story Extraction

🐛 Troubleshooting

A service fails to start

MongoDB connection errors

Redis unavailable

UserStoriesExtractor ML model errors

Port already in use

Gemini API key errors

Pull required model

Verify connectivity

📁 Project Structure

📝 Available NPM Scripts

🚀 Deployment Guide

Production Deployment Checklist

Docker Production Build

📞 Support & Contributing

Reporting Issues

Contributing

📄 License & Attribution

📋 API Endpoints

Orchestrator (Main API)

Exploration Service

User Stories (Exploration Sub-module)

Context Detection

Data Generator

Execution

Bug Analyzer

🔄 Migration Timeline (4 Weeks)

Week 1: Frontend + Orchestrator ✅

Week 2: Crawler → 3 Services

Week 3: Execution + Bug Analyzer

Week 4: Docker Optimization + Testing

🐳 Docker Commands

🔧 Development

Running services locally (without Docker)

Environment Variables

📊 Monitoring

Health Checks

Redis Queue Monitor

MongoDB

Neo4j

📚 Documentation