query-orchestration/docs/ai.generated/prompts/README.md

# Documentation Generation & Testing

This directory contains scripts and prompts for automatically generating comprehensive system documentation using Claude AI.

## Directory Structure

```
prompts/
├── README.md                                    # This file - overview of the system
├── prompt.for.system.documentation.generation.md  # Main documentation generation prompt
├── regenerate_docs.sh                          # Production documentation generation script
├── test.prompt.md                              # Test prompt for validating the pipeline
└── test_script.sh                              # Test runner for prompt pipeline validation
```

## Overview

This documentation generation system uses Claude AI to automatically analyze the codebase and produce comprehensive technical documentation. It follows a two-phase approach:

1. **Testing Phase**: Validates the pipeline with a simple test
2. **Generation Phase**: Produces full system documentation

## How It Works

### Documentation Generation Process

The system uses a comprehensive prompt (`prompt.for.system.documentation.generation.md`) that instructs Claude to:

1. **Analyze the codebase systematically**:
   - Survey documentation files and entry points
   - Map directory structure and major modules
   - Trace application startup flows
   - Identify architecture patterns

2. **Generate structured documentation**:
   - System overview and architecture
   - Service architecture and data flows
   - API documentation and schemas
   - Developer guides and operational procedures

3. **Output to temporary location**: All generated docs are placed in `./docs_temp/` for review before replacing existing documentation.

### Testing Pipeline

Before running full documentation generation, you can validate the pipeline works correctly:

```bash
# Run the test to ensure Claude integration works
./test_script.sh
```

This test:
- Creates a temporary docs directory
- Pipes a simple test prompt to Claude
- Verifies output file generation
- Confirms the pipeline is working

## Usage

### 1. Test the Pipeline (Recommended First Step)

```bash
cd docs/ai.generated/prompts
./test_script.sh
```

**Expected Output**: Creates `../../docs_temp/test_output.md` with test content.

### 2. Generate Full Documentation

```bash
cd docs/ai.generated/prompts
./regenerate_docs.sh <path_to_prompt_file>
```

**Examples**:
```bash
# Generate full system documentation
./regenerate_docs.sh ./prompt.for.system.documentation.generation.md

# Use relative path from current directory
./regenerate_docs.sh prompt.for.system.documentation.generation.md

# Test with dummy prompt
./regenerate_docs.sh test.prompt.md
```

**Process Flow**:
1. User provides prompt file path as parameter
2. Script validates prompt file exists
3. **WARNING**: User must type 'YES' to confirm expensive operation
4. Script validates prerequisites (Claude CLI)
5. Creates temporary documentation directory (`docs_temp/`)
6. Pipes the specified prompt to Claude AI
7. Claude analyzes the codebase and generates documentation
8. Script prompts user to review and optionally replace existing docs

### 3. Review and Deploy

After generation:
1. Review generated documentation in `docs_temp/`
2. When prompted, choose whether to replace existing docs
3. If approved, existing docs are backed up and new ones deployed

## Prerequisites

### Required Tools

- **Claude CLI**: Install from https://claude.ai/code
- **Bash**: For running the generation scripts
- **Project Access**: Must be run from within the project repository

### Environment Setup

The scripts automatically:
- Detect project root directory
- Create necessary temporary directories
- Handle backup of existing documentation
- Provide colored terminal output for clarity

## Script Details

### `regenerate_docs.sh`

**Purpose**: Main production script for documentation generation

**Usage**: `./regenerate_docs.sh <path_to_prompt_file>`

**Features**:
- Takes prompt file path as parameter for flexibility
- Input validation with helpful usage examples
- **WARNING prompt** requiring 'YES' confirmation for expensive operations
- Automatic prerequisite checking (Claude CLI, file existence)
- Temporary directory management
- User confirmation before replacing docs
- Automatic backup of existing documentation
- Colored output for better UX

**Safety Features**:
- Prompt file existence validation before starting
- Warning confirmation (must type exactly 'YES' to proceed)
- Backs up existing docs with timestamp
- User confirmation before destructive operations
- Comprehensive error checking and reporting

### `test_script.sh`

**Purpose**: Lightweight test to validate the Claude integration pipeline

**Features**:
- Quick validation of Claude CLI availability
- Simple test prompt execution
- File generation verification
- Pass/fail reporting

## Generated Documentation Structure

When successful, the generation process creates:

```
docs_temp/
├── 01-system-overview.md           # High-level architecture and purpose
├── 02-service-architecture.md      # Component breakdown and relationships
├── 03-api-documentation.md         # REST endpoints and schemas
├── 04-data-architecture.md         # Database schemas and data models
├── 05-getting-started.md           # Development setup and workflows
├── 06-code-organization.md         # Package structure and patterns
├── 07-configuration-management.md  # Config options and environment setup
├── 08-operations-guide.md          # Deployment and monitoring
└── README.md                       # Documentation index and navigation
```

## Troubleshooting

### Common Issues

1. **Claude CLI Not Found**
   ```bash
   Error: Claude CLI not found. Please install it first.
   ```
   **Solution**: Install Claude CLI from https://claude.ai/code

2. **No Documentation Generated**
   ```bash
   Error: No documentation was generated in docs_temp/
   ```
   **Solutions**:
   - Check Claude CLI authentication
   - Verify project structure is accessible
   - Review Claude's output for error messages

3. **Permission Issues**
   ```bash
   Permission denied: ./regenerate_docs.sh
   ```
   **Solution**: Make scripts executable:
   ```bash
   chmod +x *.sh
   ```

### Debugging

1. **Run test script first** to isolate pipeline issues
2. **Check Claude output** for specific error messages
3. **Verify working directory** is project root
4. **Ensure network connectivity** for Claude API access

## Maintenance

### Updating the Generation Prompt

The main prompt (`prompt.for.system.documentation.generation.md`) can be modified to:
- Add new documentation sections
- Change analysis methodology
- Adjust output formatting requirements
- Update verification checklist

### Script Modifications

Scripts are designed to be maintainable:
- Clear variable definitions at the top
- Comprehensive error checking
- Modular functions for key operations
- Extensive comments explaining logic

## Integration with Development Workflow

### Recommended Usage Patterns

1. **After Major Architecture Changes**: 
   ```bash
   ./regenerate_docs.sh prompt.for.system.documentation.generation.md
   ```
2. **Before Releases**: Ensure documentation matches current implementation
3. **Onboarding New Developers**: Fresh, accurate documentation for team members
4. **API Changes**: Keep endpoint documentation synchronized with code
5. **Testing Pipeline**: Always test with dummy prompt first:
   ```bash
   ./regenerate_docs.sh test.prompt.md
   ```

### CI/CD Integration Potential

The scripts could be integrated into automated workflows:
- **PR Checks**: Verify docs stay current with code changes
- **Release Automation**: Auto-update docs as part of release process
- **Scheduled Updates**: Regular documentation refreshes

## Quality Assurance

The generation process includes verification steps:
- ✅ All major packages documented
- ✅ API endpoints match route definitions
- ✅ Configuration options are complete
- ✅ Database schemas reflect models
- ✅ Deployment instructions match infrastructure
- ✅ Dependencies and versions are current

This ensures generated documentation accurately reflects the actual codebase rather than outdated or aspirational descriptions.