Testing Guide

This document outlines the comprehensive testing approach for PadawanForge, optimized for bun and focused on essential core functionality with streamlined execution.

Related Documentation: See Release Structure for complete testing requirements by release type and quality gates.

Quick Start

Run All Tests

# Test local development server
bun run test:all

# Test production deployment
bun run test:all:prod

# Test with custom URL
bun run test:all https://your-custom-domain.com

Prerequisites

# Ensure development server is running
bun run dev

# In another terminal, run tests
bun run test:all

Test Architecture

Unified Test Suite

The testing system has been streamlined from multiple separate test scripts into a single comprehensive test runner that covers all essential systems with intelligent dependency management and early exit on critical failures.

Test Configuration

// test.config.js - Bun test configuration
export default {
  testDir: './tests',
  timeout: 30000,
  retries: 2,
  workers: 1, // Sequential execution for API tests
  reporter: 'spec',
  use: {
    baseURL: process.env.TEST_BASE_URL || 'http://localhost:4321',
    timeout: 15000,
  }
};

Test Coverage

1. Health Check (Critical - Early Exit)

Purpose: Validate basic system availability before proceeding with complex tests

Coverage:

  • API server responsiveness and uptime
  • Basic endpoint availability
  • Environment variable validation
  • Database connectivity check

Example Output:

🔍 1. Health Check
✅ API Health - Response time: 45ms
✅ Environment Variables - All required vars present
✅ Database Connection - D1 accessible

2. AI Integration Testing

Purpose: Verify Cloudflare Workers AI functionality and model availability

Coverage:

  • Cloudflare Workers AI connection validation
  • Model availability verification (@cf/meta/llama-3.1-8b-instruct)
  • Response quality and structure validation
  • Error handling and fallback mechanisms
  • Rate limiting behavior

Test Flow:

// AI Connection Test
const aiResponse = await fetch('/api/ai/test-connection', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ prompt: 'Test prompt for validation' })
});

// Validate response structure
expect(aiResponse.status).toBe(200);
expect(response.data).toHaveProperty('response');
expect(response.data.model).toBe('@cf/meta/llama-3.1-8b-instruct');

3. Demo Game Flow Testing

Purpose: Comprehensive validation of the complete demo game experience

Coverage:

  • Session creation and management
  • AI statement generation (40 questions per session)
  • Game state transitions and progression
  • Real-time scoring and streak calculations
  • Session completion and summary generation
  • XP and achievement calculation accuracy
  • Guest session persistence and recovery

Test Scenarios:

describe('Demo Game Flow', () => {
  test('Complete Game Session', async () => {
    // 1. Create demo session
    const session = await createDemoSession();
    expect(session.id).toBeDefined();
    
    // 2. Generate statements
    const statements = await generateStatements(session.id, 'History');
    expect(statements).toHaveLength(40);
    
    // 3. Simulate gameplay
    const responses = simulateGameplay(statements);
    
    // 4. Complete session
    const completion = await completeSession(session.id, responses);
    expect(completion.finalScore).toBeGreaterThan(0);
    expect(completion.xpGained).toBeGreaterThan(0);
  });
});

4. Database Persistence Testing

Purpose: Verify data storage, retrieval, and analytics functionality

Coverage:

  • Demo session data storage in D1 database
  • Analytics endpoint functionality and performance
  • Performance metrics calculation accuracy
  • Topic and user analysis data integrity
  • Session response tracking and storage
  • Data migration and account conversion flows

Database Validation:

-- Test data integrity
SELECT COUNT(*) as session_count FROM demo_sessions;
SELECT AVG(accuracy_percentage) as avg_accuracy FROM demo_sessions;
SELECT COUNT(DISTINCT topic) as topic_variety FROM demo_sessions;

-- Test analytics views
SELECT * FROM session_analytics_view WHERE created_date >= DATE('now', '-7 days');

5. Statement Generation Testing

Purpose: Validate AI-powered content creation quality and consistency

Coverage:

  • AI-powered statement generation with topic specificity
  • Content quality validation (factual accuracy, clarity)
  • Statement structure verification (true/false format)
  • API parameter handling and validation
  • Difficulty level distribution (Basic/Intermediate/Advanced)
  • Topic relevance and educational value

Quality Metrics:

const validateStatements = (statements) => {
  // Structure validation
  statements.forEach(stmt => {
    expect(stmt).toHaveProperty('text');
    expect(stmt).toHaveProperty('answer');
    expect(stmt).toHaveProperty('difficulty');
    expect(stmt).toHaveProperty('topic');
    expect(typeof stmt.answer).toBe('boolean');
  });
  
  // Distribution validation
  const difficulties = statements.map(s => s.difficulty);
  const basicCount = difficulties.filter(d => d === 'basic').length;
  const intermediateCount = difficulties.filter(d => d === 'intermediate').length;
  const advancedCount = difficulties.filter(d => d === 'advanced').length;
  
  expect(basicCount).toBeGreaterThan(15); // ~50%
  expect(intermediateCount).toBeGreaterThan(8); // ~30%
  expect(advancedCount).toBeGreaterThan(5); // ~20%
};

Test Execution

Sample Successful Run

🧪 PadawanForge - Complete Test Suite
📍 Base URL: http://localhost:4321

🔍 1. Health Check
✅ API Health - Response: 200 (45ms)
✅ Environment Variables - All 12 required variables present
✅ Database Connection - D1 responsive

🤖 2. AI Integration
✅ AI Connection - Model accessible
✅ AI Model: @cf/meta/llama-3.1-8b-instruct
✅ Response Quality - Valid JSON structure
✅ Error Handling - Fallbacks operational

🎮 3. Demo Game Flow
✅ Demo Session Creation - Session ID: abc-123-def
✅ Statement Generation - 40/40 statements generated
✅ Statement Quality - All statements validated
✅ Session Completion - Score: 320, XP: 150
✅ Achievement System - 3 achievements unlocked

🗄️ 4. Database Persistence
✅ Analytics Endpoint - Response: 200
✅ Session Storage - 15 historical sessions
✅ Performance Metrics - Accuracy: 78.5%
✅ Topic Breakdown - 4 topics represented
✅ Data Integrity - All relationships valid

📝 5. Statement Generation
✅ Statement API - Generation successful
✅ Statement Quality - Content validation passed
✅ Statement Structure - Format compliance verified
✅ Difficulty Distribution - Balanced across levels

════════════════════════════════════════════════════════════
📊 TEST SUMMARY
════════════════════════════════════════════════════════════
⏱️  Total Duration: 12.3s
✅ Tests Passed: 15/15
❌ Tests Failed: 0/15
📈 Success Rate: 100.0%
🎯 Coverage: All core systems validated
════════════════════════════════════════════════════════════

Test Commands

Available Commands

# Primary commands
bun run test:all                    # Test local development server
bun run test:all:prod              # Test production deployment
bun run test:all <URL>             # Test custom URL

# Development utilities
bun run dev                        # Start development server
bun run test:watch                 # Watch mode for development
bun run test:debug                 # Debug mode with verbose output

Command Configuration

{
  "scripts": {
    "test:all": "bun scripts/test-all.js",
    "test:all:prod": "bun scripts/test-all.js https://your-domain.workers.dev",
    "test:watch": "bun scripts/test-all.js --watch",
    "test:debug": "bun scripts/test-all.js --debug"
  }
}

Test Data Management

Test Session Cleanup

// Automatic cleanup of test data
const cleanupTestSessions = async () => {
  await fetch('/api/debug/cleanup', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ 
      action: 'cleanup_test_sessions',
      olderThan: '1 hour'
    })
  });
};

Mock Data Generation

// Generate consistent test data
const generateTestSession = () => ({
  guestId: 'test-guest-' + Date.now(),
  topic: 'History',
  difficulty: 'basic',
  statements: generateMockStatements(40),
  responses: generateMockResponses(40, 0.75) // 75% accuracy
});

Performance Testing

Response Time Monitoring

  • API Endpoints: < 200ms for health checks
  • AI Generation: < 60s for 40 statements
  • Database Queries: < 100ms for analytics
  • Session Operations: < 50ms for CRUD operations

Load Testing Guidelines

# Basic load testing with curl
for i in {1..10}; do
  curl -w "%{time_total}\n" -o /dev/null -s "http://localhost:4321/api/health"
done

# AI generation stress test
for i in {1..5}; do
  bun scripts/test-ai-generation.js &
done
wait

Error Handling Testing

Common Failure Scenarios

  1. AI Service Unavailable: Test fallback mechanisms
  2. Database Connection Lost: Verify error handling
  3. Invalid Session Data: Test validation and recovery
  4. Rate Limiting: Verify graceful degradation
  5. Network Timeouts: Test retry logic

Error Recovery Validation

describe('Error Recovery', () => {
  test('AI Service Fallback', async () => {
    // Mock AI service failure
    mockAIService.mockImplementation(() => {
      throw new Error('Service unavailable');
    });
    
    const response = await generateStatements(sessionId, 'History');
    expect(response.statements).toBeDefined();
    expect(response.source).toBe('fallback');
  });
});

Continuous Integration

GitHub Actions Integration

name: Test Suite
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: oven-sh/setup-bun@v1
      - run: bun install
      - run: bun run test:all

Pre-deployment Testing

# Required before any deployment
bun run test:all:prod

# Verify all systems before release
bun run test:all https://staging.padawanforge.dev

Debugging and Troubleshooting

Debug Mode

# Enable detailed logging
bun run test:all --debug

# Output includes:
# - Request/response details
# - Timing information
# - Database query logs
# - AI model interactions

Common Issues and Solutions

AI Generation Timeouts

// Symptoms: Tests hang on AI generation
// Solution: Increase timeout and add fallback
const generateWithTimeout = async (sessionId, topic, timeout = 60000) => {
  return Promise.race([
    generateStatements(sessionId, topic),
    new Promise((_, reject) => 
      setTimeout(() => reject(new Error('Timeout')), timeout)
    )
  ]);
};

Database Connection Issues

// Symptoms: Database tests fail intermittently
// Solution: Add retry logic and connection validation
const withRetry = async (operation, retries = 3) => {
  for (let i = 0; i < retries; i++) {
    try {
      return await operation();
    } catch (error) {
      if (i === retries - 1) throw error;
      await new Promise(resolve => setTimeout(resolve, 1000 * (i + 1)));
    }
  }
};

Session State Inconsistencies

// Symptoms: Session data doesn't match expected state
// Solution: Add state validation and recovery
const validateSessionState = (session) => {
  const required = ['id', 'guestId', 'statements', 'createdAt'];
  const missing = required.filter(key => !session[key]);
  if (missing.length > 0) {
    throw new Error(`Missing required fields: ${missing.join(', ')}`);
  }
};

Test Maintenance

Regular Test Updates

  • Weekly: Review test failure rates and performance metrics
  • Monthly: Update test data and mock responses
  • Per Release: Add tests for new features and endpoints
  • Quarterly: Review and optimize test suite performance

Test Coverage Monitoring

# Generate coverage report
bun run test:coverage

# Coverage targets:
# - API endpoints: 95%
# - Core game logic: 90%
# - Error handling: 85%
# - UI components: 80%

This streamlined testing approach ensures comprehensive validation of all PadawanForge systems while maintaining fast execution times and clear reporting for efficient development workflows.

← Shared Utilities PadawanForge Task Manager & Development Progress →
PadawanForge v1.4.1