Testing & Quality Assurance

Instruction file imported from SadhviGarg/AI-observability (.github/instructions/testing.instructions.md). Copyright stays with the author.

Testing & Quality Assurance Guide

Component Structure

The tests/ directory contains all test suites:

• unit/ — Unit tests for individual functions and classes
• integration/ — Integration tests for component interactions
• performance/ — Performance and load tests
• test_routes.py — API endpoint tests

Testing Pyramid

         ╱╲
        ╱  ╲  E2E Tests (few, slow, comprehensive)
       ╱    ╲
      ╱──────╲
     ╱        ╲ Integration Tests (medium, moderate)
    ╱          ╲
   ╱────────────╲
  ╱              ╲ Unit Tests (many, fast, isolated)
 ╱________________╲

Unit Testing

Purpose: Test individual functions and classes in isolation.

Characteristics:

• Fast execution (< 1 second per test)
• Mock external dependencies
• Test both success and failure cases
• Use descriptive test names: test_model_predict_with_valid_input()

Examples:

• Data preprocessing functions
• Feature engineering calculations
• Utility functions
• Service business logic (with mocks)

Integration Testing

Purpose: Test interactions between components.

Characteristics:

• Test real component connections
• Use fixtures for shared test data
• Test data flow end-to-end
• Validate component contracts

Examples:

• API endpoint behavior with database
• Model inference with preprocessed data
• Data pipeline transformations
• Observability collectors with real components

Performance Testing

Purpose: Validate system performance under load.

Characteristics:

• Measure response times and throughput
• Test with realistic data volumes
• Identify bottlenecks and regressions
• Use performance assertions: assert response_time < 100ms

Examples:

• API endpoint response times
• Model inference latency
• Data pipeline throughput
• Dashboard rendering performance

Test Fixtures & Helpers

Reusable fixtures: Create conftest.py with shared fixtures.

Test data factories: Use factory patterns to generate test data.

Mocks & stubs: Mock external dependencies (database, models, APIs).

Test utilities: Create helper functions for common test operations.

Coverage & Metrics

Target: Aim for >80% code coverage, 100% for critical paths.

Coverage types:

• Line coverage: Which lines are executed
• Branch coverage: Which decision paths are taken
• Mutation testing: Do tests catch code changes?

Commands:

# Run tests with coverage
pytest --cov=backend tests/

# Generate HTML coverage report
pytest --cov=backend --cov-report=html tests/

Common Test Patterns

Testing API Routes

def test_api_endpoint_success(client):
    response = client.post("/api/predict", json={"input": data})
    assert response.status_code == 200
    assert response.json()["prediction"] == expected_value

Testing Models

def test_model_predict_with_valid_input(model):
    result = model.predict(input_data)
    assert isinstance(result, np.ndarray)
    assert result.shape == expected_shape

Testing Data Transformations

def test_data_transformation_produces_valid_output():
    input_df = pd.DataFrame({"x": [1, 2, 3]})
    output_df = transform_data(input_df)
    assert len(output_df) == 3
    assert "transformed" in output_df.columns

Test Organization

File naming: test_<component>.py (e.g., test_model_registry.py)

Class naming: Test<Component> (e.g., TestModelRegistry)

Method naming: test_<action>_<condition>_<expected> (e.g., test_predict_with_invalid_input_raises_error)

Assertions: Use clear, specific assertions with helpful error messages.

Integration with CI/CD

• Run tests automatically on every commit
• Fail builds if coverage drops below target
• Run performance tests to detect regressions
• Maintain test status dashboard

Common Tasks

Writing a unit test: Identify function/class, set up fixtures, execute function, assert results.

Debugging a failing test: Review test code, run test with debugger, check assertions and mocks.

Improving test coverage: Run coverage tool, identify untested paths, write targeted tests.

Testing error scenarios: Trigger errors, validate error handling, check error messages.

Performance profiling: Measure execution time, identify bottlenecks, optimize, retest.