Testing Guide¶

This document provides guidelines and instructions for testing the EVOSEAL project.

Table of Contents¶

Running Tests
Writing Tests
Test Organization
Test Coverage
Continuous Integration
Debugging Tests
Best Practices

Running Tests¶

Prerequisites¶

Make sure you have installed the development dependencies:

pip install -r requirements-dev.txt
pip install -e .  # Install package in development mode

Running Tests¶

Run All Tests¶

pytest

Run Tests by Category¶

# Unit tests
pytest tests/unit/

# Integration tests
pytest tests/integration/

# Test DGM integration
pytest tests/integration/dgm/

# Test a specific file
pytest tests/unit/test_module.py

# Test a specific function
pytest tests/unit/test_module.py::test_function_name

Run with Coverage¶

pytest --cov=evoseal --cov-report=term-missing

Run with Verbose Output¶

pytest -v

Run with Debug Output¶

pytest --log-cli-level=DEBUG

Test Organization¶

Tests follow the same structure as the source code:

tests/
├── integration/          # Integration tests
│   ├── dgm/             # DGM integration tests
│   ├── openevolve/      # OpenEvolve integration tests
│   └── seal/            # SEAL integration tests
└── unit/                # Unit tests
    ├── core/            # Core components tests
    ├── models/          # Model tests
    ├── providers/       # Provider tests
    └── utils/           # Utility function tests

Test Naming Conventions¶

Test files should be named test_*.py or *_test.py
Test functions should be named test_*
Test classes should be named Test*
Test methods should be named test_*

Writing Tests¶

Writing Good Tests¶

Isolation: Each test should be independent and not rely on the state from other tests
Descriptive Names: Test names should clearly describe what they're testing
AAA Pattern: Follow Arrange-Act-Assert pattern
Test Coverage: Aim for high test coverage, especially for critical paths
Mocks: Use mocks for external services and slow operations

Example Test¶

def test_controller_initialization():
    # Arrange
    mock_evaluator = Mock()
    mock_selector = Mock()

    # Act
    controller = Controller(evaluator=mock_evaluator, selector=mock_selector)

    # Assert
    assert controller.evaluator == mock_evaluator
    assert controller.selector == mock_selector

Fixtures¶

Use fixtures for common test setup:

import pytest

@pytest.fixture
def sample_config():
    return {
        'population_size': 100,
        'mutation_rate': 0.1,
        'crossover_rate': 0.8
    }

def test_evolution_config(sample_config):
    assert sample_config['population_size'] == 100

Test Coverage¶

To generate a coverage report:

pytest --cov=evoseal tests/

For an HTML report:

pytest --cov=evoseal --cov-report=html tests/
open htmlcov/index.html  # On macOS

Continuous Integration¶

Tests are automatically run on every push and pull request using GitHub Actions. The CI pipeline includes:

Unit tests
Integration tests
Code coverage reporting
Linting and type checking

Debugging Tests¶

PDB Debugger¶

Drop into the Python debugger on test failure:

pytest --pdb

Verbose Output¶

Get more detailed test output:

pytest -v

Test Logging¶

To see log output during tests:

pytest --log-cli-level=INFO

Best Practices¶

Isolation: Tests should be isolated and not depend on each other
Deterministic: Tests should produce the same results every time they're run
Fast: Keep tests fast to encourage frequent testing
Descriptive: Use descriptive test function names and docstrings
Edge Cases: Test edge cases and error conditions
Mocks: Use mocks for external dependencies
Fixtures: Use fixtures for common test setup
CI: Ensure all tests pass before merging to main
Arrange-Act-Assert: Structure tests with clear sections
One Assert Per Test: Each test should verify one thing
Test Naming: Use descriptive names like test_function_name_when_condition_then_result
Docstrings: Include docstrings explaining what's being tested

Performance Testing¶

For performance-critical code, consider adding benchmarks:

def test_performance(benchmark):
    result = benchmark(some_function, arg1, arg2)
    assert result is not None

Run with:

pytest --benchmark-only

Security Testing¶

Test for common vulnerabilities (injection, XSS, etc.)
Use tools like Bandit for security scanning
Never include sensitive data in test code

Resources¶

Last update: 2025-07-20
Created: 2025-06-17