Instruction file imported from jam3gw/agentic-service-bot (.cursor/rules/python-testing.mdc). Copyright stays with the author.

Python Testing Best Practices

This rule enforces best practices for writing tests in Python, with a focus on ensuring tests provide business value.

actions:

type: suggest conditions:

If test function doesn't have docstring

pattern: "def\s+test_[a-zA-Z0-9_]+\s*\([^)]\)\s:\s*(?!\s*["'])\s*\S+" message: "Add docstrings to test functions to explain what they're testing and what business value they provide"

If using assert without message

pattern: "assert\s+[^,]+" message: "Include a message with assertions to clarify what's being tested"

If test has multiple assertions

pattern: "def\s+test_[a-zA-Z0-9_]+[^:]:(?:[^\n]\n+\s*)+assert[^\n]\n+(?:[^\n]\n+\s*)*assert" message: "Consider splitting tests with multiple assertions into separate test functions"

If test is mocking external dependencies without clear purpose

pattern: "@patch\(['"][^'"]+['"]\)" message: "Ensure mocks are used purposefully and the test provides clear business value"

If test has rigid string assertions

pattern: "assert\s+["'][^"']+["']\s+in\s+.+" message: "Consider using more flexible assertions that can handle variations in wording"

If committing test files without running them

pattern: "git\s+add\s+.test_.\.py" message: "Verify that tests pass before committing them"

If mocking AWS services without proper response structure

pattern: "boto3|dynamodb|s3|lambda" message: "Ensure AWS service mocks include proper response structures to avoid ResourceNotFound errors"

If mocking at the wrong level

pattern: "@patch\('"['"]\)" message: "Consider mocking at a more direct level instead of high-level libraries"

If testing WebSocket connections

pattern: "websocket|WebSocket|ws:|wss:" message: "Ensure WebSocket tests handle connection lifecycle and message validation properly"

If testing service level permissions

pattern: "permission|authorize|role|policy" message: "Ensure service level permission tests verify both positive and negative access scenarios"

If potentially duplicating API URL prefixes

pattern: "REST_API_URL.?/api/|API_URL.?/api/" message: "Check for duplicate API prefixes in URL construction" message: |

Python Testing Best Practices

Follow these guidelines for writing tests:

Business Value First: Only write tests that provide clear business value
- Tests should verify critical business logic
- Tests should prevent regressions in important functionality
- Tests should document expected behavior for key features
Test Quality Over Quantity
- Focus on high-value tests rather than high test count
- Avoid tests that duplicate coverage without adding value
- Remove tests that are consistently failing without providing useful feedback
Practical Mocking
- Only mock external dependencies when necessary
- Ensure mocks reflect realistic behavior
- Don't create tests that are overly dependent on implementation details
- Mock at the right level: Prefer mocking the specific components you're testing rather than high-level dependencies
- Direct mocking is more reliable: Mock the specific functions or methods being called rather than their internal dependencies
- For AWS services, ensure mocks include proper response structures:
  - DynamoDB: Include proper response metadata and expected return values
  - S3: Mock both client and resource interfaces appropriately
  - Lambda: Include proper invocation response format
Flexible Assertions
- Use assertions that focus on business requirements, not implementation details
- For text validation, consider multiple ways the same concept might be expressed
- Use regular expressions or multiple conditions for flexible string matching
- Test for semantic meaning rather than exact wording
- Match implementation details: Ensure assertions match the actual implementation details, including return value types (e.g., empty dict vs None)
Appropriate Timeouts
- Set realistic timeouts for asynchronous operations
- Consider environment factors (like cold starts in serverless) when setting timeouts
- Include clear timeout messages that help diagnose issues
Verify Before Committing
- Always run tests before committing them
- Use test runners like pytest or unittest to verify functionality
- Fix failing tests before committing or document known issues
- Include test verification in your pre-commit workflow
- For Lambda functions, use the provided run_tests.py script
Effective Test Debugging
- Debug tests individually rather than running the entire suite when troubleshooting
- Create debug scripts that run specific test classes or methods in isolation
- Add detailed logging to tests to understand what's happening during execution
- Verify environment variables and mock configurations in each test
- Use descriptive test names that help identify what's being tested
Consistent Mocking Approach
- Use a consistent mocking approach throughout the test suite
- Prefer patching specific components over patching high-level dependencies
- Document your mocking strategy for complex tests
- Use setup methods to establish common mocks across multiple tests
WebSocket Testing Best Practices
- Test the full WebSocket lifecycle (connection, message exchange, disconnection)
- Verify connection establishment with proper headers and authentication
- Test message serialization/deserialization for both sending and receiving
- Implement timeout handling for asynchronous WebSocket operations
- Test reconnection logic and error handling for dropped connections
- Verify proper cleanup of resources when connections are closed
- Use mock WebSocket servers to simulate various server behaviors
- Test handling of different message types (text, binary, ping/pong)
- Verify proper handling of concurrent connections and messages
Service Level Permission Testing - Test both positive and negative permission scenarios - Verify that authorized users can access protected resources - Confirm that unauthorized users receive appropriate error responses - Test permission inheritance and role hierarchies - Verify that permission changes take effect immediately - Test cross-service permission boundaries - Validate that permission checks are applied consistently across all access points - Test permission caching and expiration behavior - Verify proper logging of permission-related events - Test permission delegation and temporary access scenarios
API URL Construction - Be careful with API URL construction to avoid duplicating prefixes - When defining base URLs like REST_API_URL, check if they already include prefixes like /api - Document the structure of base URLs in comments to make it clear what they include - Use consistent URL construction patterns across all tests - Consider creating helper functions for URL construction to ensure consistency - Verify constructed URLs match the actual API endpoints by checking API documentation - When updating API endpoints, ensure all tests are updated consistently - Don't change API endpoints to fix 502 errors: If encountering 502 Bad Gateway errors, investigate the underlying issue rather than changing endpoints (see the api-troubleshooting rule for detailed guidance)
API Test Data Setup Automation - Always automate test data setup and teardown for API tests - Use pytest fixtures to create and clean up test data - Never rely on hardcoded test data that may not exist - Create unique test data for each test run to prevent test interference - Implement proper cleanup to avoid leaving test data in the database - Use session-scoped fixtures for data that can be shared across tests - Use function-scoped fixtures for data that needs to be fresh for each test - Include clear error messages when test data setup fails - Document the test data structure in fixture docstrings - Handle both the creation and deletion of test resources - Consider using transaction rollbacks where possible to speed up tests - Ensure test data is isolated from production data
Include docstrings for all test functions that explain: - What is being tested - Why it matters (business value) - Any special test conditions
Follow the Arrange-Act-Assert (AAA) pattern
Test one concept per test function
Use descriptive assertion messages
Use fixtures for test setup and teardown
Parameterize tests for multiple input scenarios
Aim for high test coverage of critical paths, not 100% coverage
State Management Best Practices - Always ensure tests start and end in a known state - Use try/finally blocks to restore original states - Include multiple verification steps after state changes - Allow sufficient time for state changes to propagate - Document expected state transitions - Verify state through multiple channels (e.g., DynamoDB and API)
Error Handling and Debugging - Handle known issues gracefully (e.g., 502 Bad Gateway errors) - Use pytest.skip() for known transient issues - Include descriptive error messages in assertions - Log extensively for debugging purposes - Implement multiple cleanup strategies as fallbacks - Log cleanup failures without failing tests
Type Safety with DynamoDB - Use TypedDict for better type safety - Be cautious with dictionary access (use .get() with defaults) - Consider all possible types DynamoDB might return - Handle Decimal types appropriately - Use proper type casting for numeric values - Implement type-safe helper functions for common operations
Test Structure and Organization - Break down complex tests into smaller, focused functions - Use helper functions for common verification tasks - Include comprehensive docstrings - Structure tests in a logical flow (setup → test → verify → cleanup) - Keep test files organized and maintainable - Use constants for common values
Logging Best Practices - Include detailed logging at each step - Log both requests and responses - Use descriptive log messages that help with debugging - Include relevant context in log messages - Use appropriate log levels (INFO, WARNING, ERROR) - Add timestamps to important operations
Development Workflow - Ignore linter warnings during initial development - Focus on getting the test logic right first - Address linter warnings in a separate pass - Document known linter issues that can be safely ignored - Use type: ignore comments judiciously - Keep a list of technical debt items to address later

Example of State Management:

def test_service_level_permissions(test_data: Dict[str, str]) -> None:
    """Test service level permissions through the chat API."""
    customer_id = test_data['customer_id']
    dynamodb = boto3.resource('dynamodb', region_name=REGION)
    table = dynamodb.Table(CUSTOMERS_TABLE)
    
    # Store initial state
    initial_state = None
    try:
        response = table.get_item(Key={'id': customer_id})
        initial_state = response.get('Item', {}).get('level')
        
        # Set to premium for test
        table.update_item(
            Key={'id': customer_id},
            UpdateExpression="SET #lvl = :val",
            ExpressionAttributeNames={'#lvl': 'level'},
            ExpressionAttributeValues={':val': 'premium'}
        )
        
        # Test premium features
        # ... test code ...
        
    finally:
        # Restore initial state
        if initial_state:
            try:
                table.update_item(
                    Key={'id': customer_id},
                    UpdateExpression="SET #lvl = :val",
                    ExpressionAttributeNames={'#lvl': 'level'},
                    ExpressionAttributeValues={':val': initial_state}
                )
            except Exception as e:
                logging.error(f"Failed to restore initial state: {e}")

Example of Type-Safe DynamoDB Access:

from typing import TypedDict, Optional

class DeviceState(TypedDict):
    id: str
    power: str
    volume: int
    type: str
    location: str

def get_device_state(table: Table, customer_id: str) -> Optional[DeviceState]:
    """Get device state with type safety."""
    try:
        response = table.get_item(Key={'id': customer_id})
        if 'Item' not in response:
            return None
            
        device = response['Item'].get('device', {})
        return DeviceState(
            id=str(device.get('id', '')),
            power=str(device.get('power', 'off')),
            volume=int(float(device.get('volume', 0))),
            type=str(device.get('type', '')),
            location=str(device.get('location', ''))
        )
    except Exception as e:
        logging.error(f"Error getting device state: {e}")
        return None

Example of Development Workflow with Linter Warnings:

# During development, use type: ignore for known issues
def verify_device_state(response: Dict[str, Any]) -> None:
    # type: ignore  # Known issue with DynamoDB types
    device = response['Item'].get('device', {})
    
    # TODO: Fix type issues in cleanup phase
    assert device.get('power') == 'on'  # type: ignore
    
    # Document why we're ignoring the warning
    # type: ignore  # DynamoDB returns Decimal, but we know it's safe to compare
    assert int(device.get('volume')) <= 100

Example of API URL Construction:

# Define the base URL with clear documentation
# This URL already includes the /api prefix
REST_API_URL = "https://api.example.com/dev/api"

def test_get_devices():
    """Test the GET /customers/{customerId}/devices endpoint."""
    # Correct: Don't duplicate the /api prefix
    response = requests.get(
        f"{REST_API_URL}/customers/{TEST_CUSTOMER_ID}/devices"
    )
    
    # Incorrect: This would result in /api/api/customers/...
    # response = requests.get(
    #     f"{REST_API_URL}/api/customers/{TEST_CUSTOMER_ID}/devices"
    # )
    
    assert response.status_code == 200

Example of API Error Handling for 502 Errors:

def test_chat_api_with_error_handling():
    """Test the chat API with proper error handling for 502 errors.
    
    This test verifies our chat API functionality while properly
    handling potential 502 errors that may occur during testing.
    """
    try:
        response = requests.post(
            f"{REST_API_URL}/chat",
            json={"customerId": test_data['customer_id'], "message": "Hello"},
            timeout=10
        )
        
        # Check for HTTP errors
        response.raise_for_status()
        
        # Normal assertions if no errors
        assert response.status_code == 200
        data = response.json()
        assert "response" in data
        
    except requests.exceptions.HTTPError as e:
        if e.response.status_code == 502:
            logging.error("502 Bad Gateway error encountered. "
                        "This is a known issue with the chat API.")
            logging.error("Check Lambda function logs in CloudWatch for details.")
            pytest.skip("Skipping due to known 502 issue with chat API")
        else:
            # Re-raise other HTTP errors
            raise

Example of Flexible Text Assertion:

def test_error_message_for_invalid_input():
    """Test that appropriate error message is shown for invalid input.
    
    This test ensures users receive helpful guidance when they
    provide invalid input, which reduces support requests.
    """
    # Arrange
    invalid_input = "abc123"
    
    # Act
    result = process_numeric_input(invalid_input)
    
    # Assert - flexible matching for error message
    error_msg = result.get('error', '').lower()
    assert any(phrase in error_msg for phrase in 
              ['invalid input', 'not valid', 'must be numeric', 'numbers only']), \
           f"Error message should indicate invalid input, but got: {error_msg}"

Example of Test Verification Before Commit:

# For general Python tests
python -m pytest

# For Lambda function tests
cd lambda/api
python run_tests.py

# Only commit if tests pass
if [ $? -eq 0 ]; then
    git add .
    git commit -m "test: add new tests for feature X"
else
    echo "Tests failed, fixing before committing"
fi

Example of Individual Test Debugging:

# debug_tests.py - A script to run tests individually for debugging
import unittest
import os
import sys

# Set up environment variables for testing
os.environ['CUSTOMERS_TABLE'] = 'test-customers'
os.environ['SERVICE_LEVELS_TABLE'] = 'test-service-levels'

# Import the test class
from services.test_dynamodb_service import TestDynamoDBService

# Create a test suite with just one test method
suite = unittest.TestSuite()
suite.addTest(TestDynamoDBService('test_update_device_state_success'))

# Run the test with high verbosity
runner = unittest.TextTestRunner(verbosity=2)
result = runner.run(suite)

# Print detailed results
print(f"\nTest passed: {result.wasSuccessful()}")
if not result.wasSuccessful():
    print(f"Errors: {len(result.errors)}")
    for test, error in result.errors:
        print(f"\n{'-'*40}\nError in {test}:\n{error}")
    
    print(f"Failures: {len(result.failures)}")
    for test, failure in result.failures:
        print(f"\n{'-'*40}\nFailure in {test}:\n{failure}")

Example of API Test Data Setup Automation:

# conftest.py
import pytest
import boto3
import uuid
from datetime import datetime

# DynamoDB configuration
REGION = "us-west-2"
CUSTOMERS_TABLE = "dev-customers"

@pytest.fixture(scope="session")
def dynamodb_client():
    """Create a DynamoDB client for testing."""
    return boto3.resource('dynamodb', region_name=REGION)

@pytest.fixture(scope="session")
def test_data(dynamodb_client):
    """
    Create test data for API tests.
    
    This fixture creates a test customer with devices before running the tests,
    and cleans up the data after the tests are complete.
    
    Returns:
        A dictionary containing the test customer ID and device IDs
    """
    # Generate a unique customer ID for this test run
    customer_id = f"test-customer-e2e-{uuid.uuid4().hex[:8]}"
    
    # Create test customer with devices
    table = dynamodb_client.Table(CUSTOMERS_TABLE)
    customer_data = {
        'id': customer_id,
        'name': 'E2E Test Customer',
        'email': 'test@example.com',
        'serviceLevel': 'premium',
        'createdAt': datetime.now().isoformat(),
        'devices': [
            {
                'id': f"{customer_id}-device-1",
                'type': 'speaker',
                'state': 'off',
                'location': 'living_room'
            },
            {
                'id': f"{customer_id}-device-2",
                'type': 'light',
                'state': 'off',
                'location': 'bedroom'
            }
        ]
    }
    
    # Put the item in the table
    table.put_item(Item=customer_data)
    
    # Create test data dictionary
    test_data = {
        'customer_id': customer_id,
        'device_ids': [device['id'] for device in customer_data['devices']]
    }
    
    # Yield the test data to the tests
    yield test_data
    
    # Clean up after tests are complete
    table.delete_item(Key={'id': customer_id})

# In your test file:
def test_get_devices(test_data):
    """Test the GET /customers/{customerId}/devices endpoint."""
    # Get customer ID from test data fixture
    customer_id = test_data['customer_id']
    
    # Make the API request
    response = requests.get(
        f"{REST_API_URL}/customers/{customer_id}/devices"
    )
    
    # Verify response
    assert response.status_code == 200, f"Expected status code 200, got {response.status_code}"
    # Additional assertions...

Example of Proper AWS Service Mocking:

# Bad: Mocking at too high a level
@patch('boto3.resource')
def test_dynamodb_update(mock_boto3_resource):
    mock_table = MagicMock()
    mock_dynamodb = MagicMock()
    mock_dynamodb.Table.return_value = mock_table
    mock_boto3_resource.return_value = mock_dynamodb
    
    result = update_item('item-123', 'new-value')
    assert result is True

# Good: Mocking at the right level
@patch('services.dynamodb_service.dynamodb.Table')
def test_dynamodb_update(mock_table_constructor):
    """Test updating an item in DynamoDB.
    
    This test ensures our update operation correctly modifies data,
    which is critical for maintaining data integrity.
    """
    # Arrange
    mock_table = MagicMock()
    # Include proper response structure
    mock_table.update_item.return_value = {
        'ResponseMetadata': {
            'HTTPStatusCode': 200,
            'RequestId': 'EXAMPLE-REQUEST-ID'
        },
        'Attributes': {
            'id': 'item-123',
            'value': 'new-value',
            'updated_at': '2023-01-01T12:00:00Z'
        }
    }
    mock_table_constructor.return_value = mock_table
    
    # Act
    result = update_item('item-123', 'new-value')
    
    # Assert
    assert result is True, "Update operation should return True on success"
    mock_table.update_item.assert_called_once()
    # Verify the correct parameters were used
    args, kwargs = mock_table.update_item.call_args
    assert kwargs['Key'] == {'id': 'item-123'}, "Key should match the item ID"
    assert 'UpdateExpression' in kwargs, "Update should include UpdateExpression"

Example of Consistent Mocking Approach:

class TestUserService(unittest.TestCase):
    """Tests for the UserService class."""
    
    def setUp(self):
        """Set up common mocks for all tests."""
        # Patch at the right level consistently across all tests
        self.db_patcher = patch('services.user_service.database.get_connection')
        self.mock_db = self.db_patcher.start()
        self.mock_conn = MagicMock()
        self.mock_db.return_value = self.mock_conn
        
        # Create the service with dependencies properly mocked
        self.user_service = UserService()
        
    def tearDown(self):
        """Clean up patches."""
        self.db_patcher.stop()
        
    def test_get_user(self):
        """Test retrieving a user by ID."""
        # Arrange
        self.mock_conn.execute.return_value.fetchone.return_value = {
            'id': 'user-123',
            'name': 'Test User',
            'email': 'test@example.com'
        }
        
        # Act
        user = self.user_service.get_user('user-123')
        
        # Assert
        self.assertEqual(user['id'], 'user-123')
        self.mock_conn.execute.assert_called_once()

Example of WebSocket Testing:

@pytest.fixture
def mock_websocket_server():
    """Fixture to create a mock WebSocket server for testing."""
    server = MockWebSocketServer('localhost', 8765)
    server.start()
    yield server
    server.stop()

def test_websocket_connection_lifecycle(mock_websocket_server):
    """Test the full lifecycle of a WebSocket connection.
    
    This test verifies our client can properly establish, maintain,
    and close WebSocket connections, which is critical for real-time
    communication with devices.
    """
    # Arrange
    client = WebSocketClient('ws://localhost:8765')
    
    # Act - Connect
    connected = client.connect()
    
    # Assert - Connection successful
    assert connected is True, "Client should connect successfully to the WebSocket server"
    
    # Act - Send message
    success = client.send_message({"type": "command", "action": "start"})
    
    # Assert - Message sent successfully
    assert success is True, "Client should send message successfully"
    
    # Act - Receive response
    response = client.receive_message(timeout=2.0)
    
    # Assert - Response received and valid
    assert response is not None, "Client should receive a response"
    assert response.get("status") == "ok", f"Response should have 'ok' status, got: {response}"
    
    # Act - Disconnect
    disconnected = client.disconnect()
    
    # Assert - Disconnection successful
    assert disconnected is True, "Client should disconnect cleanly"
    
    # Verify server side received expected messages
    messages = mock_websocket_server.get_received_messages()
    assert len(messages) == 1, "Server should have received exactly one message"
    assert messages[0].get("type") == "command", "Message should have correct type"

Example of Service Level Permission Testing:

def test_service_level_permission_access_allowed():
    """Test that users with appropriate permissions can access protected resources.
    
    This test ensures our permission system correctly grants access to authorized users,
    which is critical for maintaining proper security boundaries while allowing
    legitimate access.
    """
    # Arrange
    user_id = "user-with-admin-role"
    resource_id = "protected-resource-123"
    permission_service = PermissionService()
    
    # Mock the permission lookup
    with patch('services.permission_service.get_user_roles') as mock_get_roles:
        mock_get_roles.return_value = ["admin", "editor"]
        
        # Act
        has_permission = permission_service.check_access(user_id, resource_id, "read")
        
        # Assert
        assert has_permission is True, "User with admin role should have read access"
        mock_get_roles.assert_called_once_with(user_id)

def test_service_level_permission_access_denied():
    """Test that users without appropriate permissions cannot access protected resources.
    
    This test verifies our permission system correctly denies access to unauthorized users,
    which is essential for maintaining security and data isolation between customers.
    """
    # Arrange
    user_id = "user-with-viewer-role"
    resource_id = "protected-resource-123"
    permission_service = PermissionService()
    
    # Mock the permission lookup
    with patch('services.permission_service.get_user_roles') as mock_get_roles:
        mock_get_roles.return_value = ["viewer"]
        
        # Act
        has_permission = permission_service.check_access(user_id, resource_id, "write")
        
        # Assert
        assert has_permission is False, "User with viewer role should not have write access"
        mock_get_roles.assert_called_once_with(user_id)
        
    # Verify proper error response when attempting to access
    with patch('services.permission_service.get_user_roles') as mock_get_roles:
        mock_get_roles.return_value = ["viewer"]
        
        # Act
        response = permission_service.attempt_resource_operation(user_id, resource_id, "write")
        
        # Assert
        assert response.get("success") is False, "Operation should not succeed"
        assert "permission denied" in response.get("error", "").lower(), \
               f"Error should indicate permission denied, got: {response.get('error')}"

Business-Critical Testing

Tests should focus on verifying functionality that delivers business value to users, rather than just testing implementation details. This means:

Test Real User Flows: Design tests around actual user scenarios and business-critical paths.
Focus on Value Delivery: Each test should verify functionality that delivers value to users or protects revenue.
End-to-End Testing: Where possible, tests should verify complete user flows rather than isolated components.
Test Service Boundaries: Always verify that service level permissions are correctly enforced, as this is critical to business models with tiered pricing.
Use Real APIs When Possible: Prefer testing against real APIs over excessive mocking for integration tests.

Bad Example:

def test_dynamodb_service_get_customer():
    # This test focuses too much on implementation details
    mock_table = MagicMock()
    mock_table.get_item.return_value = {'Item': {'id': 'customer-123', 'name': 'Test User'}}
    
    with patch('boto3.resource') as mock_boto3:
        mock_boto3.return_value.Table.return_value = mock_table
        result = get_customer('customer-123')
    
    assert result.id == 'customer-123'
    assert result.name == 'Test User'
    assert mock_table.get_item.call_count == 1

Good Example:

def test_premium_customer_can_relocate_devices():
    """
    Test that premium customers can relocate devices.
    
    This test verifies a key differentiator for the premium tier,
    which is critical for our tiered pricing model and revenue.
    """
    # Set up a premium customer
    customer = Customer('premium-customer', 'Premium User', 'premium', [
        {'id': 'device-1', 'type': 'speaker', 'location': 'living_room'}
    ])
    
    # Verify the customer can perform the premium action
    assert is_action_allowed(customer, 'device_relocation') == True
    
    # Test the actual request processing with a real user request
    response = process_request('premium-customer', 'Move my speaker to the kitchen')
    
    # Verify the response indicates success, not a service level restriction
    assert "not allowed" not in response.lower()
    assert "upgrade" not in response.lower()

Integration Testing with Real APIs

For business-critical functionality, integration tests that hit real APIs provide more confidence than unit tests with mocks. These tests should:

Verify End-to-End Flows: Test complete user journeys from request to response.
Test Against Real Endpoints: Use actual API endpoints rather than mocks when possible.
Verify Business Rules: Confirm that business rules like service level permissions are enforced.
Include Both Positive and Negative Scenarios: Test both allowed and disallowed actions.

Bad Example:

def test_api_endpoint_structure():
    # This test focuses on implementation details rather than business value
    response = requests.get(f"{API_URL}/health")
    assert response.status_code == 200
    
    data = response.json()
    assert 'status' in data
    assert 'version' in data

Good Example:

def test_basic_customer_cannot_access_premium_features():
    """
    Test that basic customers cannot access premium features.
    
    This test verifies our service tier boundaries, which are
    critical for our business model and revenue protection.
    """
    # Send a request to relocate a device (premium feature)
    response = requests.post(
        f"{API_URL}/chat",
        json={
            'customerId': 'test-basic-customer',
            'message': 'Move my speaker to the kitchen'
        }
    )
    
    # Verify the response
    assert response.status_code == 200
    data = response.json()
    
    # The response should indicate this action is not allowed and suggest an upgrade
    assert "not allowed with basic service level" in data['message'].lower()
    assert "upgrade" in data['message'].lower()
    
    # Verify that the interaction was logged in the database
    # (This could be a separate verification step)

DynamoDB Testing Best Practices

When testing DynamoDB operations, follow these guidelines:

Mock at the Table Level
- Mock the DynamoDB table directly instead of the boto3 resource
- Use consistent mocking patterns across all tests
- Set up mocks in setUp for reuse across test methods
Use Realistic Response Structures
- Include all required DynamoDB response fields
- Match the exact structure of real DynamoDB responses
- Include proper metadata and attributes
Test Error Cases
- Test item not found scenarios
- Test conditional update failures
- Test permission and access issues
Verify DynamoDB Operations
- Check that operations are called with correct parameters
- Verify update expressions and attribute values
- Ensure proper error handling

Example of DynamoDB Testing:

class TestDynamoDBService(unittest.TestCase):
    """Tests for the DynamoDB service functions."""
    
    def setUp(self):
        """Set up test environment."""
        # Create a mock table
        self.mock_table = MagicMock()
        
        # Patch the customers_table directly in the module
        self.table_patcher = patch('services.dynamodb_service.customers_table', self.mock_table)
        self.table_patcher.start()
        
    def tearDown(self):
        """Clean up after tests."""
        self.table_patcher.stop()
    
    def test_update_device_state_success(self):
        """Test updating a device state successfully."""
        # Set up mock response
        self.mock_table.get_item.return_value = {
            'Item': {
                'id': 'test-customer',
                'device': {
                    'id': 'test-device',
                    'type': 'speaker',
                    'location': 'living_room'
                }
            }
        }
        self.mock_table.update_item.return_value = {
            'ResponseMetadata': {
                'HTTPStatusCode': 200
            }
        }
        
        # Test parameters
        customer_id = 'test-customer'
        device_id = 'test-device'
        state_updates = {"power": "on"}
        
        # Call the function
        result = update_device_state(customer_id, device_id, state_updates)
        
        # Verify the result
        self.assertTrue(result)
        
        # Verify the get_item call
        self.mock_table.get_item.assert_called_once_with(Key={'id': customer_id})
        
        # Verify the update_item call
        self.mock_table.update_item.assert_called_once()
        call_args = self.mock_table.update_item.call_args[1]
        self.assertEqual(call_args['Key'], {'id': customer_id})
        self.assertIn('device.#attr_power = :val_power', call_args['UpdateExpression'])
    
    def test_update_device_state_customer_not_found(self):
        """Test updating a device state when the customer is not found."""
        # Set up mock response for customer not found
        self.mock_table.get_item.return_value = {}
        
        # Test parameters
        customer_id = 'test-customer'
        device_id = 'test-device'
        state_updates = {"power": "on"}
        
        # Call the function
        result = update_device_state(customer_id, device_id, state_updates)
        
        # Verify the result
        self.assertFalse(result)
        
        # Verify the get_item call
        self.mock_table.get_item.assert_called_once_with(Key={'id': customer_id})
        
        # Verify update_item was not called
        self.mock_table.update_item.assert_not_called()

examples:

input: |

Bad: Rigid string assertion

def test_permission_denied_message(): response = api.access_restricted_resource() assert "Permission denied" in response.text output: |

Good: Flexible string assertion

def test_permission_denied_message(): """Test that users receive appropriate message when accessing restricted resources.

  This test ensures users understand why they can't access certain resources,
  which improves user experience and reduces support tickets.
  """
  response = api.access_restricted_resource()
  error_text = response.text.lower()
  
  # Check for various ways of expressing the same concept
  permission_denied_phrases = [
      "permission denied", 
      "not authorized", 
      "access denied",
      "insufficient privileges",
      "not allowed"
  ]
  
  assert any(phrase in error_text for phrase in permission_denied_phrases), \
         f"Response should indicate permission denied, but got: {response.text}"

input: |

Bad: Test without business value justification

@patch('external_api.get_data') def test_process_data(mock_get_data): mock_get_data.return_value = {"key": "value"} result = process_data() assert result == {"processed": "value"} output: |

Good: Test with clear business value

@patch('external_api.get_data') def test_process_data(mock_get_data): """Test that data processing works correctly with API data.
```
  This test verifies that our core data processing function correctly
  transforms external API data, which is critical for our reporting
  feature used by 80% of customers.
  """
  # Arrange
  mock_get_data.return_value = {"key": "value"}
  
  # Act
  result = process_data()
  
  # Assert
  assert result == {"processed": "value"}, "Data should be correctly processed"
```

input: |

Bad: Mocking at too high a level

@patch('boto3.resource') def test_dynamodb_update(mock_boto3_resource): mock_table = MagicMock() mock_dynamodb = MagicMock() mock_dynamodb.Table.return_value = mock_table mock_boto3_resource.return_value = mock_dynamodb

  result = update_item('item-123', 'new-value')
  assert result is True

output: |

Good: Mocking at the right level

@patch('services.dynamodb_service.dynamodb.Table') def test_dynamodb_update(mock_table_constructor): """Test updating an item in DynamoDB.

  This test ensures our update operation correctly modifies data,
  which is critical for maintaining data integrity.
  """
  # Arrange
  mock_table = MagicMock()
  # Include proper response structure
  mock_table.update_item.return_value = {
      'ResponseMetadata': {
          'HTTPStatusCode': 200,
          'RequestId': 'EXAMPLE-REQUEST-ID'
      },
      'Attributes': {
          'id': 'item-123',
          'value': 'new-value',
          'updated_at': '2023-01-01T12:00:00Z'
      }
  }
  mock_table_constructor.return_value = mock_table
  
  # Act
  result = update_item('item-123', 'new-value')
  
  # Assert
  assert result is True, "Update operation should return True on success"
  mock_table.update_item.assert_called_once()
  # Verify the correct parameters were used
  args, kwargs = mock_table.update_item.call_args
  assert kwargs['Key'] == {'id': 'item-123'}, "Key should match the item ID"
  assert 'UpdateExpression' in kwargs, "Update should include UpdateExpression"

input: |

Bad: WebSocket test without proper lifecycle testing

def test_websocket(): client = WebSocketClient('ws://localhost:8765') client.connect() client.send_message({"type": "command"}) assert client.is_connected() output: |

Good: Complete WebSocket lifecycle testing

def test_websocket_connection_lifecycle(): """Test the full lifecycle of a WebSocket connection.

  This test verifies our client can properly establish, maintain,
  and close WebSocket connections, which is critical for real-time
  communication with devices.
  """
  # Arrange
  client = WebSocketClient('ws://localhost:8765')
  
  # Act - Connect
  connected = client.connect()
  
  # Assert - Connection successful
  assert connected is True, "Client should connect successfully"
  
  # Act - Send message
  success = client.send_message({"type": "command", "action": "start"})
  
  # Assert - Message sent successfully
  assert success is True, "Client should send message successfully"
  
  # Act - Receive response (with timeout)
  response = client.receive_message(timeout=2.0)
  
  # Assert - Response received and valid
  assert response is not None, "Client should receive a response"
  assert response.get("status") == "ok", "Response should have 'ok' status"
  
  # Act - Disconnect
  disconnected = client.disconnect()
  
  # Assert - Disconnection successful
  assert disconnected is True, "Client should disconnect cleanly"
  
  # Verify resources were cleaned up
  assert client.connection is None, "Connection should be properly cleaned up"

input: |

Bad: Permission test that only checks positive case

def test_user_permissions(): user = User(role="admin") assert user.can_access_admin_panel() is True output: |

Good: Permission tests for both positive and negative cases

def test_user_permission_positive_case(): """Test that admin users can access the admin panel.

  This test verifies that our permission system correctly grants
  access to users with admin privileges, which is essential for
  administrative functions.
  """
  # Arrange
  user = User(role="admin")
  
  # Act
  can_access = user.can_access_admin_panel()
  
  # Assert
  assert can_access is True, "Admin users should be able to access the admin panel"

def test_user_permission_negative_case(): """Test that non-admin users cannot access the admin panel.

  This test ensures our permission system correctly prevents
  unauthorized access to administrative functions, which is
  critical for security.
  """
  # Arrange
  user = User(role="regular")
  
  # Act
  can_access = user.can_access_admin_panel()
  
  # Assert
  assert can_access is False, "Regular users should not be able to access the admin panel"
  
  # Verify proper error response when attempting to access
  response = user.attempt_admin_panel_access()
  assert response.status_code == 403, "Should return 403 Forbidden status code"
  assert "not authorized" in response.text.lower(), "Should indicate authorization failure"

metadata: priority: high version: 1.7

About

Python Testing Best Practices

If test function doesn't have docstring

If using assert without message

If test has multiple assertions

If test is mocking external dependencies without clear purpose

If test has rigid string assertions

If committing test files without running them

If mocking AWS services without proper response structure

If mocking at the wrong level

If testing WebSocket connections

If testing service level permissions

If potentially duplicating API URL prefixes

Python Testing Best Practices

Example of State Management:

Example of Type-Safe DynamoDB Access:

Example of Development Workflow with Linter Warnings:

Example of API URL Construction:

Example of API Error Handling for 502 Errors:

Example of Flexible Text Assertion:

Example of Test Verification Before Commit:

Example of Individual Test Debugging:

Example of API Test Data Setup Automation:

Example of Proper AWS Service Mocking:

Example of Consistent Mocking Approach:

Example of WebSocket Testing:

Example of Service Level Permission Testing:

Business-Critical Testing

Bad Example:

Good Example:

Integration Testing with Real APIs

Bad Example:

Good Example:

DynamoDB Testing Best Practices

Example of DynamoDB Testing:

Bad: Rigid string assertion

Good: Flexible string assertion

Bad: Test without business value justification

Good: Test with clear business value

Bad: Mocking at too high a level

Good: Mocking at the right level

Bad: WebSocket test without proper lifecycle testing

Good: Complete WebSocket lifecycle testing

Bad: Permission test that only checks positive case

Good: Permission tests for both positive and negative cases

Use it

Manifest

Reviews