Imported from mamarrr/RMK-candidate (AGENTS.md). Install upstream with npx skills add mamarrr/RMK-candidate. Copyright stays with the author.

Repository Guidelines

Project Purpose

This repository implements the RMK data team internship 2026 test challenge. The goal is to create a reproducible probability scale: a curated list of real-world events from Estonian public data, each converted into a probability and presented graphically so readers can build intuition about common and rare events.

The project should demonstrate statistical thinking, readable code, documentation, and a clear final visual result. Prefer a focused, well-explained analysis over a large unfinished system.

Inputs

Primary inputs are Statistics Estonia tables listed in Docs/api-sources.md. Use API ingestion where possible, especially PxWeb JSON requests.

Core input tables:

• TS093 - traffic accidents with casualties by month.
• RV035 - deaths by age group, sex, and week.
• RV40 - deaths by month of death.
• RV49U - deaths by administrative unit or settlement type and sex.
• RV541U - deaths by sex, age, year of birth, and region.

Supporting sources:

• Statistics Estonia help and table pages for API details.
• Traffic accident metadata for definitions.
• Statistics Estonia and TAI death pages for methodology and context.

Raw API responses or downloaded data should go in data/raw/. Cleaned tables should go in data/processed/. Do not manually edit raw data.

Outputs

The program should calculate a clear set of named probabilities and write both human-readable and machine-readable outputs.

Target probabilities:

• Probability that a casualty traffic accident happened in each calendar month: traffic casualty accidents in month / all yearly casualty traffic accidents from TS093.
• Probability that a death occurred in each calendar month: deaths in month / all deaths in selected years from RV40.
• Probability that a death occurred in each selected week or week group: deaths in week / all deaths in selected years from RV035.
• Probability that a death belonged to each selected age group: deaths in age group / all deaths with known age from RV035 or RV541U.
• Probability that a death was male or female within selected age groups: deaths by sex and age group / all deaths in that age group from RV035 or RV541U.
• Probability that a death came from each selected region: deaths in region / all deaths in selected regions from RV49U or RV541U.

When the program is run, it should produce:

• outputs/events.csv - one row per event used in the probability scale.
• outputs/probability_scale.svg - final vector chart for the README.
• outputs/probability_scale.png - final raster chart for quick viewing.
• Optional outputs/summary.md - short text summary of the most common and rarest events.

outputs/events.csv should contain at least:

• event_name
• event_group
• probability
• one_in_n
• numerator
• denominator
• source_table
• source_url
• year_range
• notes

The chart should place events on a log probability scale and label them in plain language, for example "a death occurs in January" or "a casualty traffic accident occurs in July". The final README should show the chart, explain how to reproduce it, and state the main denominator choices behind the probabilities.

Project Structure

• Docs/ - challenge brief, API source notes, methodology, and reasoning.
• src/ - Python modules organized by DAL, BLL, and presentation concerns.
• src/models.py - shared Python dataclass definitions for structured records passed between layers, such as source metadata and calculated event probabilities.
• src/dal/ - data access layer: fetches external data, reads and writes raw and processed datasets, and hides PxWeb/API/file details from the rest of the application.
• src/bll/ - business logic layer: cleans and validates datasets, calculates probabilities, applies denominator rules, and builds the final event table.
• src/presentation/ - presentation layer: renders charts, exports summary text, and prepares outputs for the README.
• data/raw/ - unmodified source data.
• data/processed/ - generated intermediate datasets.
• outputs/ - final figures and exported tables.
• tests/ - tests for probability calculations and data transformations.

Use small modules with direct names inside the relevant layer, such as src/dal/fetch_pxweb.py, src/bll/build_events.py, and src/presentation/plot_scale.py.

Use Python dataclass types for stable records shared between modules. Define these shared classes in src/models.py rather than creating a separate domain layer. Keep the classes small and descriptive, for example an event probability record with event_name, event_group, probability, one_in_n, numerator, denominator, source_table, source_url, year_range, and notes.

Architecture

Keep a clear separation between the data access layer, business logic layer, and presentation layer:

• DAL code is responsible only for data access. It may call external APIs, read and write files under data/, cache raw responses, and normalize API response formats into data frames or records. It should not calculate probabilities or decide which events belong on the final scale.
• BLL code owns the analytical rules. It should receive data from the DAL, clean and validate it, calculate numerators, denominators, probabilities, and one_in_n, and return structured event records. It should not know how PxWeb requests are made or how charts are styled.
• Presentation code owns user-facing outputs. It should take final event records from the BLL and write outputs/events.csv, outputs/probability_scale.svg, outputs/probability_scale.png, and optional summaries. It should not fetch source data or change denominator logic.
• Shared data structures should live in src/models.py as dataclasses. Do not add a separate domain layer unless the project grows enough to need richer domain concepts beyond simple records.

Top-level scripts or entry points may orchestrate the layers, but should keep themselves thin: call DAL functions to load data, pass those results into BLL functions, then pass final events into presentation functions.

Development Standards

Use Python and keep all steps reproducible from code: fetch data, process it, calculate probabilities, and save outputs. Follow PEP 8, use snake_case, and write short functions with clear inputs and outputs.

Document every external dataset with a source URL. When adding tests, use pytest and focus on probability formulas, denominator handling, missing values, and generated event-table validity.