Imported from meowmeow-uwu/mushroom-classification (web/AGENTS.md). Install upstream with npx skills add meowmeow-uwu/mushroom-classification --skill web. Copyright stays with the author.

AGENTS.md

Project Context

This repository contains the web demo for mushroom classification at the edge/browser.

The web application does not send images to an AI backend for inference.

Main flow:

Image Upload / Camera
        ↓
Image Preview
        ↓
Preprocessing
        ↓
ONNX Runtime Web
        ↓
model.onnx
        ↓
Top-K Prediction
        ↓
Confidence + Inference Time

The purpose of the demo is to show that the exported ONNX model can run directly on the client device.

Main Goal

Build a small, reliable demo application that:

1. accepts a mushroom image;
2. preprocesses it exactly as required by the trained model;
3. runs inference in the browser using ONNX Runtime Web;
4. shows Top-K predicted species;
5. shows confidence scores;
6. shows inference time;
7. clearly indicates that inference is executed locally.

Do not turn this project into a full SaaS application.

Scope

P0 — Required

Upload image
↓
Preview image
↓
Preprocess
↓
ONNX inference
↓
Top-3 predictions
↓
Confidence
↓
Inference time

Required UI information:

• predicted species;
• Top-3 predictions;
• confidence;
• active model name;
• inference time;
• runtime: ONNX Runtime Web;
• execution: local device/browser;
• image sent to server: no.

P1 — Optional

• camera input;
• session-only prediction history;
• model selector;
• compare multiple exported models.

Example:

Same Image
   ↓
┌────────────┬────────────┬────────────┐
│ ResNet50   │ ViT-B/16   │ Hybrid     │
│ 81%        │ 87%        │ 94%        │
│ 50 ms      │ 95 ms      │ 130 ms     │
└────────────┴────────────┴────────────┘

P2 — Optional

Robustness demo:

Original
Blur
Dark
Noise

The same image can be transformed in the browser and re-evaluated to show how prediction confidence changes.

Out of Scope

Do not add these unless explicitly requested:

• login/register;
• authentication;
• database;
• user accounts;
• admin dashboard;
• Spring Boot backend;
• FastAPI inference server;
• microservices;
• CRUD features;
• cloud image upload;
• persistent prediction history;
• complex analytics;
• model training inside the web repository.

Recommended Stack

React
TypeScript
Vite
ONNX Runtime Web

Keep dependencies minimal.

Repository Responsibilities

This repository owns:

Web UI
Image input
Image preprocessing
ONNX model loading
Browser inference
Postprocessing
Top-K display
Inference timing
Optional model comparison

This repository does not own:

Dataset preparation
Model training
Model evaluation pipeline
ONNX export implementation
GAN training
Synthetic data generation

Team Interface

The web side depends on the training/export side through a fixed model contract.

Expected files:

model.onnx
labels.json
model-config.json

Recommended model-config.json:

{
  "modelName": "ResNet50",
  "inputSize": [224, 224],
  "channels": 3,
  "colorFormat": "RGB",
  "layout": "NCHW",
  "mean": [0.485, 0.456, 0.406],
  "std": [0.229, 0.224, 0.225],
  "outputType": "logits",
  "topK": 3
}

labels.json example:

[
  "Agaricus_x",
  "Amanita_y",
  "Boletus_z"
]

Do not hard-code preprocessing assumptions if they can be supplied through model-config.json.

Critical Rule: Preprocessing Must Match Training

The following values must match the training/export pipeline exactly:

• input width;
• input height;
• channel count;
• RGB/BGR order;
• tensor layout;
• resize strategy;
• normalization mean;
• normalization standard deviation;
• output class order;
• whether the model output is logits or probabilities.

A correct model with incorrect preprocessing can produce incorrect predictions.

Suggested Architecture

src/
├── components/
│   ├── ImageUploader.tsx
│   ├── ImagePreview.tsx
│   ├── PredictionResult.tsx
│   ├── TopKList.tsx
│   ├── ModelInfo.tsx
│   └── ModelSelector.tsx
│
├── inference/
│   ├── loadModel.ts
│   ├── preprocess.ts
│   ├── predict.ts
│   └── postprocess.ts
│
├── model/
│   ├── modelConfig.ts
│   └── labels.ts
│
├── utils/
│   └── timing.ts
│
├── App.tsx
└── main.tsx

Static model assets:

public/
└── models/
    ├── model.onnx
    ├── labels.json
    └── model-config.json

If multiple models are supported:

public/models/
├── resnet50/
├── vit-b16/
└── hybrid/

Each directory should contain its own:

model.onnx
labels.json
model-config.json

Inference Workflow

User selects image
        ↓
Decode image
        ↓
Resize
        ↓
Convert pixels to tensor
        ↓
Normalize
        ↓
Create ONNX input tensor
        ↓
Run inference session
        ↓
Read output
        ↓
Softmax if output is logits
        ↓
Sort probabilities
        ↓
Take Top-K
        ↓
Render result

Implementation Rules

Model Loading

• load the ONNX model once when possible;
• reuse the inference session;
• show a loading state while the model is being initialized;
• show a clear error if model loading fails.

Image Handling

• validate supported image formats;
• reject invalid input cleanly;
• preview the selected image before inference;
• avoid unnecessary image uploads or network calls.

Inference

• measure inference time around the actual inference call;
• do not include UI rendering time in the inference metric;
• keep preprocessing and postprocessing deterministic;
• do not silently change model input assumptions.

Results

Show at least:

Top prediction
Confidence
Top-3 predictions
Inference time
Model name
Execution location

Do not present the result as medical or food-safety advice.

Recommended notice:

This model is for research and demonstration only. Do not use its prediction to decide whether a mushroom is safe to eat.

Mock-First Development

The web work should not wait for the final model.

Start with:

type Prediction = {
  label: string;
  confidence: number;
};

const mockPrediction: Prediction[] = [
  { label: "Amanita muscaria", confidence: 0.924 },
  { label: "Amanita pantherina", confidence: 0.048 },
  { label: "Amanita rubescens", confidence: 0.017 }
];

Build the complete UI with mock data first.

Then replace:

mockPredict()

with:

onnxPredict()

when model.onnx is available.

Error Handling

Handle at least:

• unsupported image;
• failed model load;
• invalid model input shape;
• inference failure;
• missing labels;
• mismatch between output size and label count;
• browser/runtime incompatibility.

Errors should be visible to the user and useful for debugging.

Performance

Prefer:

Load model once
↓
Reuse session
↓
Run repeated inference

Avoid:

Select image
↓
Reload model
↓
Infer

Keep the demo responsive.

Definition of Done

P0 is complete when:

• the application loads;
• an image can be selected;
• the image is previewed;
• preprocessing matches the model contract;
• model.onnx runs in the browser;
• Top-3 predictions are displayed;
• confidence is displayed;
• inference time is displayed;
• no AI inference request is sent to a backend;
• the application handles basic errors;
• the research-only disclaimer is visible.