Imported from kotuku-group/kotuku (src/tiri/jit/src/parser/AGENTS.md). Install upstream with npx skills add kotuku-group/kotuku --skill parser. Copyright stays with the author.

Parser Implementation Notes

This file captures parser-specific practices and gotchas for the Tiri JIT parser that ships inside Kōtuku. The parser has been modernised to C++20 and implements a two-phase architecture: AST construction followed by IR emission.

The parser is built as an amalgamated build, with all source files included in a single compilation unit at parser.cpp. This simplifies dependency management and ensures consistent compiler settings across the entire parser codebase.

Architecture Overview

The parser follows a classic two-phase compilation model:

1. AST Building Phase (ast/) - Parses source tokens into a typed Abstract Syntax Tree
2. IR Emission Phase (ir_emitter/) - Lowers the AST to LuaJIT bytecode

This separation allows the AST and bytecode emission to evolve independently while sharing a single AST boundary.

Parser File Structure

Core Infrastructure

Lexer

Parser Context and Configuration

AST Building (ast/ subdirectory)

IR Emission (ir_emitter/ subdirectory)

Register and Control Flow Management

Type System

Legacy/Support Files

Key Classes

ParserContext

Central context object that bundles:

• LexState reference for tokenisation
• FuncState reference for bytecode state
• ParserDiagnostics for error collection
• TokenStreamAdapter for token consumption
• ParserConfig for behaviour options

AstBuilder

Constructs a typed AST from the token stream. Entry point is parse_chunk() which returns a BlockStmt root node. The builder is stateless between chunks and does not touch FuncState or emit bytecode.

IrEmitter

Lowers AST nodes to LuaJIT bytecode. Entry point is emit_chunk() which consumes a BlockStmt and populates FuncState with bytecode instructions. Uses RegisterAllocator and ControlFlowGraph for resource management.

RegisterAllocator

Manages register allocation with RAII-based RegisterSpan for automatic cleanup. Tracks the register floor and provides debugging assertions for register balance.

ControlFlowGraph

Manages jump patching for branching constructs. Provides ControlFlowEdge for forward jumps that are resolved when targets become known.

Key Implementation Patterns

Type Safety with Concepts

// Use concepts for compile-time validation
template<BytecodeOpcode Op>
static inline BCPOS bcemit_ABC(FuncState *, Op, BCREG, BCREG, BCREG);

RAII for Resource Management

// Automatic scope cleanup
FuncScope bl;
ScopeGuard scope_guard(fs, &bl, FuncScopeFlag::None);
// ... parse statements ...
// Automatic cleanup on scope exit

Result Types for Error Handling

// ParserResult<T> wraps success/failure without exceptions
ParserResult<ExprNodePtr> result = parse_expression();
if (not result.ok()) return ParserResult<StmtNodePtr>::failure(result.error_ref());

AST Node Ownership

// Nodes own children through unique_ptr
using ExprNodePtr = std::unique_ptr<ExprNode>;
using StmtNodePtr = std::unique_ptr<StmtNode>;
using ExprNodeList = std::vector<ExprNodePtr>;

Modern Container Usage

// Prefer std::span for array access
auto uvmap_range = std::span(fs->uvmap.data(), fs->nuv);
for (auto uv_idx : uvmap_range) { ... }

// Use std::string_view for string parameters
static GCstr * keepstr(std::string_view str);

Quick Reference

File Locations

• Parser source: src/tiri/jit/src/parser/
• AST definitions: src/tiri/jit/src/parser/ast/
• IR emission: src/tiri/jit/src/parser/ir_emitter/
• Bytecode reference: src/tiri/jit/BYTECODE.md
• Tiri tests: src/tiri/tests/
• Troubleshooting Guide: src/tiri/jit/src/parser/TROUBLESHOOTING.md

Common Tasks

• Adding an AST node: Define struct in nodes.h, add AstNodeKind enum entry, implement parsing in builder_*.cpp, implement emission in ir_emitter/emit_*.cpp
• Adding an operator: Add to AstBinaryOperator/AstUnaryOperator enum, handle in expressions.cpp, emit in operator_emitter.cpp
• Register allocation: Use RegisterAllocator::reserve() for temporary registers, ensure balance with ensure_register_floor()
• Understanding concepts: See parse_concepts.h for type constraints
• Expression builders: See parse_types.h for constexpr factories

Key Headers

AST Node Hierarchy

The AST uses a discriminated union pattern:

ExprNode (AstNodeKind discriminator)
├── Literals: Nil, Boolean, Number, String
├── References: Identifier, Vararg
├── Operations: Unary, Binary, Ternary, Update, Presence
├── Access: Member, Index, SafeMember, SafeIndex
├── Calls: Call, SafeCall, Pipe
├── Constructors: Table, Function, Range
└── Extensions: Choose, ResultFilter

StmtNode (AstNodeKind discriminator)
├── Declarations: LocalDecl, GlobalDecl, LocalFunction, Function
├── Control Flow: If, While, Repeat, NumericFor, GenericFor
├── Jumps: Break, Continue, Return
├── Blocks: Block, Defer
└── Expressions: ExpressionStmt, Assignment, ConditionalShorthand

Type System Overview

The parser includes optional static type analysis:

• TiriType enum defines supported types (Nil, Boolean, Number, String, Table, Function, etc.)
• FunctionReturnTypes captures declared return type annotations
• TypeCheckScope tracks variable types within scopes
• Type mismatches can be configured as errors or warnings via ParserConfig

Type annotations use the syntax local x: number = 42 and function return types use : <type1, type2>.