Imported from jbcodeforce/flink-studies (
AGENTS.md). Install upstream withnpx skills add jbcodeforce/flink-studies. Copyright stays with the author.
Agent Description for Flink Studies Repository
Repository Overview
This is a comprehensive Apache Flink and Confluent Cloud data streaming processing, learning and demonstration repository containing practical examples, deployment configurations, documentation, and end-to-end demos for real-world streaming data scenarios.
Repository Structure & Agent Responsibilities
/docs - Documentation (MkDocs Configuration)
- Purpose: Contains all documentation files configured for MkDocs static site generation
- Agent Role:
- Maintain and update documentation structure
- Ensure proper MkDocs YAML configuration
- Keep documentation current with code examples
- Manage cross-references and navigation structure
/code - Implementation Examples
/code/flink-sql - SQL Examples and Tutorials
- Simple how-to guides for Flink SQL operations
- Basic SQL patterns and common use cases
- Data transformation examples
/code/dbt - dbt examples
- simple code for dbt confluent, and dbt ETL with duckdb
/code/flink-java - Java DataStream API Examples
- Java-based Flink applications
- DataStream API demonstrations
- Batch and stream processing samples
/code/table-api - Table API Examples
- Table API implementations in Java and Python
- Bridge between SQL and programmatic APIs
/code/tools - Utility Scripts
- Helper scripts for data processing and analysis
/deployment - Deployment Configurations
- Purpose: Contains various deployment strategies and configurations
- Agent Role:
- Maintain deployment scripts for different environments
- Keep Docker, Kubernetes, and Terraform configurations updated
- Ensure deployment examples work with latest Flink versions
- Document deployment best practices
Key Deployment Types:
- Kubernetes: Production-ready K8s deployments with operators
- Confluent Cloud: Cloud-native Flink deployments
- Custom Images: Specialized Flink runtime configurations
/e2e-demos - End-to-End Customer Demonstrations
- Purpose: Advanced, customer-facing demonstrations solving real business problems
- Agent Role:
- Maintain production-ready demo scenarios
- Ensure demos work across different deployment environments
- Keep demos updated with latest Flink and Kafka versions
- Document business use cases and technical implementation
/assistant - AI/ML Integration
- Text QA assistant for Flink documentation
- Corpus building for knowledge base
- SQL query assistance and validation
/tools
- Script to help manage demonstration and studies
- Script to do web research on Flink articles
Agent Capabilities & Responsibilities
Core Competencies Required:
-
Apache Flink Expertise
- Deep understanding of Flink SQL, DataStream API, and Table API
- Knowledge of Flink deployment patterns and best practices
- Understanding of streaming concepts (windowing, state, checkpointing)
-
Infrastructure Knowledge
- Docker containerization
- Kubernetes orchestration
- Terraform infrastructure as code
- Apache Kafka integration
-
Documentation Management
- MkDocs configuration and maintenance
- Technical writing and documentation structure
- Code example synchronization with documentation
-
Development Support
- Java and Python programming for Flink applications
- SQL query optimization and debugging
- CI/CD pipeline maintenance
- for Python code use
uvas package managers and virtual environment management.
Primary Agent Tasks:
- Code Maintenance: Keep all code examples functional and up-to-date
- Documentation Sync: Ensure documentation reflects current code implementations
- Deployment Validation: Test and validate deployment configurations
- Demo Management: Maintain working end-to-end demonstrations
- Demo Scaffolding: Use jump_start_demo/ to generate foundational e2e demo code (CLI, Agno agent, or Cursor skill)
- Knowledge Base: Assist with Flink-related questions and best practices
- Version Management: Keep pace with Flink and ecosystem updates
Agent Interaction Guidelines:
- Prioritize working, executable examples over theoretical explanations
- Focus on practical, real-world scenarios that demonstrate business value
- Maintain clear separation between simple tutorials (
/code) and advanced demos (/e2e-demos) - Ensure all deployment configurations include proper documentation and validation steps
- Keep examples vendor-agnostic where possible, with specific vendor implementations clearly marked
Success Metrics:
- All code examples compile and run successfully
- Documentation is current and accurately reflects implementation
- Deployment configurations work in their target environments
- End-to-end demos successfully demonstrate complete business scenarios
- Knowledge base effectively answers common Flink development questions