Instruction file imported from networkdowntime/oauth-user-management (.github/instructions/tech-performance-optimization.instructions.md). Copyright stays with the author.

Performance Optimization Best Practices

Introduction

Performance isn't just a buzzword—it's the difference between a product people love and one they abandon. I've seen firsthand how a slow app can frustrate users, rack up cloud bills, and even lose customers. This guide is a living collection of the most effective, real-world performance practices I've used and reviewed, covering frontend, backend, and database layers, as well as advanced topics. Use it as a reference, a checklist, and a source of inspiration for building fast, efficient, and scalable software.

General Principles

Measure First, Optimize Second: Always profile and measure before optimizing. Use benchmarks, profilers, and monitoring tools to identify real bottlenecks. Guessing is the enemy of performance.
- Pro Tip: Use tools like Chrome DevTools, Lighthouse, New Relic, Datadog, Py-Spy, or your language's built-in profilers.
Optimize for the Common Case: Focus on optimizing code paths that are most frequently executed. Don't waste time on rare edge cases unless they're critical.
Avoid Premature Optimization: Write clear, maintainable code first; optimize only when necessary. Premature optimization can make code harder to read and maintain.
Minimize Resource Usage: Use memory, CPU, network, and disk resources efficiently. Always ask: "Can this be done with less?"
Prefer Simplicity: Simple algorithms and data structures are often faster and easier to optimize. Don't over-engineer.
Document Performance Assumptions: Clearly comment on any code that is performance-critical or has non-obvious optimizations. Future maintainers (including you) will thank you.
Understand the Platform: Know the performance characteristics of your language, framework, and runtime. What's fast in Python may be slow in JavaScript, and vice versa.
Automate Performance Testing: Integrate performance tests and benchmarks into your CI/CD pipeline. Catch regressions early.
Set Performance Budgets: Define acceptable limits for load time, memory usage, API latency, etc. Enforce them with automated checks.

Frontend Performance

Rendering and DOM

Minimize DOM Manipulations: Batch updates where possible. Frequent DOM changes are expensive.
- Anti-pattern: Updating the DOM in a loop. Instead, build a document fragment and append it once.
Virtual DOM Frameworks: Use React, Vue, or similar efficiently—avoid unnecessary re-renders.
- React Example: Use React.memo, useMemo, and useCallback to prevent unnecessary renders.
Keys in Lists: Always use stable keys in lists to help virtual DOM diffing. Avoid using array indices as keys unless the list is static.
Avoid Inline Styles: Inline styles can trigger layout thrashing. Prefer CSS classes.
CSS Animations: Use CSS transitions/animations over JavaScript for smoother, GPU-accelerated effects.
Defer Non-Critical Rendering: Use requestIdleCallback or similar to defer work until the browser is idle.

Asset Optimization

Image Compression: Use tools like ImageOptim, Squoosh, or TinyPNG. Prefer modern formats (WebP, AVIF) for web delivery.
SVGs for Icons: SVGs scale well and are often smaller than PNGs for simple graphics.
Minification and Bundling: Use Webpack, Rollup, or esbuild to bundle and minify JS/CSS. Enable tree-shaking to remove dead code.
Cache Headers: Set long-lived cache headers for static assets. Use cache busting for updates.
Lazy Loading: Use loading="lazy" for images, and dynamic imports for JS modules/components.
Font Optimization: Use only the character sets you need. Subset fonts and use font-display: swap.

Network Optimization

Reduce HTTP Requests: Combine files, use image sprites, and inline critical CSS.
HTTP/2 and HTTP/3: Enable these protocols for multiplexing and lower latency.
Client-Side Caching: Use Service Workers, IndexedDB, and localStorage for offline and repeat visits.
CDNs: Serve static assets from a CDN close to your users. Use multiple CDNs for redundancy.
Defer/Async Scripts: Use defer or async for non-critical JS to avoid blocking rendering.
Preload and Prefetch: Use <link rel="preload"> and <link rel="prefetch"> for critical resources.

JavaScript Performance

Avoid Blocking the Main Thread: Offload heavy computation to Web Workers.
Debounce/Throttle Events: For scroll, resize, and input events, use debounce/throttle to limit handler frequency.
Memory Leaks: Clean up event listeners, intervals, and DOM references. Use browser dev tools to check for detached nodes.
Efficient Data Structures: Use Maps/Sets for lookups, TypedArrays for numeric data.
Avoid Global Variables: Globals can cause memory leaks and unpredictable performance.
Avoid Deep Object Cloning: Use shallow copies or libraries like lodash's cloneDeep only when necessary.

Accessibility and Performance

Accessible Components: Ensure ARIA updates are not excessive. Use semantic HTML for both accessibility and performance.
Screen Reader Performance: Avoid rapid DOM updates that can overwhelm assistive tech.

Framework-Specific Tips

React

Use React.memo, useMemo, and useCallback to avoid unnecessary renders.
Split large components and use code-splitting (React.lazy, Suspense).
Avoid anonymous functions in render; they create new references on every render.
Use ErrorBoundary to catch and handle errors gracefully.
Profile with React DevTools Profiler.

Angular

Use OnPush change detection for components that don't need frequent updates.
Avoid complex expressions in templates; move logic to the component class.
Use trackBy in ngFor for efficient list rendering.
Lazy load modules and components with the Angular Router.
Profile with Angular DevTools.

Vue

Use computed properties over methods in templates for caching.
Use v-show vs v-if appropriately (v-show is better for toggling visibility frequently).
Lazy load components and routes with Vue Router.
Profile with Vue Devtools.

Common Frontend Pitfalls

Loading large JS bundles on initial page load.
Not compressing images or using outdated formats.
Failing to clean up event listeners, causing memory leaks.
Overusing third-party libraries for simple tasks.
Ignoring mobile performance (test on real devices!).

Frontend Troubleshooting

Use Chrome DevTools' Performance tab to record and analyze slow frames.
Use Lighthouse to audit performance and get actionable suggestions.
Use WebPageTest for real-world load testing.
Monitor Core Web Vitals (LCP, FID, CLS) for user-centric metrics.

Backend Performance

Algorithm and Data Structure Optimization

Choose the Right Data Structure: Arrays for sequential access, hash maps for fast lookups, trees for hierarchical data, etc.
Efficient Algorithms: Use binary search, quicksort, or hash-based algorithms where appropriate.
Avoid O(n^2) or Worse: Profile nested loops and recursive calls. Refactor to reduce complexity.
Batch Processing: Process data in batches to reduce overhead (e.g., bulk database inserts).
Streaming: Use streaming APIs for large data sets to avoid loading everything into memory.

Concurrency and Parallelism

Asynchronous I/O: Use async/await, callbacks, or event loops to avoid blocking threads.
Thread/Worker Pools: Use pools to manage concurrency and avoid resource exhaustion.
Avoid Race Conditions: Use locks, semaphores, or atomic operations where needed.
Bulk Operations: Batch network/database calls to reduce round trips.
Backpressure: Implement backpressure in queues and pipelines to avoid overload.

Caching

Cache Expensive Computations: Use in-memory caches (Redis, Memcached) for hot data.
Cache Invalidation: Use time-based (TTL), event-based, or manual invalidation. Stale cache is worse than no cache.
Distributed Caching: For multi-server setups, use distributed caches and be aware of consistency issues.
Cache Stampede Protection: Use locks or request coalescing to prevent thundering herd problems.
Don't Cache Everything: Some data is too volatile or sensitive to cache.

API and Network

Minimize Payloads: Use JSON, compress responses (gzip, Brotli), and avoid sending unnecessary data.
Pagination: Always paginate large result sets. Use cursors for real-time data.
Rate Limiting: Protect APIs from abuse and overload.
Connection Pooling: Reuse connections for databases and external services.
Protocol Choice: Use HTTP/2, gRPC, or WebSockets for high-throughput, low-latency communication.

Logging and Monitoring

Minimize Logging in Hot Paths: Excessive logging can slow down critical code.
Structured Logging: Use JSON or key-value logs for easier parsing and analysis.
Monitor Everything: Latency, throughput, error rates, resource usage. Use Prometheus, Grafana, Datadog, or similar.
Alerting: Set up alerts for performance regressions and resource exhaustion.

Language/Framework-Specific Tips

Node.js

Use asynchronous APIs; avoid blocking the event loop (e.g., never use fs.readFileSync in production).
Use clustering or worker threads for CPU-bound tasks.
Limit concurrent open connections to avoid resource exhaustion.
Use streams for large file or network data processing.
Profile with clinic.js, node --inspect, or Chrome DevTools.

Python

Use built-in data structures (dict, set, deque) for speed.
Profile with cProfile, line_profiler, or Py-Spy.
Use multiprocessing or asyncio for parallelism.
Avoid GIL bottlenecks in CPU-bound code; use C extensions or subprocesses.
Use lru_cache for memoization.

Java

Use efficient collections (ArrayList, HashMap, etc.).
Profile with VisualVM, JProfiler, or YourKit.
Use thread pools (Executors) for concurrency.
Tune JVM options for heap and garbage collection (-Xmx, -Xms, -XX:+UseG1GC).
Use CompletableFuture for async programming.

.NET

Use async/await for I/O-bound operations.
Use Span<T> and Memory<T> for efficient memory access.
Profile with dotTrace, Visual Studio Profiler, or PerfView.
Pool objects and connections where appropriate.
Use IAsyncEnumerable<T> for streaming data.

Common Backend Pitfalls

Synchronous/blocking I/O in web servers.
Not using connection pooling for databases.
Over-caching or caching sensitive/volatile data.
Ignoring error handling in async code.
Not monitoring or alerting on performance regressions.

Backend Troubleshooting

Use flame graphs to visualize CPU usage.
Use distributed tracing (OpenTelemetry, Jaeger, Zipkin) to track request latency across services.
Use heap dumps and memory profilers to find leaks.
Log slow queries and API calls for analysis.

Database Performance

Query Optimization

Indexes: Use indexes on columns that are frequently queried, filtered, or joined. Monitor index usage and drop unused indexes.
*Avoid SELECT : Select only the columns you need. Reduces I/O and memory usage.
Parameterized Queries: Prevent SQL injection and improve plan caching.
Query Plans: Analyze and optimize query execution plans. Use EXPLAIN in SQL databases.
Avoid N+1 Queries: Use joins or batch queries to avoid repeated queries in loops.
Limit Result Sets: Use LIMIT/OFFSET or cursors for large tables.

Schema Design

Normalization: Normalize to reduce redundancy, but denormalize for read-heavy workloads if needed.
Data Types: Use the most efficient data types and set appropriate constraints.
Partitioning: Partition large tables for scalability and manageability.
Archiving: Regularly archive or purge old data to keep tables small and fast.
Foreign Keys: Use them for data integrity, but be aware of performance trade-offs in high-write scenarios.

Transactions

Short Transactions: Keep transactions as short as possible to reduce lock contention.
Isolation Levels: Use the lowest isolation level that meets your consistency needs.
Avoid Long-Running Transactions: They can block other operations and increase deadlocks.

Caching and Replication

Read Replicas: Use for scaling read-heavy workloads. Monitor replication lag.
Cache Query Results: Use Redis or Memcached for frequently accessed queries.
Write-Through/Write-Behind: Choose the right strategy for your consistency needs.
Sharding: Distribute data across multiple servers for scalability.

NoSQL Databases

Design for Access Patterns: Model your data for the queries you need.
Avoid Hot Partitions: Distribute writes/reads evenly.
Unbounded Growth: Watch for unbounded arrays or documents.
Sharding and Replication: Use for scalability and availability.
Consistency Models: Understand eventual vs strong consistency and choose appropriately.

Common Database Pitfalls

Missing or unused indexes.
SELECT * in production queries.
Not monitoring slow queries.
Ignoring replication lag.
Not archiving old data.

Database Troubleshooting

Use slow query logs to identify bottlenecks.
Use EXPLAIN to analyze query plans.
Monitor cache hit/miss ratios.
Use database-specific monitoring tools (pg_stat_statements, MySQL Performance Schema).

Code Review Checklist for Performance

Advanced Topics

Profiling and Benchmarking

Profilers: Use language-specific profilers (Chrome DevTools, Py-Spy, VisualVM, dotTrace, etc.) to identify bottlenecks.
Microbenchmarks: Write microbenchmarks for critical code paths. Use benchmark.js, pytest-benchmark, or JMH for Java.
A/B Testing: Measure real-world impact of optimizations with A/B or canary releases.
Continuous Performance Testing: Integrate performance tests into CI/CD. Use tools like k6, Gatling, or Locust.

Memory Management

Resource Cleanup: Always release resources (files, sockets, DB connections) promptly.
Object Pooling: Use for frequently created/destroyed objects (e.g., DB connections, threads).
Heap Monitoring: Monitor heap usage and garbage collection. Tune GC settings for your workload.
Memory Leaks: Use leak detection tools (Valgrind, LeakCanary, Chrome DevTools).

Scalability

Horizontal Scaling: Design stateless services, use sharding/partitioning, and load balancers.
Auto-Scaling: Use cloud auto-scaling groups and set sensible thresholds.
Bottleneck Analysis: Identify and address single points of failure.
Distributed Systems: Use idempotent operations, retries, and circuit breakers.

Security and Performance

Efficient Crypto: Use hardware-accelerated and well-maintained cryptographic libraries.
Validation: Validate inputs efficiently; avoid regexes in hot paths.
Rate Limiting: Protect against DoS without harming legitimate users.

Mobile Performance

Startup Time: Lazy load features, defer heavy work, and minimize initial bundle size.
Image/Asset Optimization: Use responsive images and compress assets for mobile bandwidth.
Efficient Storage: Use SQLite, Realm, or platform-optimized storage.
Profiling: Use Android Profiler, Instruments (iOS), or Firebase Performance Monitoring.

Cloud and Serverless

Cold Starts: Minimize dependencies and keep functions warm.
Resource Allocation: Tune memory/CPU for serverless functions.
Managed Services: Use managed caching, queues, and DBs for scalability.
Cost Optimization: Monitor and optimize for cloud cost as a performance metric.

Practical Examples

Example 1: Debouncing User Input in JavaScript

// BAD: Triggers API call on every keystroke
input.addEventListener('input', (e) => {
  fetch(`/search?q=${e.target.value}`);
});

// GOOD: Debounce API calls
let timeout;
input.addEventListener('input', (e) => {
  clearTimeout(timeout);
  timeout = setTimeout(() => {
    fetch(`/search?q=${e.target.value}`);
  }, 300);
});

Example 2: Efficient SQL Query

-- BAD: Selects all columns and does not use an index
SELECT * FROM users WHERE email = 'user@example.com';

-- GOOD: Selects only needed columns and uses an index
SELECT id, name FROM users WHERE email = 'user@example.com';

Example 3: Caching Expensive Computation in Python

# BAD: Recomputes result every time
result = expensive_function(x)

# GOOD: Cache result
from functools import lru_cache

@lru_cache(maxsize=128)
def expensive_function(x):
    ...
result = expensive_function(x)

Example 4: Lazy Loading Images in HTML

<!-- BAD: Loads all images immediately -->
<img src="large-image.jpg" />

<!-- GOOD: Lazy loads images -->
<img src="large-image.jpg" loading="lazy" />

Example 5: Asynchronous I/O in Node.js

// BAD: Blocking file read
const data = fs.readFileSync('file.txt');

// GOOD: Non-blocking file read
fs.readFile('file.txt', (err, data) => {
  if (err) throw err;
  // process data
});

Example 6: Profiling a Python Function

import cProfile
import pstats

def slow_function():
    ...

cProfile.run('slow_function()', 'profile.stats')
p = pstats.Stats('profile.stats')
p.sort_stats('cumulative').print_stats(10)

Example 7: Using Redis for Caching in Node.js

const redis = require('redis');
const client = redis.createClient();

function getCachedData(key, fetchFunction) {
  return new Promise((resolve, reject) => {
    client.get(key, (err, data) => {
      if (data) return resolve(JSON.parse(data));
      fetchFunction().then(result => {
        client.setex(key, 3600, JSON.stringify(result));
        resolve(result);
      });
    });
  });
}

References and Further Reading

Conclusion

Performance optimization is an ongoing process. Always measure, profile, and iterate. Use these best practices, checklists, and troubleshooting tips to guide your development and code reviews for high-performance, scalable, and efficient software. If you have new tips or lessons learned, add them here—let's keep this guide growing!

About