Welcome to the first part of this series, What You Need to Succeed - Essential Concepts for Serverless Success. Serverless is an exciting and powerful approach to building scalable and efficient systems. However, success requires understanding core concepts and implementing the right patterns. In this post, we’ll explore four key areas to set you up for success: Design Patterns, Backoff and Retry, Reserved Concurrency, and the Circuit Breaker pattern.
1. Design Patterns
What are design patterns?
Software Design patterns are inspired by urban and building architecture, by Christopher Alexander’s Pattern Language.
They gained prominence through the "Gang of Four" book, Design Patterns: Elements of Reusable Object-Oriented Software.
Think of them as reusable solutions to common problems in software development - a set of blueprints that guide you without forcing rigid rules. In serverless architectures, design patterns help simplify complexity, improve communication among developers, and promote maintainable and flexible code.
Why Design Patterns Matter in Serverless
Design patterns refer to general software design solutions and not necessarily serverless-specific architectural patterns. However, they play an essential role in serverless architectures too. Beyond the obvious benefits:
- Reusable Solutions: Solve recurring challenges efficiently.
- Shared Language: Foster collaboration and alignment within your team.
- Maintainability and Flexibility: Reduce duplication and ease future changes.
->Teams will struggle to adapt their architectural approach when the codebase is poorly maintained. Without clean and manageable code, meaningful improvements become nearly impossible.
In serverless development, you will learn and evolve continuously, so maintaining flexibility to adjust, replace services, or enhance segregation over time is essential to long-term success. And you can't improve what you can't change!
Categories of Design Patterns
- Creational: Focus on how objects are created. (e.g., Singleton).
- Structural: Deal with the composition of objects and classes. (e.g., Adapter).
- Behavioral: Concerned with communication between objects. (e.g., Observer).
When to Apply Patterns
- Improve Maintainability: Patterns reduce code duplication and improve structure.
- Encourage Flexibility: Enable future changes with reasonable effort.
- Enhance Collaboration: Patterns act as a shared vocabulary for developers.
Bonus Tip: Don’t overuse patterns; let them emerge naturally as you identify recurring problems. Use them as tools, not rules.
Bonus Tip 2: Avoid patterns when the problem is simple, the team is unfamiliar with the pattern, or when it violates YAGNI ("You Aren’t Gonna Need It").
2. Backoff and Retry: Handling Failures Gracefully
In distributed systems, failures are inevitable. APIs can fail due to network issues or server overload, and your serverless architecture must handle these gracefully to ensure reliability.
The Core Idea
When a request fails, pause for a moment (backoff), then retry. This prevents your app from overwhelming the server and increases the chance of success.
What To Do
- Exponential Backoff: Gradually increase the waiting time between retries to avoid server overload.
- Retry Limits: Stop retries after a defined threshold to prevent infinite loops.
- Idempotent Operations: Ensure retries don’t create duplicate effects or errors.
- Fallback Mechanisms: Use circuit breakers (discussed below) or alternative workflows when retries fail.
Retry Limit Reached?
Log the failure, alert stakeholders, and trigger fallback or circuit breaker mechanisms to handle the issue.
Bonus Tip: AWS SDKs like boto3 often include built-in retry and backoff mechanisms. Leverage these to simplify implementation.
3. Reserved Concurrency: Protecting Critical Functions
Serverless environments (especially on AWS) offer incredible scalability, but you need to know a thing or two. A poorly designed function can consume all available capacity, affecting other parts of your system.
The Problem: Shared Limits
By default, all Lambda functions in an AWS account share the same concurrency limit (1,000 parallel executions per account).
or see it like this:
If one function uses up all of these. When this limit is reached, new lambda executions will be rejected.
This is called rate limiting, an important concept in distributed systems. Not Only for AWS lambda, but all serverless services. Common error messages you might encounter include:
RateLimitExceededExceptionTooManyRequestsExceptionThrottlingException
Individual Reserved Concurrency
Reserved concurrency allows you to set execution limits for individual functions, ensuring critical parts of your system have guaranteed capacity.
Why It’s Important
- System Stability: Prevent one function from overwhelming your account.
- Critical Functions: Safeguard system-critical workflows by isolating their concurrency limits.
Additional Tip
Enforce that new Lambda functions cannot be created without defining reserved concurrency to ensure system-wide stability.
4. Circuit Breaker: Preventing Cascading Failures
Failures in distributed systems can snowball. For example, when a subsystem struggles, continuous retries by upstream services can exacerbate the issue, leading to cascading failures.
If left undetected, continuous calls to an unresponsive subsystem can delay recovery, cause total failure, or even trigger cascading failures.
What Is a Circuit Breaker?
A circuit breaker is a pattern that detects persistent failures and temporarily halts further requests to the failing subsystem. This gives it time to recover without added pressure.
How It Works
- Open State: All is normal; requests flow freely.
- Closed State: After repeated failures, requests are not made anymore.
- Half-Open State: After the block, and a delay, a few test requests are allowed. If successful, the circuit reopens; otherwise, it stays closed.
Why Use Circuit Breakers?
- Stability: Protect struggling subsystems from additional load.
- Prevention: Avoid cascading failures across your architecture.
- Recovery: Allow time for subsystems to stabilize before resuming normal operations.
Bonus Tip: Combine circuit breakers with logging and monitoring to detect early signs of trouble in your subsystems.
Key Takeaways
- Design Patterns: Use these as tools to solve recurring problem.
- Backoff and Retry: Handle failures gracefully with exponential backoff and retry limits.
- Reserved Concurrency: Protect critical functions by isolating their concurrency limits.
- Circuit Breaker: Detect and respond to persistent failures to prevent cascading issues.
By mastering these patterns, you’ll be well on your way to building resilient and scalable systems on AWS. Stay tuned for Part 2, where we’ll dive into more essential patterns for serverless success.
