Ready to build better conversations?
Simple to set up. Easy to use. Powerful integrations.
Get free accessReady to build better conversations?
Simple to set up. Easy to use. Powerful integrations.
Get free accessIn an era where providing the best-possible user experience is paramount, our journey toward excellence in ensuring seamless accessibility to our products and support documents encounters a recurring challenge: the need to rotate public-facing endpoints.
This challenge is particularly prominent in the dynamic landscape of modern technology, where changes are constant.
To meet this challenge head-on and to maintain uninterrupted service for our users, we have implemented two robust strategies that involve the redirection of traffic from old URLs to newer endpoints.
Our objective in this technical article is to unveil our current solutions for this intricate traffic redirection project, emphasising the duality of our approach.
We recognise that many of you might be grappling with similar issues and we aim to shed light on our strategies by showcasing a recent use case.
In this particular instance, we deprecated our knowledge base site platform and transitioned our support pages to a new platform.
Crucially, we ensured that all previous URLs remain accessible and automatically redirect users to the relevant articles.
S3 Traffic Redirection Using Metadata
In our pursuit of efficient traffic redirection, we leverage a key feature of AWS S3 static website hosting—the S3 redirection parameter.
This parameter allows us to add a metadata field to our static website, effectively overriding the page header and enabling seamless redirection from old URLs to their corresponding new destinations.
To ensure reliability and consistency, we prefer performing this redirection programmatically rather than directly through the AWS console.
We offer two viable methods for achieving this goal:
Terraform: When dealing with a limited number of URLs to redirect, we recommend using Terraform to edit the S3 bucket for website hosting. This method is particularly suited for smaller-scale redirection needs.
Python Script and AWS CLI: For scenarios involving multiple URL redirection requests, our Python script, coupled with the AWS CLI, provides a swift and efficient solution. This script takes an input CSV file containing pairs of old and new URLs, automatically adding the necessary redirect properties to each S3 object.
Here we provide a snippet of the python script we created for the static website redirection using the S3 property, this scripts receives as input a csv file with original endpoint and target endpoints for the redirection:
Python snippet for static website redirection using S3 metadata
Drawbacks of S3 traffic redirection using metadata
While our S3 traffic redirection using metadata solution offers several advantages, it's essential to be aware of its potential limitations:
Complexity for Large-Scale Redirections: Managing numerous redirections using Terraform or the Python script can become challenging and less scalable.
Maintenance Overhead: As the number of redirections grows, maintaining and organizing Terraform configurations can become cumbersome.
Lack of Real-Time Redirection: S3 redirections are static, so changes may not take effect immediately, leading to delays in URL updates.
Limited Dynamic Behavior: This method is less flexible in handling dynamic scenarios or conditions based on user locations since S3 is hosted in a non-controlled AWS region, so a user in Australia will have a difference performance than a user in the United States for example.
Organizations should consider these drawbacks when choosing the S3 Traffic Redirection approach, especially for scenarios involving extensive redirection needs or dynamic requirements.
Alternative solutions like Lambda@Edge may be more suitable in such cases and it is our current preferred choice for most use cases.
CloudFront Traffic Redirection Empowered by Lambda@Edge
A more innovative solution harnesses the power of Lambda@Edge to seamlessly intercept viewer requests at CloudFront's edge locations. With Lambda@Edge, we have the capability to execute customized logic, enabling dynamic traffic redirection based on URL patterns, query parameters, and other criteria.
This dynamic approach ensures that redirected requests are intelligently routed to the relevant support knowledge base articles or URLs, enhancing the user experience.
Another key feature here is that, since we are using Lambda@Edge, we can guarantee best performance, since the lambda will be triggered in the region closer to the viewer' request.
Overview of the used cloud resources for the Lambda@Edge redirection.
Benefits of Using Lambda@Edge
Reduced Latency: By executing redirection logic at the edge locations, we minimize the round-trip time between the viewer and the origin server, resulting in significantly lower latency for redirected requests.
Flexibility and Customization: Lambda@Edge empowers us to craft intricate redirection rules that align precisely with our unique requirements. We can extract information from requested URLs, conduct pattern matching, and dynamically decide where to redirect traffic for the most fitting destination.
Scalability: Lambda@Edge functions automatically adapt to varying traffic loads, ensuring our redirection system remains consistently available and performs optimally under diverse traffic conditions.
Hands Down Implementation
Create the Lambda@Edge Function: Begin by developing the Lambda@Edge function responsible for traffic redirection. Utilize regular expressions and pattern-matching techniques to effectively manage diverse URL patterns and extract relevant information. This step is especially crucial for paths with multiple languages.
Organize Redirections in an S3 Bucket: Create an S3 bucket object that houses the list of redirections in a JSON file or similar. This organizational approach proves invaluable when dealing with a substantial number of pages to redirect, such as those on a knowledge base site.
Deploy the Function to a Specified Edge Location: After creating the Lambda@Edge function, deploy it to the designated edge location within AWS. This ensures that it's strategically positioned to handle viewer requests efficiently.
Configure CloudFront for Lambda@Edge Usage: Configure CloudFront to utilize the Lambda@Edge function effectively. Create a behavior within the desired CloudFront distribution to specify the paths intended for redirection. Wildcards can be employed for this purpose to broaden the scope of redirection.
Implement as Code for Predictability and Maintainability: To guarantee the predictability and ease of maintenance of your infrastructure, implement the entire solution as code. This approach ensures consistency and simplifies future updates and modifications.
Bellow is a part of one of the functions we use for origin response redirection using CloudFront, some of the internal logic was removed to protect key features and for security purposes. For this specific use case, we went with Javascript.
Javascript Lambda@Edge function that handles redirections and returns the origin response
Understanding the Difference Between Origin Request and Origin Response in Behavior Configurations
In Amazon CloudFront, the ability to finely tune how your content is served to viewers is crucial. Two key settings that play a pivotal role in this process are "Origin Request" and "Origin Response."
These settings determine how CloudFront interacts with your origin server, shaping the way content is delivered to users. Let's delve into the key differences between these two configurations.
Origin Request
Outbound Requests: The "Origin Request" setting governs the communication between CloudFront and your origin server when CloudFront needs to retrieve content that isn't cached or needs to be updated.
Manipulating Outbound Requests: You can use "Origin Request" to modify or enhance outbound requests sent from CloudFront to your origin server. This is particularly useful when you want to add headers, cookies, or query parameters to the request before it reaches the origin.
Origin Response
Inbound Responses: The "Origin Response" setting comes into play when CloudFront receives a response from your origin server. It allows you to modify or enhance the response before it's delivered to the viewer.
Manipulating Inbound Responses: You can use "Origin Response" to process the response from your origin server, altering headers, caching behaviour, or even the content itself. This step enables you to tailor the response to meet specific requirements.
Key Differences
Timing: The most fundamental difference is the timing of their application. "Origin Request" operates before CloudFront sends a request to the origin server, while "Origin Response" operates after CloudFront receives a response from the origin server.
Modification Scope: "Origin Request" primarily focuses on modifying the request sent to the origin server, whereas "Origin Response" deals with modifying the response received from the origin server.
In summary, "Origin Request" and "Origin Response" settings in CloudFront behavior configurations provide the flexibility needed to tailor the content delivery process to your specific needs.
Understanding when and how to use each of these settings is essential for optimizing the performance, security, and functionality of your content delivery network for our current use case it is required to change the Origin Request since the response server is many times deprecated and no longer accessible.
Flow Diagram
Bellow is a flow diagram of a request to an endpoint on the deprecated knowledge base and the redirection using Lambda@Edge response.
Flow Chart for traffic redirection using Lambda@Edge
Challenges and Workarounds
Lambda@Edge, combined with CloudFront, provides a powerful solution for dynamic content delivery and customization. However, like any technology, it comes with its own set of challenges, particularly in terms of monitoring and troubleshooting. Here, we'll explore some common challenges and offer solutions to overcome them:
1. Monitoring Challenges
Limited Visibility: Lambda@Edge functions execute at the edge locations, making it challenging to have comprehensive visibility into their performance and execution.
Solution: Implement robust logging and monitoring strategies. Use CloudWatch Logs to capture and analyze function execution logs. Utilize CloudWatch Metrics to monitor key performance indicators.
2. Cold Starts
Cold Start Latency: Lambda@Edge functions can experience cold starts, which result in increased latency for the first request to a particular edge location.
Solution: Mitigate cold starts by optimising your Lambda functions. Reduce the function's initialization time by minimizing dependencies and optimizing code. Additionally you can implement a warm-up strategy by periodically invoking your function to keep it available on the spot.
3. Retrieving the redirection file from S3 on edge an edge location
Edge Locations: Using edge locations brings a challenge on retrieving the redirection file from the S3 bucket depending on where the bucket is located. This could limit the performance increase we gained from using edge locations.
Solution: To address this challenge, we employed two distinct solutions. Initially, we leveraged caching to maintain an updated file readily accessible at edge locations, effectively minimizing latency for multiple object fetching actions. This approach ensured optimal response quality during peak hours and accommodated our expanding user base. Subsequently, we implemented S3 file transfer acceleration, resulting in a further reduction of latency. Looking ahead, we plan to assess the viability of incorporating the S3 high-performance storage feature to enhance our overall quality of service.
4. Cost Optimization
Cost Considerations: Running Lambda@Edge functions at scale may lead to increased operational costs.
Solution: Regularly review and optimize Lambda@Edge functions to reduce unnecessary execution. Use AWS Cost Explorer to analyze the costs and implement cost-saving strategies, such as rate limiting or caching.
By addressing these challenges with the suggested solutions, you can effectively employ Lambda@Edge with CloudFront while ensuring reliable performance, robust monitoring, and cost-effective operations. Lambda@Edge provides powerful capabilities for dynamic content delivery, and with careful planning and management, you can harness its full potential while mitigating potential issues.
The path forward
AWS keeps fueling our innovation with additional features and solutions for our challenges, a possible path forward for our redirection solution is to use a new feature on CloudFront the KeyValueStore, this would allow us to replicate the redirection file on CloudFront itself and avoid using the S3 bucket for file storage and the lambda@edge solution.
It would simplify our infrastructure and possibly even improve the redirection response time due to the removal of the object fetching from storage.
This solution would have us drop the Lambda@Edge redirection logic and opt for a native CloudFront function associated with each of the behavior rules needed. The values would be fetched directly from the key-value store within CloudFront and the redirection implemented via a simple javaScript CloudFront function that would work at edge locations similar to what the Lambda@Edge offers.
As always we will need to fully evaluate this solution and it’s limitations before moving forward to ensure that our customers have always the best solutions available to them.
Also, it is essential for us is to always streamline our infrastructure, reducing our costs and carbon footprint by using the right tools for the right job.
Conclusion: Elevating Client Experience Through Innovation
In this article, we've unveiled two robust solutions—S3 Traffic Redirection Using Metadata and CloudFront Traffic Redirection with Lambda@Edge—aimed at elevating our clients' experiences.
With S3 Traffic Redirection, we ensure seamless transitions between old and new URLs, even at scale. Terraform and Python scripts enable us to deliver reliable and efficient URL redirection.
But we don't stop there. CloudFront Traffic Redirection powered by Lambda@Edge takes personalisation to the next level. By intercepting viewer requests and enabling dynamic content delivery, we empower clients to optimise, secure, and personalise user experiences.
Our commitment to innovation remains unwavering. These technologies are more than tools; they are the foundation of exceptional user experiences.
As we conclude, we emphasise our dedication to enhancing client experiences. In this dynamic tech landscape, we're here to shape the future of user interactions, ensuring that every interaction is seamless, secure, and unforgettable.
Published on December 15, 2023.
