The short answer? DNS.
The long answer is a bit more involved.
Let’s take a closer look at what’s actually happening.
For our example, we’ll use Python with boto3, but the problem exists in every programming language and AWS SDK.
client = boto3.client(
"s3"
)
body = "Hello Intenics!".encode()
client.put_object(Bucket=BUCKET, Key=key, Body=body, ContentType="text/plain")
url = client.generate_presigned_url(
"get_object",
Params={"Bucket": BUCKET_NAME, "Key": BUCKET_KEY},
ExpiresIn=EXPIRES_IN,
)This will typically result in a link with following schema:
This type of URL is what AWS calls a "virtual-hosted" URL - the new recommended way to handle S3 paths. In this format, your bucket gets its own subdomain, which unfortunately depends on AWS’s DNS TTL. And yes, that can be pretty annoying. In our experience it usually needs between 0 and 4 hours.
Luckily, the fix is simple: switch back to the old "path-style" S3 URLs. To make boto3 generate those, you just need to adjust your client configuration like this:
client = boto3.client(
"s3",
config=Config(
s3={"addressing_style": "path"}
),
)Now our presigned URL should look like this:
Why did we do this?
In CI/CD environments that use feature branches, you can’t depend on DNS updates every time you need to run tests against your system. When the DNS lags, the deployed frontend breaks, and both integration and end-to-end tests fail. For short-lived feature branches and any automated testing setup, this is simply unacceptable.
Why AWS wants us to use the Virtual-Hosted Style
AWS moved toward virtual-hosted style URLs as the standard for accessing S3 objects because they solve a few limitations of the old path-style format. There are three major benefits
Direct Lookups
With the bucket name in the hostname (e.g., my-bucket.s3.eu-central-1.amazonaws.com), AWS can route requests directly to the correct region without extra lookups.
Better SSL compatibility
Virtual-hosted URLs make it possible to use HTTPS with custom bucket names while still validating against AWS’s wildcard SSL certificates.
Custom DNS
You can have your bucket on your own subdomain.
So we are using the deprecated format?!
Yes. In a blog post back in 2020 AWS announced that path-style URLs are considered legacy. While they still work for now, new features are being built assuming virtual-hosted style.
Furthermore the blog post states that they will announce a full deprecation a year before it happens. This was five years ago and we are still able to run on the path style, so we have no need to assume it will change soon.
Some last words
If you’ve ever wondered why your shiny new S3 presigned URL works perfectly after lunch but not right after deployment - it’s probably DNS dragging its feet.
AWS’s virtual-hosted style is the modern, official way to go, and it comes with some nice perks. But when speed and reliability matter more than following the latest recommendation – like in CI/CD pipelines and short-lived feature branches – the old path style format is still a choice to consider.
Until AWS actually pulls the plug (and they’ve been saying “soon” for five years), there’s no harm in using the workaround where it makes your life easier. Your tests will run, your frontend won’t break, and you won’t have to schedule your deployments around DNS propagation.
