Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SignatureDoesNotMatch when default port explicitly defined on S3 requests #1244

Open
1 task
thill opened this issue Jan 17, 2025 · 0 comments
Open
1 task

SignatureDoesNotMatch when default port explicitly defined on S3 requests #1244

thill opened this issue Jan 17, 2025 · 0 comments
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@thill
Copy link

thill commented Jan 17, 2025

Describe the bug

It appears that when I explicitly set the default port in an s3 endpoint, the payload signature is calculated using hostname:port, but only hostname is set in the header, leading to a signature validation failure server-side. I encountered this issue when using rook ceph, but I don't know to what extent it impacts other use-cases.

When I run an echo server on localhost:80 and run the example code below, here is the signature trace log:

2025-01-17T04:12:16.283375Z TRACE invoke{service=s3 operation=ListObjectsV2 sdk_invocation_id=2708980}:try_op:try_attempt: aws_sigv4::http_request::sign: canonical_request=GET
/test/
list-type=2
host:localhost:80
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20250117T041216Z
x-amz-user-agent:aws-sdk-rust/1.3.4 ua/2.1 api/s3/1.69.0 os/macos lang/rust/1.81.0 m/E md/http#hyper-0.x

host;x-amz-content-sha256;x-amz-date;x-amz-user-agent
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

And here is the request as was received by the server:

HttpRequest HTTP/1.1 GET:/test/

query: ?"list-type=2"

headers:
"x-amz-user-agent": "aws-sdk-rust/1.3.4 ua/2.1 api/s3/1.69.0 os/macos lang/rust/1.81.0 m/E md/http#hyper-0.x"
"amz-sdk-invocation-id": "65542b93-96b0-4351-81cd-40515a90e6ca"
"authorization": "*redacted*"
"amz-sdk-request": "attempt=1; max=1"
"x-amz-date": "20250117T041216Z"
"x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"user-agent": "aws-sdk-rust/1.3.4 os/macos lang/rust/1.81.0"
"host": "localhost"

Notice that host:localhost:80 is used in the signing process, but the host header received was simply localhost. This leads a server to attempt to use a different value to validate the signature than was used by this library.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

Setting http://hostname:80 should calculate the signature that ends up being specified in the Host header.

Current Behavior

It looks like hostname:80 is used to calculate the signature, and hostname is set as the host header. I get a SignatureDoesNotMatch from a ceph server when :80 is included, but not when :80 is removed.

Reproduction Steps

Code:

    let endpoint = "[http://localhost:80](http://localhost/)";
    let bucket = "test";

    let s3_config = aws_sdk_s3::config::Builder::new()
        .behavior_version(BehaviorVersion::latest())
        .credentials_provider(EnvironmentVariableCredentialsProvider::new())
        .region(Region::new("us-east-1"))
        .force_path_style(true)
        .endpoint_url(endpoint);
    let s3_config = s3_config.build();
    let client = aws_sdk_s3::Client::from_conf(s3_config);

    let request = client.list_objects_v2().bucket(bucket);
    let response = request.send().await.unwrap();

    println!("{:?}", response);

Possible Solution

Either:

  1. strip the default port away from the host and use it to calculate the signature
  2. or ensure the host header set on the hyper request matches the header used to calculate the signature

Additional Information/Context

No response

Version

cargo tree | grep aws-
│ ├── aws-config v1.5.14
│ │ ├── aws-credential-types v1.2.1
│ │ │ ├── aws-smithy-async v1.2.4
│ │ │ ├── aws-smithy-runtime-api v1.7.3
│ │ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ │ ├── aws-smithy-types v1.2.12
│ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ ├── aws-runtime v1.5.4
│ │ │ ├── aws-credential-types v1.2.1 (*)
│ │ │ ├── aws-sigv4 v1.2.7
│ │ │ │ ├── aws-credential-types v1.2.1 (*)
│ │ │ │ ├── aws-smithy-eventstream v0.60.6
│ │ │ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ │ ├── aws-smithy-http v0.60.12
│ │ │ │ │ ├── aws-smithy-eventstream v0.60.6 (*)
│ │ │ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ ├── aws-smithy-eventstream v0.60.6 (*)
│ │ │ ├── aws-smithy-http v0.60.12 (*)
│ │ │ ├── aws-smithy-runtime v1.7.7
│ │ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ │ ├── aws-smithy-http v0.60.12 (*)
│ │ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-types v1.3.4
│ │ │ │ ├── aws-credential-types v1.2.1 (*)
│ │ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ ├── aws-sdk-sso v1.54.0
│ │ │ ├── aws-credential-types v1.2.1 (*)
│ │ │ ├── aws-runtime v1.5.4 (*)
│ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ ├── aws-smithy-http v0.60.12 (*)
│ │ │ ├── aws-smithy-json v0.61.2
│ │ │ │ └── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-smithy-runtime v1.7.7 (*)
│ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-types v1.3.4 (*)
│ │ ├── aws-sdk-ssooidc v1.55.0
│ │ │ ├── aws-credential-types v1.2.1 (*)
│ │ │ ├── aws-runtime v1.5.4 (*)
│ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ ├── aws-smithy-http v0.60.12 (*)
│ │ │ ├── aws-smithy-json v0.61.2 (*)
│ │ │ ├── aws-smithy-runtime v1.7.7 (*)
│ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-types v1.3.4 (*)
│ │ ├── aws-sdk-sts v1.55.0
│ │ │ ├── aws-credential-types v1.2.1 (*)
│ │ │ ├── aws-runtime v1.5.4 (*)
│ │ │ ├── aws-smithy-async v1.2.4 (*)
│ │ │ ├── aws-smithy-http v0.60.12 (*)
│ │ │ ├── aws-smithy-json v0.61.2 (*)
│ │ │ ├── aws-smithy-query v0.60.7
│ │ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-smithy-runtime v1.7.7 (*)
│ │ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ │ ├── aws-smithy-xml v0.60.9
│ │ │ ├── aws-types v1.3.4 (*)
│ │ ├── aws-smithy-async v1.2.4 (*)
│ │ ├── aws-smithy-http v0.60.12 (*)
│ │ ├── aws-smithy-json v0.61.2 (*)
│ │ ├── aws-smithy-runtime v1.7.7 (*)
│ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ ├── aws-smithy-types v1.2.12 (*)
│ │ ├── aws-types v1.3.4 (*)
│ ├── aws-credential-types v1.2.1 (*)
│ ├── aws-sdk-s3 v1.69.0
│ │ ├── aws-credential-types v1.2.1 (*)
│ │ ├── aws-runtime v1.5.4 (*)
│ │ ├── aws-sigv4 v1.2.7 (*)
│ │ ├── aws-smithy-async v1.2.4 (*)
│ │ ├── aws-smithy-checksums v0.62.0
│ │ │ ├── aws-smithy-http v0.60.12 (*)
│ │ │ ├── aws-smithy-types v1.2.12 (*)
│ │ ├── aws-smithy-eventstream v0.60.6 (*)
│ │ ├── aws-smithy-http v0.60.12 (*)
│ │ ├── aws-smithy-json v0.61.2 (*)
│ │ ├── aws-smithy-runtime v1.7.7 (*)
│ │ ├── aws-smithy-runtime-api v1.7.3 (*)
│ │ ├── aws-smithy-types v1.2.12 (*)
│ │ ├── aws-smithy-xml v0.60.9 (*)
│ │ ├── aws-types v1.3.4 (*)
├── aws-config v1.5.14 (*)
├── aws-credential-types v1.2.1 (*)
├── aws-sdk-s3 v1.69.0 (*)

Environment details (OS name and version, etc.)

Linux x86 and macOS M3

Logs

No response
@thill thill added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@thill