Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@uppy/aws-s3: always set S3 meta to UppyFile & include key #5602

Merged
merged 3 commits into from
Jan 15, 2025
Merged

@uppy/aws-s3: always set S3 meta to UppyFile & include key #5602

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4ef4c82
@uppy/aws-s3: always set S3 to UppyFile & include key
Murderlon Jan 14, 2025
779f9a5
Fix test
Murderlon Jan 14, 2025
1b2fc79
Merge branch 'main' into s3-meta-in-files
Murderlon Jan 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 16 additions & 9 deletions packages/@uppy/aws-s3/src/HTTPCommunicationQueue.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,15 +276,21 @@
signal,
}).abortOn(signal)) as unknown as B // todo this doesn't make sense

// location will be missing from result if CORS is not correctly set up on the bucket.
return 'location' in result ? result : (
{
// todo `url` is not really the final location URL of the resulting file, it's just the base URL of the bucket
// https://github.com/transloadit/uppy/issues/5388
location: removeMetadataFromURL(url),
...result,
}
const key = fields?.key
if (!key) {
console.error(

Check warning on line 281 in packages/@uppy/aws-s3/src/HTTPCommunicationQueue.ts

View workflow job for this annotation

GitHub Actions / Lint JavaScript/TypeScript 

Unexpected console statement
'Expected `fields.key` to be returend but the backend/Companion',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we remove this console.log? make it into a comment instead?

// Expected `fields.key` to be returned by Companion. This means the Companion version is too old

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe that makes more sense yes 👍

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does anyone mind please explaining why I'm getting this error after updating?
Everything was working before (and still does) but now this comes up as an error in my react overlay.

Using AwsS3 with getUploadParameters.

I think it has something to do with this return:

      return await res.json().then((data: DataObject) => {
        return {
          method: data?.method,
          url: data?.url,
          fields: {}, // Add an empty fields property
          expires: undefined, // Add expires property with value undefined
          headers: {
            'Content-Type': file.type,
            'Content-Disposition': `${disposition}; filename="${sanitizedFilename}"`,
          },
        };
      });

Can I just add key: '' inside empty fields object?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to worry about if everything works. The log was removed here: #5607

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Murderlon Okay thanks, this is resolved in 4.2.3
I was just more concerned that maybe I did miss passing the fields.key or something since it's asserted as required

)
}
this.#setS3MultipartState(file, { key: key! })

return {
...result,
location:
(result.location as string | undefined) ?? removeMetadataFromURL(url),
bucket: fields?.bucket,
key,
}
}

async uploadFile(
Expand Down Expand Up @@ -393,7 +399,8 @@

try {
signature = await this.#fetchSignature(this.#getFile(file), {
uploadId,
// Always defined for multipart uploads
uploadId: uploadId!,
key,
partNumber,
body: chunkData,
Expand Down
2 changes: 1 addition & 1 deletion packages/@uppy/aws-s3/src/MultipartUploader.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ interface MultipartUploaderOptions<M extends Meta, B extends Body> {
file: UppyFile<M, B>
log: Uppy<M, B>['log']

uploadId: string
uploadId?: string
key: string
}

Expand Down
7 changes: 6 additions & 1 deletion packages/@uppy/aws-s3/src/index.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,10 @@ describe('AwsS3Multipart', () => {
getUploadParameters: () => ({
method: 'POST',
url: 'https://bucket.s3.us-east-2.amazonaws.com/',
fields: {},
fields: {
key: 'file',
bucket: 'https://bucket.s3.us-east-2.amazonaws.com/',
},
}),
})
const scope = nock(
Expand Down Expand Up @@ -89,6 +92,8 @@ describe('AwsS3Multipart', () => {
ETag: 'test',
etag: 'test',
location: 'http://example.com',
key: 'file',
bucket: 'https://bucket.s3.us-east-2.amazonaws.com/',
},
status: 200,
uploadURL: 'http://example.com',
Expand Down
6 changes: 3 additions & 3 deletions packages/@uppy/aws-s3/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,7 +510,7 @@ export default class AwsS3Multipart<
return this.#client
.get<
AwsS3Part[]
>(`s3/multipart/${encodeURIComponent(uploadId)}?key=${filename}`, { signal })
>(`s3/multipart/${encodeURIComponent(uploadId!)}?key=${filename}`, { signal })
.then(assertServerError)
}

Expand All @@ -524,7 +524,7 @@ export default class AwsS3Multipart<
throwIfAborted(signal)

const filename = encodeURIComponent(key)
const uploadIdEnc = encodeURIComponent(uploadId)
const uploadIdEnc = encodeURIComponent(uploadId!)
return this.#client
.post<B>(
`s3/multipart/${uploadIdEnc}/complete?key=${filename}`,
Expand Down Expand Up @@ -633,7 +633,7 @@ export default class AwsS3Multipart<
this.#assertHost('abortMultipartUpload')

const filename = encodeURIComponent(key)
const uploadIdEnc = encodeURIComponent(uploadId)
const uploadIdEnc = encodeURIComponent(uploadId!)
return this.#client
.delete<void>(`s3/multipart/${uploadIdEnc}?key=${filename}`, undefined, {
signal,
Expand Down
4 changes: 3 additions & 1 deletion packages/@uppy/aws-s3/src/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ export function throwIfAborted(signal?: AbortSignal | null): void {
}
}

export type UploadResult = { key: string; uploadId: string }
export type UploadResult = { key: string; uploadId?: string; bucket?: string }
export type UploadResultWithSignal = UploadResult & { signal?: AbortSignal }
export type MultipartUploadResult = UploadResult & { parts: AwsS3Part[] }
export type MultipartUploadResultWithSignal = MultipartUploadResult & {
Expand All @@ -25,4 +25,6 @@ export type UploadPartBytesResult = {

export interface AwsBody extends Body {
location: string
key: string
bucket: string
}
3 changes: 3 additions & 0 deletions packages/@uppy/companion/src/server/controllers/s3.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ module.exports = function s3 (config) {
res.json({
key: data.Key,
uploadId: data.UploadId,
bucket: data.Bucket
})
}, next)
}
Expand Down Expand Up @@ -360,6 +361,8 @@ module.exports = function s3 (config) {
})).then(data => {
res.json({
location: data.Location,
key: data.Key,
bucket: data.Bucket
})
}, next)
}
Expand Down
Loading