Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resumable File Uploads Fail Beyond 6MB with TUS in Supabase Storage #563

Open
2 tasks done
yaschet opened this issue Oct 4, 2024 · 2 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@yaschet
Copy link

yaschet commented Oct 4, 2024

Bug report

  • I confirm this is a bug with Supabase, not with my own application.
  • I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

I’m having a weird issue with file uploads using FilePond and TUS with Supabase Storage. Files under 6MB upload just fine, but anything larger gets stuck at a certain percentage, usually around the 6MB chunk size.

To Reproduce

  1. Open the Supabase Local CLI Dashboard.
  2. Create a bucket in Storage.
  3. Attempt to upload a file that is under 6MB and check if it uploads successfully.
  4. Attempt to upload a file that is above 6/7 MB.
  5. Take note if the upload process stalls at a specific percentage between 0 and 99%.
  6. Error shows: Failed to upload 7MB example.jpg: tus: failed to resume upload, caused by [object ProgressEvent], originated from request (method: HEAD, url: https://127.0.0.1:8443/storage/v1/upload/resumable/RklMRV9VUExPQURfQlVDS0VULzdNQiBleGFtcGxlLmpwZy9jZTI2YzMyOC1mNjU4LTQxYWQtYjdkZS0yZTM2NTBhNzk1OTE, response code: n/a, response text: n/a, request id: n/a).

Expected behavior

I thought the larger files would upload without any problems, especially because Resumable Uploads with TUS are designed to handle large file loads. It shouldn't fail for simple files under 6MB at all. Otherwise, my app will be useless. Our users could upload files larger than hundreds of megabytes, so this is not acceptable.

Screenshots

I have a video recording showing the issue in my Next.js app, but here's a quick text log of what I see from the Supabase CLI local dashboard.

Uploading 1 file... 0s remaining – 84.79%
Do not close the browser until it's completed

And then it fails with:

Failed to upload 7MB example.jpg: tus: failed to resume upload, caused by [object ProgressEvent], originated from request (method: HEAD, url: [insert URL here]), response code: n/a, response text: n/a, request id: n/a
Supabase.Storage.Fails.to.Upload.Files.above.6MB.mp4

System information

  • OS: MacOS Ventura 13.2.1 - MacBook Pro M1 Pro.
  • Browser: Arc Browser (Version 1.63.0 (54427) - Chromium Engine Version 129.0.6668.90).
  • Version of supabase-js: "@supabase/supabase-js": "^2.45.4"
  • Version of Supabase CLI: 1.200.3
  • Version of Node.js: v22.8.0

Additional context

I disabled the built-in chunking in FilePond, thinking it might help, but the issue still persists.

Here’s a snippet of my upload code for context:

<FilePond
    files={field.value}
    oninit={() => setIsFilepondReady(true)}
    onupdatefiles={(fileItems) =>
        field.onChange(fileItems.map((fileItem) => fileItem.file))
    }
    server={{
        process: async (fieldName, file, _metadata, load, error, progress, abort) => {
            const formData = new FormData();
            formData.append(fieldName, file, file.name);

            const fileName = file.name;
            const uniqueFileName = `${new Date().getTime()}-${fileName}`;
            const generatedFileName = `${userId}/${uniqueFileName}`;

            const upload = new tus.Upload(file, {
                endpoint: `${env.NEXT_PUBLIC_SUPABASE_URL}/storage/v1/upload/resumable`,
                retryDelays: FileUploadProps.chunkRetryDelays,
                headers: {
                    authorization: `Bearer ${session?.access_token}`,
                    'x-upsert': 'true',
                },
                uploadDataDuringCreation: true,
                uploadLengthDeferred: false,
                removeFingerprintOnSuccess: true,
                metadata: {
                    bucketName: env.NEXT_PUBLIC_FILE_UPLOAD_BUCKET,
                    objectName: generatedFileName,
                    contentType: 'image/png',
                    cacheControl: '3600',
                },
                chunkSize: 6 * 1024 * 1024, // It must be set to 6MB for now
                onError(caughtError) {
                    error(caughtError.message);
                },
                onProgress(bytesUploaded, bytesTotal) {
                    progress(true, bytesUploaded, bytesTotal);
                },
                onSuccess() {
                    console.log(
                        'Download %s from %s',
                        upload?.options?.metadata?.objectName,
                        upload?.url,
                    );
                    load(upload?.url?.split('/')?.pop() ?? '');
                },
            });

            upload
                .findPreviousUploads()
                .then(function (previousUploads) {
                    if (previousUploads.length) {
                        upload.resumeFromPreviousUpload(previousUploads[0]);
                    }
                    upload.start();
                });

            return {
                abort: () => {
                    upload.abort();
                    abort();
                },
            };
        },
    }}
    {...FileUploadProps}
/>

Thanks for any help or insights you can provide!

@fenos
Copy link
Contributor

fenos commented Dec 18, 2024

I believe that this comment should fix this issue:
#572 (comment)

@yaschet
Copy link
Author

yaschet commented Dec 24, 2024

Hey @fenos,

I'm not self-hosting Supabase, but rather using the CLI for Local Development. I managed to make the resumable file uploads by turning off TLS in the config.toml file (#538 comment).

Likewise, I guess your solution will work to make Kong use TLS safely by changing the port in the env. I'll try it soon and see if it works.

All the best!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants