[Issue] Problems uploading non-English filenames

[tsyoo-31564815]

hello. I'm having a problem with uploading files.

English filenames are uploaded, but other language filenames such as Japanese, Korean, Chinese, etc. are not uploaded. Below is the error message. If anyone knows how to upload a file name in a language other than English, please let me know.

Thank you for reading my problem.

========== Error log ==========
backend-core | 2023-09-20 05:19:50,062 [INFO] repository.chat.create_chat: New chat entry in chats table for user d186b5c5-d331-4044-9ba5-11e9903d4a18
backend-core | 2023-09-20 05:19:50,126 [INFO] repository.chat.create_chat: Insert response [{'chat_id': 'f353246c-f519-4483-b076-cd22d5460d6b', 'user_id': 'd186b5c5-d331-4044-9ba5-11e9903d4a18', 'creation_time': '2023-09-20T05:19:50.246073', 'history': None, 'chat_name': 'New Chat'}]
backend-core | INFO: 172.25.0.1:60592 - "POST /chat HTTP/1.1" 200 OK
backend-core | INFO: 172.25.0.1:60592 - "OPTIONS /upload?brain_id=586fc51f-e7a2-43fd-8726-324186e8b42a&chat_id=f353246c-f519-4483-b076-cd22d5460d6b HTTP/1.1" 200 OK
backend-core | {'statusCode': 400, 'error': 'Invalid Input', 'message': 'The object name contains invalid characters'}
backend-core | {'statusCode': 400, 'error': 'Invalid Input', 'message': 'The object name contains invalid characters'}
backend-core | INFO: 172.25.0.1:60592 - "POST /upload?brain_id=586fc51f-e7a2-43fd-8726-324186e8b42a&chat_id=f353246c-f519-4483-b076-cd22d5460d6b HTTP/1.1" 200 OK
worker | [2023-09-20 05:19:50,921: INFO/MainProcess] Task process_file_and_notify[2df5b4cb-5ed5-4328-8898-785d5719a39e] received
backend-core | INFO: 172.25.0.1:60600 - "GET /user/identity HTTP/1.1" 200 OK
worker | [2023-09-20 05:19:52,631: WARNING/ForkPoolWorker-8] --- Logging error ---
worker | [2023-09-20 05:19:52,634: WARNING/ForkPoolWorker-8] Traceback (most recent call last):
worker | [2023-09-20 05:19:52,634: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/storage3/_sync/file_api.py", line 49, in _request
worker | response.raise_for_status()
worker | [2023-09-20 05:19:52,634: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/httpx/models.py", line 749, in raise_for_status
worker | raise HTTPStatusError(message, request=request, response=self)
worker | [2023-09-20 05:19:52,634: WARNING/ForkPoolWorker-8] httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://wvjonsttyvctbnnvjsgk.supabase.co/storage/v1/object/quivr/586fc51f-e7a2-43fd-8726-324186e8b42a/who%20wants%20to%20go%20home.txt'
worker | For more information check: https://httpstatuses.com/400
worker | [2023-09-20 05:19:52,634: WARNING/ForkPoolWorker-8]
worker | During handling of the above exception, another exception occurred:
worker | [2023-09-20 05:19:52,634: WARNING/ForkPoolWorker-8] Traceback (most recent call last):
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 451, in trace_task
worker | R = retval = fun(*args, **kwargs)
worker | ^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 734, in protected_call
worker | return self.run(*args, **kwargs)
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] File "/code/celery_worker.py", line 83, in process_file_and_notify
worker | res = supabase_client.storage.from("quivr").download(file_name)
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/storage3/_sync/file_api.py", line 323, in download
worker | response = self._request(
worker | ^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/storage3/_sync/file_api.py", line 51, in _request
worker | raise StorageException(
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] storage3.utils.StorageException: {'statusCode': 400, 'error': 'not_found', 'message': 'Object not found'}
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8]
worker | During handling of the above exception, another exception occurred:
worker | [2023-09-20 05:19:52,635: WARNING/ForkPoolWorker-8] Traceback (most recent call last):
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/logging/init.py", line 1110, in emit
worker | msg = self.format(record)
worker | ^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/logging/init.py", line 953, in format
worker | return fmt.format(record)
worker | ^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/utils/log.py", line 146, in format
worker | msg = super().format(record)
worker | ^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/logging/init.py", line 695, in format
worker | record.exc_text = self.formatException(record.exc_info)
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/utils/log.py", line 142, in formatException
worker | r = super().formatException(ei)
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/logging/init.py", line 645, in formatException
worker | traceback.print_exception(ei[0], ei[1], tb, None, sio)
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/traceback.py", line 124, in print_exception
worker | te = TracebackException(type(value), value, tb, limit=limit, compact=True)
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/traceback.py", line 692, in init
worker | self.stack = StackSummary._extract_from_extended_frame_gen(
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/traceback.py", line 416, in _extract_from_extended_frame_gen
worker | for f, (lineno, end_lineno, colno, end_colno) in frame_gen:
worker | [2023-09-20 05:19:52,636: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/traceback.py", line 353, in _walk_tb_with_full_positions
worker | positions = _get_code_position(tb.tb_frame.f_code, tb.tb_lasti)
worker | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,637: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/traceback.py", line 366, in _get_code_position
worker | positions_gen = code.co_positions()
worker | ^^^^^^^^^^^^^^^^^
worker | [2023-09-20 05:19:52,637: WARNING/ForkPoolWorker-8] AttributeError: '_Code' object has no attribute 'co_positions'
worker | [2023-09-20 05:19:52,637: WARNING/ForkPoolWorker-8] Call stack:
worker | [2023-09-20 05:19:52,640: WARNING/ForkPoolWorker-8] File "/usr/local/bin/celery", line 8, in
worker | sys.exit(main())
worker | [2023-09-20 05:19:52,640: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/main.py", line 15, in main
worker | sys.exit(_main())
worker | [2023-09-20 05:19:52,640: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/bin/celery.py", line 217, in main
worker | return celery(auto_envvar_prefix="CELERY")
worker | [2023-09-20 05:19:52,640: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in call
worker | return self.main(*args, **kwargs)
worker | [2023-09-20 05:19:52,640: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
worker | rv = self.invoke(ctx)
worker | [2023-09-20 05:19:52,640: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
worker | return _process_result(sub_ctx.command.invoke(sub_ctx))
worker | [2023-09-20 05:19:52,641: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
worker | return ctx.invoke(self.callback, **ctx.params)
worker | [2023-09-20 05:19:52,641: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
worker | return __callback(*args, **kwargs)
worker | [2023-09-20 05:19:52,641: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
worker | return f(get_current_context(), *args, **kwargs)
worker | [2023-09-20 05:19:52,641: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/bin/base.py", line 134, in caller
worker | return f(ctx, *args, **kwargs)
worker | [2023-09-20 05:19:52,642: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/bin/worker.py", line 351, in worker
worker | worker.start()
worker | [2023-09-20 05:19:52,642: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/worker/worker.py", line 203, in start
worker | self.blueprint.start(self)
worker | [2023-09-20 05:19:52,642: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/bootsteps.py", line 116, in start
worker | step.start(parent)
worker | [2023-09-20 05:19:52,643: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/bootsteps.py", line 365, in start
worker | return self.obj.start()
worker | [2023-09-20 05:19:52,643: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/concurrency/base.py", line 129, in start
worker | self.on_start()
worker | [2023-09-20 05:19:52,643: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/concurrency/prefork.py", line 109, in on_start
worker | P = self._pool = Pool(processes=self.limit,
worker | [2023-09-20 05:19:52,643: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/concurrency/asynpool.py", line 463, in init
worker | super().init(processes, *args, **kwargs)
worker | [2023-09-20 05:19:52,643: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/pool.py", line 1046, in init
worker | self._create_worker_process(i)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/concurrency/asynpool.py", line 480, in _create_worker_process
worker | return super()._create_worker_process(i)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/pool.py", line 1158, in _create_worker_process
worker | w.start()
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/process.py", line 124, in start
worker | self._popen = self._Popen(self)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/context.py", line 333, in _Popen
worker | return Popen(process_obj)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/popen_fork.py", line 24, in init
worker | self._launch(process_obj)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/popen_fork.py", line 79, in _launch
worker | code = process_obj._bootstrap()
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/process.py", line 327, in _bootstrap
worker | self.run()
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/process.py", line 114, in run
worker | self._target(*self._args, **self._kwargs)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/pool.py", line 292, in call
worker | sys.exit(self.workloop(pid=pid))
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/billiard/pool.py", line 362, in workloop
worker | result = (True, prepare_result(fun(*args, **kwargs)))
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 649, in fast_trace_task
worker | R, I, T, Rstr = tasks[task].trace(
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 468, in trace_task
worker | I, R, state, retval = on_error(task_request, exc, uuid)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 379, in on_error
worker | R = I.handle_error_state(
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 178, in handle_error_state
worker | return {
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 237, in handle_failure
worker | self._log_error(task, req, einfo)
worker | [2023-09-20 05:19:52,644: WARNING/ForkPoolWorker-8] File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 265, in _log_error
worker | logger.log(policy.severity, policy.format.strip(), context,
worker | [2023-09-20 05:19:52,645: WARNING/ForkPoolWorker-8] Message: 'Task %(name)s[%(id)s] %(description)s: %(exc)s'
worker | Arguments: {'hostname': 'celery@e74f476742de', 'id': '2df5b4cb-5ed5-4328-8898-785d5719a39e', 'name': 'process_file_and_notify', 'exc': "StorageException({'statusCode': 400, 'error': 'not_found', 'message': 'Object not found'})", 'traceback': 'Traceback (most recent call last):\n File "/usr/local/lib/python3.11/site-packages/storage3/_sync/file_api.py", line 49, in _request\n response.raise_for_status()\n File "/usr/local/lib/python3.11/site-packages/httpx/models.py", line 749, in raise_for_status\n raise HTTPStatusError(message, request=request, response=self)\nhttpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://wvjonsttyvctbnnvjsgk.supabase.co/storage/v1/object/quivr/586fc51f-e7a2-43fd-8726-324186e8b42a/who%20wants%20to%20go%20home.txt\'\nFor more information check: https://httpstatuses.com/400\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 451, in trace_task\n R = retval = fun(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 734, in protected_call\n return self.run(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/code/celery_worker.py", line 83, in process_file_and_notify\n res = supabase_client.storage.from("quivr").download(file_name)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/site-packages/storage3/_sync/file_api.py", line 323, in download\n response = self._request(\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/site-packages/storage3/_sync/file_api.py", line 51, in _request\n raise StorageException(\nstorage3.utils.StorageException: {'statusCode': 400, 'error': 'not_found', 'message': 'Object not found'}\n', 'args': '[]', 'kwargs': "{'file_name': '586fc51f-e7a2-43fd-8726-324186e8b42a/집에 가고 싶은 사람.txt', 'file_original_name': '집에 가고 싶은 사람.txt', 'enable_summarization': False, 'brain_id': UUID('586fc51f-e7a2-43fd-8726-324186e8b42a'), 'openai_api_key': '[blank]', 'notification_id': UUID('04af3496-e72a-495a-ad9c-ce6311240f99')}", 'description': 'raised unexpected', 'internal': False}

[xiaoxuan220]

Same error

[scaniusin]

The same problem with English files is also on the website. https://www.quivr.app/chat/

[azulika]

Same. It's happening with Japanese characters too. Apparently it's Supabase's fault supabase/storage#273

[github-actions]

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.