Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MongoDB 5.0+ requires a CPU with AVX support #42

Closed
ghost opened this issue Oct 29, 2023 · 4 comments
Closed

MongoDB 5.0+ requires a CPU with AVX support #42

ghost opened this issue Oct 29, 2023 · 4 comments
Labels
wontfix This will not be worked on

Comments

@ghost
Copy link

ghost commented Oct 29, 2023

Expected Behavior

I can access the program from the web interface and it works

Actual Behavior

This has the same behavior as the one described in #32 .
Accessing the web interface gives me a fully white page.
Mongo complains about the CPU not having AVX support (see logs)

Steps to reproduce the problem

I am using the following compose.yml

version: '3.9'
services:
    dcef:
        image: 'slada/dcef:main'
        ports:
            - '21011:21011'
        container_name: dcef
        volumes:
            - 'dcef_cache:/dcef/cache'
            - './exports:/dcef/exports'
        restart: unless-stopped

volumes:
    dcef_cache:
  1. Pull the docker image with docker pull slada/dcef:main
  2. Create and start the container with sudo docker compose up -d --force-recreate
  3. Try to access the web interface

Specifications

Environment

  • Operating system: Manjaro 23.0.2 Uranos with x86_64 Linux 5.15.131-1-MANJARO Kernel
  • CPU Architecture: x86_64 (PC)

DiscordChatExporter-frontend

DiscordChatExporter

  • Version (CLI --version): v2.42.0

How did you export the data?

  • Export format (CLI --format): Json
  • Downloaded assets (CLI --media): Yes
  • Markdown processing (CLI --markdown): No
  • Other CLI options (optional): --dateformat dd/MM/yyyy HH:mm:ss, --reuse-media

Logs

Is there any relevant information in logs? If so, please provide them:

WARNING: MongoDB 5.0+ requires a CPU with AVX support, and your current system does not appear to have that!
  see https://jira.mongodb.org/browse/SERVER-54407
  see also https://www.mongodb.com/community/forums/t/mongodb-5-0-cpu-intel-g4650-compatibility/116610/2
  see also https://github.com/docker-library/mongo/issues/485#issuecomment-891991814

main_mongo loaded
Connecting to database...
[Preprocess] took 30.1 seconds
Traceback (most recent call last):
  File "/dcef/backend/preprocess/main_mongo.py", line 145, in <module>
    main(input_dir, output_dir)
  File "/dcef/backend/preprocess/main_mongo.py", line 100, in main
    wipe_database(database)
  File "/dcef/backend/preprocess/main_mongo.py", line 32, in wipe_database
    whitelisted_guild_ids = config.find_one({"key": "whitelisted_guild_ids"})
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/collection.py", line 1459, in find_one
    for result in cursor.limit(-1):
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1248, in next
    if len(self.__data) or self._refresh():
                           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1139, in _refresh
    self.__session = self.__collection.database.client._ensure_session()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
    return self.__start_session(True, causal_consistency=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1685, in __start_session
    self._topology._check_implicit_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
    self._check_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 554, in _check_session_support
    self._select_servers_loop(
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 238, in _select_servers_loop
    raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 653e9b253a02030314937ff4, topology_type: Unknown, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [Errno 111] Connection refused')>]>
/dcef/run_container.sh: line 6:    13 Illegal instruction     (core dumped) mongod --dbpath "/dcef/cache/db/" --wiredTigerCacheSizeGB 1.5  (wd: /dcef)
############################################################
# Open http://127.0.0.1:21011/ in your browser to view GUI #
############################################################
2023/10/29 17:49:56 [notice] 18#18: using the "epoll" event method
2023/10/29 17:49:56 [notice] 18#18: nginx/1.18.0 (Ubuntu)
2023/10/29 17:49:56 [notice] 18#18: OS: Linux 5.15.131-1-MANJARO
2023/10/29 17:49:56 [notice] 18#18: getrlimit(RLIMIT_NOFILE): 1073741816:1073741816
2023/10/29 17:49:56 [notice] 18#18: start worker processes
2023/10/29 17:49:56 [notice] 18#18: start worker process 19
INFO:     Started server process [17]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:58000 (Press CTRL+C to quit)
INFO:     [CENSORED]:0 - "GET /guilds HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.11/dist-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.11/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 237, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dcef/backend/fastapi/app.py", line 66, in get_guilds
    whitelisted_guild_ids = get_whitelisted_guild_ids()
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dcef/backend/fastapi/helpers.py", line 18, in get_whitelisted_guild_ids
    whitelisted_guild_ids = collection_config.find_one({"key": "whitelisted_guild_ids"})["value"]
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/collection.py", line 1459, in find_one
    for result in cursor.limit(-1):
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1248, in next
    if len(self.__data) or self._refresh():
                           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/cursor.py", line 1139, in _refresh
    self.__session = self.__collection.database.client._ensure_session()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1740, in _ensure_session
    return self.__start_session(True, causal_consistency=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pymongo/mongo_client.py", line 1685, in __start_session
    self._topology._check_implicit_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 538, in _check_implicit_session_support
    self._check_session_support()
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 554, in _check_session_support
    self._select_servers_loop(
  File "/usr/local/lib/python3.11/dist-packages/pymongo/topology.py", line 238, in _select_servers_loop
    raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 127.0.0.1:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 653e9b4525d89af64a3da447, topology_type: Unknown, servers: [<ServerDescription ('127.0.0.1', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('127.0.0.1:27017: [Errno 111] Connection refused')>]>
2023/10/29 17:53:32 [error] 19#19: *3 open() "/dcef/cache/preprocess/fonts/ggsans-normal-400.woff2" failed (2: No such file or directory), client: [CENSORED], server: localhost, request: "GET /data/fonts/ggsans-normal-400.woff2 HTTP/1.1", host: "[CENSORED]:21011", referrer: "http://[CENSORED]:21011/_app/immutable/assets/_layout-92ac35d3.css"
2023/10/29 17:53:32 [error] 19#19: *4 open() "/dcef/cache/preprocess/fonts/ggsans-normal-700.woff2" failed (2: No such file or directory), client: [CENSORED], server: localhost, request: "GET /data/fonts/ggsans-normal-700.woff2 HTTP/1.1", host: "[CENSORED]:21011", referrer: "http://[CENSORED]:21011/_app/immutable/assets/_layout-92ac35d3.css"
@ghost
Copy link
Author

ghost commented Oct 29, 2023

Also, note that the output of grep flags -m1 /proc/cpuinfo | grep avx is empty, as i indeed don't have avx support.

@slatinsky
Copy link
Owner

Hello, thanks for detailed report.

It looks like only Mongo 4.x supports your CPU. I have considered using older mongodb (latest version is 4.1.4 before they changed the license), but the latest version was considerably faster and it offered official precompiled binaries for windows.

The latest mongodb version without AVX is 4.4.6. There are unofficial workarounds to build mongodb without avx, but nothing official.

If you are running the system under a hypervisor, check if really your processor doesn't support this extended instruction set. Because you would need to have CPU older than 8 years to not have AVX, which is unlikely.

@ghost
Copy link
Author

ghost commented Oct 31, 2023

If you are running the system under a hypervisor, check if really your processor doesn't support this extended instruction set. Because you would need to have CPU older than 8 years to not have AVX, which is unlikely.

A lot of server CPUs for low-consumptions servers also don't have AVX, which is my case.

Will the software work with an older Mongo version?

@slatinsky
Copy link
Owner

You can try to change base docker image in Dockerfile, but I won't support this old mongodb version, sorry.

Often you can ask your hosting provider to move your VPS to another server, where AVX is supported.

Closing as wont-fix.

@slatinsky slatinsky added the wontfix This will not be worked on label Nov 5, 2023
@slatinsky slatinsky closed this as not planned Won't fix, can't repro, duplicate, stale Nov 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@slatinsky