Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable experimental async use of driver #901

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

feat: enable experimental async use of driver #901

wants to merge 6 commits into from

Conversation

Mause
Copy link
Owner

@Mause Mause commented Feb 14, 2024

No description provided.

@sinaiy
Copy link

sinaiy commented Mar 24, 2024

FYI:
I am using the duckdb_engine as Async engine

I had to patch
duckdb_engine.duckdb_engine.Dialect.get_pool_class
in order to be able to set an Async Connection pool

@Mause
Copy link
Owner Author

Mause commented Apr 7, 2024

FYI: I am using the duckdb_engine as Async engine

I had to patch duckdb_engine.duckdb_engine.Dialect.get_pool_class in order to be able to set an Async Connection pool

That shouldn't be required, you should just be able to pass poolclass=... to the create_engine(...) call

@sinaiy
Copy link

sinaiy commented Apr 8, 2024

latest version of SQLAlchemy is more explicit
it adds a verification that the async status of the Dialect matches the async status of the connection pool.

I followed the psycopg3 engine example, and created AsyncDuckDbDialect as a subclass of the original dialect

import duckdb_engine

class AsyncDialect(duckdb_engine.Dialect):
    is_async = True

    @classmethod
    def get_pool_class(cls, url):
        async_fallback = url.query.get("async_fallback", False)

        if util.asbool(async_fallback):
            return pool.FallbackAsyncAdaptedQueuePool
        else:
            return pool.AsyncAdaptedQueuePool


duckdb_engine.Dialect.get_async_dialect_cls = lambda cls: AsyncDialect

@Mause Mause changed the title chore: async test feat: enable experimental async use of driver Apr 16, 2024
@Mause Mause marked this pull request as ready for review April 16, 2024 02:18
@kodiakhq kodiakhq bot removed the automerge label Apr 18, 2024
Copy link
Contributor

kodiakhq bot commented Apr 18, 2024

This PR currently has a merge conflict. Please resolve this and then re-add the automerge label.

@ShravanSunder
Copy link

ShravanSunder commented Jul 9, 2024

the method @sinaiy mentioned works for me in the mean time. here it is with type hints

import logging
import typing as t

import duckdb_engine
from pydantic_core import from_json, to_json
from sqlalchemy import URL, pool, util
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine

# Assuming the logger is already configured elsewhere
logger = logging.getLogger(__name__)


class AsyncDialect(duckdb_engine.Dialect):
  is_async = True

  @classmethod
  def get_pool_class(cls, url: URL):
    async_fallback = url.query.get("async_fallback", False)

    if util.asbool(async_fallback):
      return pool.FallbackAsyncAdaptedQueuePool
    else:
      return pool.AsyncAdaptedQueuePool


duckdb_engine.Dialect.get_async_dialect_cls = lambda cls: AsyncDialect  # type: ignore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants