Separating infinity server capability from reranking capability

[noamgershi]

Feature request

Hi
I want to suggest that this repo will be separated to its reranking-feature and its fastAPI server-feature, and its reranking-feature will be in a separate pypi dependency (other then infinity_emb).

I would like to be able to get a pypi dependency which contains only the reranking/embedding/etc features, but without the fastAPI server feature.
Meaning - only the content of file engine.py, and I will create the AsyncEmbeddingEngine and I will call the rerank() method directly, without having the infinity_server.py file at all.

Motivation

Basically I don't want features like the sending StartupTelemetry to PostHog at all, and also I want to be able to have control on the server creation/parameters

Your contribution

Not currently

[michaelfeil]

@noamgershi Understandable.

This feature request is already fully implemented and documented. Also multiple docs and in the readme there is docs on this!

Basically I don't want features like the sending StartupTelemetry to PostHog at all, and also I want to be able to have control on the server creation/parameters.

• They are only send via docker container and cli. No telemetry is send elsewise. telemetry is important to deprecate old and unused features, e.g. diskcache or ct2.
• Telemetry can be disabled via https://michaelfeil.eu/infinity/0.0.73/telemetry/

I want to use infinity from python. This is also possible. The pip package is modular.
install the dependencies without docker: https://github.com/michaelfeil/infinity/blob/944643bf3c83654d831ef5223e925eda4f1f0c6d/libs/infinity_emb/pyproject.toml#L102C2-L120C2

pip install "infinity_emb[optimum,torch,einops,cache,vision]"
would do the job, and not install any pydantic packages.
Then you should be able to run the examples from here:
https://michaelfeil.eu/infinity/0.0.73/python_engine
The infinity engine is part of the public api of the python package, here is an example copied from the docs.

import asyncio
from infinity_emb import AsyncEngineArray, EngineArgs, AsyncEmbeddingEngine
query = "What is the python package infinity_emb?"
docs = ["This is a document not related to the python package infinity_emb, hence...", 
    "Paris is in France!",
    "infinity_emb is a package for sentence embeddings and rerankings using transformer models in Python!"]
array = AsyncEmbeddingEngine.from_args(
  [EngineArgs(model_name_or_path = "mixedbread-ai/mxbai-rerank-xsmall-v1", engine="torch")]
)

async def rerank(engine: AsyncEmbeddingEngine): 
    async with engine:
        ranking, usage = await engine.rerank(query=query, docs=docs)
        print(list(zip(ranking, docs)))
    # or handle the async start / stop yourself.
    await engine.astart()
    ranking, usage = await engine.rerank(query=query, docs=docs)
    await engine.astop()

asyncio.run(rerank(array[0]))

Lastly I am happy you submit this request, but not excited of the underlying message of:

I do not want to contribute x I also don't want to contribute analytics x I did not check the docs x please add and maintain another package.

I saw this is the first issue opening in Github. It would be grateful to see an adoption of a oss mindset and contributing back!

I'll move this to discussions, hopefully it hints more people to the public docs that have similar requests (which is reasonable).

[noamgershi]

Thanx for the reply
This is exactly what I was looking for. I will check the partial dependency installation.

I did reviewed the documentation, but didn't realized that there are submodules and that I can download only part of the dependency...

I will be glad to to contribute. But I'm still learning Python and am not yet at a level where I feel confident making meaningful contributions.

[michaelfeil]

Awesome! Its a good idea to just call the astart method once, and keep the model up during the duration of the server