Brand new AI provider Feedback | 全新 AI Provider 管理模块反馈

[arvinxx]

We released a brand new AI Provider management module in version v1.44.0, bringing a completely new experience for managing AI service providers. Based on extensive user feedback, we introduced the following in this new Provider version:

• Create custom service providers (supporting different SDK formats, such as multiple OpenAI format providers), create custom models;
• Capabilities and parameter customization for all models (currently only FC and Vision are available, more will be opened in the future based on community demand);
• Custom sorting by service provider, enable custom sorting of models;
• Supported full-type models at the framework level (language models, embedding models, TTS/STT models, text-to-image models). This framework enables us to support richer modal AI models in the future, such as text-to-video and music models.

As the new feature for the beginning of 2025, we expect this new module to encompass all possibilities for the coming year, making LobeChat the most versatile client in the AI Chat field~

Important

Note: This AI Provider module upgrade is only available under the server DB. Future iterations will support pglite mode. The dexie db will maintain its original implementation and will be completely removed in v2.0.

At the same time, the AI Provider page has undergone a complete rewrite from data storage to front-end implementation in order to achieve better performance and interaction experience. Therefore, at this stage, the logic related to the original environment variables may not be implemented or may have flaws. If you encounter any scenarios that do not meet your expectations, please provide feedback below, and I will strive to complete the fix within T+1 days.

Therefore, when this post is closed, if there are still no feedback on the environment variable issue, it will be removed as a deprecated feature in V2.

---

我们在 v1.44.0 版本发布了全新的 AI Provider 管理模块，带来了全新的 AI 服务商管理体验。结合诸多用户反馈，我们在这一全新的 Provider 版本中，带来了：

• 创建自定义服务商（支持不同 SDK 格式，例如多个 OpenAI 格式的服务商）、创建自定义模型；
• 所有模型的能力、参数自定义（目前仅开放 FC 和 Vision，未来将会根据社区诉求开放更多参数，例如模型是否支持流式等）；
• 服务商自定义排序、启用模型的自定义排序；
• 在框架层支持了全量类型的模型（语言模型、Embedding模型、TTS/STT 模型、文生图模型。这一框架使得我们未来也将能支持上文生视频、音乐模型等更丰富模态的 AI 模型。

Important

注意事项：本次 AI Provider 模块升级，只在服务端 DB 下可用，后续迭代会支持 pglite 模式。而 dexie db 仍保持原有实现，后续将在 v2.0 中完全移除。

由于AI Provider 这个页面从数据存储到前端实现是一次完全的重写，以期获得更好的操作性能和交互体验。因此数据上是完全独立的两套，我们不做任何迁移，升级后你需要自行重新配置对应的 Provider。同时现阶段的实现中，和原有 环境变量 相关的部分的逻辑可能没有实现或者存在瑕疵。如果遇到不符合你的预期场景，请在下方反馈，我会争取在当天评估是否对齐原有逻辑，如需对齐， T+1 天内完成对齐。

待本帖关闭时，如果仍然没有反馈的环境变量问题，也将作为废弃功能在 V2 中移除。

作为 2025 年的开年新特性，我们期望这个新的模块能承载未来一年的全部可能性，让 LobeChat 变成 AI Chat 领域最全面的 AI 客户端 🚀

[YANANEER]

• OpenAI的模型列表里总有几个模型关不掉，关掉后点击获取模型列表又会自动启用。
• 模型自定义排序好像只在AI服务商的列表里有效。

我只在环境变量里配置了OPENAI_API_KEY和OPENAI_PROXY_URL

---

1.44.1更新后模型自定义排序有似乎效了，但那些多出来的模型排序优先级比这个自定义排序的优先级高

---

1.45.3，上述问题已经完全解决

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• There are always several models in the OpenAI model list that cannot be turned off. After turning them off, click to get the model list and they will be automatically enabled again.
• Custom sorting of models seems to be completely useless.

[SpeedupMaster]

• 环境变量中 *_MODEL_LIST -all 功能失效，无法仅启用需要的模型
• 是否可以在环境变量里模型排序，之前是可以通过*_MODEL_LIST里添加模型的先后顺序来排
• 是否能在环境变量里添加自定义服务商？

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• The *_MODEL_LIST -all function in the environment variable is invalid and cannot enable only the required models.
• Is it possible to sort models in environment variables? Previously, it was possible to sort the models by adding them in *_MODEL_LIST.
• Is it possible to add a custom service provider in environment variables?

[sqkkyzx]

1. AI服务商列表中启用的模型与对话页面的可选模型不匹配：

对话页面的模型选项会显示AI服务商中未启用的模型，甚至将模型全部禁用也会显示模型。

备注1：测试过将服务商禁用，对话页面对应服务商会消失，再启用服务商（未启用任何模型），对话页面对应服务商会显示，并且依然存在三个未启用的模型。
备注2：其他服务商也有类似问题。

2. OpenAI（使用AIHUBMIX）点击获取模型列表按钮时，会显示并自动启用其他一些模型，比如 Claude 3.5

备注：猜测是否与 AIHUBMIX 的模型范围限制有关，我从模型范围限制中删除Claude模型的访问权限，然后，在AI服务商列表中删除Claude模型，等待大约10分钟后，再次点击获取模型列表按钮时，不会再出现 Claude 的模型了。

3. 希望保留一些提供商的环境变量功能

像OLLAMA这类，希望禁用下载模型的功能，否则不小心运行太大的模型会对服务器造成影响。或者有一个对所有用户生效的配置全局AI提供商。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

1. The model enabled in the AI service provider list does not match the optional model on the Conversation page:

The model option on the Dialogue page will display models that are not enabled in the AI Service Provider, and even disabling all models will display the models.

Note 1: After testing, if the service provider is disabled, the corresponding service provider on the dialogue page will disappear. If the service provider is enabled again (no model is enabled), the corresponding service provider on the dialogue page will be displayed, and there are still three unenabled models.
Note 2: Other service providers have similar problems.

2. When OpenAI (using AIHUBMIX) clicks the Get model list button, some other models will be displayed and automatically enabled, such as Claude 3.5

Note: Guessing whether it is related to the Model Range Limitation of AIHUBMIX, I removed the access rights of the Claude model from the Model Range Limitation, and then deleted the Claude model from the AI Service Provider list. After waiting for about 10 minutes, when you click the Get Model List button again, Claude's model will no longer appear.

3. Hope to retain some of the provider’s environment variable functionality

Like OLLAMA, I hope to disable the function of downloading models, otherwise accidentally running a model that is too large will affect the server. Or have a configured global AI provider that takes effect for all users.

[SAnBlog]

• 环境变量中 *_MODEL_LIST -all 功能失效，无法仅启用需要的模型
• 是否可以在环境变量里模型排序，之前是可以通过*_MODEL_LIST里添加模型的先后顺序来排
• 是否能在环境变量里添加自定义服务商？

对于环境变量的需求还是很重要的,对于用户来说并不懂配置,希望能兼容原先的环境变量
1.如*_MODEL_LIST
2.在Openai服务商中自定义其他模型等

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• The *_MODEL_LIST -all function in the environment variable is invalid and cannot only enable the required models.
• Is it possible to sort models in environment variables? Previously, it was possible to sort by the order of adding models in *_MODEL_LIST
• Is it possible to add a custom service provider in environment variables?

The need for environment variables is still very important. For users who do not know how to configure, they hope to be compatible with the original environment variables.

1. Such as *_MODEL_LIST
2. Customize other models in the Openai service provider, etc.

[SAnBlog]

• 环境变量中 *_MODEL_LIST -all 功能失效，无法仅启用需要的模型
• 是否可以在环境变量里模型排序，之前是可以通过*_MODEL_LIST里添加模型的先后顺序来排
• 是否能在环境变量里添加自定义服务商？

对于环境变量的需求还是很重要的,对于用户来说并不懂配置,希望能兼容原先的环境变量 1.如*_MODEL_LIST 2.在Openai服务商中自定义其他模型等

2 是上文的<是否能在环境变量里添加自定义服务商>,设置里提供了自定义服务商,这就不用在原先的Openai服务商中配置,希望能够在变量里能添加

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• The *_MODEL_LIST -all function in the environment variable is invalid and cannot enable only the required models.
• Is it possible to sort models in environment variables? Previously, it was possible to sort by the order of adding models in *_MODEL_LIST
• Is it possible to add a custom service provider in environment variables?

The need for environment variables is still very important. For users who do not know how to configure, they hope to be compatible with the original environment variables 1. Such as *_MODEL_LIST 2. Customize other models in the Openai service provider, etc.

2 is the above ? Custom service providers are provided in the settings, so there is no need to configure them in the original Openai service provider. I hope that I can add them to the variables.

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

The problem of sorting and refreshing not taking effect has been fixed in #5330

[arvinxx]

环境变量中 _MODEL_LIST -all 功能失效，无法仅启用需要的模型
是否可以在环境变量里模型排序，之前是可以通过_MODEL_LIST里添加模型的先后顺序来排

@SpeedupMaster MODEL_LIST 环境变量会保持原样，今天会兼容上，之前什么样，现在也应该是什么样；

是否能在环境变量里添加自定义服务商？

不考虑在环境变量中添加自定义服务商，能配置的东西太少了，未来可能会考虑加一个全局范围的服务商设定，面向管理员来控制。

还有就是tts和embedding这种非聊天模型也会展示在聊天框的下拉模型列表里

@coulsontl 这个是 by design ，后续这个 ai provider 的页面还会加入文生图、文生视频等各类模型，provider 也不再仅局限于 chat 类的 provider。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

The *_MODEL_LIST -all function in the environment variable is invalid and cannot enable only the required models.
Is it possible to sort the models in the environment variables? Previously, it was possible to sort the models by adding them in *_MODEL_LIST.

@SpeedupMaster MODEL_LIST environment variables will remain the same and will be compatible today. What was before should be what it is now;

Is it possible to add a custom service provider in environment variables?

I don’t consider adding a custom service provider in the environment variables. There are too few things that can be configured. In the future, I may consider adding a global service provider setting for administrators to control.

In addition, non-chat models such as tts and embedding will also be displayed in the drop-down model list of the chat box.

@coulsontl This is by design. In the future, various models such as Vincent pictures and Vincent videos will be added to the AI ​​provider page. The provider is no longer limited to chat providers.

[gemnioo]

the cross-region inference in aws bedrock still makes user confused; the naming issues need to be addressed

Ref: #4659

[Sun-drenched]

docker部署环境下OPENAI_MODEL_LIST环境变量下各项设置失效。
————————————————————————————————
看到之前的回复了，已解决。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Various settings under the OPENAI_MODEL_LIST environment variable in the docker deployment environment are invalid.

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Looking forward to whether the function of environment variable preconfiguration and custom service provider will be provided in the future ❤️

[TonyBlur]

当前版本(1.45.0)还存在获取模型列表后，仍然会显示当前API默认的部分模型，但是实际/v1/models中并没有获取到的模型，且无法删除，如下图所示

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In the current version (1.45.0), after obtaining the model list, some of the default models of the current API will still be displayed, but the actual models are not obtained in /v1/models and cannot be deleted, as shown in the figure below

[ysicing]

@arvinxx Azure 好像也有问题

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@arvinxx Azure seems to have a problem too

[MTshameless]

So does hugging face.

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

So does hugging face.

[darkelfzdy]

原先的模型配置信息全都没有了，还能恢复吗？lobech并不提供配置导出或备份功能，每次大升级就清空配置信息，这太不科学了

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

All the original model configuration information is gone. Can it still be restored? Lobech does not provide configuration export or backup functions. It is too unscientific to clear the configuration information every time there is a major upgrade.

[arvinxx]

原先的模型配置信息全都没有了，还能恢复吗？

你可以进入 /settings/llm 拿回之前的配置，也可以降级到之前版本继续使用。

lobech并不提供配置导出或备份功能，每次大升级就清空配置信息，这太不科学了

目前没有哪次是完全清空配置的，总会有让你恢复配置的手段。既然选择自部署了，就请接受自部署可能带来的不稳定因素。如果你要追求稳定性，完全可以用我们的Cloud版，我们会确保我们的付费用户配置不会清空。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

All the original model configuration information is gone. Can it still be restored? Lobech does not provide configuration export or backup functions. It is too unscientific to clear the configuration information every time there is a major upgrade.

You can enter /settings/llm to get back the previous configuration

[coulsontl]

在 v1.45.3 中，已经按照批量插入，不做已有配置项更新的逻辑来处理了。

那需要提供一个重置按钮了，我后面对接了一个OneAPI 经常会增加删除模型，现在只增不删了，一个一个手动删除很麻烦。如果能提供多选批量删除就更好了。建议删除可以使用软删除，这样如果不小心删了，下次点击刷新的时候可以再把之前的配置找回来不需要重新添加。

而且模型列表的加载好慢，大概270多个模型需要8秒，电脑是Mac M1性能也不差

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In v1.45.3, it has been processed according to the logic of batch insertion and no update of existing configuration items.

Then we need to provide a reset button. I connected to a OneAPI later and often added and deleted models. Now I can only add but not delete. It is very troublesome to delete manually.

Moreover, the model list is very slow to load. It takes about 8 seconds for more than 270 models. The computer is a Mac M1 and the performance is not bad.

[W3XA56Y3RDUKBS]

环境：

部署时间： 2025.01.09, 20:01 北京时间
使用docker部署数据库版
compose.yaml:

  lobe:
    image: lobehub/lobe-chat-database:latest
    container_name: lobe-chat
    ports:
      - '3210:3210'
    depends_on:
      - postgresql
      - minio
      - logto
    env_file:
      - .env
    restart: always

.env:

ENABLED_OLLAMA=0
OPENAI_MODEL_LIST=-all,+gpt-4o-mini=gpt-4o-mini,+gpt-4o-2024-11-20=gpt-4o,+claude-3-5-haiku-20241022=claude-3-5-haiku,+claude-3-5-sonnet-20241022=claude-3-5-sonnet,+deepseek-chat,+gemini-2.0-flash-exp,+flux-schnell=Flux 1 Schnell [画图]

版本显示为：

问题：

但是用户登录后，还是显示默认的几个openai的模型，我的OPENAI_MODEL_LIST没有生效。并且用户可以通过点击fetch获取全部的列表。请问如何操作可以使模型列表默认展示我在环境变量中设置好的列表？以及如何能禁止用户使用fetch models的功能？

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

environment:

Deployment time: 2025.01.09, 20:01 Beijing time
Deploy the database version using docker
compose.yaml:

  lobe:
    image: lobehub/lobe-chat-database:latest
    container_name: lobe-chat
    ports:
      - '3210:3210'
    depends_on:
      -postgresql
      - minio
      -logto
    env_file:
      - .env
    restart: always

.env:

ENABLED_OLLAMA=0
OPENAI_MODEL_LIST=-all,+gpt-4o-mini=gpt-4o-mini,+gpt-4o-2024-11-20=gpt-4o,+claude-3-5-haiku-20241022=claude-3 -5-haiku,+claude-3-5-sonnet-20241022=claude-3-5-sonnet,+deepseek-chat,+gemini-2.0-flash-exp,+flux-schnell=Flux 1 Schnell [Drawing]

The version is shown as:

question:

However, after the user logs in, several default openai models are still displayed, and my OPENAI_MODEL_LIST does not take effect. And users can get the entire list by clicking fetch. How can I make the model list display the list I set in the environment variable by default? And how to prohibit users from using the fetch models function?

[arvinxx]

So does hugging face.

Hf在1.45.4也修复了

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

So does hugging face.

Hf is also fixed in 1.45.4