Autogen Studio: LLM model defined per agent - makes it VERY hard to switch to test models! Need duplicate agents for every model??

[ewebgh33]

Installed Autogen Studio today
Seems really cool. But a big downside I've already noticed, is that if we want to do a research task on GPT4, then also try Mixtral, or whatever, we have to either
a) go edit the agents, and delete the model GPT4, add in the model for something else
or
b) duplicate the agents so we have a research assistant that is basically identical in all aspects except the model they use.

Am I the only one that thinks this is clunky?
There has to be a better way.

If I want to run a group of say, 3-5 agents, that 10 agents to try 2 models. 15 agents to try 3 models. etc.
UI is gonna get messy real quick.

Now say I want to edit that agent. But I have 2-3 or more of them, that are identical again except for the agent specified.
(or I'm editing the LLM priority order every time I want to use a different model).

I've had this running less than 30mins and I've already hit this clunkiness of multiple agents that do the same thing, or spending tedious minutes each time re-configuring what LLM each agent should use.

Hoping others may have an insight into a better way the LLMs could be defined?

My solution? Not sure...
But maybe this is why other tools use a flowchart style workflow to build agents, because then you can easily see the order of operations AND quickly specify whatever you need per node.
If I could see the workflow as an actual flow of nodes, they could have dropdowns. Say one overall dropdown or node where you could quickly specify default LLM for any nodes in which the LLM was not explicitly defined, and a dropdown IN the nodes if you wanted an override one, such that an agent used a different LLM to the default for the flow.

Current implementation is way too many boxes, modals, windows in which I need to go edit LLM!

[ewebgh33]

Case in point, this demo from 2 days ago
https://www.youtube.com/watch?v=mUEFwUU0IfE&t=160s
He has to go make new agents just so they can use a local model. There doesn't seem to be a global default, and if there was, there's also no way to quickly/easily override one or more agents to use different LLMs. The buttons with X to close, inability to drag them, is poor.

Adding another note
In CrewAI I have also had to specify a model for each agent
But in a .py file / piece of code, I can really easily do a find-replace to change, say openhermes to mistral.

In Autogen you can't do a find-replace now in the UI, you have to go clicking in and out of all the modal windows and edit the options in a dozen places.

[squartochi]

I am very new to autogen studio and never used autogen.

So there seems to be many bugs in the IU. Does not seem to save a good number of settings. I have been looking everywhere for the agent files or file created from the web UI. Does anyone know where I can find and edit these?

[franklinbaldo]

If use autogenstudio ui --appdir . You will get the database

[gagb]

@victordibia

[victordibia]

@EmmaWebGH ,

autogenstudio dev here.

You are right. We are currently working on refactoring the authoring UI.
Yes, a flow based UI could solve many of these issues and we will be looking into it.

@samershi , @JingyaChen fyi

Thanks.

[ewebgh33]

Just saw the new version where we can predefine agents! Yes!!! Thank you team