Add model selection to the user interface

[Aaronminer1]

With all the models in a race for the top spot could you add say a dropdown menu to select the model you want to use i.e. openai, anthropic, lmstudio, or local model rather than having to change code to change models. also add a pop up to insert the api key if needed or not since lm studios dose not require an api key. with this we could switch quickly between models for different task

[iFurySt]

yes, this feature was already planned and in progress :)

[rajofearth]

It's a good quality of life feature just show this in Matthew Berman's Tutorial/Review.

[marikan114]

Hi!

There is a dropdown menu for model selection (the frontend settings), but I don't see "getgroq/mixtral-8x7b-32768" listed on the menu even if it was in the config.toml. It is supported by litellm but is not listed on API link. I walked around it by Js fiddling on the console: localStorage.setItem("model","groq/mixtral-8x7b-32768,") and it worked!

It might be a good idea to allow custom entries or add the one from config.toml LLM_MODEL to the dropdown.

Best.

[iFurySt]

actually, i didn't find groq/mixtral-8x7b-32768 in the litellm 🤔

[marikan114]

Thanks, @iFurySt! That's why it is not listed on the API link. However, it is supported based on this documentation link. It's more of a litellm issue than an openDevin issue.

However, I would suggest creating a custom entry or pushing the LLM_MODEL from the config file to the selection menu. I did this from the Javascript console, and it worked.

config.toml:

LLM_API_KEY="gsk_hidden"
LLM_MODEL="groq/mixtral-8x7b-32768"

What do you think ?

[iFurySt]

i've just checked the other related PR #591 , it seems the litellm's question about this. i'll submit a PR to merge model_list and model_cost later. i think the merged list is complete list 🤔