Built-In Token Tracker

[eWOOD29]

I love this program, and really appreciate the hard work of the developers! I would really appreciate the implementation of a feature within the UI that would display the number of currently used tokens, along with the token limit of the currently-used model if possible. I'm aware I can check the logs for this number which works well, but it would be significantly easier if I was able to see this number directly within the interface. Thank you for all of your development efforts and I look forward to seeing future improvements in LibreChat!

[danny-avila]

I’m working on something related (tracking token spend per message), and may look into UI changes here after.

Thing is it’s tricky. The number of currently used tokens is pretty broad and can vary within a conversation. Different models affect the context window, as well as message branching. Also you are not charged per message rather you are charged per context, which is made up of messages.

you may type a 10 token message as an 11th message in the convo, but your spend for that 11th message would be that message’s tokens + however many prior messages’ tokens could fit in that context.

Something easier, which what I’m working on will make easier to trace, will be to say how much a conversation has cost so far, regardless of context (in prompt and completion tokens)

[danny-avila]

Also there is summarization now implemented which affects token spend

[eWOOD29]

Thank you so much for the reply! It definitely seems like a challenge to build out a feature like this, especially with such a modular program. Either way, I appreciate your time and look forward to future progress!

[danny-avila]

Hi @eWOOD29 I wanted to let you know this is now live in the app as of #1018

[danny-avila]

not the UI piece, but all the backend stuff that will enable the UI display of tokens used

[eWOOD29]

Awesome, thank you so much for the hard work and for keeping me updated!