Very different quality of response when using vision model via openai compatible endpoint and /extra/generate/stream endpoint

[lobsterchan27]

Maybe someone could help me understand whats going on in the /v1/chat/completions endpoint? I'd like to get similar quality of response from the streaming endpoint. Is it applying a chatml format on the passed content? Doesn't really seem like the difference should be so drastic only from that.

The image being used for testing: https://files.catbox.moe/ahz7sg.webp
when using the openai compatible endpoint the response is as expected.

/v1/chat/completions

Input: {"model": "koboldcpp_current", "messages": [{"role": "user", "content": [{"type": "text", "text": "Whats in this image?"}, {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/...

Output: The given visual depicts a digital rendering of an animated Pokemon -like creature, dressed in maid attire. It is posing in a walking stance with its hands on its hips, showcasing its blue fur, white belly, and wearing a white bow tie around its neck. The background is transparent, highlighting the focus on the character itself. This image seems to be related to a video game or anime series, possibly featuring this particular creature as a playable character or a supporting persona.

/extra/generate/stream

Input: {"prompt": "Whats in this image?", "max_length": 200, "images": ["/9j/4AAQSkZJRgABAQAAAQABAAD/...

Output: a pokemon character dressed in maid outfit

Commenters agree
Nobody commented yet Write first

Similar news

a pokemon character dressed in maid outfit - pornhub

nintendo reveals pokemon go, the first pokemon game for mobile - ign

pokemon go release date, features, news, and everything you need to know - ign

pokemon go is a fun and engaging game that can be played on mobile devices - ign

nintendo presents the pokemon company international in new york - ign

pokemon go is a new mobile game with an augmented reality - ign

pokemon go is a fun and engaging game that can be played on mobile devices - ign

pokemon go is a fun and engaging game that can be played on mobile devices - ign

the official pokemon website has revealed new details about the pokemon go game - ign

pokemon go

---

edit: after updating to 1.64.1 the results are better but still very incoherent and no where near the quality you would expect.

/v1/chat/completions

Output: The given visual represents a digital rendering of an anime - styled rabbit - like creature dressed in a maid outfit, with blue colored fur and accessories. It is posed in a dynamic stance, as if about to move, while wearing a white apron and a bow tie. The background is white, and there's a subtle shadow effect creating a three-dimensional appearance. This seems to be a screenshot or promotional artwork for a video game character or a mobile game, possibly from the popular Pokemon franchise.

/extra/generate/stream

Output: a character wearing an outfit with blue and white
animations

a pokemon character named lunala is shown wearing a maid dress and holding a broom

the pokemon go event has begun, and the game's events include a new event called the'pokemon go event,' which features a rare pokemon character, lunala

this event runs from november 3 to 10, with a special event that rewards players with a new reward for completing a special challenge, the lunala

the event will also feature a new pokemon trainer character, a rare event for trainers, and the chance to earn some special event rewards

the pokemon go event is an ongoing event that runs from november 3 to 10, and it features a special event for trainers called the'pokemon go event,' which features a rare pokemon character, lunala

this event runs from november 3 to 10, with

---

so its somewhat more coherent but not useful at all. please if anyone could help me solve this issue whether its hidden prompting format or settings via the openai endpoint. or if its something i could be doing wrong? the only thing i can think of thats visible is the data tag in the image string is removed. but the /extra/generate/stream does not work with the data tag in the string.

Thanks!

[LostRuins]

LLAVA in 1.64 was bugged.

1.64.1 should have fixed the issue.

[lobsterchan27]

i did update to 1.64.1 i put the a 2nd set of request/response after updating after the edit: comment above. im still having issues.