How to use it?

[souvikqb]

Can someone please share a step-by-step procedure to get started?

I would prefer a Non-Gradio Approach (CLI or HF Pipeline)

[Zeqiang-Lai]

Here is an example

from minidalle3.model import MiniDALLE3

model = MiniDALLE3('gpt3.5', prompt_path="minidalle3/prompts/prompt-v2.txt")
message = [model.system_message, 'I have read a story where it talks about an "astronaut riding a horse" -- What does it look like ?']
images = []
text, image = state['model'].ask(state['messages'], state['images'])
if image is not None:
   image = image.image

[souvikqb]

Here is an example

from minidalle3.model import MiniDALLE3

model = MiniDALLE3('gpt3.5', prompt_path="minidalle3/prompts/prompt-v2.txt")
message = [model.system_message, 'I have read a story where it talks about an "astronaut riding a horse" -- What does it look like ?']
images = []
text, image = state['model'].ask(state['messages'], state['images'])
if image is not None:
   image = image.image

Thank for the code snippet, I'll definitely try it out and let you know.

2. I was trying to access the Demo but for some reason it was not loading, can you please check it once?

3. I wanted to learn whether this model is allowed for commercial use or for just research purposes?

[Zeqiang-Lai]

1. The demo can be accessed normally on my computer.
2. We are still working on the advanced version of minidall3 👨‍💻 and have not released any model yet. Currently, the demo is built with open techniques (ChatGPT + SDXL + IP-Adapter). I think SDXL and ChatGPT are allowed for commercial use. IP-adapter is built on SD, so it might be allowed for commercial use as well but I am not sure about it.

[souvikqb]

1. The demo can be accessed normally on my computer.

For the Demo I'm getting this, the page keeps on loading but nothing is displayed --

2. We are still working on the advanced version of minidall3 👨‍💻 and have not released any model yet. Currently, the demo is built with open techniques (ChatGPT + SDXL + IP-Adapter). I think SDXL and ChatGPT are allowed for commercial use. IP-adapter is built on SD, so it might be allowed for commercial use as well but I am not sure about it.

Would love to have this as an HF Pipeline
What I'm looking for is-

1. Any easy way to download and load the model on GPU
2. Have some parameters to play with to get different outputs
3. Chat based memory to help it remember things and keep the conversation flowing (if possible)

[souvikqb]

@Zeqiang-Lai
Hey Can you check this Colab Notebook - https://colab.research.google.com/drive/1MR5raiKXpqhHjcakcMtcl-izoShtE6nm?usp=sharing

I was unable to get started with the above snippet for some reason.

If this Colab works out, you can also add a Colab Demo for people to try out!

[souvikqb]

Hi any updates? @Zeqiang-Lai

[souvikqb]

Hey!

I was able to get the Demo working

Can you work on a Demo Colab Notebook as well? @Zeqiang-Lai

[Zeqiang-Lai]

I am sorry that I am very busy for these moments, we will include the Colab demo for our next version.

[souvikqb]

Sure, please let me know once you release an update