推理中将多张图像cat在一起作为输入，出现CUDA out of memory

[aabbc-cell]

推理中，我使用1-2张图像cat在一起作为输入时，使用8张v100-32G（device_map="auto"）能正常进行对话，但使用3张及以上图像（例如10张图像）cat在一起作为输入时，报错：
t）orch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 9.14 GiB. GPU 0 has a total capacity of 31.75 GiB of which 8.85 GiB is free. Process 18099 has 22.89 GiB memory in use. Of the allocated memory 22.12 GiB is allocated by PyTorch, and 414.56 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
这是因为device_map="auto"的分配不均吗？但我自己重新设置device_map时仍报错CUDA out of memory

[czczup]

拼起来之后的图片大概有多少个patch? 如果图片数量很多的话，可以试试load_image函数里max_num给小点，原本给的例子里是6，可以试试4。