[BUG]: GROK-1 does not support do_sample

[vsmelov]

🐛 Describe the bug

inference_tp.py does not support --do_sample mode

if do_sample=False (default) it gives me warnings and same result every time

do_sample=False

torchrun --standalone --nproc_per_node 8 inference_tp.py --pretrained grok-1 --max_new_tokens 300     --text "I will writ
e you very detailed explanations of what is Ethereum and how it works. Ethereum is"

warnings:

/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:381: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.3` -- this flag i
s only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:386: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.95` -- this flag is onl
y used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.
  warnings.warn(

same output every run (seems like temperature is inactive):

I will write you very detailed explanations of what is Ethereum and how it works. Ethereum is a decentralized platform that runs smart contracts: applications that run exactly as programmed
 without any possibility of downtime, censorship, fraud or third party interference. These apps run on a custom built blockchain, an enormously powerful shared global infrastructure that ca
n move value around and represent the ownership of property. This enables developers to create markets, store registries of debts or promises, move funds in accordance with instructions giv
en long in the past (like a will or a futures contract) and many other things that have not been invented yet, all without a middle man or counterparty risk. The project was bootstrapped vi
a an ether presale in August 2014 by fans all around the world. It is developed by the Ethereum Foundation, a Swiss nonprofit, with contributions from great minds across the globe. On tradi
tional server architectures, every application has to set up its own servers that run their own code in isolated silos, making sharing of data hard. If a single app is compromised or goes o
ffline, many users and other apps are affected. On a blockchain, anyone can set up a node that replicates the necessary data for all nodes to reach an agreement and be compensated by users
and app developers. This allows user data to remain private and apps to be decentralized like the Internet was supposed to work. The Ethereum Wallet is a gateway to decentralized applicatio
ns on the Ethereum blockchain. It allows you to hold and secure ether and other crypto-assets built on Ethereum, as well as write, deploy and use smart contracts.

## 1 review for Buy

Run with --do_sample

torchrun --standalone --nproc_per_node 8 inference_tp.py --pretrained grok-1 --do_sample --max_new_tokens 300     --text "I will writ
e you very detailed explanations of what is Ethereum and how it works. Ethereum is"

Gives no output but an error at the end

Exception ignored in: <function HybridParallelPlugin.__del__ at 0x7f821062e950>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/colossalai/booster/plugin/hybrid_parallel_plugin.py", line 1136, in __del__
  File "/opt/conda/lib/python3.10/site-packages/colossalai/cluster/process_group_mesh.py", line 61, in destroy_mesh_process_groups
TypeError: 'NoneType' object is not callable

Environment

No response