4 x n2d-standard-16 (16 vCPU, 64 GB RAM) [Grok-1] #23

b4rtaz started this conversation in Results

b4rtaz
Apr 17, 2024
Maintainer

I successfully started the inference of Grok-1 on 4 x 16 vCPU, 64 GB RAM (4 x Google Cloud n2d-standard-16). Achieved 1.8 tokens/second. 🎉

Metric	4 VM
Avg generation time	549.80 ms
Avg inference time	485.87 ms
Avg transfer time	62.89 ms

Screenshot 2024-04-04 at 23 52 56

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment