Alternatives

Products that do what Tokenflood – simulate arbitrary loads on instruction-tuned LLMs does

Hi everyone, I just released an open source load testing tool for LLMs: https://github.com/twerkmeister/tokenflood === What is it and what problems does it solve? === Tokenflood is a load testing tool for instruction-tuned LLMs hat can simulate arbitrary LLM loads in terms of prompt, prefix, and output lengths and requests per second. Instead of first collecting prompt data for different load types, you can configure the desired parameters for your load test and you are good to go. It also let's you assess the latency effects of potential prompt parameter changes before…

1. Not indexed yet — check back in a few minutes.

Ranked by how close each launch is in meaning, then by votes. Refine with a description →