Find the right model for your GPU. Configure vLLM. Measure real capacity.
uv syncInside the cloned repo. See setup instructions →
Python 3.12+ · NVIDIA GPUs · Built for vLLM
BUILT ON THE STACK
YOU ALREADY USE.
01 / FROM POSSIBILITY TO PERFORMANCE
One toolkit to connect your hardware,
your model, and your next deployment.
Search a model family. Get recommendations matched to your environment.
EXAMPLE · QWEN / 24 GiB FREE / FP16
Weight estimates only · illustrative.
Pick a profile. Get a vLLM command ready to review and run.
EXAMPLE · BALANCED PROFILE
vllm serve Qwen/Qwen2.5-7B-Instruct \
--tensor-parallel-size 1 \
--dtype float16 \
--gpu-memory-utilization 0.9 \
--max-model-len 4096 \
--generation-config vllm \
--kv-cache-dtype auto \
--max-num-seqs 64 \
--max-num-batched-tokens 8192 \
--enable-prefix-caching \
--enable-chunked-prefill
Example: 1 GPU · balanced profile. Review before running.
Sweep the load. See where throughput stops scaling.
ILLUSTRATIVE CONCURRENCY SWEEP
16 workers Last healthy level
+3.9% throughput at 32 workers · sample data.
Estimates exclude KV cache and runtime memory. Verify by running your model.
02 / YOUR TERMINAL. YOUR WORKFLOW.
Choose a step. Copy the command. Run it locally.
# Start with the hardware you have.
Illustrative shortlist, not live results. Run locally for your hardware. Estimates exclude KV cache and runtime memory.
03 / MAKE EVERY GPU COUNT
Open source. Hardware aware. Built for developers.
Explore the repository ↗Read the docs →$ git clone https://github.com/nguyenvmthien/infercap.git $ cd infercap $ uv sync
Requires Python 3.12+ and a compatible vLLM / PyTorch platform.
02 / Find a model for your GPU →