KVInfer

152M · GPT-2 · AVX2 + OpenMP · KV-Cache

Live Performance

Throughput —

TTFT —

Tokens out —

Last latency —

Throughput History (tok/s)

Session avg —

Session peak —

Session

Turns 0

Total tokens 0

Tokens in engine —

Server RAM —

System Prompt

Generation

Temperature 0.70

Top-K 40

Max tokens 200

KVInfer · 152M

Chat

Idle

152M · GPT-2 Decoder-Only · Custom C++ inference engine with AVX2 SIMD, OpenMP parallelism & persistent session KV-cache.

152M params AVX2 SIMD OpenMP KV Cache Streaming

Enter to send · Shift+Enter for newline

Runs 5 built-in prompts and measures throughput, TTFT, and per-token latency.