I have been running Qwen3.8 27B locally on two RTX 3060 12GB GPUs, and the results have been surprisingly good.
That's 24GB of total VRAM, and with the right llama.cpp configuration I'm getting around 40–50 tokens/sec, while reaching around 200K context with the IQ4_XS model.
More importantly, the model is good enough that I've started using it for actual coding, reasoning and agentic work rather than just testing it.
The Setup
Nothing particularly exotic:
- Ryzen 7 5700G
- 32GB DDR4 RAM
- 2 × RTX 3060 12GB
- Ubuntu
- llama.cpp
- Qwen3.8 27B IQ4_XS
I connect the display to the iGPU so both RTX 3060s can be dedicated to inference.
The key is how the two GPUs are configured. I'm using tensor splitting rather than simply splitting the model by layers.
My llama.cpp Command
This is the command I'm using:
llama-server \
-m ~/lm/unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-IQ4_XS.gguf \
--host 0.0.0.0 \
--port 8080 \
--split-mode tensor \
--tensor-split 1,1 \
--flash-attn on \
--batch-size 2048 \
--parallel 1 \
--models-max 1 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--ctx-size 177000 \
--n-gpu-layers 99 \
--ubatch-size 128 \
--temp 1.0 \
--top-p 0.95 \
--top-k 20 \
--min-p 0.0 \
--presence-penalty 0.0 \
--repeat-penalty 1.0 \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--jinja \
--reasoning on \
--fit on
The important part is:
--split-mode tensor
--tensor-split 1,1
This makes both RTX 3060s participate in the computation.
I'm also using Flash Attention, Q8 KV cache and MTP.
The --ctx-size 177000 shown above is from my original configuration. With the IQ4_XS model, I've subsequently been able to push the context to around 200K tokens.
Performance
With this setup, generation is generally around 40–50 tok/s.
My sustained measurements are around 44 tok/s, with some segments going above 50 tok/s.
Prompt processing is also fast enough that large prompts don't become a major bottleneck.
One of the most interesting findings was how much tensor splitting matters.
Another user with a similar dual-3060 setup reported going from roughly 20–25 tok/s to 40+ tok/s after switching to tensor splitting.
So having 24GB of VRAM isn't the whole story. How you distribute the workload across the GPUs matters.
200K Context on 24GB
This is probably the most impressive part of the setup for me.
Using the IQ4_XS model, I can reach around 200K context on the two 12GB cards.
That gives me enough room for large codebases, long conversations and agentic workflows without constantly worrying about running out of context.
I also tested around 100K context and was still getting 30+ tok/s during generation.
For extremely long-context workloads, I keep Muse Glimmer 30B around as another option. Its sliding-window KV cache makes it particularly useful when I need very large context.
MTP
I'm also using Qwen's MTP support through llama.cpp.
In my runs, the MTP acceptance rate was around 70%, with an average accepted draft length of roughly 2.4 tokens.
It isn't the only reason the model is fast, but it helps push the generation speed further.
What Matters More Than the Benchmark
The biggest surprise for me isn't actually the 40–50 tok/s.
It's Qwen3.8 itself.
Compared with Qwen3.6 27B, I find the improvement very noticeable, particularly in reasoning.
It is also surprisingly good at agentic tasks. It doesn't always just answer the question and stop. During a task, it can notice another problem or a better approach and bring it up.
That makes a difference when you're using it as a coding agent rather than simply chatting with it.
Using It for Coding
I've been using Qwen3.8 with OpenCode for agentic coding.
My workflow is essentially:
Plan → Build/Execute
The model can inspect the project, reason through the problem, modify files and iterate.
I'm also running the machine remotely, using Termius, with ntfy notifications when the agent needs attention or finishes.
At that point, it starts feeling less like a local LLM experiment and more like having my own AI coding machine.
The Limits
There are still obvious limitations.
24GB isn't a lot once you start pushing very large contexts, and the quantization and KV-cache configuration become important.
Thermals are another consideration. During a long reasoning run, one of my Ventus cards reached around 87°C.
And while 16GB can run smaller configurations, having 24GB gives much more breathing room for a 27B model and large context.
Final Thoughts
I didn't build this system expecting two RTX 3060s to compete with the latest high-end GPUs.
The interesting part is how much they can actually do together.
Two RTX 3060 12GB cards + Qwen3.8 27B IQ4_XS + llama.cpp gives me:
- Around 40–50 tok/s
- Up to 200K context
- ~70% MTP acceptance in my runs
- Strong reasoning
- Good agentic coding performance
- 24GB total VRAM
For hardware that is relatively inexpensive and widely available, that's a pretty impressive local AI setup.