Muse Glimmer 30B on Dual RTX 3060 (12GB Each)

Alongside Qwen3.8 27B, Muse Glimmer 30B has become my other go-to model on my dual RTX 3060 setup.

I mainly use it when I need very long context or longer agentic sessions.

The interesting part is that with 24GB of total VRAM, I can run Muse Glimmer at its full 262K context while still having roughly 1GB of VRAM available on each GPU.

The Setup

My local AI machine is:

• Ryzen 7 5700G
• 32GB DDR4 RAM
• 2 × RTX 3060 12GB
• 24GB total VRAM
• Ubuntu
• llama.cpp
• Muse Glimmer 30B UD Q4_K_XL

The display is connected to the iGPU, so the NVIDIA GPUs can be dedicated to inference.

I also don't allow other Ubuntu processes to use the NVIDIA GPUs, giving the model as much VRAM as possible.

My llama.cpp Command

This is the configuration I'm using:

llama-server \
-m ~/llm/unsloth/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
--host 0.0.0.0 \
--port 8080 \
--split-mode tensor \
--tensor-split 1,1 \
--flash-attn on \
--batch-size 2048 \
--parallel 1 \
--models-max 1 \
--cache-type-k f16 \
--cache-type-v f16 \
--ctx-size 262144 \
--n-gpu-layers 999 \
--spec-draft-model ~/llm/unsloth/dflash-kquant.gguf \
--spec-draft-ngl 999 \
--spec-draft-n-max 15 \
--spec-type draft-dflash \
--override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144 \
--fit off \
--no-warmup \
--temp 1.0 \
--top-p 0.95 \
--top-k 64 \
--reasoning-preserve

With this configuration, I'm getting 30+ tok/s.

262K Context on 24GB

This is the main reason I keep Muse Glimmer around.

The model supports a maximum context of 262,144 tokens, and I can actually run that context on my dual RTX 3060 setup.

Even at that context size, I still have roughly 1GB of VRAM remaining on each GPU.

That's where Muse Glimmer's architecture becomes particularly interesting.

Muse Glimmer uses a sliding-window KV cache for most layers, which makes its KV cache much more memory-efficient at very large context sizes.

On a 24GB setup, that makes a huge difference.

Why Long Context Actually Matters

A huge context window isn't useful just because the number looks impressive.

For agentic work, context gets consumed very quickly:

• Source code
• Previous conversation
• Tool output
• Terminal output
• Files being inspected
• Search results
• Previous changes
• Reasoning

All of that needs to stay available if you want an agent to work through a large task without constantly losing earlier information.

This is where Muse Glimmer makes sense for me.

When I need a very long-running session, I switch to Muse Glimmer rather than trying to squeeze everything into Qwen3.8.

Agentic Work

Muse Glimmer is also good at agentic tasks and fixing setup issues.

That's important because a large context window by itself isn't enough.

The model still needs to be capable of working through a problem, keeping track of what it has already done, and continuing through multiple steps.

I've found Muse Glimmer particularly useful for this kind of work.

My Local Model Setup

I don't keep just one model loaded and expect it to do everything.

I run llama.cpp with a model preset file that essentially acts as a router, allowing me to load the models I need on demand.

That gives me a simple division of work.

Qwen3.8 27B

• Faster generation
• Strong reasoning
• Coding
• Agentic work
• Around 40–50 tok/s on my setup

Muse Glimmer 30B

• 262K context
• Efficient KV cache
• Long-running agent sessions
• Good agentic performance
• Useful for fixing setup problems
• 30+ tok/s on my setup

This works better for me than trying to find one model that is best at everything.

Why the Dual 3060 Setup Works

The important thing isn't just having 24GB of VRAM.

It's using the VRAM efficiently.

I'm using tensor splitting:

--split-mode tensor
--tensor-split 1,1

Both RTX 3060s participate in inference.

The display is connected to the iGPU, and I keep other processes from consuming NVIDIA VRAM.

With Muse Glimmer, the sliding-window KV architecture then makes it possible to use a 262K context without consuming the entire 24GB just on KV cache.

Final Thoughts

Muse Glimmer isn't simply another 30B model in my collection.

It fills a very specific role.

When I want speed and reasoning, I use Qwen3.8 27B.

When I need very large context and a long-running agent, I use Muse Glimmer 30B.

On my dual RTX 3060 system, Muse Glimmer can run at 30+ tok/s with a full 262K context, while still leaving roughly 1GB of VRAM free on each GPU.

For a 24GB consumer-GPU machine, that's what makes Muse Glimmer particularly interesting to me.