llama.cpp 0.3.0 release notes

Overview

llama.cpp 0.3.0 introduces the dots3-note multimodal model (with a new DSA-ISWA KV cache), MTP support for GLM-4.5-Air, and tensor-split (-sm tensor) plus multi-sequence rollback fixes for DeepSeek 4. ggml is bumped to v0.22.0 (meta-backend tensor split, per-op Metal kernels with parallel compilation, non-in-place ggml_clamp), while mtmd gains dots3-note vision/audio, WebP decoding and a Pillow-accurate resize. The server adds a LLAMA_SERVER_SLOTS_N_DIFF debug knob, and the web UI gets tabbed chat navigation.

New models

Add dots3-note model with a new DSA-ISWA KV cache type (#27060)

Core changes

DeepSeek 4: add tensor-split mode via -sm tensor (#26490)
DeepSeek 4: fix rollback with multiple sequences (#26756)
Fix meta tensor split state propagation for tensor parallel (#27574)
GLM-4.5-Air: add MTP (multi-token prediction) support (#26534)
bailingmoe3: support DSpark (#27508)
mamba2: flatten in/out projections to dispatch GEMM instead of GEMV (#27513)
Models: use ggml_rope_set_offset in deepseek2/4, dflash, minicpm3 and plm (#27382)
Grammar: parse \- in char classes as a literal hyphen (#27591)
Common: add json.h abstraction (#27511) with a clang LTO fix (#27575)
Common: fit moved out of the server and now takes n_streams into account (#27496)
Common: fix draft-mtp with embeddings (#27400)
Arg: remove the -no-cnv CLI option (#27542)

Multi-modality changes

Support dots3-note vision and audio (#27524)
Support WebP images via ffmpeg (#27520)
Fix loading videos with the moov atom at the end of the file (#27596)
Use a Pillow-accurate resize algorithm and correct resize_algo for all models (#27594)
Use ggml_rope_set_offset in the CLIP graph (#27521)

Server changes

Add LLAMA_SERVER_SLOTS_N_DIFF env var to widen the slot debug diff window (#27600)
Slot fitting logic moved to the common fit, now accounting for n_streams (#27496)
Adopt the common json.h abstraction (#27511)

UI changes

Tabbed navigation for chat conversations (#27263)
Fix keyboard shortcuts for the chat tabs navigation (#27609)

ggml changes

ggml bumped to v0.22.0 (ggml/1607):
- This release adds tensor-split support to the multi-backend (meta) backend with improved split-state propagation, reworks the Metal kernels into per-op sources with parallel compilation, and fixes ggml_clamp to be a proper non-in-place op. It also brings new ops
  (POOL_1D, PAD_REFLECT_1D), Q2_K SYCL kernels, MoE bias fusion on OpenCL, and assorted fixes across the CUDA, Metal, SYCL, Vulkan, OpenCL and WebGPU backends.

Assets

Nightly build: b10621

More info

Changelog since v0.2.0

c1d0e7a00 llama.cpp : bump version to 0.3.0 (#27696) 81191affa sync : ggml 93882361f ggml : bump version to 0.22.0 (ggml/1607) eb25b7263 grammar : parse - in char classes as literal hyphen (#27591) 814d84bc9 sycl : mark tq2_0 as not supported (#27660) 5ea87ddad webgpu : fix handling of infinity values during ARGSORT and TOP_K (#27538) f280b2698 metal : per-device tuned (Q, NE) for flash-attn vec (#26570) b615f5b4b metal: per-op source split + parallel compile (#26561) b3c3b96a1 misc : read repetition_penalty from generation_config.json (#27659) 758443071 tests : disable DOTS3NOTE arch test for WebGPU (#27654) 71cc86fa4 convert: fix GLM regression in index_tensors (#27655) a14dba686 ggml : shorten virtual device naming in CUDA and Metal (#27608) c1c766da5 webgpu : reorder includes since V that appears in common_decls.tmpl may be defined as K in flash_attn_decls.tmpl if KV_OVERLAP (#27545) 160c6b0bd mtmd: video: fix moov atom at the end of file (#27596) 985b14912 ci : apply ccache-clear with older/min/dry-run to all ccache jobs (#27602) 6036c635e ggml : fix ggml_clamp (#27644) a130532ae mamba2 : Flatten in/out projections to dispatch GEMM instead of GEMV (#27513) bf0a29cc1 Deepseek 4: -sm tensor (#26490) c060ca974 model : support MTP in GLM-4.5-Air (#26534) ccc8fd2ba readme : update links (#27617) d05f89562 fix: Change chat tabs nav shortcuts (#27609) 8d9af2563 test : fix multi-GPU server tests (#27614) 4a08fa297 test: move tools/parser to tests (#27548) 56db501e7 mtmd: use pillow-accurate algo, correct resize_algo for all models (#27594) 95b8e33e1 ci : add test-llama-archs tensor split for Metal (#27598) a278dcef0 contrib : recommend waiting for CI before merging (#27603) e8eed4525 server : add LLAMA_SERVER_SLOTS_N_DIFF (#27600) ba8e0eddf common : skip device_info loop if it's not going to be printed (#26692) b0539c43e DeepseekV4: fix rollback with multi-seq (#26756) d3371929b [Tensor parallel] Fix meta tensor split state propagation (#27574) 8144f3192 ui: Chat Conversation Tabbed navigation (#27263) 6657ded4f vendor : update subprocess.h (#27409) 29ea9412a cuda : add POOL_1D support (#27573) 70adb1b4c common: json.h: fix clang lto (#27575) 3f545becc vulkan : added the PAD_REFLECT_1D operation (#26586) b21e4de74 mtmd: use ggml_rope_set_offset (#27521) d9f918d2d common: add json.h abstraction (#27511) 2fb989b9e fit: also take into account n_streams (#27496) 9fee29e94 arg: remove -no-cnv from cli [no ci] (#27542) e85caa81e ci : Restore ROCm job for Ubuntu (#27399) 2115b73d8 model : support DSpark for bailingmoe3 (#27508) 54ee5ee64 mtmd: support dots3-note vision+audio (#27524) 3a653fea9 ci : add older, min and dry-run options to ccache-clear (#27504) 369e1cd61 ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (#24575) 2c6b141ef common : fix draft-mtp with embeddings (#26352, #27299) (#27400) 867229003 sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (#27490) 3aeb92462 readme : fix server badge alt (#27533) 2100e5926 readme : update badges (#27531) d775b8967 mtmd: support webp via ffmpeg (#27520) 3af988fab opencl: fold the gpt-oss MoE per-expert bias adds into the epilogue (op/kernel fusion) (#26431) 9a286ac98 docs: improve Windows build instructions (#27381) a3b9c23ea ci : fix empty release_id in make-release upload step (#27516) 5a32f7b66 model: add dots3-note (#27060) 873e5d8e3 model: use ggml_rope_set_offset() (#27382) d7fa69b7d ci : run ccache-clear as the last step of release jobs (#27503)