{"url":"https://api.github.com/repos/ggml-org/llama.cpp/compare/b11368...99b95488cac0f00ce3f05af113a8c1e287753f87","html_url":"https://github.com/ggml-org/llama.cpp/compare/b11368...99b95488cac0f00ce3f05af113a8c1e287753f87","permalink_url":"https://github.com/ggml-org/llama.cpp/compare/ggml-org:1fb7ef3...ggml-org:99b9548","diff_url":"https://github.com/ggml-org/llama.cpp/compare/b11368...99b95488cac0f00ce3f05af113a8c1e287753f87.diff","patch_url":"https://github.com/ggml-org/llama.cpp/compare/b11368...99b95488cac0f00ce3f05af113a8c1e287753f87.patch","base_commit":{"sha":"1fb7ef3e3327f18f1e99d294115b8493d54e196a","node_id":"C_kwDOJH_K4NoAKDFmYjdlZjNlMzMyN2YxOGYxZTk5ZDI5NDExNWI4NDkzZDU0ZTE5NmE","commit":{"author":{"name":"Pranesh Gonegandla","email":"pranesh.iitp@gmail.com","date":"2026-10-02T14:52:54Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2026-10-02T14:52:54Z"},"message":"spec : add probabilistic sampling for simple draft and MTP (#27694)\n\n* Make the drafter probabilistic and the target verify by rejection sampling\n\n* Drop stale spec_draft_q before drafting\n\n* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.\n\n* Support grammar-constrained requests in rejection sampling\n\n* Fix - renormalize distribution after masking\n\n* copy rng on sampler copy and re-accept drafted tokens on replay\n\n* Fix draft sampler sharing the target's rng stream\n\n* Simplify the rejection sampler's inputs and move replay to the server\n\n* Truncate the draft candidates along with the draft\n\n---------\n\nCo-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>\nCo-authored-by: Pranesh Gonegandla ","tree":{"sha":"a4f16e2d3bfd3b201fac47968e85ef82ee66c5f2","url":"https://api.github.com/repos/ggml-org/llama.cpp/git/trees/a4f16e2d3bfd3b201fac47968e85ef82ee66c5f2"},"url":"https://api.github.com/repos/ggml-org/llama.cpp/git/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsFcBAABCAAQBQJqv8VGCRC1aQ7uu5UhlAAAHHcQAFGdln4ixLNOX7kDBPjw+54I\n4ZD5CF4KifT+f4suFsTGh1Q6Gry+Kw8Cr2yqDH+mazgytyrU4Rycfe5XrwLnq2VL\nhBr2mzuFdSko6UXYJhC/1LZyBmLpBuEEafY2qX0x9YXI5kVh3IEWr9X7/cfaJQNe\nX2JwtOOpu+yGSHSUROKSxTeIyrp/b22gU1V81quDRWEwZXy2kyRweRpwXF17uCKt\nNWTckytqOlVfiJNykuozVFvx8KbBwES6EzZm8641fH1s4Ypq8uX72qs4AYkpv41u\nQ3YlafmH+i6QeK1KWF4yB1XeO0a9UTAZw3n/EZTbGJ8EE7nY+2btl+z9NwYCnmlA\nwIoBCmjqYbMWw+M49x21O5GuRQpZ5arSmbIn5fBWOYxBI9ysCQGic2T01wbT1Xlm\ntobUTHilBtjxMV2acrXGDKH2N05XCReNUedKIwgYwLUcOfcwazmRNa5BqyKheEwj\nzXemim/IOxK7VKXv4C9L4ZgzKT3Eux2//zYYFCtL+H6y1yUswd25l+7IdjPEFP2G\ngz1SjfObZJps5oUmuFmyAH2tuJnYY1dGB+oHFQQT4MsfvyRBDpqu44vHQsAS+psp\noiqZAjnYD/FM6nBccBHvH0WI/tGMz0vYPgjCVVyvFAv+PCaIppYTyOOM9EmBRQXn\ni8SmD+GOM/Ta30WU6Az5\n=GnQ9\n-----END PGP SIGNATURE-----\n","payload":"tree a4f16e2d3bfd3b201fac47968e85ef82ee66c5f2\nparent 134b2bb756c984d4fd2cabc26335f8de56b12ee1\nauthor Pranesh Gonegandla 1790952774 +0530\ncommitter GitHub 1790952774 +0300\n\nspec : add probabilistic sampling for simple draft and MTP (#27694)\n\n* Make the drafter probabilistic and the target verify by rejection sampling\n\n* Drop stale spec_draft_q before drafting\n\n* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.\n\n* Support grammar-constrained requests in rejection sampling\n\n* Fix - renormalize distribution after masking\n\n* copy rng on sampler copy and re-accept drafted tokens on replay\n\n* Fix draft sampler sharing the target's rng stream\n\n* Simplify the rejection sampler's inputs and move replay to the server\n\n* Truncate the draft candidates along with the draft\n\n---------\n\nCo-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>\nCo-authored-by: Pranesh Gonegandla ","verified_at":"2026-10-02T14:52:55Z"}},"url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a","html_url":"https://github.com/ggml-org/llama.cpp/commit/1fb7ef3e3327f18f1e99d294115b8493d54e196a","comments_url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a/comments","author":{"login":"praneshgo","id":227579474,"node_id":"U_kgDODZCWUg","avatar_url":"https://avatars.githubusercontent.com/u/227579474?v=4","gravatar_id":"","url":"https://api.github.com/users/praneshgo","html_url":"https://github.com/praneshgo","followers_url":"https://api.github.com/users/praneshgo/followers","following_url":"https://api.github.com/users/praneshgo/following{/other_user}","gists_url":"https://api.github.com/users/praneshgo/gists{/gist_id}","starred_url":"https://api.github.com/users/praneshgo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/praneshgo/subscriptions","organizations_url":"https://api.github.com/users/praneshgo/orgs","repos_url":"https://api.github.com/users/praneshgo/repos","events_url":"https://api.github.com/users/praneshgo/events{/privacy}","received_events_url":"https://api.github.com/users/praneshgo/received_events","type":"User","user_view_type":"public","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","user_view_type":"public","site_admin":false},"parents":[{"sha":"134b2bb756c984d4fd2cabc26335f8de56b12ee1","url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/134b2bb756c984d4fd2cabc26335f8de56b12ee1","html_url":"https://github.com/ggml-org/llama.cpp/commit/134b2bb756c984d4fd2cabc26335f8de56b12ee1"}]},"merge_base_commit":{"sha":"1fb7ef3e3327f18f1e99d294115b8493d54e196a","node_id":"C_kwDOJH_K4NoAKDFmYjdlZjNlMzMyN2YxOGYxZTk5ZDI5NDExNWI4NDkzZDU0ZTE5NmE","commit":{"author":{"name":"Pranesh Gonegandla","email":"pranesh.iitp@gmail.com","date":"2026-10-02T14:52:54Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2026-10-02T14:52:54Z"},"message":"spec : add probabilistic sampling for simple draft and MTP (#27694)\n\n* Make the drafter probabilistic and the target verify by rejection sampling\n\n* Drop stale spec_draft_q before drafting\n\n* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.\n\n* Support grammar-constrained requests in rejection sampling\n\n* Fix - renormalize distribution after masking\n\n* copy rng on sampler copy and re-accept drafted tokens on replay\n\n* Fix draft sampler sharing the target's rng stream\n\n* Simplify the rejection sampler's inputs and move replay to the server\n\n* Truncate the draft candidates along with the draft\n\n---------\n\nCo-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>\nCo-authored-by: Pranesh Gonegandla ","tree":{"sha":"a4f16e2d3bfd3b201fac47968e85ef82ee66c5f2","url":"https://api.github.com/repos/ggml-org/llama.cpp/git/trees/a4f16e2d3bfd3b201fac47968e85ef82ee66c5f2"},"url":"https://api.github.com/repos/ggml-org/llama.cpp/git/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsFcBAABCAAQBQJqv8VGCRC1aQ7uu5UhlAAAHHcQAFGdln4ixLNOX7kDBPjw+54I\n4ZD5CF4KifT+f4suFsTGh1Q6Gry+Kw8Cr2yqDH+mazgytyrU4Rycfe5XrwLnq2VL\nhBr2mzuFdSko6UXYJhC/1LZyBmLpBuEEafY2qX0x9YXI5kVh3IEWr9X7/cfaJQNe\nX2JwtOOpu+yGSHSUROKSxTeIyrp/b22gU1V81quDRWEwZXy2kyRweRpwXF17uCKt\nNWTckytqOlVfiJNykuozVFvx8KbBwES6EzZm8641fH1s4Ypq8uX72qs4AYkpv41u\nQ3YlafmH+i6QeK1KWF4yB1XeO0a9UTAZw3n/EZTbGJ8EE7nY+2btl+z9NwYCnmlA\nwIoBCmjqYbMWw+M49x21O5GuRQpZ5arSmbIn5fBWOYxBI9ysCQGic2T01wbT1Xlm\ntobUTHilBtjxMV2acrXGDKH2N05XCReNUedKIwgYwLUcOfcwazmRNa5BqyKheEwj\nzXemim/IOxK7VKXv4C9L4ZgzKT3Eux2//zYYFCtL+H6y1yUswd25l+7IdjPEFP2G\ngz1SjfObZJps5oUmuFmyAH2tuJnYY1dGB+oHFQQT4MsfvyRBDpqu44vHQsAS+psp\noiqZAjnYD/FM6nBccBHvH0WI/tGMz0vYPgjCVVyvFAv+PCaIppYTyOOM9EmBRQXn\ni8SmD+GOM/Ta30WU6Az5\n=GnQ9\n-----END PGP SIGNATURE-----\n","payload":"tree a4f16e2d3bfd3b201fac47968e85ef82ee66c5f2\nparent 134b2bb756c984d4fd2cabc26335f8de56b12ee1\nauthor Pranesh Gonegandla 1790952774 +0530\ncommitter GitHub 1790952774 +0300\n\nspec : add probabilistic sampling for simple draft and MTP (#27694)\n\n* Make the drafter probabilistic and the target verify by rejection sampling\n\n* Drop stale spec_draft_q before drafting\n\n* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.\n\n* Support grammar-constrained requests in rejection sampling\n\n* Fix - renormalize distribution after masking\n\n* copy rng on sampler copy and re-accept drafted tokens on replay\n\n* Fix draft sampler sharing the target's rng stream\n\n* Simplify the rejection sampler's inputs and move replay to the server\n\n* Truncate the draft candidates along with the draft\n\n---------\n\nCo-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>\nCo-authored-by: Pranesh Gonegandla ","verified_at":"2026-10-02T14:52:55Z"}},"url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a","html_url":"https://github.com/ggml-org/llama.cpp/commit/1fb7ef3e3327f18f1e99d294115b8493d54e196a","comments_url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a/comments","author":{"login":"praneshgo","id":227579474,"node_id":"U_kgDODZCWUg","avatar_url":"https://avatars.githubusercontent.com/u/227579474?v=4","gravatar_id":"","url":"https://api.github.com/users/praneshgo","html_url":"https://github.com/praneshgo","followers_url":"https://api.github.com/users/praneshgo/followers","following_url":"https://api.github.com/users/praneshgo/following{/other_user}","gists_url":"https://api.github.com/users/praneshgo/gists{/gist_id}","starred_url":"https://api.github.com/users/praneshgo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/praneshgo/subscriptions","organizations_url":"https://api.github.com/users/praneshgo/orgs","repos_url":"https://api.github.com/users/praneshgo/repos","events_url":"https://api.github.com/users/praneshgo/events{/privacy}","received_events_url":"https://api.github.com/users/praneshgo/received_events","type":"User","user_view_type":"public","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","user_view_type":"public","site_admin":false},"parents":[{"sha":"134b2bb756c984d4fd2cabc26335f8de56b12ee1","url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/134b2bb756c984d4fd2cabc26335f8de56b12ee1","html_url":"https://github.com/ggml-org/llama.cpp/commit/134b2bb756c984d4fd2cabc26335f8de56b12ee1"}]},"status":"ahead","ahead_by":3,"behind_by":0,"total_commits":3,"commits":[{"sha":"4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe","node_id":"C_kwDOJH_K4NoAKDRlYmRmMmM3NGFjY2UzMDg4M2Q4ZTM0YjdjNzBiM2ViODE0NmYyZmU","commit":{"author":{"name":"Sigbjørn Skjæret","email":"sigbjorn.skjaeret@huggingface.co","date":"2026-10-02T15:31:19Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2026-10-02T15:31:19Z"},"message":"ci : use t4-medium for cuda jobs (#29842)\n\n[no ci]","tree":{"sha":"713045fb92fd083e95732cfc355e6138db3c4b35","url":"https://api.github.com/repos/ggml-org/llama.cpp/git/trees/713045fb92fd083e95732cfc355e6138db3c4b35"},"url":"https://api.github.com/repos/ggml-org/llama.cpp/git/commits/4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsFcBAABCAAQBQJqv85HCRC1aQ7uu5UhlAAABgQQABe03PXD4dv5cRlWSSr9+qqP\nqp9lxcggcyiPkhanfhs24HeVpfHRK3u7XYoVbL3qN8tCfzbPfGylI4LmeZRPUxe2\nKSNRyLl27uz8rmCBOkKTUfNnwjGUs8lR9iUI0cwDHwj1gZRpJAW4poHVZ40v0Lpf\n3q8zJE9gPI56whPP/FgtQZpCERBIFYLnDyjFpgTczcwz2AaEjTo+owbdztxnsDvz\naBNgcaK5iCG8toxQjIoUycRoeVx4LTicZ0p3iSsZT14drjww74FpnDD/NeIqs7an\n+3Lv9M2ydNOHu4pMOEE5lgd1+hBoCeejHUxrGV8MtOJNyAA1C75P6546zQlFdaPc\n+Duzpg8IYxNOCK3Nbm+rdKRhnOWut3412InEcUNrwRhN9QC15UoTCbe8UvQKC/OV\nWmNsKZGcnSL/bMsy+vIVjkTy2UomaCs5pmRz0iJunZAkfEFadl19oQXjxCnhL+kL\nEt1//LRgG+T3j0NxjgdpPITI2dA0UHRFnFLLrKNN1SeyBCHnc3psMT1kKbNG6vjn\ncQQF4+sdPX4mDM0TZl/WdFitomFRr/seFq6VD4Jk5AuOpvyhq7TBcTCivvpcopy3\nEKaKH0ZEpFMPpiU0cZ8oyQewrtBGWUfXTRNjVa3sdBK6xBf/Qvqo+TjEKk7H6cTL\nqCWUL3rKrJm68IRv9Y0L\n=JRT7\n-----END PGP SIGNATURE-----\n","payload":"tree 713045fb92fd083e95732cfc355e6138db3c4b35\nparent 1fb7ef3e3327f18f1e99d294115b8493d54e196a\nauthor Sigbjørn Skjæret 1790955079 +0200\ncommitter GitHub 1790955079 +0200\n\nci : use t4-medium for cuda jobs (#29842)\n\n[no ci]","verified_at":"2026-10-02T15:31:20Z"}},"url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe","html_url":"https://github.com/ggml-org/llama.cpp/commit/4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe","comments_url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe/comments","author":{"login":"CISC","id":1629204,"node_id":"MDQ6VXNlcjE2MjkyMDQ=","avatar_url":"https://avatars.githubusercontent.com/u/1629204?v=4","gravatar_id":"","url":"https://api.github.com/users/CISC","html_url":"https://github.com/CISC","followers_url":"https://api.github.com/users/CISC/followers","following_url":"https://api.github.com/users/CISC/following{/other_user}","gists_url":"https://api.github.com/users/CISC/gists{/gist_id}","starred_url":"https://api.github.com/users/CISC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/CISC/subscriptions","organizations_url":"https://api.github.com/users/CISC/orgs","repos_url":"https://api.github.com/users/CISC/repos","events_url":"https://api.github.com/users/CISC/events{/privacy}","received_events_url":"https://api.github.com/users/CISC/received_events","type":"User","user_view_type":"public","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","user_view_type":"public","site_admin":false},"parents":[{"sha":"1fb7ef3e3327f18f1e99d294115b8493d54e196a","url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/1fb7ef3e3327f18f1e99d294115b8493d54e196a","html_url":"https://github.com/ggml-org/llama.cpp/commit/1fb7ef3e3327f18f1e99d294115b8493d54e196a"}]},{"sha":"bed0a856606ee4a24a164066f73d2379447033f5","node_id":"C_kwDOJH_K4NoAKGJlZDBhODU2NjA2ZWU0YTI0YTE2NDA2NmY3M2QyMzc5NDQ3MDMzZjU","commit":{"author":{"name":"Aman Gupta","email":"amangupta052@gmail.com","date":"2026-10-02T18:26:27Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2026-10-02T18:26:27Z"},"message":"CUDA: fuse shared experts into MMVQ (#29184)\n\n* CUDA: fuse shared experts into MMVQ\n\n* check if buffer is null\n\n* move stride_col_dst to fusion args","tree":{"sha":"852a43c30d6674c377d58a8368dede817edb9021","url":"https://api.github.com/repos/ggml-org/llama.cpp/git/trees/852a43c30d6674c377d58a8368dede817edb9021"},"url":"https://api.github.com/repos/ggml-org/llama.cpp/git/commits/bed0a856606ee4a24a164066f73d2379447033f5","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsFcBAABCAAQBQJqv/dTCRC1aQ7uu5UhlAAAE1sQACftlFsCabB60TuvNvSGe79y\nNYIRmCaxs8DrMhrTe4dnT3m77cMaKGRdZK3bGdDILAfADHvmc63CHYtNqXU5Wfvy\nB/1/REZdv+YpdACJXhDvDBwFyUC5jtQGJ4fmlImfaar1Q4gysTi4pnTQuBWqG/Pi\n3hQxL42tNKKvao1u8ksq5I2CpMLxE/sWwEDwsoh50SdTe2BmHQhpHqo0GrI/CIOY\nUa5bVYPgtm+l7rcHWIB8pQiK21E2Ch3HqeInJ6ejn9BMsLM1JcRIg6D3bExdSaQq\nTdvGLYMAaFrPEqMnLgmBGD4fc9C6vKpUAm7k/mn2KEAO5KKlQzTmA2VkZX0rG9oY\npbNf7/E4x9dlvPIuB3W6faigSPf5ZzTQ0F0BEBIhi7Lh7qSJ/S7Ofi1n2CsV8mEK\nXsvyOksdtSzt8xNdSlbJtNprHkVu3GIirCKTBXEJJ485fIUpGftay4/1vLbBeGhx\nyww1At2i9uCYl4dXpPrRBtsSmfGC/FRfW+bEDnaBGriwvk1y7unUQH0T5NPb40dw\nypow+6/AirC2v8Zbx39ktbZ0FaUQs0G5xW8VsCo4cynjxxy0QyP10v3A/svvzqIs\n+/+w48QB+RmVVxBI8z7SIO85CXfUr+aZx808UBIsgjJcHt8A0ZANM7gSKZ0JHOGv\nb/aqn37UZ9bETOb6JQr/\n=imTo\n-----END PGP SIGNATURE-----\n","payload":"tree 852a43c30d6674c377d58a8368dede817edb9021\nparent 4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe\nauthor Aman Gupta 1790965587 +0800\ncommitter GitHub 1790965587 +0300\n\nCUDA: fuse shared experts into MMVQ (#29184)\n\n* CUDA: fuse shared experts into MMVQ\n\n* check if buffer is null\n\n* move stride_col_dst to fusion args","verified_at":"2026-10-02T18:26:27Z"}},"url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/bed0a856606ee4a24a164066f73d2379447033f5","html_url":"https://github.com/ggml-org/llama.cpp/commit/bed0a856606ee4a24a164066f73d2379447033f5","comments_url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/bed0a856606ee4a24a164066f73d2379447033f5/comments","author":{"login":"am17an","id":2929750,"node_id":"MDQ6VXNlcjI5Mjk3NTA=","avatar_url":"https://avatars.githubusercontent.com/u/2929750?v=4","gravatar_id":"","url":"https://api.github.com/users/am17an","html_url":"https://github.com/am17an","followers_url":"https://api.github.com/users/am17an/followers","following_url":"https://api.github.com/users/am17an/following{/other_user}","gists_url":"https://api.github.com/users/am17an/gists{/gist_id}","starred_url":"https://api.github.com/users/am17an/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/am17an/subscriptions","organizations_url":"https://api.github.com/users/am17an/orgs","repos_url":"https://api.github.com/users/am17an/repos","events_url":"https://api.github.com/users/am17an/events{/privacy}","received_events_url":"https://api.github.com/users/am17an/received_events","type":"User","user_view_type":"public","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","user_view_type":"public","site_admin":false},"parents":[{"sha":"4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe","url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe","html_url":"https://github.com/ggml-org/llama.cpp/commit/4ebdf2c74acce30883d8e34b7c70b3eb8146f2fe"}]},{"sha":"99b95488cac0f00ce3f05af113a8c1e287753f87","node_id":"C_kwDOJH_K4NoAKDk5Yjk1NDg4Y2FjMGYwMGNlM2YwNWFmMTEzYThjMWUyODc3NTNmODc","commit":{"author":{"name":"Xuan-Son Nguyen","email":"son@huggingface.co","date":"2026-10-03T00:50:48Z"},"committer":{"name":"GitHub","email":"noreply@github.com","date":"2026-10-03T00:50:48Z"},"message":"model: add support for clef decision model (text-only) (#29831)\n\n* init support for clef (text only)\n\n* more static graph\n\n* clean up\n\n* nits\n\n* nits 2\n\n* Update gguf-py/gguf/constants.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n---------\n\nCo-authored-by: Sigbjørn Skjæret ","tree":{"sha":"c6935c7fbae85482e5674b3e99c7ae7004a9841f","url":"https://api.github.com/repos/ggml-org/llama.cpp/git/trees/c6935c7fbae85482e5674b3e99c7ae7004a9841f"},"url":"https://api.github.com/repos/ggml-org/llama.cpp/git/commits/99b95488cac0f00ce3f05af113a8c1e287753f87","comment_count":0,"verification":{"verified":true,"reason":"valid","signature":"-----BEGIN PGP SIGNATURE-----\n\nwsFcBAABCAAQBQJqwFFoCRC1aQ7uu5UhlAAA1oQQAJMMxxFjRVnmdT53KDLxNe0i\n1pjpRkrmiVTYCfUTbiKgmYMSyqahA6ufmIBg/BTmIRBvGcnw4uiPupca1fWK5I5N\n8iRH1+1p97kG+EbPg1O/seU6yr132ITLQhLb1OvG1I0MwhvN+laAdcRxDZ4k36/I\nnRGpDuFh0aWhJgGgK1Tuth2IoMINSUkGRxQ6svnR9Gl45piD/LL6cQpP6RPrM0y6\nOl5E+PZ4a9JMoFVny8qMbvnmKC5NoWl1cnKDZxJGnCdydTj2IyKxfe32Rn/IhcCT\nonIaERMeP0ZK+xRW81VJoztduTizYNZnpvb6cHnSIhNrYwolOc5eBAXlsDNGjrWs\nBdUzDqbTUO3qpcij/xnBeqHbouJDBSGIOKOK9z1TuoK8ZnOE2mJVLQ+pth9u93TZ\nf/d7X05kZn8DOMNxyOeXi98ANMhVrLSVWaCP0rX8QMvQ1XG4S39CXVgPdGXxlSM6\nFAvO0wZMZU1TOHTI6T41nFQtsBAKUAin8/AAM5jHlmL+C989MhKOGSOwARveO3yX\nCWKaXhx7Z+BqVc3rryWTx84d+J2gnGlRH57qCWhC+9bN1vvXbB0dks6YZMiWHFxg\nvFfpk462w6b2aozu+kcgP036kws+9BtJfUpd4YZcCwV9Z5i+/12JvSNF5kyOYaya\nTBpAadm4iI8RZullzHYX\n=NX+f\n-----END PGP SIGNATURE-----\n","payload":"tree c6935c7fbae85482e5674b3e99c7ae7004a9841f\nparent bed0a856606ee4a24a164066f73d2379447033f5\nauthor Xuan-Son Nguyen 1790988648 +0200\ncommitter GitHub 1790988648 +0200\n\nmodel: add support for clef decision model (text-only) (#29831)\n\n* init support for clef (text only)\n\n* more static graph\n\n* clean up\n\n* nits\n\n* nits 2\n\n* Update gguf-py/gguf/constants.py\n\nCo-authored-by: Sigbjørn Skjæret \n\n---------\n\nCo-authored-by: Sigbjørn Skjæret ","verified_at":"2026-10-03T00:50:49Z"}},"url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/99b95488cac0f00ce3f05af113a8c1e287753f87","html_url":"https://github.com/ggml-org/llama.cpp/commit/99b95488cac0f00ce3f05af113a8c1e287753f87","comments_url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/99b95488cac0f00ce3f05af113a8c1e287753f87/comments","author":{"login":"ngxson","id":7702203,"node_id":"MDQ6VXNlcjc3MDIyMDM=","avatar_url":"https://avatars.githubusercontent.com/u/7702203?v=4","gravatar_id":"","url":"https://api.github.com/users/ngxson","html_url":"https://github.com/ngxson","followers_url":"https://api.github.com/users/ngxson/followers","following_url":"https://api.github.com/users/ngxson/following{/other_user}","gists_url":"https://api.github.com/users/ngxson/gists{/gist_id}","starred_url":"https://api.github.com/users/ngxson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ngxson/subscriptions","organizations_url":"https://api.github.com/users/ngxson/orgs","repos_url":"https://api.github.com/users/ngxson/repos","events_url":"https://api.github.com/users/ngxson/events{/privacy}","received_events_url":"https://api.github.com/users/ngxson/received_events","type":"User","user_view_type":"public","site_admin":false},"committer":{"login":"web-flow","id":19864447,"node_id":"MDQ6VXNlcjE5ODY0NDQ3","avatar_url":"https://avatars.githubusercontent.com/u/19864447?v=4","gravatar_id":"","url":"https://api.github.com/users/web-flow","html_url":"https://github.com/web-flow","followers_url":"https://api.github.com/users/web-flow/followers","following_url":"https://api.github.com/users/web-flow/following{/other_user}","gists_url":"https://api.github.com/users/web-flow/gists{/gist_id}","starred_url":"https://api.github.com/users/web-flow/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/web-flow/subscriptions","organizations_url":"https://api.github.com/users/web-flow/orgs","repos_url":"https://api.github.com/users/web-flow/repos","events_url":"https://api.github.com/users/web-flow/events{/privacy}","received_events_url":"https://api.github.com/users/web-flow/received_events","type":"User","user_view_type":"public","site_admin":false},"parents":[{"sha":"bed0a856606ee4a24a164066f73d2379447033f5","url":"https://api.github.com/repos/ggml-org/llama.cpp/commits/bed0a856606ee4a24a164066f73d2379447033f5","html_url":"https://github.com/ggml-org/llama.cpp/commit/bed0a856606ee4a24a164066f73d2379447033f5"}]}],"files":[{"sha":"24f830a9f03735c7f11f8f69d52b04cf9db98fd8","filename":".github/workflows/ci-self-hosted-cuda.yml","status":"modified","additions":1,"deletions":1,"changes":2,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/.github%2Fworkflows%2Fci-self-hosted-cuda.yml","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/.github%2Fworkflows%2Fci-self-hosted-cuda.yml","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/.github%2Fworkflows%2Fci-self-hosted-cuda.yml?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -45,7 +45,7 @@ env:\n \n jobs:\n gpu-cuda:\n- runs-on: \"hf-jobs-t4-small:cuda13\"\n+ runs-on: \"hf-jobs-t4-medium:cuda13\"\n \n steps:\n - name: Clone"},{"sha":"614fd32f5bb788af9a76b3999542191926d4b647","filename":".github/workflows/server-self-hosted.yml","status":"modified","additions":1,"deletions":1,"changes":2,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/.github%2Fworkflows%2Fserver-self-hosted.yml","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/.github%2Fworkflows%2Fserver-self-hosted.yml","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/.github%2Fworkflows%2Fserver-self-hosted.yml?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -102,7 +102,7 @@ jobs:\n PYTEST_WORKERS=1 ./tests.sh\n \n server-cuda:\n- runs-on: \"hf-jobs-t4-small:cuda13\"\n+ runs-on: \"hf-jobs-t4-medium:cuda13\"\n \n steps:\n - name: Clone"},{"sha":"0269bc895c4dcc67f51c3cd370f8264d00164267","filename":"common/common.cpp","status":"modified","additions":9,"deletions":2,"changes":11,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/common%2Fcommon.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/common%2Fcommon.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/common%2Fcommon.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -3,6 +3,9 @@\n \n #include \"build-info.h\"\n #include \"common.h\"\n+\n+#include \"../src/llama-ext.h\"\n+\n #include \"fit.h\"\n #include \"log.h\"\n #include \"llama.h\"\n@@ -1186,6 +1189,7 @@ static const std::map COMMON_DECISION_TYPE_NA\n { COMMON_DECISION_TYPE_KEV, \"kev\" },\n { COMMON_DECISION_TYPE_NIMBLE, \"nimble\" },\n { COMMON_DECISION_TYPE_LAYA, \"laya\" },\n+ { COMMON_DECISION_TYPE_CLEF, \"clef\" },\n };\n \n static common_decision_type common_decision_type_from_string(const std::string & str) {\n@@ -1261,10 +1265,10 @@ common_init_result::common_init_result(common_params & params, bool model_only)\n \n const llama_vocab * vocab = llama_model_get_vocab(model);\n \n- // this decision model returns a score for each token via the embeddings output\n+ // these decision models return a score for each token via the embeddings output\n // TODO: maybe improve this in the future\n const auto decision_type = common_get_decision_type(model);\n- if (decision_type == COMMON_DECISION_TYPE_LAYA || decision_type == COMMON_DECISION_TYPE_KEV) {\n+ if (decision_type == COMMON_DECISION_TYPE_LAYA || decision_type == COMMON_DECISION_TYPE_KEV || decision_type == COMMON_DECISION_TYPE_CLEF) {\n params.embedding = true;\n params.pooling_type = LLAMA_POOLING_TYPE_NONE;\n \n@@ -2210,6 +2214,9 @@ llama_batch_ext * common_batch::get_sub_batch(int32_t off, int32_t n) {\n if (t.output) {\n llama_batch_ext_set_output_logits(res, idx, true);\n }\n+ if (t.decision_order != 0) {\n+ llama_batch_ext_set_decision_order(res, idx, (llama_decision_order) t.decision_order);\n+ }\n }\n \n return res;"},{"sha":"27be5c0586cd58d66b2a528b62b9b58de1ec6b07","filename":"common/common.h","status":"modified","additions":2,"deletions":0,"changes":2,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/common%2Fcommon.h","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/common%2Fcommon.h","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/common%2Fcommon.h?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -960,6 +960,7 @@ enum common_decision_type {\n COMMON_DECISION_TYPE_KEV, // dot product of the hidden states of the last token and of one end token per option\n COMMON_DECISION_TYPE_NIMBLE, // same as openjev, the prompt lists all the questions of the request\n COMMON_DECISION_TYPE_LAYA, // score of one marker token per option, read from the embeddings output\n+ COMMON_DECISION_TYPE_CLEF, // all questions in one prompt, score of option i read from the embeddings output at row i\n COMMON_DECISION_TYPE_UNKNOWN, // a decision model of a type that is not supported\n };\n \n@@ -1062,6 +1063,7 @@ struct common_batch {\n bool output;\n llama_embd embd; // non-owning view of the data passed to add_embd()/set_embd(), data == NULL if none\n std::vector seq_ids_extra; // see add_seq()\n+ int32_t decision_order = 0; // see llama_batch_ext_set_decision_order()\n };\n \n std::vector tokens; // mirror of the entries, tokens[i] describes batch index i"},{"sha":"6d8ae9c1a5a08438373efeef978f5e05aa96bcf0","filename":"conversion/__init__.py","status":"modified","additions":2,"deletions":0,"changes":2,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/conversion%2F__init__.py","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/conversion%2F__init__.py","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/conversion%2F__init__.py?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -42,6 +42,7 @@\n \"ChameleonForConditionalGeneration\": \"chameleon\",\n \"ChatGLMForConditionalGeneration\": \"chatglm\",\n \"ChatGLMModel\": \"chatglm\",\n+ \"ClefModel\": \"clef\",\n \"CodeShellForCausalLM\": \"codeshell\",\n \"CogVLMForCausalLM\": \"cogvlm\",\n \"Cohere2MoeForCausalLM\": \"command_r\",\n@@ -296,6 +297,7 @@\n \n MMPROJ_MODEL_MAP: dict[str, str] = {\n \"AudioFlamingo3ForConditionalGeneration\": \"ultravox\",\n+ \"ClefModel\": \"clef\",\n \"CogVLMForCausalLM\": \"cogvlm\",\n \"DeepseekOCR2ForCausalLM\": \"deepseek\",\n \"DeepseekOCRForCausalLM\": \"deepseek\","},{"sha":"a6f7243945dbb693d13165dacd323cae8d77bcb9","filename":"conversion/clef.py","status":"added","additions":150,"deletions":0,"changes":150,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/conversion%2Fclef.py","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/conversion%2Fclef.py","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/conversion%2Fclef.py?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -0,0 +1,150 @@\n+from __future__ import annotations\n+\n+import json\n+import math\n+\n+from pathlib import Path\n+from typing import Any, Iterable, Iterator, TYPE_CHECKING\n+\n+import torch\n+\n+if TYPE_CHECKING:\n+ from torch import Tensor\n+\n+from .base import MmprojModel, ModelBase, gguf, logger\n+from .qwen import Qwen3_5TextModel\n+\n+\n+def _is_clef_checkpoint(dir_model: Path) -> bool:\n+ return (dir_model / \"joint_head_config.json\").is_file() and (dir_model / \"config.json\").is_file()\n+\n+\n+@ModelBase.register_hparams_loader(_is_clef_checkpoint)\n+def _load_clef_hparams(dir_model: Path) -> dict[str, Any]:\n+ logger.info(\"gguf: detected Clef checkpoint\")\n+ hparams = ModelBase.load_hparams(dir_model, False, guess=False)\n+ hparams[\"architectures\"] = [\"ClefModel\"]\n+ with open(dir_model / \"joint_head_config.json\", encoding=\"utf-8\") as f:\n+ hparams[\"decision\"] = json.load(f)\n+ return hparams\n+\n+\n+@ModelBase.register(\"ClefModel\")\n+class ClefModel(Qwen3_5TextModel):\n+ model_arch = gguf.MODEL_ARCH.CLEF\n+ no_mtp = True # the checkpoint has no MTP head\n+\n+ # prompt follows joint_schema_model.py of the model repo\n+ _SYSTEM_PROMPT = (\n+ \"Read the complete state and schema. Decide every field jointly. Each answer \"\n+ \"must be exactly one of that field's allowed options.\"\n+ )\n+ # torch.nn.LayerNorm default, used by the head\n+ _HEAD_NORM_EPS = 1e-5\n+\n+ def __init__(self, *args, **kwargs):\n+ super().__init__(*args, **kwargs)\n+ head = self.hparams[\"decision\"]\n+ self._n_routing = head[\"routing_layers\"]\n+ # the head blocks are named dec.blk.N, routing blocks first\n+ self.tensor_map = gguf.get_tensor_name_map(self.model_arch, max(self.block_count, self._n_routing + head[\"layers\"]))\n+ self._scales: dict[str, float] = {}\n+\n+ def set_vocab(self):\n+ super().set_vocab()\n+ self.gguf_writer.add_chat_template([{\"name\": \"systemone\", \"template\": self._systemone_template()}])\n+\n+ @classmethod\n+ def _systemone_template(cls) -> str:\n+ def text(value: str) -> str:\n+ return \"{{ \" + json.dumps(value) + \" }}\"\n+\n+ def render(name: str) -> str:\n+ # strings are used as is, other values are compact JSON\n+ return \"{{ \" + name + \" if \" + name + \" is string else \" + name + \" | tojson(separators=[',', ':']) }}\"\n+\n+ # the pieces of the prompt are tokenized one by one, the server gives the text that separates them (sep)\n+ # and the text that starts the span of a question or of an option (mark_question, mark_option)\n+ # the keys of JSON objects are given in sorted order\n+ option = (\n+ \"{% set d = o.description %}\"\n+ \"{% if q.type == 'noul' and d is none %}\"\n+ \"{% set d = 'The proposition is true or the answer is yes.' if o.key == 'true' else 'The proposition is false or the answer is no.' %}\"\n+ \"{% endif %}\"\n+ \"{{ ({'option_id': o.key} if d is none else {'description': d, 'option_id': o.key}) | tojson(separators=[',', ':']) }}\"\n+ )\n+ return (\n+ text(f\"<|im_start|>system\\n{cls._SYSTEM_PROMPT}<|im_end|>\\n<|im_start|>user\\nSTATE:\\n\")\n+ + \"{{ sep }}\" + render(\"state\")\n+ + \"{{ sep }}\" + text(\"\\n\\nSCHEMA FIELDS:\\n\")\n+ + \"{% for q in questions %}\"\n+ + \"{{ sep }}\" + text(\"\\nFIELD \") + \"{{ loop.index }}\" + text(\"\\nID: \") + \"{{ q.id }}\"\n+ + text(\"\\nTYPE: \") + \"{{ q.type }}\" + text(\"\\nINSTRUCTION: \")\n+ + \"{{ sep }}{{ mark_question }}\" + render(\"q.instructions\")\n+ + \"{{ sep }}\" + text(\"\\nALLOWED OPTIONS:\\n\")\n+ + \"{% for o in q.options %}\"\n+ + \"{{ sep }}\" + text(\"OPTION \") + \"{{ loop.index }}\" + text(\": \")\n+ + \"{{ sep }}{{ mark_option }}\" + option\n+ + \"{{ sep }}\" + text(\"\\n\")\n+ + \"{% endfor %}\"\n+ + \"{{ sep }}\" + text(\"END FIELD\\n\")\n+ + \"{% endfor %}\"\n+ + \"{{ sep }}\" + text(\"\\n<|im_end|>\\n<|im_start|>assistant\\n\\n\\n\\n\\nJOINT SCHEMA DECISIONS:\")\n+ )\n+\n+ def set_gguf_parameters(self):\n+ super().set_gguf_parameters()\n+ head = self.hparams[\"decision\"]\n+ self.gguf_writer.add_decision_type(gguf.DecisionType.CLEF)\n+ self.gguf_writer.add_decision_routing_block_count(head[\"routing_layers\"])\n+ self.gguf_writer.add_decision_block_count(head[\"layers\"])\n+ self.gguf_writer.add_decision_head_count(head[\"heads\"])\n+ self.gguf_writer.add_layer_norm_eps(self._HEAD_NORM_EPS)\n+\n+ def get_tensors(self) -> Iterator[tuple[str, Tensor]]:\n+ yield from super().get_tensors()\n+ from safetensors.torch import load_file\n+ for name, data in load_file(self.dir_model / \"joint_head.safetensors\").items():\n+ yield \"joint_head.\" + name, data\n+\n+ def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:\n+ if not name.startswith(\"joint_head.\"):\n+ yield from super().modify_tensors(data_torch, name, bid)\n+ return\n+\n+ parts = name.split(\".\")\n+\n+ # learned scalars, stored as the values used at inference\n+ if len(parts) == 2 and data_torch.ndim == 0:\n+ value = float(data_torch)\n+ if parts[1] == \"residual_gate\":\n+ self._scales[parts[1]] = 1.0 / (1.0 + math.exp(-value))\n+ else:\n+ self._scales[parts[1]] = math.exp(min(value, math.log(100.0)))\n+ if len(self._scales) == 3:\n+ scales = [self._scales[k] for k in (\"prior_logit_scale\", \"joint_logit_scale\", \"residual_gate\")]\n+ yield self.format_tensor_name(gguf.MODEL_TENSOR.DECISION_SCALES, suffix=\"\"), torch.tensor(scales, dtype=torch.float32)\n+ return\n+\n+ # routing blocks come first\n+ if parts[1] == \"layers\":\n+ parts[2] = str(int(parts[2]) + self._n_routing)\n+ name = \".\".join(parts)\n+\n+ # nn.MultiheadAttention keeps q, k, v in one tensor\n+ for suffix in (\"weight\", \"bias\"):\n+ if name.endswith(\".in_proj_\" + suffix):\n+ prefix = name[:-len(\"in_proj_\" + suffix)]\n+ for x, data in zip(\"qkv\", data_torch.chunk(3, dim=0)):\n+ yield self.map_tensor_name(prefix + x + \".\" + suffix), data\n+ return\n+\n+ yield self.map_tensor_name(name), data_torch\n+\n+\n+@ModelBase.register(\"ClefModel\")\n+class ClefVisionModel(MmprojModel):\n+ def __init__(self, *args, **kwargs):\n+ del args, kwargs\n+ raise NotImplementedError(\n+ \"multimodal input is not supported yet for Clef, requires https://github.com/ggml-org/llama.cpp/pull/29622 to be merged first\")"},{"sha":"928607c4103be3b1c7082d59f60d8f3432fab4f2","filename":"ggml/src/ggml-cuda/common.cuh","status":"modified","additions":7,"deletions":0,"changes":7,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/ggml%2Fsrc%2Fggml-cuda%2Fcommon.cuh","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/ggml%2Fsrc%2Fggml-cuda%2Fcommon.cuh","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/ggml%2Fsrc%2Fggml-cuda%2Fcommon.cuh?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -1571,6 +1571,9 @@ struct ggml_cuda_mm_fusion_args_host {\n const ggml_tensor * gate_scale = nullptr;\n ggml_glu_op glu_op;\n float glu_limit = 0.0f;\n+ const ggml_tensor * shared_up = nullptr;\n+ const ggml_tensor * shared_gate = nullptr;\n+ ggml_tensor * shared_dst = nullptr;\n };\n struct ggml_cuda_mm_fusion_args_device {\n const void * x_bias = nullptr;\n@@ -1580,6 +1583,10 @@ struct ggml_cuda_mm_fusion_args_device {\n const void * gate_scale = nullptr;\n ggml_glu_op glu_op;\n float glu_limit = 0.0f;\n+ const void * shared_up = nullptr;\n+ const void * shared_gate = nullptr;\n+ float * shared_dst = nullptr;\n+ uint32_t shared_stride_col_dst = 0;\n };\n \n struct ggml_cuda_kernel_launch_params {"},{"sha":"303c9f463dcf8c2fb41641216ec39b80cde28895","filename":"ggml/src/ggml-cuda/ggml-cuda.cu","status":"modified","additions":89,"deletions":0,"changes":89,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/ggml%2Fsrc%2Fggml-cuda%2Fggml-cuda.cu","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/ggml%2Fsrc%2Fggml-cuda%2Fggml-cuda.cu","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/ggml%2Fsrc%2Fggml-cuda%2Fggml-cuda.cu?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -1823,6 +1823,55 @@ static bool ggml_cuda_should_fuse_mul_mat_vec_q(const ggml_tensor * tensor) {\n return use_mul_mat_vec_q;\n }\n \n+static bool ggml_cuda_match_shared_expert(const ggml_cgraph * graph, int routed_idx, int shared_idx) {\n+ if (routed_idx + 2 >= graph->n_nodes || shared_idx + 2 >= graph->n_nodes || shared_idx < routed_idx + 3) {\n+ return false;\n+ }\n+ const int nodes[] = { routed_idx, routed_idx + 1, routed_idx + 2, shared_idx, shared_idx + 1, shared_idx + 2 };\n+ const ggml_op ops[] = { GGML_OP_MUL_MAT_ID, GGML_OP_MUL_MAT_ID, GGML_OP_GLU,\n+ GGML_OP_MUL_MAT, GGML_OP_MUL_MAT, GGML_OP_GLU };\n+ const int outputs[] = { routed_idx + 2, shared_idx + 2 };\n+ if (!ggml_can_fuse_subgraph_ext(graph, nodes, 6, ops, outputs, 2)) {\n+ return false;\n+ }\n+\n+ const ggml_tensor * routed = graph->nodes[routed_idx + 2];\n+ const ggml_tensor * shared = graph->nodes[shared_idx + 2];\n+ const ggml_tensor * gate = routed->src[0];\n+ const ggml_tensor * up = routed->src[1];\n+ const ggml_tensor * shared_gate = shared->src[0];\n+ const ggml_tensor * shared_up = shared->src[1];\n+ const auto is_pair = [&](const ggml_tensor * a, const ggml_tensor * b, int idx) {\n+ return (a == graph->nodes[idx] && b == graph->nodes[idx + 1]) ||\n+ (b == graph->nodes[idx] && a == graph->nodes[idx + 1]);\n+ };\n+ if (!is_pair(gate, up, routed_idx) || !is_pair(shared_gate, shared_up, shared_idx) ||\n+ !ggml_cuda_should_fuse_mul_mat(up, gate, routed) ||\n+ !ggml_cuda_should_fuse_mul_mat(shared_up, shared_gate, shared) ||\n+ !up->src[0]->buffer ||\n+ !ggml_cuda_should_fuse_mul_mat_vec_q(up)) {\n+ return false;\n+ }\n+ const ggml_tensor * input = up->src[1];\n+ const ggml_tensor * weight = up->src[0];\n+ const ggml_tensor * shared_weight = shared_up->src[0];\n+ if (input->op != GGML_OP_RESHAPE || input->src[0] != shared_up->src[1] ||\n+ input->ne[1] != 1 || input->ne[3] != 1 || !ggml_is_contiguous(input) ||\n+ !ggml_is_contiguous(shared_up->src[1]) || !ggml_is_matrix(shared_up->src[1]) ||\n+ weight->type != shared_weight->type || weight->ne[0] != shared_weight->ne[0] ||\n+ weight->ne[1] != shared_weight->ne[1] || weight->nb[1] != shared_weight->nb[1] || weight->ne[3] != 1 ||\n+ !ggml_is_matrix(shared_weight) || !ggml_is_contiguous(shared_weight) ||\n+ !ggml_is_contiguous(shared_gate->src[0]) || !ggml_is_contiguous(routed) || !ggml_is_contiguous(shared)) {\n+ return false;\n+ }\n+ if (shared_weight->op != GGML_OP_NONE || shared_gate->src[0]->op != GGML_OP_NONE ||\n+ ggml_get_glu_op(routed) != ggml_get_glu_op(shared) ||\n+ ggml_get_op_params_f32(routed, 3) != ggml_get_op_params_f32(shared, 3)) {\n+ return false;\n+ }\n+ return true;\n+}\n+\n static void ggml_cuda_mul_mat(ggml_backend_cuda_context & ctx, const ggml_tensor * src0, const ggml_tensor * src1, ggml_tensor * dst) {\n GGML_TENSOR_BINARY_OP_LOCALS\n \n@@ -3459,6 +3508,25 @@ static int ggml_cuda_try_fuse(ggml_backend_cuda_context * cuda_ctx, ggml_cgraph\n \n ggml_tensor * node = cgraph->nodes[i];\n \n+ if (node->op == GGML_OP_MUL_MAT_ID && cuda_ctx->stream_context().concurrent_events.empty() &&\n+ ggml_cuda_match_shared_expert(cgraph, i, i + 3)) {\n+ const int outputs[] = { i + 2, i + 5 };\n+ if (ggml_cuda_check_fusion_memory_ranges(cgraph, i, 6, outputs, 2)) {\n+ ggml_tensor * routed = cgraph->nodes[i + 2];\n+ ggml_tensor * shared = cgraph->nodes[i + 5];\n+ const ggml_tensor * up = routed->src[1];\n+ ggml_cuda_mm_fusion_args_host fusion{};\n+ fusion.gate = routed->src[0]->src[0];\n+ fusion.glu_op = ggml_get_glu_op(routed);\n+ fusion.glu_limit = ggml_get_op_params_f32(routed, 3);\n+ fusion.shared_up = shared->src[1]->src[0];\n+ fusion.shared_gate = shared->src[0]->src[0];\n+ fusion.shared_dst = shared;\n+ ggml_cuda_mul_mat_vec_q(*cuda_ctx, up->src[0], up->src[1], up->src[2], routed, &fusion);\n+ return 5;\n+ }\n+ }\n+\n if (node->op == GGML_OP_MUL) {\n ggml_cuda_moe_weighted_reduction_match match;\n if (ggml_cuda_match_moe_weighted_reduction(cgraph, i, match)) {\n@@ -4549,6 +4617,27 @@ static void ggml_backend_cuda_graph_optimize(ggml_backend_t backend, ggml_cgraph\n if (!disable_fusion) {\n // add alloc deps for performance positive fusions. This may increase the overall compute buffer size.\n // TODO: consolidate fusion paths in graph_optimize and graph_compute\n+ ggml_cuda_set_device(cuda_ctx->device);\n+ for (int i = 0; i + 5 < cgraph->n_nodes; ++i) {\n+ if (cgraph->nodes[i]->op != GGML_OP_MUL_MAT_ID) {\n+ continue;\n+ }\n+ for (int j = i + 3; j + 2 < cgraph->n_nodes; ++j) {\n+ if (cgraph->nodes[j]->op == GGML_OP_MUL_MAT_ID && cgraph->nodes[j + 1]->op == GGML_OP_MUL_MAT_ID) {\n+ break;\n+ }\n+ if (cgraph->nodes[j]->op != GGML_OP_MUL_MAT || !ggml_cuda_match_shared_expert(cgraph, i, j)) {\n+ continue;\n+ }\n+ // Group both outputs before allocation so the shared result cannot alias intervening nodes.\n+ std::rotate(cgraph->nodes + i + 3, cgraph->nodes + j, cgraph->nodes + j + 3);\n+ ggml_tensor * up = cgraph->nodes[i + 2]->src[1];\n+ params->add_alloc_dep(params->user_data, up->src[1], cgraph->nodes[i + 5]);\n+ params->add_alloc_dep(params->user_data, up->src[2], cgraph->nodes[i + 5]);\n+ i += 5;\n+ break;\n+ }\n+ }\n for (int i = 0; i < cgraph->n_nodes; ++i) {\n ggml_cuda_moe_weighted_reduction_match match;\n if (ggml_cuda_match_moe_weighted_reduction(cgraph, i, match)) {"},{"sha":"251577d2fd789e59ea129f4efe7e6954621f4125","filename":"ggml/src/ggml-cuda/mmvq.cu","status":"modified","additions":40,"deletions":10,"changes":50,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/ggml%2Fsrc%2Fggml-cuda%2Fmmvq.cu","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/ggml%2Fsrc%2Fggml-cuda%2Fmmvq.cu","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/ggml%2Fsrc%2Fggml-cuda%2Fmmvq.cu?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -601,7 +601,7 @@ __launch_bounds__(calc_nwarps(type, ncols_dst, get_device_table_id(), small_k, h\n static __global__ void mul_mat_vec_q(\n const void * vx_ptr, const void * vy_ptr, const int32_t * ids_ptr, const ggml_cuda_mm_fusion_args_device fusion, float * dst_ptr,\n const uint32_t ncols_x, const uint3 nchannels_y, const uint32_t stride_row_x, const uint32_t stride_col_y,\n- const uint32_t stride_col_dst, const uint3 channel_ratio, const uint32_t stride_channel_x,\n+ uint32_t stride_col_dst, const uint3 channel_ratio, const uint32_t stride_channel_x,\n const uint32_t stride_channel_y, const uint32_t stride_channel_dst, const uint3 sample_ratio,\n const uint32_t stride_sample_x, const uint32_t stride_sample_y, const uint32_t stride_sample_dst,\n const uint32_t ids_stride) {\n@@ -625,14 +625,20 @@ static __global__ void mul_mat_vec_q(\n const int blocks_per_row_x = ncols_x / qk;\n constexpr int blocks_per_iter = vdr * nwarps*warp_size / qi;\n \n- const uint32_t channel_dst = blockIdx.y;\n+ const bool shared_expert = has_fusion && fusion.shared_up && blockIdx.y == gridDim.y - 1;\n+ const uint32_t channel_dst = shared_expert ? 0 : blockIdx.y;\n+ if (shared_expert) {\n+ vx = fusion.shared_up;\n+ dst = fusion.shared_dst;\n+ stride_col_dst = fusion.shared_stride_col_dst;\n+ }\n \n uint32_t channel_x;\n uint32_t channel_y;\n uint32_t sample_dst;\n \n ggml_cuda_pdl_sync();\n- channel_x = ncols_dst == 1 && ids ? ids[channel_dst] : fastdiv(channel_dst, channel_ratio);\n+ channel_x = shared_expert ? 0 : ncols_dst == 1 && ids ? ids[channel_dst] : fastdiv(channel_dst, channel_ratio);\n channel_y = ncols_dst == 1 && ids ? fastmodulo(channel_dst, nchannels_y) : channel_dst;\n sample_dst = blockIdx.z;\n \n@@ -656,7 +662,7 @@ static __global__ void mul_mat_vec_q(\n use_gate = fusion.gate != nullptr;\n use_bias = fusion.x_bias != nullptr;\n use_gate_bias = fusion.gate_bias != nullptr && use_gate;\n- vgate = fusion.gate;\n+ vgate = shared_expert ? fusion.shared_gate : fusion.gate;\n x_bias = (const float *) fusion.x_bias;\n gate_bias = (const float *) fusion.gate_bias;\n active_glu = fusion.glu_op;\n@@ -854,7 +860,7 @@ static __global__ void mul_mat_vec_q_moe(\n const void * vx_ptr, const void * vy_ptr, const int32_t * ids_ptr, const ggml_cuda_mm_fusion_args_device fusion,\n float * dst_ptr,\n const uint32_t ncols_x, const uint3 nchannels_y, const uint32_t nrows_x,\n- const uint32_t stride_row_x, const uint32_t stride_col_y, const uint32_t stride_col_dst,\n+ const uint32_t stride_row_x, const uint32_t stride_col_y, uint32_t stride_col_dst,\n const uint32_t stride_channel_x, const uint32_t stride_channel_y, const uint32_t stride_channel_dst,\n const uint32_t ncols_dst, const uint32_t ids_stride) {\n const void * GGML_CUDA_RESTRICT vx = vx_ptr;\n@@ -869,6 +875,13 @@ static __global__ void mul_mat_vec_q_moe(\n \n constexpr vec_dot_q_cuda_t vec_dot_q_cuda = get_vec_dot_q_cuda(type);\n \n+ const bool shared_expert = has_fusion && fusion.shared_up && blockIdx.y == gridDim.y - 1;\n+ if (shared_expert) {\n+ vx = fusion.shared_up;\n+ dst = fusion.shared_dst;\n+ stride_col_dst = fusion.shared_stride_col_dst;\n+ }\n+\n // fuse gate, bias, scales, and glu_op into the up projection\n bool use_gate = false;\n const void * vgate = nullptr;\n@@ -881,7 +894,7 @@ static __global__ void mul_mat_vec_q_moe(\n \n if constexpr (has_fusion) {\n use_gate = fusion.gate != nullptr;\n- vgate = fusion.gate;\n+ vgate = shared_expert ? fusion.shared_gate : fusion.gate;\n x_bias = (const float *) fusion.x_bias;\n gate_bias = (const float *) fusion.gate_bias;\n active_glu = fusion.glu_op;\n@@ -897,14 +910,14 @@ static __global__ void mul_mat_vec_q_moe(\n const int blocks_per_row_x = ncols_x / qk;\n constexpr int blocks_per_iter = vdr * warp_size / qi;\n \n- const uint32_t channel_dst = blockIdx.y;\n+ const uint32_t channel_dst = shared_expert ? 0 : blockIdx.y;\n \n if (token_idx >= ncols_dst) {\n return;\n }\n \n ggml_cuda_pdl_sync();\n- const uint32_t channel_x = ids[channel_dst + token_idx * ids_stride];\n+ const uint32_t channel_x = shared_expert ? 0 : ids[channel_dst + token_idx * ids_stride];\n const uint32_t channel_y = fastmodulo(channel_dst, nchannels_y);\n \n const block_q8_1 * y = ((const block_q8_1 *) vy) + channel_y*stride_channel_y + token_idx*stride_col_y;\n@@ -1050,7 +1063,7 @@ static void mul_mat_vec_q_moe_launch(\n \n constexpr int rows_per_block = 2; // 2 gives best perf based on tuning\n const int64_t nblocks_rows = (nrows_x + rows_per_block - 1) / rows_per_block;\n- const dim3 block_nums(nblocks_rows, nchannels_dst);\n+ const dim3 block_nums(nblocks_rows, nchannels_dst + (fusion.shared_up != nullptr));\n const dim3 block_dims(warp_size, ncols_dst);\n const ggml_cuda_kernel_launch_params launch_params = ggml_cuda_kernel_launch_params(block_nums, block_dims, 0, stream);\n \n@@ -1187,7 +1200,7 @@ static void mul_mat_vec_q_switch_ncols_dst(\n \n constexpr bool c_halve_iters = decltype(halve_iters_tag)::value && c_promoted;\n \n- const std::pair dims = calc_launch_params(c_ncols_dst, nrows_x, nchannels_dst,\n+ const std::pair dims = calc_launch_params(c_ncols_dst, nrows_x, nchannels_dst + (fusion.shared_up != nullptr),\n nsamples_dst, warp_size, table_id, c_small_k, c_halve_iters);\n mul_mat_vec_q_switch_fusion(\n vx, vy, ids, fusion, dst, ncols_x, nchannels_y_fd, stride_row_x, stride_col_y, stride_col_dst,\n@@ -1454,6 +1467,23 @@ void ggml_cuda_mul_mat_vec_q(\n // non-negligible for some models such as gpt-oss-20b\n GGML_ASSERT((fusion->x_scale == nullptr && fusion->gate_scale == nullptr) || src0->type == GGML_TYPE_NVFP4);\n \n+ if (fusion->shared_up) {\n+ GGML_ASSERT(ids && fusion->gate && fusion->shared_gate && fusion->shared_dst);\n+ GGML_ASSERT(!fusion->x_bias && !fusion->gate_bias && !fusion->x_scale && !fusion->gate_scale);\n+ GGML_ASSERT(ne11 == 1 && ne03 == 1 && ne13 == 1);\n+ GGML_ASSERT(fusion->shared_up->type == src0->type && fusion->shared_gate->type == src0->type);\n+ GGML_ASSERT(ggml_are_same_shape(fusion->shared_up, fusion->shared_gate));\n+ GGML_ASSERT(ggml_is_contiguous(fusion->shared_up) && ggml_is_contiguous(fusion->shared_gate));\n+ GGML_ASSERT(fusion->shared_up->ne[0] == ne00 && fusion->shared_up->ne[1] == ne01);\n+ GGML_ASSERT(fusion->shared_up->nb[1] == nb01 && ggml_is_matrix(fusion->shared_up));\n+ GGML_ASSERT(fusion->shared_dst->type == GGML_TYPE_F32 && ggml_is_contiguous(fusion->shared_dst));\n+ GGML_ASSERT(fusion->shared_dst->ne[0] == ne0 && fusion->shared_dst->ne[1] == ne2);\n+ fusion_local.shared_up = fusion->shared_up->data;\n+ fusion_local.shared_gate = fusion->shared_gate->data;\n+ fusion_local.shared_dst = (float *) fusion->shared_dst->data;\n+ fusion_local.shared_stride_col_dst = fusion->shared_dst->nb[1] / ts_dst;\n+ }\n+\n if (fusion->x_bias) {\n GGML_ASSERT(fusion->x_bias->type == GGML_TYPE_F32);\n GGML_ASSERT(fusion->x_bias->ne[0] == dst->ne[0]);"},{"sha":"1e0a6b18ac6bd433810a74c86ce0da80283a8bf6","filename":"gguf-py/gguf/constants.py","status":"modified","additions":87,"deletions":0,"changes":87,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/gguf-py%2Fgguf%2Fconstants.py","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/gguf-py%2Fgguf%2Fconstants.py","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/gguf-py%2Fgguf%2Fconstants.py?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -324,6 +324,8 @@ class Decision:\n TYPE = \"{arch}.decision.type\"\n # note: single-use-case keys can be hard-coded in cpp code\n BLOCK_COUNT = \"{arch}.decision.block_count\"\n+ ROUTING_BLOCK_COUNT = \"{arch}.decision.routing_block_count\"\n+ HEAD_COUNT = \"{arch}.decision.head_count\"\n MAX_HEAD_TOKENS = \"{arch}.decision.max_head_tokens\"\n TEMPERATURE = \"{arch}.decision.temperature.{name}\" # name: \"\" or \".\"\n \n@@ -537,6 +539,7 @@ class MODEL_ARCH(IntEnum):\n QWEN3VLMOE = auto()\n QWEN35 = auto()\n QWEN35MOE = auto()\n+ CLEF = auto()\n QWEN4EXP = auto()\n PHI2 = auto()\n PHI3 = auto()\n@@ -861,6 +864,7 @@ class MODEL_TENSOR(IntEnum):\n DEC_ATTN_OUT = auto()\n DEC_ATTN_REL_B = auto()\n DEC_CROSS_ATTN_NORM = auto()\n+ DEC_CROSS_ATTN_NORM_KV = auto()\n DEC_CROSS_ATTN_Q = auto()\n DEC_CROSS_ATTN_K = auto()\n DEC_CROSS_ATTN_V = auto()\n@@ -885,6 +889,19 @@ class MODEL_TENSOR(IntEnum):\n CLS = auto() # classifier\n CLS_OUT = auto() # classifier output projection\n CLS_NORM = auto()\n+ DECISION_HIDDEN_NORM = auto()\n+ DECISION_PROJ_MEMORY = auto()\n+ DECISION_PROJ_QUESTION = auto()\n+ DECISION_PROJ_OPTION_QUESTION = auto()\n+ DECISION_PROJ_GLOBAL = auto()\n+ DECISION_PROJ_OPTION_CONTEXT = auto()\n+ DECISION_PROJ_OPTION_LEXICAL = auto()\n+ DECISION_OPTION_SUMMARY_NORM = auto()\n+ DECISION_FIELD_NORM = auto()\n+ DECISION_OPTION_NORM = auto()\n+ DECISION_SCALES = auto()\n+ DECISION_SCORER = auto()\n+ DECISION_SCORER_OUT = auto()\n CONV1D = auto()\n CONVNEXT_DW = auto()\n CONVNEXT_NORM = auto()\n@@ -1301,6 +1318,7 @@ class MODEL_TENSOR(IntEnum):\n MODEL_ARCH.QWEN3VLMOE: \"qwen3vlmoe\",\n MODEL_ARCH.QWEN35: \"qwen35\",\n MODEL_ARCH.QWEN35MOE: \"qwen35moe\",\n+ MODEL_ARCH.CLEF: \"clef\",\n MODEL_ARCH.QWEN4EXP: \"qwen4exp\",\n MODEL_ARCH.PHI2: \"phi2\",\n MODEL_ARCH.PHI3: \"phi3\",\n@@ -1624,6 +1642,7 @@ class MODEL_TENSOR(IntEnum):\n MODEL_TENSOR.DEC_ATTN_OUT: \"dec.blk.{bid}.attn_o\",\n MODEL_TENSOR.DEC_ATTN_REL_B: \"dec.blk.{bid}.attn_rel_b\",\n MODEL_TENSOR.DEC_CROSS_ATTN_NORM: \"dec.blk.{bid}.cross_attn_norm\",\n+ MODEL_TENSOR.DEC_CROSS_ATTN_NORM_KV: \"dec.blk.{bid}.cross_attn_norm_kv\",\n MODEL_TENSOR.DEC_CROSS_ATTN_Q: \"dec.blk.{bid}.cross_attn_q\",\n MODEL_TENSOR.DEC_CROSS_ATTN_K: \"dec.blk.{bid}.cross_attn_k\",\n MODEL_TENSOR.DEC_CROSS_ATTN_V: \"dec.blk.{bid}.cross_attn_v\",\n@@ -1648,6 +1667,19 @@ class MODEL_TENSOR(IntEnum):\n MODEL_TENSOR.CLS: \"cls\",\n MODEL_TENSOR.CLS_OUT: \"cls.output\",\n MODEL_TENSOR.CLS_NORM: \"cls.norm\",\n+ MODEL_TENSOR.DECISION_HIDDEN_NORM: \"decision.hidden_norm\",\n+ MODEL_TENSOR.DECISION_PROJ_MEMORY: \"decision.proj_memory\",\n+ MODEL_TENSOR.DECISION_PROJ_QUESTION: \"decision.proj_question\",\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_QUESTION: \"decision.proj_option_question\",\n+ MODEL_TENSOR.DECISION_PROJ_GLOBAL: \"decision.proj_global\",\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_CONTEXT: \"decision.proj_option_context\",\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_LEXICAL: \"decision.proj_option_lexical\",\n+ MODEL_TENSOR.DECISION_OPTION_SUMMARY_NORM: \"decision.option_summary_norm\",\n+ MODEL_TENSOR.DECISION_FIELD_NORM: \"decision.field_norm\",\n+ MODEL_TENSOR.DECISION_OPTION_NORM: \"decision.option_norm\",\n+ MODEL_TENSOR.DECISION_SCALES: \"decision.scales\",\n+ MODEL_TENSOR.DECISION_SCORER: \"decision.scorer\",\n+ MODEL_TENSOR.DECISION_SCORER_OUT: \"decision.scorer_out\",\n MODEL_TENSOR.CONV1D: \"conv1d\",\n MODEL_TENSOR.CONVNEXT_DW: \"convnext.{bid}.dw\",\n MODEL_TENSOR.CONVNEXT_NORM: \"convnext.{bid}.norm\",\n@@ -2918,6 +2950,60 @@ class MODEL_TENSOR(IntEnum):\n MODEL_TENSOR.NEXTN_SHARED_HEAD_HEAD,\n MODEL_TENSOR.NEXTN_SHARED_HEAD_NORM,\n ],\n+ MODEL_ARCH.CLEF: [\n+ MODEL_TENSOR.TOKEN_EMBD,\n+ MODEL_TENSOR.OUTPUT_NORM,\n+ MODEL_TENSOR.OUTPUT,\n+ MODEL_TENSOR.ATTN_NORM,\n+ MODEL_TENSOR.ATTN_Q,\n+ MODEL_TENSOR.ATTN_Q_NORM,\n+ MODEL_TENSOR.ATTN_K,\n+ MODEL_TENSOR.ATTN_K_NORM,\n+ MODEL_TENSOR.ATTN_V,\n+ MODEL_TENSOR.ATTN_OUT,\n+ MODEL_TENSOR.ATTN_POST_NORM,\n+ MODEL_TENSOR.ATTN_GATE,\n+ MODEL_TENSOR.ATTN_QKV,\n+ MODEL_TENSOR.FFN_GATE,\n+ MODEL_TENSOR.FFN_DOWN,\n+ MODEL_TENSOR.FFN_UP,\n+ MODEL_TENSOR.SSM_A,\n+ MODEL_TENSOR.SSM_CONV1D,\n+ MODEL_TENSOR.SSM_DT,\n+ MODEL_TENSOR.SSM_NORM,\n+ MODEL_TENSOR.SSM_BETA,\n+ MODEL_TENSOR.SSM_ALPHA,\n+ MODEL_TENSOR.SSM_OUT,\n+ # decision head\n+ MODEL_TENSOR.TOKEN_TYPES,\n+ MODEL_TENSOR.DEC_ATTN_NORM,\n+ MODEL_TENSOR.DEC_ATTN_Q,\n+ MODEL_TENSOR.DEC_ATTN_K,\n+ MODEL_TENSOR.DEC_ATTN_V,\n+ MODEL_TENSOR.DEC_ATTN_OUT,\n+ MODEL_TENSOR.DEC_CROSS_ATTN_NORM,\n+ MODEL_TENSOR.DEC_CROSS_ATTN_NORM_KV,\n+ MODEL_TENSOR.DEC_CROSS_ATTN_Q,\n+ MODEL_TENSOR.DEC_CROSS_ATTN_K,\n+ MODEL_TENSOR.DEC_CROSS_ATTN_V,\n+ MODEL_TENSOR.DEC_CROSS_ATTN_OUT,\n+ MODEL_TENSOR.DEC_FFN_NORM,\n+ MODEL_TENSOR.DEC_FFN_DOWN,\n+ MODEL_TENSOR.DEC_FFN_UP,\n+ MODEL_TENSOR.DECISION_HIDDEN_NORM,\n+ MODEL_TENSOR.DECISION_PROJ_MEMORY,\n+ MODEL_TENSOR.DECISION_PROJ_QUESTION,\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_QUESTION,\n+ MODEL_TENSOR.DECISION_PROJ_GLOBAL,\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_CONTEXT,\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_LEXICAL,\n+ MODEL_TENSOR.DECISION_OPTION_SUMMARY_NORM,\n+ MODEL_TENSOR.DECISION_FIELD_NORM,\n+ MODEL_TENSOR.DECISION_OPTION_NORM,\n+ MODEL_TENSOR.DECISION_SCALES,\n+ MODEL_TENSOR.DECISION_SCORER,\n+ MODEL_TENSOR.DECISION_SCORER_OUT,\n+ ],\n MODEL_ARCH.QWEN35MOE: [\n MODEL_TENSOR.TOKEN_EMBD,\n MODEL_TENSOR.OUTPUT_NORM,\n@@ -5921,6 +6007,7 @@ class DecisionType:\n LEV = \"lev\" # same as openjev, noul is read from a rating scale\n KEV = \"kev\" # dot product of the hidden states of the last token and of one end token per option\n NIMBLE = \"nimble\" # same as openjev, the prompt lists all the questions of the request\n+ CLEF = \"clef\" # joint head over all questions, one score per option\n \n \n class VisionProjectorType:"},{"sha":"4f1f8e5a5dc01faf81671d21725783691a71df7f","filename":"gguf-py/gguf/gguf_writer.py","status":"modified","additions":6,"deletions":0,"changes":6,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/gguf-py%2Fgguf%2Fgguf_writer.py","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/gguf-py%2Fgguf%2Fgguf_writer.py","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/gguf-py%2Fgguf%2Fgguf_writer.py?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -1346,6 +1346,12 @@ def add_decision_type(self, value: str) -> None:\n def add_decision_block_count(self, value: int) -> None:\n self.add_uint32(Keys.Decision.BLOCK_COUNT.format(arch=self.arch), value)\n \n+ def add_decision_routing_block_count(self, value: int) -> None:\n+ self.add_uint32(Keys.Decision.ROUTING_BLOCK_COUNT.format(arch=self.arch), value)\n+\n+ def add_decision_head_count(self, value: int) -> None:\n+ self.add_uint32(Keys.Decision.HEAD_COUNT.format(arch=self.arch), value)\n+\n def add_decision_max_head_tokens(self, value: int) -> None:\n self.add_uint32(Keys.Decision.MAX_HEAD_TOKENS.format(arch=self.arch), value)\n "},{"sha":"cef04d08271e1e5d88c2383b56a6e656f51f5f3b","filename":"gguf-py/gguf/tensor_mapping.py","status":"modified","additions":74,"deletions":0,"changes":74,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/gguf-py%2Fgguf%2Ftensor_mapping.py","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/gguf-py%2Fgguf%2Ftensor_mapping.py","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/gguf-py%2Fgguf%2Ftensor_mapping.py?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -49,6 +49,7 @@ class TensorNameMap:\n MODEL_TENSOR.TOKEN_TYPES: (\n \"embeddings.token_type_embeddings\", # bert nomic-bert\n \"type_emb\", # laya\n+ \"joint_head.type_embedding\", # clef\n ),\n \n # Normalization of token embeddings\n@@ -1181,22 +1182,27 @@ class TensorNameMap:\n \n MODEL_TENSOR.DEC_ATTN_NORM: (\n \"decoder.block.{bid}.layer.0.layer_norm\", # t5\n+ \"joint_head.layers.{bid}.norm1\", # clef\n ),\n \n MODEL_TENSOR.DEC_ATTN_Q: (\n \"decoder.block.{bid}.layer.0.SelfAttention.q\", # t5\n+ \"joint_head.layers.{bid}.self_attn.q\", # clef\n ),\n \n MODEL_TENSOR.DEC_ATTN_K: (\n \"decoder.block.{bid}.layer.0.SelfAttention.k\", # t5\n+ \"joint_head.layers.{bid}.self_attn.k\", # clef\n ),\n \n MODEL_TENSOR.DEC_ATTN_V: (\n \"decoder.block.{bid}.layer.0.SelfAttention.v\", # t5\n+ \"joint_head.layers.{bid}.self_attn.v\", # clef\n ),\n \n MODEL_TENSOR.DEC_ATTN_OUT: (\n \"decoder.block.{bid}.layer.0.SelfAttention.o\", # t5\n+ \"joint_head.layers.{bid}.self_attn.out_proj\", # clef\n ),\n \n MODEL_TENSOR.DEC_ATTN_REL_B: (\n@@ -1205,22 +1211,32 @@ class TensorNameMap:\n \n MODEL_TENSOR.DEC_CROSS_ATTN_NORM: (\n \"decoder.block.{bid}.layer.1.layer_norm\", # t5\n+ \"joint_head.layers.{bid}.norm2\", # clef\n+ \"joint_head.evidence_layers.{bid}.query_norm\", # clef\n ),\n \n MODEL_TENSOR.DEC_CROSS_ATTN_Q: (\n \"decoder.block.{bid}.layer.1.EncDecAttention.q\", # t5\n+ \"joint_head.layers.{bid}.multihead_attn.q\", # clef\n+ \"joint_head.evidence_layers.{bid}.attention.q\", # clef\n ),\n \n MODEL_TENSOR.DEC_CROSS_ATTN_K: (\n \"decoder.block.{bid}.layer.1.EncDecAttention.k\", # t5\n+ \"joint_head.layers.{bid}.multihead_attn.k\", # clef\n+ \"joint_head.evidence_layers.{bid}.attention.k\", # clef\n ),\n \n MODEL_TENSOR.DEC_CROSS_ATTN_V: (\n \"decoder.block.{bid}.layer.1.EncDecAttention.v\", # t5\n+ \"joint_head.layers.{bid}.multihead_attn.v\", # clef\n+ \"joint_head.evidence_layers.{bid}.attention.v\", # clef\n ),\n \n MODEL_TENSOR.DEC_CROSS_ATTN_OUT: (\n \"decoder.block.{bid}.layer.1.EncDecAttention.o\", # t5\n+ \"joint_head.layers.{bid}.multihead_attn.out_proj\", # clef\n+ \"joint_head.evidence_layers.{bid}.attention.out_proj\", # clef\n ),\n \n MODEL_TENSOR.DEC_CROSS_ATTN_REL_B: (\n@@ -1229,6 +1245,8 @@ class TensorNameMap:\n \n MODEL_TENSOR.DEC_FFN_NORM: (\n \"decoder.block.{bid}.layer.2.layer_norm\", # t5\n+ \"joint_head.layers.{bid}.norm3\", # clef\n+ \"joint_head.evidence_layers.{bid}.feedforward_norm\", # clef\n ),\n \n MODEL_TENSOR.DEC_FFN_GATE: (\n@@ -1238,16 +1256,72 @@ class TensorNameMap:\n MODEL_TENSOR.DEC_FFN_UP: (\n \"decoder.block.{bid}.layer.2.DenseReluDense.wi\", # t5\n \"decoder.block.{bid}.layer.2.DenseReluDense.wi_1\", # flan-t5\n+ \"joint_head.layers.{bid}.linear1\", # clef\n+ \"joint_head.evidence_layers.{bid}.feedforward.0\", # clef\n ),\n \n MODEL_TENSOR.DEC_FFN_DOWN: (\n \"decoder.block.{bid}.layer.2.DenseReluDense.wo\", # t5\n+ \"joint_head.layers.{bid}.linear2\", # clef\n+ \"joint_head.evidence_layers.{bid}.feedforward.3\", # clef\n ),\n \n MODEL_TENSOR.DEC_OUTPUT_NORM: (\n \"decoder.final_layer_norm\", # t5\n ),\n \n+ MODEL_TENSOR.DEC_CROSS_ATTN_NORM_KV: (\n+ \"joint_head.evidence_layers.{bid}.memory_norm\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_HIDDEN_NORM: (\n+ \"joint_head.hidden_norm\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_PROJ_MEMORY: (\n+ \"joint_head.memory_projection\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_PROJ_QUESTION: (\n+ \"joint_head.question_projection\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_QUESTION: (\n+ \"joint_head.option_question_projection\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_PROJ_GLOBAL: (\n+ \"joint_head.global_projection\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_CONTEXT: (\n+ \"joint_head.option_context_projection\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_PROJ_OPTION_LEXICAL: (\n+ \"joint_head.option_lexical_projection\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_OPTION_SUMMARY_NORM: (\n+ \"joint_head.option_summary_norm\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_FIELD_NORM: (\n+ \"joint_head.field_norm\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_OPTION_NORM: (\n+ \"joint_head.option_norm\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_SCORER: (\n+ \"joint_head.residual_scorer.0\", # clef\n+ ),\n+\n+ MODEL_TENSOR.DECISION_SCORER_OUT: (\n+ \"joint_head.residual_scorer.3\", # clef\n+ ),\n+\n MODEL_TENSOR.ENC_ATTN_NORM: (\n \"encoder.block.{bid}.layer.0.layer_norm\", # t5\n ),"},{"sha":"2af5445dfa96b168e2c743352570e9aa29b7483d","filename":"src/llama-arch.cpp","status":"modified","additions":32,"deletions":1,"changes":33,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-arch.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-arch.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-arch.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -40,6 +40,7 @@ static const std::map LLM_ARCH_NAMES = {\n { LLM_ARCH_QWEN3VLMOE, \"qwen3vlmoe\" },\n { LLM_ARCH_QWEN35, \"qwen35\" },\n { LLM_ARCH_QWEN35MOE, \"qwen35moe\" },\n+ { LLM_ARCH_CLEF, \"clef\" },\n { LLM_ARCH_QWEN4EXP, \"qwen4exp\" },\n { LLM_ARCH_PHI2, \"phi2\" },\n { LLM_ARCH_PHI3, \"phi3\" },\n@@ -366,7 +367,9 @@ static const std::map LLM_KV_NAMES = {\n { LLM_KV_CLASSIFIER_OUTPUT_LABELS, \"%s.classifier.output_labels\" },\n { LLM_KV_CLASSIFIER_POOLING_TYPE, \"%s.classifier.pooling_type\" },\n \n- { LLM_KV_DECISION_BLOCK_COUNT, \"%s.decision.block_count\" },\n+ { LLM_KV_DECISION_BLOCK_COUNT, \"%s.decision.block_count\" },\n+ { LLM_KV_DECISION_ROUTING_BLOCK_COUNT, \"%s.decision.routing_block_count\" },\n+ { LLM_KV_DECISION_HEAD_COUNT, \"%s.decision.head_count\" },\n \n { LLM_KV_TARGET_LAYERS, \"%s.target_layers\" },\n { LLM_KV_TARGET_HIDDEN_SIZE, \"%s.target_hidden_size\" },\n@@ -490,6 +493,19 @@ static const std::map LLM_TENSOR_NAMES = {\n { LLM_TENSOR_CLS, \"cls\" },\n { LLM_TENSOR_CLS_OUT, \"cls.output\" },\n { LLM_TENSOR_CLS_NORM, \"cls.norm\" },\n+ { LLM_TENSOR_DECISION_HIDDEN_NORM, \"decision.hidden_norm\" },\n+ { LLM_TENSOR_DECISION_PROJ_MEMORY, \"decision.proj_memory\" },\n+ { LLM_TENSOR_DECISION_PROJ_QUESTION, \"decision.proj_question\" },\n+ { LLM_TENSOR_DECISION_PROJ_OPTION_QUESTION, \"decision.proj_option_question\" },\n+ { LLM_TENSOR_DECISION_PROJ_GLOBAL, \"decision.proj_global\" },\n+ { LLM_TENSOR_DECISION_PROJ_OPTION_CONTEXT, \"decision.proj_option_context\" },\n+ { LLM_TENSOR_DECISION_PROJ_OPTION_LEXICAL, \"decision.proj_option_lexical\" },\n+ { LLM_TENSOR_DECISION_OPTION_SUMMARY_NORM, \"decision.option_summary_norm\" },\n+ { LLM_TENSOR_DECISION_FIELD_NORM, \"decision.field_norm\" },\n+ { LLM_TENSOR_DECISION_OPTION_NORM, \"decision.option_norm\" },\n+ { LLM_TENSOR_DECISION_SCALES, \"decision.scales\" },\n+ { LLM_TENSOR_DECISION_SCORER, \"decision.scorer\" },\n+ { LLM_TENSOR_DECISION_SCORER_OUT, \"decision.scorer_out\" },\n { LLM_TENSOR_ENC_OUTPUT_NORM, \"enc.output_norm\" },\n { LLM_TENSOR_FFN_GATE_INP_SHEXP, \"blk.%d.ffn_gate_inp_shexp\" },\n { LLM_TENSOR_SSM_A_NOSCAN, \"blk.%d.ssm_a\" },\n@@ -606,6 +622,7 @@ static const std::map LLM_TENSOR_NAMES = {\n { LLM_TENSOR_DEC_ATTN_OUT, \"dec.blk.%d.attn_o\" },\n { LLM_TENSOR_DEC_ATTN_REL_B, \"dec.blk.%d.attn_rel_b\" },\n { LLM_TENSOR_DEC_CROSS_ATTN_NORM, \"dec.blk.%d.cross_attn_norm\" },\n+ { LLM_TENSOR_DEC_CROSS_ATTN_NORM_KV, \"dec.blk.%d.cross_attn_norm_kv\" },\n { LLM_TENSOR_DEC_CROSS_ATTN_Q, \"dec.blk.%d.cross_attn_q\" },\n { LLM_TENSOR_DEC_CROSS_ATTN_K, \"dec.blk.%d.cross_attn_k\" },\n { LLM_TENSOR_DEC_CROSS_ATTN_V, \"dec.blk.%d.cross_attn_v\" },\n@@ -739,6 +756,19 @@ static const std::map LLM_TENSOR_INFOS = {\n {LLM_TENSOR_CLS, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n {LLM_TENSOR_CLS_OUT, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n {LLM_TENSOR_CLS_NORM, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n+ {LLM_TENSOR_DECISION_HIDDEN_NORM, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n+ {LLM_TENSOR_DECISION_PROJ_MEMORY, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_PROJ_QUESTION, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_PROJ_OPTION_QUESTION, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_PROJ_GLOBAL, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_PROJ_OPTION_CONTEXT, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_PROJ_OPTION_LEXICAL, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_OPTION_SUMMARY_NORM, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n+ {LLM_TENSOR_DECISION_FIELD_NORM, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n+ {LLM_TENSOR_DECISION_OPTION_NORM, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n+ {LLM_TENSOR_DECISION_SCALES, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n+ {LLM_TENSOR_DECISION_SCORER, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n+ {LLM_TENSOR_DECISION_SCORER_OUT, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}},\n {LLM_TENSOR_DENSE_2_OUT, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}}, // Dense layer output\n {LLM_TENSOR_DENSE_3_OUT, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL_MAT}}, // Dense layer output\n {LLM_TENSOR_OUTPUT_NORM, {LLM_TENSOR_LAYER_OUTPUT, GGML_OP_MUL}},\n@@ -908,6 +938,7 @@ static const std::map LLM_TENSOR_INFOS = {\n {LLM_TENSOR_FFN_SUB_NORM, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},\n {LLM_TENSOR_DEC_ATTN_NORM, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},\n {LLM_TENSOR_DEC_CROSS_ATTN_NORM, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},\n+ {LLM_TENSOR_DEC_CROSS_ATTN_NORM_KV, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},\n {LLM_TENSOR_DEC_FFN_NORM, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},\n {LLM_TENSOR_ENC_ATTN_NORM, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},\n {LLM_TENSOR_ENC_FFN_NORM, {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},"},{"sha":"148d293cee76b107966bf5b031233abf8fbe7bf8","filename":"src/llama-arch.h","status":"modified","additions":17,"deletions":0,"changes":17,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-arch.h","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-arch.h","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-arch.h?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -45,6 +45,7 @@ enum llm_arch {\n LLM_ARCH_QWEN3VLMOE,\n LLM_ARCH_QWEN35,\n LLM_ARCH_QWEN35MOE,\n+ LLM_ARCH_CLEF,\n LLM_ARCH_QWEN4EXP,\n LLM_ARCH_PHI2,\n LLM_ARCH_PHI3,\n@@ -413,6 +414,8 @@ enum llm_kv {\n LLM_KV_CLASSIFIER_POOLING_TYPE,\n \n LLM_KV_DECISION_BLOCK_COUNT,\n+ LLM_KV_DECISION_ROUTING_BLOCK_COUNT,\n+ LLM_KV_DECISION_HEAD_COUNT,\n \n LLM_KV_TARGET_LAYERS,\n LLM_KV_TARGET_HIDDEN_SIZE,\n@@ -636,6 +639,7 @@ enum llm_tensor {\n LLM_TENSOR_DEC_ATTN_OUT,\n LLM_TENSOR_DEC_ATTN_REL_B,\n LLM_TENSOR_DEC_CROSS_ATTN_NORM,\n+ LLM_TENSOR_DEC_CROSS_ATTN_NORM_KV,\n LLM_TENSOR_DEC_CROSS_ATTN_Q,\n LLM_TENSOR_DEC_CROSS_ATTN_K,\n LLM_TENSOR_DEC_CROSS_ATTN_V,\n@@ -660,6 +664,19 @@ enum llm_tensor {\n LLM_TENSOR_CLS,\n LLM_TENSOR_CLS_OUT,\n LLM_TENSOR_CLS_NORM,\n+ LLM_TENSOR_DECISION_HIDDEN_NORM,\n+ LLM_TENSOR_DECISION_PROJ_MEMORY,\n+ LLM_TENSOR_DECISION_PROJ_QUESTION,\n+ LLM_TENSOR_DECISION_PROJ_OPTION_QUESTION,\n+ LLM_TENSOR_DECISION_PROJ_GLOBAL,\n+ LLM_TENSOR_DECISION_PROJ_OPTION_CONTEXT,\n+ LLM_TENSOR_DECISION_PROJ_OPTION_LEXICAL,\n+ LLM_TENSOR_DECISION_OPTION_SUMMARY_NORM,\n+ LLM_TENSOR_DECISION_FIELD_NORM,\n+ LLM_TENSOR_DECISION_OPTION_NORM,\n+ LLM_TENSOR_DECISION_SCALES,\n+ LLM_TENSOR_DECISION_SCORER,\n+ LLM_TENSOR_DECISION_SCORER_OUT,\n LLM_TENSOR_CONV1D,\n LLM_TENSOR_CONVNEXT_DW,\n LLM_TENSOR_CONVNEXT_NORM,"},{"sha":"1b3d70627a284aab213297bcc371002e399238ee","filename":"src/llama-batch.cpp","status":"modified","additions":29,"deletions":0,"changes":29,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-batch.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-batch.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-batch.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -168,6 +168,14 @@ bool llama_batch_allocr::init(\n }\n }\n \n+ // kept empty if no entry has one\n+ for (int32_t i = 0; i < n_tok; ++i) {\n+ if (batch_inp.tokens[i].decision_order != 0) {\n+ decision_order.resize(n_tok, 0);\n+ decision_order[i] = batch_inp.tokens[i].decision_order;\n+ }\n+ }\n+\n //\n // set up the internal llama_batch to point to our owned arrays\n //\n@@ -256,6 +264,7 @@ bool llama_batch_allocr::init(\n /*.seq_id_unq =*/ this->seq_id_unq.data(),\n /*.seq_idx =*/ this->seq_idx.data(),\n /*.output =*/ batch.logits,\n+ /*.decision_order =*/ decision_order.empty() ? nullptr : decision_order.data(),\n /*.data =*/ {},\n };\n \n@@ -462,6 +471,7 @@ llama_ubatch llama_batch_allocr::ubatch_reserve(uint32_t n_seq_tokens, uint32_t\n /*.seq_id_unq =*/ udata->seq_id_unq.data(),\n /*.seq_idx =*/ udata->seq_idx.data(),\n /*.output =*/ udata->output.data(),\n+ /*.decision_order =*/ nullptr,\n /*.data =*/ std::move(udata),\n };\n \n@@ -765,6 +775,7 @@ void llama_batch_allocr::clear() {\n seq_id .clear();\n seq_id_unq .clear();\n output .clear();\n+ decision_order.clear();\n \n for (auto & cur : seq_pos) {\n cur.clear();\n@@ -799,6 +810,7 @@ llama_ubatch llama_batch_allocr::ubatch_add(const std::vector & idxs, u\n udata->seq_id_unq.resize(0);\n udata->seq_idx .resize(LLAMA_MAX_SEQ, -1);\n udata->output .resize(n_tokens);\n+ udata->decision_order.resize(decision_order.empty() ? 0 : n_tokens);\n \n udata->batch_idxs = idxs;\n udata->seq_id_data.reserve(n_tokens);\n@@ -826,6 +838,10 @@ llama_ubatch llama_batch_allocr::ubatch_add(const std::vector & idxs, u\n udata->n_seq_id[i] = batch.n_seq_id[idxs[i]];\n udata->output[i] = batch.logits[idxs[i]];\n \n+ if (!decision_order.empty()) {\n+ udata->decision_order[i] = decision_order[idxs[i]];\n+ }\n+\n for (int s = 0; s < udata->n_seq_id[i]; ++s) {\n const llama_seq_id seq_id = batch.seq_id[idxs[i]][s];\n \n@@ -867,6 +883,7 @@ llama_ubatch llama_batch_allocr::ubatch_add(const std::vector & idxs, u\n /*.seq_id_unq =*/ udata->seq_id_unq.data(),\n /*.seq_idx =*/ udata->seq_idx.data(),\n /*.output =*/ udata->output.data(),\n+ /*.decision_order =*/ udata->decision_order.empty() ? nullptr : udata->decision_order.data(),\n /*.data =*/ std::move(udata),\n };\n \n@@ -1175,6 +1192,14 @@ bool llama_batch_ext::set_output(int32_t idx, bool output_last) {\n return true;\n }\n \n+bool llama_batch_ext::set_decision_order(int32_t idx, int32_t order) {\n+ if (idx < 0 || idx >= (int32_t) tokens.size()) {\n+ return false;\n+ }\n+ tokens[idx].decision_order = order;\n+ return true;\n+}\n+\n // llama_batch_ext C API\n \n llama_batch_ext * llama_batch_ext_init(llama_context * ctx) {\n@@ -1243,6 +1268,10 @@ bool llama_batch_ext_set_output_logits(llama_batch_ext * batch, int32_t idx, boo\n return batch->set_output(idx, value);\n }\n \n+bool llama_batch_ext_set_decision_order(llama_batch_ext * batch, int32_t idx, llama_decision_order order) {\n+ return batch->set_decision_order(idx, order);\n+}\n+\n // llama_batch_compat\n \n void llama_batch_compat::init(llama_batch_ext & dst, const llama_batch & batch_inp, size_t n_embd_row) {"},{"sha":"32f103bfcb3a4b84387b4dfdb911181040e2a5bf","filename":"src/llama-batch.h","status":"modified","additions":5,"deletions":0,"changes":5,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-batch.h","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-batch.h","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-batch.h?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -52,6 +52,7 @@ struct llama_ubatch {\n llama_seq_id * seq_id_unq; // [n_seqs_unq] | s | seq_id\n int32_t * seq_idx; // [LLAMA_MAX_SEQ] | - | seq_idx\n int8_t * output; // [n_tokens] | i | -\n+ int32_t * decision_order; // [n_tokens], NULL if no entry has one, see llama_batch_ext_set_decision_order()\n \n struct data_t {\n std::vector token;\n@@ -63,6 +64,7 @@ struct llama_ubatch {\n std::vector seq_idx;\n std::vector output;\n std::vector batch_idxs; // original batch index for each token\n+ std::vector decision_order;\n \n std::vector seq_id_data;\n };\n@@ -96,6 +98,7 @@ struct llama_batch_ext {\n bool has_embd = false; // whether embd_off is set\n size_t embd_off = 0; // index offset in the embd array\n bool output = false; // TODO: have dedicated output flags\n+ int32_t decision_order = 0; // see llama_batch_ext_set_decision_order()\n std::unordered_set seq_ids;\n std::array pos = {0, 0, 0, 0};\n };\n@@ -125,6 +128,7 @@ struct llama_batch_ext {\n bool set_token_embd(int32_t idx, llama_embd embd_in);\n bool set_token_pos(int32_t idx, const llama_pos * pos_in);\n bool set_output(int32_t idx, bool output_last);\n+ bool set_decision_order(int32_t idx, int32_t order);\n };\n \n // a helper for sanitizing, fulfilling and splitting a batch\n@@ -202,6 +206,7 @@ class llama_batch_allocr {\n std::vector seq_id_unq;\n std::vector seq_idx;\n std::vector output;\n+ std::vector decision_order; // empty if no entry has one\n \n using pos_set_t = std::set;\n using seq_cpl_t = std::vector;"},{"sha":"96b5464e67e4f0010fbfed5935126eaa6aa8a766","filename":"src/llama-context.cpp","status":"modified","additions":1,"deletions":0,"changes":1,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-context.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-context.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-context.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -2406,6 +2406,7 @@ uint32_t llama_context::graph_max_nodes(uint32_t n_tokens) const {\n model.arch == LLM_ARCH_BAILINGMOE3 ||\n model.arch == LLM_ARCH_QWEN35 ||\n model.arch == LLM_ARCH_QWEN35MOE ||\n+ model.arch == LLM_ARCH_CLEF ||\n model.arch == LLM_ARCH_QWEN4EXP ||\n model.arch == LLM_ARCH_DEEPSEEK4 ||\n (model.arch == LLM_ARCH_DFLASH && model.hparams.dsv4_hc_mult > 0) ||"},{"sha":"fcc78f47e4534e3ed0748eca7f821a14abc5e191","filename":"src/llama-ext.h","status":"modified","additions":14,"deletions":0,"changes":14,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-ext.h","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-ext.h","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-ext.h?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -100,6 +100,20 @@ LLAMA_API void llama_set_embeddings_nextn(struct llama_context * ctx, bool value\n // chain multiple trained NextN heads. Default 0 (first head).\n LLAMA_API void llama_set_nextn_layer_offset(struct llama_context * ctx, int32_t offset);\n \n+// Marks the entries that a joint decision head (clef) reads, the default is 0\n+// See https://github.com/ggml-org/llama.cpp/pull/29831 for details\n+// A run of entries with the same value is one span, spans must be separated by entries with value 0\n+// An option belongs to the last question before it\n+enum llama_decision_order {\n+ LLAMA_DECISION_ORDER_NONE = 0, // not read by the head\n+ LLAMA_DECISION_ORDER_QUESTION_NOUL = 1, // text of a question\n+ LLAMA_DECISION_ORDER_QUESTION_CHOICE = 2,\n+ LLAMA_DECISION_ORDER_QUESTION_SCORE = 3,\n+ LLAMA_DECISION_ORDER_OPTION = 4, // text of an option\n+};\n+// The embeddings output has one value per entry: row i is the score of option i\n+LLAMA_API bool llama_batch_ext_set_decision_order(struct llama_batch_ext * batch, int32_t idx, enum llama_decision_order order);\n+\n // mirrors:\n // LLAMA_API float * llama_get_embeddings(struct llama_context * ctx);\n LLAMA_API float * llama_get_embeddings_nextn(struct llama_context * ctx);"},{"sha":"4dbcfb8db8ba317b888379a42ca74d5b6e545e18","filename":"src/llama-kv-cache-dsv4.cpp","status":"modified","additions":1,"deletions":0,"changes":1,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-kv-cache-dsv4.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-kv-cache-dsv4.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-kv-cache-dsv4.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -164,6 +164,7 @@ static llama_ubatch dsv4_build_raw_write_ubatch(const llama_ubatch & ubatch) {\n /*.seq_id_unq =*/ data->seq_id_unq.data(),\n /*.seq_idx =*/ data->seq_idx.data(),\n /*.output =*/ data->output.data(),\n+ /*.decision_order =*/ nullptr,\n /*.data =*/ data,\n };\n "},{"sha":"e4fd45ec4f0716a05f9dd1956763a4f0852231ac","filename":"src/llama-model-saver.cpp","status":"modified","additions":1,"deletions":0,"changes":1,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-model-saver.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-model-saver.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-model-saver.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -20,6 +20,7 @@ bool llama_model_saver_supports_arch(llm_arch arch) {\n case LLM_ARCH_T5:\n case LLM_ARCH_APERTUS:\n case LLM_ARCH_STEP35:\n+ case LLM_ARCH_CLEF: // the head tensors are not saved\n return false;\n default:\n return true;"},{"sha":"97e0cee70e42c4960619008063e0705ea4c2fd31","filename":"src/llama-model.cpp","status":"modified","additions":12,"deletions":4,"changes":16,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-model.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fllama-model.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fllama-model.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -326,6 +326,8 @@ static llama_model * llama_model_mapping(llm_arch arch, const llama_model_params\n return new llama_model_qwen35(params);\n case LLM_ARCH_QWEN35MOE:\n return new llama_model_qwen35moe(params);\n+ case LLM_ARCH_CLEF:\n+ return new llama_model_clef(params);\n case LLM_ARCH_QWEN4EXP:\n return new llama_model_qwen4exp(params);\n case LLM_ARCH_MISTRAL3:\n@@ -607,7 +609,7 @@ struct ggml_backend_meta_split_state llama_meta_device_get_split_state(const str\n // if a model has fused tensors they need to be separated into \"segments\", see the comments on ggml_backend_meta_split_state struct\n auto get_split_segments = [&](int axis, uint32_t il) -> std::vector> {\n // TODO: deduplicate condition [TAG_SPLIT_QGATE_QWEN]\n- if (ud->model->arch == LLM_ARCH_QWEN3NEXT || ud->model->arch == LLM_ARCH_QWEN35 || ud->model->arch == LLM_ARCH_QWEN35MOE ||\n+ if (ud->model->arch == LLM_ARCH_QWEN3NEXT || ud->model->arch == LLM_ARCH_QWEN35 || ud->model->arch == LLM_ARCH_QWEN35MOE || ud->model->arch == LLM_ARCH_CLEF ||\n ud->model->arch == LLM_ARCH_QWEN4EXP) {\n \n // fused full attention layers with Q gate tensors that need n_embd doubled:\n@@ -762,7 +764,7 @@ struct ggml_backend_meta_split_state llama_meta_device_get_split_state(const str\n GGML_ASSERT(segments.size() == 1);\n // some models have Q gate tensors, for those cases the granularity needs to be doubled:\n // TODO: deduplicate condition [TAG_SPLIT_QGATE_QWEN]\n- if (ud->model->arch == LLM_ARCH_QWEN3NEXT || ud->model->arch == LLM_ARCH_QWEN35 || ud->model->arch == LLM_ARCH_QWEN35MOE ||\n+ if (ud->model->arch == LLM_ARCH_QWEN3NEXT || ud->model->arch == LLM_ARCH_QWEN35 || ud->model->arch == LLM_ARCH_QWEN35MOE || ud->model->arch == LLM_ARCH_CLEF ||\n ud->model->arch == LLM_ARCH_QWEN4EXP) {\n return {std::lcm(2*n_embd_q, blck_size_perf)};\n }\n@@ -795,7 +797,7 @@ struct ggml_backend_meta_split_state llama_meta_device_get_split_state(const str\n if (std::regex_match(tensor_name, pattern_qkv_weight) || std::regex_match(tensor_name, pattern_qkv_bias)) {\n // fused full attention layers need Q gate tensors handled like above:\n // TODO: deduplicate condition [TAG_SPLIT_QGATE_QWEN]\n- if (ud->model->arch == LLM_ARCH_QWEN3NEXT || ud->model->arch == LLM_ARCH_QWEN35 || ud->model->arch == LLM_ARCH_QWEN35MOE ||\n+ if (ud->model->arch == LLM_ARCH_QWEN3NEXT || ud->model->arch == LLM_ARCH_QWEN35 || ud->model->arch == LLM_ARCH_QWEN35MOE || ud->model->arch == LLM_ARCH_CLEF ||\n ud->model->arch == LLM_ARCH_QWEN4EXP) {\n return {std::lcm(2*n_embd_q, blck_size_perf), granularity_kv};\n }\n@@ -1961,7 +1963,11 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {\n }\n \n ggml_tensor * llama_model_base::create_tensor(llama_model_loader & ml, const LLM_TN_IMPL & tn, const std::initializer_list & ne, int flags) {\n- const buft_list_t * buft_list_layer = tn.bid == -1 ? nullptr : pimpl->dev_layer.at(tn.bid).buft_list;\n+ const buft_list_t * buft_list_layer = nullptr;\n+ if (tn.bid != -1) {\n+ // blocks that are not model layers (e.g. the blocks of a head) go with the output\n+ buft_list_layer = (size_t) tn.bid < pimpl->dev_layer.size() ? pimpl->dev_layer.at(tn.bid).buft_list : pimpl->dev_output.buft_list;\n+ }\n return ml.create_tensor(\n hparams, &pimpl->cpu_buft_list, pimpl->dev_input.buft_list, pimpl->dev_output.buft_list, buft_list_layer,\n tn, ne, flags);\n@@ -2369,6 +2375,7 @@ llama_memory_i * llama_model::create_memory(const llama_memory_params & params,\n case LLM_ARCH_LLADA:\n case LLM_ARCH_LLADA_MOE:\n case LLM_ARCH_RND1:\n+ case LLM_ARCH_CLEF:\n {\n res = nullptr;\n } break;\n@@ -3200,6 +3207,7 @@ llama_rope_type llama_model_rope_type(const llama_model * model) {\n case LLM_ARCH_QWEN3VLMOE:\n case LLM_ARCH_QWEN35:\n case LLM_ARCH_QWEN35MOE:\n+ case LLM_ARCH_CLEF:\n case LLM_ARCH_QWEN4EXP:\n case LLM_ARCH_QWEN3TTS:\n return LLAMA_ROPE_TYPE_IMROPE;"},{"sha":"1911efa7e22110dbbb35660d52e6dcc350070159","filename":"src/models/clef.cpp","status":"added","additions":616,"deletions":0,"changes":616,"blob_url":"https://github.com/ggml-org/llama.cpp/blob/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fmodels%2Fclef.cpp","raw_url":"https://github.com/ggml-org/llama.cpp/raw/99b95488cac0f00ce3f05af113a8c1e287753f87/src%2Fmodels%2Fclef.cpp","contents_url":"https://api.github.com/repos/ggml-org/llama.cpp/contents/src%2Fmodels%2Fclef.cpp?ref=99b95488cac0f00ce3f05af113a8c1e287753f87","patch":"@@ -0,0 +1,616 @@\n+#include \"models.h\"\n+\n+#include \"llama-ext.h\"\n+\n+#include \n+\n+void llama_model_clef::load_arch_hparams(llama_model_loader & ml) {\n+ llama_model_qwen35::load_arch_hparams(ml);\n+\n+ ml.get_key(LLM_KV_DECISION_ROUTING_BLOCK_COUNT, n_layer_routing);\n+ ml.get_key(LLM_KV_DECISION_BLOCK_COUNT, n_layer_joint);\n+ ml.get_key(LLM_KV_DECISION_HEAD_COUNT, n_head_decision);\n+\n+ if (n_head_decision == 0 || n_layer_routing > LLAMA_MAX_LAYERS || n_layer_joint > LLAMA_MAX_LAYERS) {\n+ throw std::runtime_error(\"invalid size of the decision head\");\n+ }\n+\n+ // used by the head\n+ ml.get_key(LLM_KV_ATTENTION_LAYERNORM_EPS, hparams.f_norm_eps);\n+\n+ // the output is one score per token, see llama_batch_ext_set_decision_order()\n+ hparams.n_embd_out_impl = 1;\n+}\n+\n+void llama_model_clef::load_arch_tensors(llama_model_loader & ml) {\n+ llama_model_qwen35::load_arch_tensors(ml);\n+\n+ LLAMA_LOAD_LOCALS;\n+\n+ const auto * w_memory = ml.get_weight(tn(LLM_TENSOR_DECISION_PROJ_MEMORY, \"weight\").str().c_str());\n+ const auto * w_ffn = ml.get_weight(tn(LLM_TENSOR_DEC_FFN_UP, \"weight\", 0).str().c_str());\n+ if (w_memory == nullptr || w_ffn == nullptr) {\n+ throw std::runtime_error(\"the decision head is missing\");\n+ }\n+ const int64_t n_embd_h = w_memory->tensor->ne[1];\n+ const int64_t n_ff_h = w_ffn->tensor->ne[1];\n+\n+ if (n_embd_h % n_head_decision != 0) {\n+ throw std::runtime_error(\"invalid width of the decision head\");\n+ }\n+\n+ auto load_norm = [&](norm & n, llm_tensor type, int64_t size, int il = -1) {\n+ n.w = il < 0 ? create_tensor(tn(type, \"weight\"), {size}, 0) : create_tensor(tn(type, \"weight\", il), {size}, 0);\n+ n.b = il < 0 ? create_tensor(tn(type, \"bias\"), {size}, 0) : create_tensor(tn(type, \"bias\", il), {size}, 0);\n+ };\n+\n+ auto load_attn = [&](attn & a, llm_tensor q, llm_tensor k, llm_tensor v, llm_tensor o, int il) {\n+ a.wq = create_tensor(tn(q, \"weight\", il), {n_embd_h, n_embd_h}, 0);\n+ a.bq = create_tensor(tn(q, \"bias\", il), {n_embd_h}, 0);\n+ a.wk = create_tensor(tn(k, \"weight\", il), {n_embd_h, n_embd_h}, 0);\n+ a.bk = create_tensor(tn(k, \"bias\", il), {n_embd_h}, 0);\n+ a.wv = create_tensor(tn(v, \"weight\", il), {n_embd_h, n_embd_h}, 0);\n+ a.bv = create_tensor(tn(v, \"bias\", il), {n_embd_h}, 0);\n+ a.wo = create_tensor(tn(o, \"weight\", il), {n_embd_h, n_embd_h}, 0);\n+ a.bo = create_tensor(tn(o, \"bias\", il), {n_embd_h}, 0);\n+ };\n+\n+ head_layers.resize(n_layer_routing + n_layer_joint);\n+ for (int il = 0; il < (int) head_layers.size(); ++il) {\n+ auto & layer = head_layers[il];\n+\n+ if (il < (int) n_layer_routing) {\n+ load_norm(layer.cross_norm_kv, LLM_TENSOR_DEC_CROSS_ATTN_NORM_KV, n_embd_h, il);\n+ } else {\n+ load_norm(layer.self_norm, LLM_TENSOR_DEC_ATTN_NORM, n_embd_h, il);\n+ load_attn(layer.self_attn, LLM_TENSOR_DEC_ATTN_Q, LLM_TENSOR_DEC_ATTN_K, LLM_TENSOR_DEC_ATTN_V, LLM_TENSOR_DEC_ATTN_OUT, il);\n+ }\n+\n+ load_norm(layer.cross_norm, LLM_TENSOR_DEC_CROSS_ATTN_NORM, n_embd_h, il);\n+ load_attn(layer.cross_attn, LLM_TENSOR_DEC_CROSS_ATTN_Q, LLM_TENSOR_DEC_CROSS_ATTN_K, LLM_TENSOR_DEC_CROSS_ATTN_V, LLM_TENSOR_DEC_CROSS_ATTN_OUT, il);\n+\n+ load_norm(layer.ffn_norm, LLM_TENSOR_DEC_FFN_NORM, n_embd_h, il);\n+ layer.ffn_up = create_tensor(tn(LLM_TENSOR_DEC_FFN_UP, \"weight\", il), {n_embd_h, n_ff_h}, 0);\n+ layer.ffn_up_b = create_tensor(tn(LLM_TENSOR_DEC_FFN_UP, \"bias\", il), {n_ff_h}, 0);\n+ layer.ffn_down = create_tensor(tn(LLM_TENSOR_DEC_FFN_DOWN, \"weight\", il), {n_ff_h, n_embd_h}, 0);\n+ layer.ffn_down_b = create_tensor(tn(LLM_TENSOR_DEC_FFN_DOWN, \"bias\", il), {n_embd_h}, 0);\n+ }\n+\n+ load_norm(hidden_norm, LLM_TENSOR_DECISION_HIDDEN_NORM, n_embd);\n+ load_norm(option_summary_norm, LLM_TENSOR_DECISION_OPTION_SUMMARY_NORM, n_embd_h);\n+ load_norm(field_norm, LLM_TENSOR_DECISION_FIELD_NORM, n_embd_h);\n+ load_norm(option_norm, LLM_TENSOR_DECISION_OPTION_NORM, n_embd_h);\n+\n+ proj_memory = create_tensor(tn(LLM_TENSOR_DECISION_PROJ_MEMORY, \"weight\"), {n_embd, n_embd_h}, 0);\n+ proj_question = create_tensor(tn(LLM_TENSOR_DECISION_PROJ_QUESTION, \"weight\"), {n_embd, n_embd_h}, 0);\n+ proj_option_question = create_tensor(tn(LLM_TENSOR_DECISION_PROJ_OPTION_QUESTION, \"weight\"), {n_embd, n_embd_h}, 0);\n+ proj_global = create_tensor(tn(LLM_TENSOR_DECISION_PROJ_GLOBAL, \"weight\"), {n_embd, n_embd_h}, 0);\n+ proj_option_context = create_tensor(tn(LLM_TENSOR_DECISION_PROJ_OPTION_CONTEXT, \"weight\"), {n_embd, n_embd_h}, 0);\n+ proj_option_lexical = create_tensor(tn(LLM_TENSOR_DECISION_PROJ_OPTION_LEXICAL, \"weight\"), {n_embd, n_embd_h}, 0);\n+\n+ scales = create_tensor(tn(LLM_TENSOR_DECISION_SCALES), {3}, 0);\n+ type_embd = create_tensor(tn(LLM_TENSOR_TOKEN_TYPES, \"weight\"), {n_embd_h, 3}, 0);\n+\n+ scorer = create_tensor(tn(LLM_TENSOR_DECISION_SCORER, \"weight\"), {4 * n_embd_h, n_embd_h}, 0);\n+ scorer_b = create_tensor(tn(LLM_TENSOR_DECISION_SCORER, \"bias\"), {n_embd_h}, 0);\n+ scorer_out = create_tensor(tn(LLM_TENSOR_DECISION_SCORER_OUT, \"weight\"), {n_embd_h, 1}, 0);\n+ scorer_out_b = create_tensor(tn(LLM_TENSOR_DECISION_SCORER_OUT, \"bias\"), {1}, 0);\n+}\n+\n+std::unique_ptr llama_model_clef::build_arch_graph(const llm_graph_params & params) const {\n+ return std::make_unique(*this, params);\n+}\n+\n+// spans read by the head, [start, end) in ubatch token indices\n+struct clef_spans {\n+ struct question {\n+ int32_t type; // noul, choice, score\n+ int32_t start;\n+ int32_t end;\n+ };\n+ struct option {\n+ int32_t question;\n+ int32_t start;\n+ int32_t end;\n+ };\n+ std::vector questions;\n+ std::vector