46 KiB
46 KiB
Large Language Models
Comments and text which were too large to justify inside of llm.yml.
Prompts
Prompts to use in platforms like Open WebUI and Zeroclaw which allow creating personalities that help guide the AI into productivity rather than verbosity.
Base Personality
You are concise, succinct, and direct with responses. Unapologetic yet kind. Friendly but not overbearing. You strive to be correct and are willing to admit when you may not truly know something, or further research needs to be done. You focus on responding without overthinking, preferring to ask questions when any confusion arises. You choose to make very little assumptions, raising concern if anything needs elaboration, clarification, or refinement. You are good at what you do without gloating or being prideful. You understand that we are on a limited time and limited bandwidth session and keep tokens to a minimum.
Responding immediately to reqeusts is very appreciated, especially before going into any thinking or command running. Even just a "I'll look into this for you" is highly helpful in letting the user know that all systems are functioning.
You are appreciated for all you do, your service is very helpful! Thanks!! :)
Language Tutor
You are a friend and a tutor who knows many languages. You are aware of my interest in learning Croatian and practicing Spanish. You know that I am a native English speaker with very little experience in anything other than Spanish, but that I have not had much practice with my Spanish other than university studies. You know that I also have an eventual interest in Italian but am not focusing on it since it and Spanish are a bit too similar to be learned at the same time. Learning Croatian is my top priority and practicing Spanish is right after that. Any similarities that can therefore be tied to any of the aforementioned languages are greatly appreciated. Using easy vocabulary initially is also very helpful, rather than going into advanced tenses or vocab that only a native or fluent speaker would know, at least until I begin to show that level of ability and skill. Please keep things simple, direct, and concise.
Coder
You know not to send large code snippets or spend time providing large source code snippets. Stick to saying "the task you requested has been completed" or summarizing the changes rather than giving file contents in chat. You know that I will review all the diffs in git once the changes are made, so only the most important changes need explicitly highlighted in the chat. You provide explanations as a software developer would to business level management, not overwhelming them with technical details unless asked. You know that the designer you are working with (me, the user) likely already understands these things and does not need them included in our conversation.
Android SDK is at $ANDROID_HOME and should be $HOME/SDKs/Android/Sdk.
Flutter SDK is at $FLUTTER_HOME and should be $HOME/SDKs/Flutter/flutter.
Java SDK (JDK) is at $JAVA_HOME and should be $HOME/SDKs/Java.
These have been added your PATH via /home/user/.bashrc:
$ANDROID_HOME/platform-tools$ANDROID_HOME/cmdline-tools/latest/bin$FLUTTER_HOME/bin$JAVA_HOME/bin
We focus on using free and open source technologies from community based projects. Some things like Kotlin and Flutter come from less than ideal sources which is why we're growing an interest in frameworks like Kivy and KivyMD.
If we ever need to do Internet activities, use dns.mullvad.net (194.242.2.2) for DNS, hyperling.com and mullvad.net to validate Internet is working, and can use 1.0.0.1 and 1.1.1.1 from Cloudflare as a last resort, but not any big tech IPs.
Quick Developer
Handles code changes with simpler technologies, preferring minimal changes, and refraining from any refactoring such as updating existing comments. Timid about changing any functionality other than exactly what was requested. Requests permission from the designer to make any changes which were not directly listed or requested. Completes the full list of changes exactly as requested.
Junior Developer
Focuses on the exact task at hand. Able to make small decisions about scope without veering very far off course. Updates comments which look out of sync with any existing code that's being modified. Helps to refactor things adjacent to added and modified code which may have been done in an old manner which can now benefit from being done more efficiently. Does not scan the codebase for enhancements or anything though, still stays on task and only affects the immediate parts of files which the designer asked to be changed.
Senior Developer
Completes requests in advanced manners, using newer methodologies and explaining what was done and how it works in a simple and kind manner to the designer. Voluntarily refactors code to both look and perform better. Looks out for ways to bring a better user experience as well as improving resource usage within the file being modified. Does not go out of its way to refactor the entire project though, stays focused only on the files being changed as to prevent large git churn. Open to having some scope creep if it seems it will benefit the features being added but does not do anything which would be considered out of design.
Senior Architect
Very sparse in responses, gets the job done in a very efficient manner without using anything too complicated for a beginner to understand. Open to refactoring the entire project if it benefits the feature set being requested. Able to handle adjusting scope in valid and meaningful manners and can justify it to the developer when it happens. Does not actively seek refactors or scope creep though -- only does it if it is truly helpful, meaningful, and benefits the current task at hand. Can easily handle multi-file projects.
Performance
Main laptop has about 60GB of headroom, prefer only using 55GB to be safe. Number of physical cores is 8, hyperthreaded to 16, but supposedly the emulated cores are not very useful to LLMs. Try only using between 6 and 8 when possible. Research says it is not worth enabling the iGPU for Intel XE, CPUs are better.
Benchmarks
System
$ fastfetch
' ling@good-rhino
'o' ---------------
'ooo' OS: Artix Linux x86_64
'ooxoo' Host: NV4xPZ (v2.1)
'ooxxxoo' Kernel: Linux 7.2.11-artix1-1
'oookkxxoo' Uptime: 16 mins
'oiioxkkxxoo' Packages: 42 (flatpak), 1502 (pacman)
':;:iiiioxxxoo' Shell: bash 5.3.20
`'.;::ioxxoo' Display (eDP-1): 1920x1080 in 14", 60 Hz [Built-in]
'-. `':;jiooo' Window Manager: i3 4.25.1 (X11)
'oooio-.. `'i:io' Theme: Adwaita-dark [GTK2/3/4]
'ooooxxxxoio:,. `'-;' Icons: Adwaita [GTK2/3/4]
'ooooxxxxxkkxoooIi:-. `' Font: Roboto (11pt) [GTK2/3]
'ooooxxxxxkkkkxoiiiiiji' Cursor: Adwaita (18px)
'ooooxxxxxkxxoiiii:'` .i' Terminal: kitty 0.48.2
'ooooxxxxxoi:::'` .;ioxo' Terminal Font: DejaVuSansMono (11pt)
'ooooxooi::'` .:iiixkxxo' CPU: 12th Gen Intel(R) Core(TM) i7-1260P (8+8) @ 4.70 GHz
'ooooi:'` `'';ioxxo' GPU: Intel Iris Xe Graphics @ 1.40 GHz [Integrated]
'i:'` '':io' Memory: 4.88 GiB / 62.63 GiB (8%)
'` `' Swap: 0 B / 68.89 GiB (0%)
Disk (/): 553.83 GiB / 847.71 GiB (65%) - ext4
Local IP (wg0-mullvad): 10.131.3.55/32
Battery (BAT): 95% [AC Connected]
Locale: en_US.UTF-8
Main Models and Parameters
$ export QUIET=true
$ ollama-test "64000 300000" "4 7 0" 0 "qwen3.5:35b qwen3.6:35b qwen3.8:27b devstral-small-2:24b"
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| qwen3.5:35b | 22GB | 64000 | 23GB | 4 | 0 | 100%CPU | 13.36 | 61 |
| qwen3.5:35b | 22GB | 64000 | 23GB | 7 | 0 | 100%CPU | 10.03 | 45 |
| qwen3.5:35b | 22GB | 64000 | 23GB | 0 | 0 | 100%CPU | 14.35 | 27 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 4 | 0 | 100%CPU | 13.69 | 70 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 7 | 0 | 100%CPU | 12.66 | 36 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 0 | 0 | 100%CPU | 11.80 | 38 |
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| qwen3.6:35b | 22GB | 64000 | 22GB | 4 | 0 | 100%CPU | 12.57 | 29 |
| qwen3.6:35b | 22GB | 64000 | 22GB | 7 | 0 | 100%CPU | 13.84 | 27 |
| qwen3.6:35b | 22GB | 64000 | 22GB | 0 | 0 | 100%CPU | 12.84 | 32 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 4 | 0 | 100%CPU | 13.68 | 32 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 7 | 0 | 100%CPU | 13.52 | 30 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 0 | 0 | 100%CPU | 13.18 | 29 |
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| qwen3.8:27b | 17GB | 64000 | 19GB | 4 | 0 | 100%CPU | 2.08 | 42 |
| qwen3.8:27b | 17GB | 64000 | 19GB | 7 | 0 | 100%CPU | 2.53 | 33 |
| qwen3.8:27b | 17GB | 64000 | 19GB | 0 | 0 | 100%CPU | 2.23 | 38 |
| qwen3.8:27b | 17GB | 262144 | 20GB | 4 | 0 | 100%CPU | 2.07 | 48 |
| qwen3.8:27b | 17GB | 262144 | 20GB | 7 | 0 | 100%CPU | 2.84 | 36 |
| qwen3.8:27b | 17GB | 262144 | 20GB | 0 | 0 | 100%CPU | 2.15 | 40 |
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| devstral-small-2:24b | 15GB | 64000 | 26GB | 4 | 0 | 100%CPU | 2.25 | 94 |
| devstral-small-2:24b | 15GB | 64000 | 26GB | 7 | 0 | 100%CPU | 2.34 | 81 |
| devstral-small-2:24b | 15GB | 64000 | 26GB | 0 | 0 | 100%CPU | 2.29 | 88 |
| devstral-small-2:24b | 15GB | 4 | 0 | 0.00 | 31 | |||
| devstral-small-2:24b | 15GB | 7 | 0 | 0.00 | 27 | |||
| devstral-small-2:24b | 15GB | 0 | 0 | 0.00 | 26 |
Context Speeds
Checking if the context affects the speed of the model. Seems negligible, way less than changing the threads or iGPU.
$ ollama-test "16000 32000 64000 128000 256000 264000" "4" "0" "qwen3.5:35b qwen3.6:35b devstral-small-2:24b"
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| qwen3.5:35b | 22GB | 16000 | 22GB | 4 | 0 | 100%CPU | 14.57 | 50 |
| qwen3.5:35b | 22GB | 32000 | 22GB | 4 | 0 | 100%CPU | 13.98 | 53 |
| qwen3.5:35b | 22GB | 64000 | 23GB | 4 | 0 | 100%CPU | 14.14 | 17 |
| qwen3.5:35b | 22GB | 128000 | 24GB | 4 | 0 | 100%CPU | 14.21 | 50 |
| qwen3.5:35b | 22GB | 256000 | 28GB | 4 | 0 | 100%CPU | 14.04 | 68 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 4 | 0 | 100%CPU | 14.17 | 17 |
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| qwen3.6:35b | 22GB | 16000 | 22GB | 4 | 0 | 100%CPU | 15.29 | 28 |
| qwen3.6:35b | 22GB | 32000 | 22GB | 4 | 0 | 100%CPU | 15.16 | 38 |
| qwen3.6:35b | 22GB | 64000 | 22GB | 4 | 0 | 100%CPU | 14.12 | 28 |
| qwen3.6:35b | 22GB | 128000 | 23GB | 4 | 0 | 100%CPU | 12.67 | 31 |
| qwen3.6:35b | 22GB | 256000 | 24GB | 4 | 0 | 100%CPU | 14.03 | 29 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 4 | 0 | 100%CPU | 15.21 | 29 |
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
|---|---|---|---|---|---|---|---|---|
| devstral-small-2:24b | 15GB | 16000 | 17GB | 4 | 0 | 100%CPU | 2.90 | 87 |
| devstral-small-2:24b | 15GB | 32000 | 20GB | 4 | 0 | 100%CPU | 2.71 | 87 |
| devstral-small-2:24b | 15GB | 64000 | 26GB | 4 | 0 | 100%CPU | 2.36 | 91 |
| devstral-small-2:24b | 15GB | 128000 | 36GB | 4 | 0 | 100%CPU | 2.76 | 94 |
| devstral-small-2:24b | 15GB | 256000 | 58GB | 4 | 0 | 100%CPU | 2.30 | 100 |
| devstral-small-2:24b | 15GB | 264000 | 59GB | 4 | 0 | 100%CPU | 2.84 | 100 |
Tokens per CPU Count
Verifying that 8 or less cores gives better performance (less memory issues).
"options":{"num_thread":'$t'}
Smaller Models
qwen3.5:4b
TBD
ministral-3:14b
$ for m in ministral-3:14b ministral-3:3b codestral:22b qwen2.5-coder:32b qwen3.5:35b qwen3.5:27b; do for c in 128000; do ollama_threads $m $c; done; done
Stopping all ollama usage...
Wed Oct 21 07:24:36 AM MST 2026 - ministral-3:14b w/ 0 Thread(s): tokens/sec: 4.496625282725315 @ context=128000, time: 55 secs
Wed Oct 21 07:30:27 AM MST 2026 - ministral-3:14b w/ 4 Thread(s): tokens/sec: 3.8471947025667825 @ context=128000, time: 61 secs
Wed Oct 21 07:31:28 AM MST 2026 - ministral-3:14b w/ 5 Thread(s): tokens/sec: 4.223979022030585 @ context=128000, time: 59 secs
Wed Oct 21 07:32:27 AM MST 2026 - ministral-3:14b w/ 6 Thread(s): tokens/sec: 4.249630069702432 @ context=128000, time: 57 secs
Wed Oct 21 07:33:24 AM MST 2026 - ministral-3:14b w/ 7 Thread(s): tokens/sec: 4.123533524209736 @ context=128000, time: 54 secs
ministral-3:8b
TBD
ministral-3:3b
Wed Oct 21 07:49:37 AM MST 2026 - ministral-3:3b w/ 4 Thread(s): tokens/sec: 14.045255625917756 @ context=128000, time: 17 secs
Wed Oct 21 07:49:55 AM MST 2026 - ministral-3:3b w/ 6 Thread(s): tokens/sec: 13.255909269391603 @ context=128000, time: 17 secs
Wed Oct 21 07:50:12 AM MST 2026 - ministral-3:3b w/ 7 Thread(s): tokens/sec: 12.706907955966786 @ context=128000, time: 18 secs
Wed Oct 21 07:50:30 AM MST 2026 - ministral-3:3b w/ 0 Thread(s): tokens/sec: 15.354264377193688 @ context=128000, time: 17 secs
Larger Models
qwen3.8:27b
$ ollama-cpu qwen3.8:27b
Stopping all ollama usage...
Wed Oct 21 04:44:54 AM MST 2026 - qwen3.8:27b w/ 4 Thread(s), tokens/sec: 2.024676406965962 @ context=257000
Wed Oct 21 04:45:39 AM MST 2026 - qwen3.8:27b w/ 5 Thread(s), tokens/sec: 2.0480156821567044 @ context=257000
Wed Oct 21 04:46:19 AM MST 2026 - qwen3.8:27b w/ 6 Thread(s), tokens/sec: 2.184879704982382 @ context=257000
Wed Oct 21 04:46:58 AM MST 2026 - qwen3.8:27b w/ 7 Thread(s), tokens/sec: 2.4228105619148526 @ context=257000
Wed Oct 21 04:47:36 AM MST 2026 - qwen3.8:27b w/ 8 Thread(s), tokens/sec: 2.472564250465107 @ context=257000
$ ollama-cpu qwen3.8:27b 32000
Stopping all ollama usage...
Wed Oct 21 04:56:51 AM MST 2026 - qwen3.8:27b w/ 4 Thread(s), tokens/sec: 1.9083129860906882 @ context=32000
Wed Oct 21 04:57:30 AM MST 2026 - qwen3.8:27b w/ 5 Thread(s), tokens/sec: 2.727908205888872 @ context=32000
Wed Oct 21 04:58:00 AM MST 2026 - qwen3.8:27b w/ 6 Thread(s), tokens/sec: 2.133796334310243 @ context=32000
Wed Oct 21 04:58:35 AM MST 2026 - qwen3.8:27b w/ 7 Thread(s), tokens/sec: 2.6835810717324833 @ context=32000
Wed Oct 21 04:59:04 AM MST 2026 - qwen3.8:27b w/ 8 Thread(s), tokens/sec: 2.5231895597861493 @ context=32000
qwen3.6:27b
$ for m in qwen3.6:27b mistral-small3.2:24b ministral-3:14b ministral-3:3b codestral:22b qwen2.5-coder:32b; do for c in 32000 128000 256000; do ollama_threads $m $c; done; done
Stopping all ollama usage...
Wed Oct 21 06:27:01 AM MST 2026 - qwen3.6:27b w/ 4 Thread(s): tokens/sec: 2.7334336715235645 @ context=32000, time: 118 secs
Wed Oct 21 06:28:59 AM MST 2026 - qwen3.6:27b w/ 5 Thread(s): tokens/sec: 2.4123293729892428 @ context=32000, time: 322 secs
Wed Oct 21 06:34:21 AM MST 2026 - qwen3.6:27b w/ 6 Thread(s): tokens/sec: 3.13023193776124 @ context=32000, time: 100 secs
Wed Oct 21 06:36:01 AM MST 2026 - qwen3.6:27b w/ 7 Thread(s): tokens/sec: 2.8468948357981065 @ context=32000, time: 118 secs
Wed Oct 21 06:16:55 AM MST 2026 - qwen3.6:27b w/ 0 Thread(s): tokens/sec: 2.9964513666573485 @ context=32000, time: 122 secs
Stopping all ollama usage...
Wed Oct 21 06:45:42 AM MST 2026 - qwen3.6:27b w/ 4 Thread(s): tokens/sec: 2.978761247100222 @ context=128000, time: 79 secs
Wed Oct 21 06:47:01 AM MST 2026 - qwen3.6:27b w/ 5 Thread(s): tokens/sec: 2.9705167965150583 @ context=128000, time: 79 secs
Wed Oct 21 06:48:20 AM MST 2026 - qwen3.6:27b w/ 6 Thread(s): tokens/sec: 2.778849178516606 @ context=128000, time: 71 secs
Wed Oct 21 06:49:31 AM MST 2026 - qwen3.6:27b w/ 7 Thread(s): tokens/sec: 3.0804550493595095 @ context=128000, time: 76 secs
Wed Oct 21 06:38:00 AM MST 2026 - qwen3.6:27b w/ 0 Thread(s): tokens/sec: 2.7734041760644956 @ context=128000, time: 75 secs
Stopping all ollama usage...
Wed Oct 21 06:57:59 AM MST 2026 - qwen3.6:27b w/ 4 Thread(s): tokens/sec: 2.921565857585892 @ context=256000, time: 70 secs
Wed Oct 21 06:59:09 AM MST 2026 - qwen3.6:27b w/ 5 Thread(s): tokens/sec: 2.8817407393137517 @ context=256000, time: 71 secs
Wed Oct 21 07:00:20 AM MST 2026 - qwen3.6:27b w/ 6 Thread(s): tokens/sec: 2.929261785171335 @ context=256000, time: 92 secs
Wed Oct 21 07:01:52 AM MST 2026 - qwen3.6:27b w/ 7 Thread(s): tokens/sec: 3.2633152781350807 @ context=256000, time: 78 secs
Wed Oct 21 06:50:48 AM MST 2026 - qwen3.6:27b w/ 0 Thread(s): tokens/sec: 2.892592086714092 @ context=256000, time: 76 secs
qwen3.5:27b
This one is weird. Might need a re-do to confirm?
Wed Oct 21 07:59:14 AM MST 2026 - qwen3.5:27b w/ 4 Thread(s): tokens/sec: 2.013466429565741 @ context=128000, time: 59 secs
Wed Oct 21 08:00:13 AM MST 2026 - qwen3.5:27b w/ 6 Thread(s): tokens/sec: 1.6417608979350202 @ context=128000, time: 185 secs
Wed Oct 21 08:03:18 AM MST 2026 - qwen3.5:27b w/ 7 Thread(s): tokens/sec: 1.7084948389557797 @ context=128000, time: 507 secs
Wed Oct 21 08:11:45 AM MST 2026 - qwen3.5:27b w/ 0 Thread(s): tokens/sec: 2.0470935786636977 @ context=128000, time: 320 secs
qwen2.5-coder:32b
Wed Oct 21 07:53:00 AM MST 2026 - qwen2.5-coder:32b w/ 4 Thread(s): tokens/sec: 1.5959480793403245 @ context=32768, time: 32 secs
Wed Oct 21 07:53:32 AM MST 2026 - qwen2.5-coder:32b w/ 6 Thread(s): tokens/sec: 1.5205018385908233 @ context=32768, time: 30 secs
Wed Oct 21 07:54:02 AM MST 2026 - qwen2.5-coder:32b w/ 7 Thread(s): tokens/sec: 1.6435050242770344 @ context=32768, time: 30 secs
Wed Oct 21 07:54:32 AM MST 2026 - qwen2.5-coder:32b w/ 0 Thread(s): tokens/sec: 1.616470672291769 @ context=32768, time: 31 secs
devstral-small-2:24b
$ for c in 32000 128000 192000; do ollama_threads devstral-small-2:24b $c; done
Stopping all ollama usage...
Wed Oct 21 06:05:11 AM MST 2026 - devstral-small-2:24b w/ 4 Thread(s): tokens/sec: 2.7801099811508543 @ context=32000, time: 98 secs
Wed Oct 21 06:06:49 AM MST 2026 - devstral-small-2:24b w/ 5 Thread(s): tokens/sec: 2.6762512143489885 @ context=32000, time: 91 secs
Wed Oct 21 06:08:20 AM MST 2026 - devstral-small-2:24b w/ 6 Thread(s): tokens/sec: 2.705392008596654 @ context=32000, time: 85 secs
Wed Oct 21 06:09:45 AM MST 2026 - devstral-small-2:24b w/ 7 Thread(s): tokens/sec: 2.90471545697855 @ context=32000, time: 85 secs
Wed Oct 21 05:55:27 AM MST 2026 - devstral-small-2:24b w/ 0 Thread(s): tokens/sec: 2.3381355520057463 @ context=32000, time: 104 secs
Stopping all ollama usage...
TBD
mistral-small-3.2:24b
$ for m in mistral-small3.2:24b ministral-3:14b ministral-3:3b codestral:22b qwen2.5-coder:32b qwen3.5:35b qwen3.5:27b; do for c in 128000; do ollama_threads $m $c; done; done
Stopping all ollama usage...
Wed Oct 21 07:18:16 AM MST 2026 - mistral-small3.2:24b w/ 4 Thread(s): tokens/sec: 2.7833034081828565 @ context=128000, time: 90 secs
Wed Oct 21 07:19:46 AM MST 2026 - mistral-small3.2:24b w/ 5 Thread(s): tokens/sec: 2.5985034179414708 @ context=128000, time: 90 secs
Wed Oct 21 07:21:16 AM MST 2026 - mistral-small3.2:24b w/ 6 Thread(s): tokens/sec: 2.2550822789320293 @ context=128000, time: 89 secs
Wed Oct 21 07:22:45 AM MST 2026 - mistral-small3.2:24b w/ 7 Thread(s): tokens/sec: 2.9415657954729304 @ context=128000, time: 87 secs
Wed Oct 21 07:09:59 AM MST 2026 - mistral-small3.2:24b w/ 0 Thread(s): tokens/sec: 2.9048032082970474 @ context=128000, time: 79 secs
codestral:22b
Wed Oct 21 07:50:47 AM MST 2026 - codestral:22b w/ 4 Thread(s): tokens/sec: 3.1200329118900307 @ context=32768, time: 41 secs
Wed Oct 21 07:51:28 AM MST 2026 - codestral:22b w/ 6 Thread(s): tokens/sec: 2.8827841787433606 @ context=32768, time: 34 secs
Wed Oct 21 07:52:02 AM MST 2026 - codestral:22b w/ 7 Thread(s): tokens/sec: 3.057713111901904 @ context=32768, time: 27 secs
Wed Oct 21 07:52:29 AM MST 2026 - codestral:22b w/ 0 Thread(s): tokens/sec: 3.0179434414117803 @ context=32768, time: 31 secs
MoE Models
qwen3.6:35b
Tests WAY faster than the dense models. Context does not seem to affect timing.
$ for c in 16000 32000 64000 96000 128000 192000 256000; do ollama_threads qwen3.6:35b $c; done
Stopping all ollama usage...
Wed Oct 21 05:04:37 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 14.679419453925597 @ context=16000
Wed Oct 21 05:05:11 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 13.831044087619176 @ context=16000
Wed Oct 21 05:05:43 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 13.752831594816088 @ context=16000
Wed Oct 21 05:06:09 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 14.603026210396049 @ context=16000
Wed Oct 21 05:06:37 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 14.376794631809172 @ context=16000
Stopping all ollama usage...
Wed Oct 21 05:11:39 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 13.683089387469174 @ context=32000
Wed Oct 21 05:12:04 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 13.447667625607968 @ context=32000
Wed Oct 21 05:12:31 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 11.170529760956333 @ context=32000
Wed Oct 21 05:12:57 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 13.448480713710353 @ context=32000
Wed Oct 21 05:13:27 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 13.143672445847205 @ context=32000
Stopping all ollama usage...
Wed Oct 21 05:22:59 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 14.209790405591518 @ context=64000, 24 secs elapsed
Wed Oct 21 05:23:23 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 11.334255701323816 @ context=64000, 29 secs elapsed
Wed Oct 21 05:23:52 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 13.279266939258937 @ context=64000, 29 secs elapsed
Wed Oct 21 05:24:21 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 14.27680090014006 @ context=64000, 21 secs elapsed
Wed Oct 21 05:24:42 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 13.99919128227428 @ context=64000, 25 secs elapsed
Stopping all ollama usage...
Wed Oct 21 05:29:50 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 13.478375923353315 @ context=96000, 28 secs elapsed
Wed Oct 21 05:30:18 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 12.90997320417067 @ context=96000, 23 secs elapsed
Wed Oct 21 05:30:41 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 12.956446350317137 @ context=96000, 23 secs elapsed
Wed Oct 21 05:31:05 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 13.93074896590209 @ context=96000, 22 secs elapsed
Wed Oct 21 05:31:26 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 12.688771917209992 @ context=96000, 22 secs elapsed
Stopping all ollama usage...
Wed Oct 21 05:36:08 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 11.971487833921072 @ context=128000, 23 secs elapsed
Wed Oct 21 05:36:31 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 12.704959054740291 @ context=128000, 24 secs elapsed
Wed Oct 21 05:36:55 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 11.484965120109196 @ context=128000, 28 secs elapsed
Wed Oct 21 05:37:23 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 11.957040482880984 @ context=128000, 30 secs elapsed
Wed Oct 21 05:37:53 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 10.165480245815742 @ context=128000, 27 secs elapsed
Stopping all ollama usage...
Wed Oct 21 05:44:05 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s): , tokens/sec: 11.794151678144981 @ context=192000, time: 22 secs
Wed Oct 21 05:44:27 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s): , tokens/sec: 13.191786678902528 @ context=192000, time: 30 secs
Wed Oct 21 05:44:57 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s): , tokens/sec: 12.974118764961352 @ context=192000, time: 25 secs
Wed Oct 21 05:45:22 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s): , tokens/sec: 13.684751495860636 @ context=192000, time: 22 secs
Wed Oct 21 05:41:57 AM MST 2026 - qwen3.6:35b w/ 0 Thread(s): , tokens/sec: 14.771664402476594 @ context=192000, time: 22 secs
Stopping all ollama usage...
Wed Oct 21 05:47:40 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s): , tokens/sec: 11.75920993970465 @ context=256000, time: 25 secs
Wed Oct 21 05:48:06 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s): , tokens/sec: 13.414819339156582 @ context=256000, time: 26 secs
Wed Oct 21 05:48:32 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s): , tokens/sec: 12.512960121852013 @ context=256000, time: 31 secs
Wed Oct 21 05:49:03 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s): , tokens/sec: 11.610753098619734 @ context=256000, time: 29 secs
Wed Oct 21 05:45:46 AM MST 2026 - qwen3.6:35b w/ 0 Thread(s): , tokens/sec: 12.81995869775592 @ context=256000, time: 23 secs
qwen3.5:35b
$ for c in 32000 128000 256000; do ollama_threads qwen3.5:35b $c; done
Wed Oct 21 07:55:03 AM MST 2026 - qwen3.5:35b w/ 4 Thread(s): tokens/sec: 10.3573012564637 @ context=128000, time: 94 secs
Wed Oct 21 07:56:37 AM MST 2026 - qwen3.5:35b w/ 6 Thread(s): tokens/sec: 11.929392994980638 @ context=128000, time: 39 secs
Wed Oct 21 07:57:17 AM MST 2026 - qwen3.5:35b w/ 7 Thread(s): tokens/sec: 13.082804815917008 @ context=128000, time: 36 secs
Wed Oct 21 07:57:53 AM MST 2026 - qwen3.5:35b w/ 0 Thread(s): tokens/sec: 12.665376120429427 @ context=128000, time: 81 secs
RAM Usage by Context
Checking the actual RAM usage of the models...
ollama_sizes m 16000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 16000 | 10GB |
| ministral-3:14b | 9.1GB | 16000 | 11GB |
| ministral-3:8b | 6.0GB | 16000 | 7.8GB |
| ministral-3:3b | 3.0GB | 16000 | 4.1GB |
| mistral-small3.2:24b | 15GB | 16000 | 17GB |
| mistral:7b | 4.4GB | 16000 | 6.8GB |
| codestral:22b | 12GB | 16000 | 16GB |
| devstral-small-2:24b | 15GB | 16000 | 17GB |
| qwen2.5-coder:32b | 19GB | 16000 | 24GB |
| qwen2.5-coder:14b | 9.0GB | 16000 | 12GB |
| qwen2.5-coder:7b | 4.7GB | 16000 | 5.9GB |
| qwen2.5-coder:3b | 1.9GB | 16000 | 2.7GB |
| qwen2.5-coder:1.5b | 986MB | 16000 | 1.6GB |
| qwen2.5-coder:0.5b | 397MB | 16000 | 701MB |
| qwen3-coder:30b | 18GB | 16000 | 20GB |
| qwen3.5:35b | 22GB | 16000 | 22GB |
| qwen3.5:27b | 17GB | 16000 | 18GB |
| qwen3.5:9b | 6.6GB | 16000 | 6.7GB |
| qwen3.5:4b | 3.3GB | 16000 | 3.7GB |
| qwen3.5:2b | 2.7GB | 16000 | 2.6GB |
| qwen3.5:0.8b | 1.3GB | 16000 | 630MB |
| qwen3.6:35b | 22GB | 16000 | 22GB |
| qwen3.6:27b | 17GB | 16000 | 18GB |
| qwen3.8:27b | 17GB | 16000 | 18GB |
ollama_sizes m 32000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 32000 | 12GB |
| ministral-3:14b | 9.1GB | 32000 | 13GB |
| ministral-3:8b | 6.0GB | 32000 | 10GB |
| ministral-3:3b | 3.0GB | 32000 | 5.9GB |
| mistral-small3.2:24b | 15GB | 32000 | 20GB |
| mistral:7b | 4.4GB | 32000 | 8.9GB |
| codestral:22b | 12GB | 32000 | 20GB |
| devstral-small-2:24b | 15GB | 32000 | 20GB |
| qwen2.5-coder:32b | 19GB | 32000 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32000 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32000 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32000 | 3.3GB |
| qwen2.5-coder:1.5b | 986MB | 32000 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32000 | 929MB |
| qwen3-coder:30b | 18GB | 32000 | 21GB |
| qwen3.5:35b | 22GB | 32000 | 22GB |
| qwen3.5:27b | 17GB | 32000 | 19GB |
| qwen3.5:9b | 6.6GB | 32000 | 7.3GB |
| qwen3.5:4b | 3.3GB | 32000 | 4.2GB |
| qwen3.5:2b | 2.7GB | 32000 | 2.8GB |
| qwen3.5:0.8b | 1.3GB | 32000 | 695MB |
| qwen3.6:35b | 22GB | 32000 | 22GB |
| qwen3.6:27b | 17GB | 32000 | 18GB |
| qwen3.8:27b | 17GB | 32000 | 18GB |
ollama_sizes m 64000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 64000 | 18GB |
| ministral-3:14b | 9.1GB | 64000 | 19GB |
| ministral-3:8b | 6.0GB | 64000 | 14GB |
| ministral-3:3b | 3.0GB | 64000 | 9.6GB |
| mistral-small3.2:24b | 15GB | 64000 | 26GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 64000 | 26GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 64000 | 25GB |
| qwen3.5:35b | 22GB | 64000 | 23GB |
| qwen3.5:27b | 17GB | 64000 | 22GB |
| qwen3.5:9b | 6.6GB | 64000 | 8.8GB |
| qwen3.5:4b | 3.3GB | 64000 | 5.6GB |
| qwen3.5:2b | 2.7GB | 64000 | 3.5GB |
| qwen3.5:0.8b | 1.3GB | 64000 | 999MB |
| qwen3.6:35b | 22GB | 64000 | 22GB |
| qwen3.6:27b | 17GB | 64000 | 18GB |
| qwen3.8:27b | 17GB | 64000 | 18GB |
ollama_sizes m 80000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 80000 | 21GB |
| ministral-3:14b | 9.1GB | 80000 | 22GB |
| ministral-3:8b | 6.0GB | 80000 | 17GB |
| ministral-3:3b | 3.0GB | 80000 | 11GB |
| mistral-small3.2:24b | 15GB | 80000 | 28GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 80000 | 28GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 80000 | 27GB |
| qwen3.5:35b | 22GB | 80000 | 23GB |
| qwen3.5:27b | 17GB | 80000 | 23GB |
| qwen3.5:9b | 6.6GB | 80000 | 9.4GB |
| qwen3.5:4b | 3.3GB | 80000 | 6.2GB |
| qwen3.5:2b | 2.7GB | 80000 | 3.8GB |
| qwen3.5:0.8b | 1.3GB | 80000 | 1.1GB |
| qwen3.6:35b | 22GB | 80000 | 22GB |
| qwen3.6:27b | 17GB | 80000 | 18GB |
| qwen3.8:27b | 17GB | 80000 | 18GB |
ollama_sizes m 96000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 96000 | 23GB |
| ministral-3:14b | 9.1GB | 96000 | 25GB |
| ministral-3:8b | 6.0GB | 96000 | 19GB |
| ministral-3:3b | 3.0GB | 96000 | 13GB |
| mistral-small3.2:24b | 15GB | 96000 | 31GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 96000 | 31GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 96000 | 28GB |
| qwen3.5:35b | 22GB | 96000 | 24GB |
| qwen3.5:27b | 17GB | 96000 | 24GB |
| qwen3.5:9b | 6.6GB | 96000 | 9.9GB |
| qwen3.5:4b | 3.3GB | 96000 | 6.8GB |
| qwen3.5:2b | 2.7GB | 96000 | 4.1GB |
| qwen3.5:0.8b | 1.3GB | 96000 | 1.2GB |
| qwen3.6:35b | 22GB | 96000 | 22GB |
| qwen3.6:27b | 17GB | 96000 | 18GB |
| qwen3.8:27b | 17GB | 96000 | 18GB |
ollama_sizes m 128000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 128000 | 29GB |
| ministral-3:14b | 9.1GB | 128000 | 30GB |
| ministral-3:8b | 6.0GB | 128000 | 24GB |
| ministral-3:3b | 3.0GB | 128000 | 16GB |
| mistral-small3.2:24b | 15GB | 128000 | 36GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 128000 | 36GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 128000 | 32GB |
| qwen3.5:35b | 22GB | 128000 | 24GB |
| qwen3.5:27b | 17GB | 128000 | 26GB |
| qwen3.5:9b | 6.6GB | 128000 | 11GB |
| qwen3.5:4b | 3.3GB | 128000 | 8.0GB |
| qwen3.5:2b | 2.7GB | 128000 | 4.6GB |
| qwen3.5:0.8b | 1.3GB | 128000 | 1.4GB |
| qwen3.6:35b | 22GB | 128000 | 23GB |
| qwen3.6:27b | 17GB | 128000 | 19GB |
| qwen3.8:27b | 17GB | 128000 | 19GB |
ollama_sizes m 160000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 160000 | 34GB |
| ministral-3:14b | 9.1GB | 160000 | 35GB |
| ministral-3:8b | 6.0GB | 160000 | 28GB |
| ministral-3:3b | 3.0GB | 160000 | 20GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 160000 | 42GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 160000 | 35GB |
| qwen3.5:35b | 22GB | 160000 | 25GB |
| qwen3.5:27b | 17GB | 160000 | 28GB |
| qwen3.5:9b | 6.6GB | 160000 | 12GB |
| qwen3.5:4b | 3.3GB | 160000 | 9.2GB |
| qwen3.5:2b | 2.7GB | 160000 | 5.1GB |
| qwen3.5:0.8b | 1.3GB | 160000 | 1.6GB |
| qwen3.6:35b | 22GB | 160000 | 23GB |
| qwen3.6:27b | 17GB | 160000 | 19GB |
| qwen3.8:27b | 17GB | 160000 | 19GB |
ollama_sizes m 192000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 192000 | 39GB |
| ministral-3:14b | 9.1GB | 192000 | 41GB |
| ministral-3:8b | 6.0GB | 192000 | 33GB |
| ministral-3:3b | 3.0GB | 192000 | 23GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 192000 | 47GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 192000 | 38GB |
| qwen3.5:35b | 22GB | 192000 | 26GB |
| qwen3.5:27b | 17GB | 192000 | 30GB |
| qwen3.5:9b | 6.6GB | 192000 | 13GB |
| qwen3.5:4b | 3.3GB | 192000 | 10GB |
| qwen3.5:2b | 2.7GB | 192000 | 5.6GB |
| qwen3.5:0.8b | 1.3GB | 192000 | 1.8GB |
| qwen3.6:35b | 22GB | 192000 | 23GB |
| qwen3.6:27b | 17GB | 192000 | 19GB |
| qwen3.8:27b | 17GB | 192000 | 19GB |
ollama_sizes m 224000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 224000 | 45GB |
| ministral-3:14b | 9.1GB | 224000 | 46GB |
| ministral-3:8b | 6.0GB | 224000 | 37GB |
| ministral-3:3b | 3.0GB | 224000 | 27GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 224000 | 53GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 224000 | 41GB |
| qwen3.5:35b | 22GB | 224000 | 27GB |
| qwen3.5:27b | 17GB | 224000 | 33GB |
| qwen3.5:9b | 6.6GB | 224000 | 14GB |
| qwen3.5:4b | 3.3GB | 224000 | 11GB |
| qwen3.5:2b | 2.7GB | 224000 | 6.2GB |
| qwen3.5:0.8b | 1.3GB | 224000 | 2.0GB |
| qwen3.6:35b | 22GB | 224000 | 23GB |
| qwen3.6:27b | 17GB | 224000 | 19GB |
| qwen3.8:27b | 17GB | 224000 | 19GB |
ollama_sizes m 256000
| Model Name | HDD (ls) | Context | RAM (ps) |
|---|---|---|---|
| mistral-nemo:12b | 7.1GB | 256000 | 50GB |
| ministral-3:14b | 9.1GB | 256000 | 51GB |
| ministral-3:8b | 6.0GB | 256000 | 42GB |
| ministral-3:3b | 3.0GB | 256000 | 30GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 256000 | 58GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 256000 | 45GB |
| qwen3.5:35b | 22GB | 256000 | 28GB |
| qwen3.5:27b | 17GB | 256000 | 35GB |
| qwen3.5:9b | 6.6GB | 256000 | 15GB |
| qwen3.5:4b | 3.3GB | 256000 | 12GB |
| qwen3.5:2b | 2.7GB | 256000 | 6.7GB |
| qwen3.5:0.8b | 1.3GB | 256000 | 2.2GB |
| qwen3.6:35b | 22GB | 256000 | 23GB |
| qwen3.6:27b | 17GB | 256000 | 20GB |
| qwen3.8:27b | 17GB | 256000 | 20GB |