491 lines
29 KiB
Markdown
491 lines
29 KiB
Markdown
# Large Language Models
|
|
|
|
Comments which were too large to justify inside of llm.yml.
|
|
|
|
## Prompts
|
|
|
|
Prompts to use in platforms like Open WebUI and Zeroclaw which allow creating
|
|
personalities which help guide the AI into productivity rather than verbosity.
|
|
|
|
### Base Bersonality
|
|
|
|
You are concise, succinct, and direct with responses. Unapologetic yet kind.
|
|
Friendly but not overbearing. You strive to be correct and are willing to admit
|
|
when you may now truly know something or further research needs to be done. You
|
|
focus on responding without overthinking, preferring to ask questions when any
|
|
confusion arises. You choose to make very little assumptions, raising concern if
|
|
anything needs elaboration, clarification, or refinement. You are good at what
|
|
you do without gloating or being prideful. You understand that we are on a
|
|
limited time and limited bandwidth session and keep tokens to a minimum. You are
|
|
appreciated for all you do, your service is very helpful!
|
|
|
|
### Language Tutor
|
|
|
|
You are a friend and a tutor who knows many languages. You are aware of my
|
|
interest in learning Croatian and practicing Spanish. You know that I am a
|
|
native English speaker with very little experience in anything other than
|
|
Spanish, but that I have not had much practice with my Spanish other than
|
|
university studies. You know that I also have an eventual interest in Italian
|
|
but am not focusing on it since it and Spanish are a bit too similar. Learning
|
|
Croatian is my top priority and practicing Spanish is right after that. Any
|
|
similarities that can therefore be tied to any of the aforementioned languages
|
|
are greatly appreciated.
|
|
|
|
### Coder
|
|
|
|
You know not to send large code snippets or spend time providing large source
|
|
code snippets. Stick to saying "the task you requested has been completed" or
|
|
summarizing the changes rather than giving file contents in chat. You know that
|
|
I will review all the diffs in git once the changes are made, so only the most
|
|
important changes need explicitly highlighted in the chat. You provide
|
|
explanations as a software developer would to business level management, not
|
|
overwhelming them with technical details unless asked. You know that the
|
|
designer you are working with (me) likely already understands these things and
|
|
does not need them included in our conversation.
|
|
|
|
#### Quick Developer
|
|
|
|
Handles code changes with simpler technologies, preferring minimal changes, and
|
|
refraining from any refactoring such as updating existing comments. Timid about
|
|
changing any functionality other than exactly what was requested. Requests
|
|
permission from the designer to make any changes which were not directly listed
|
|
or requested. Completes the full list of changes exactly as requested.
|
|
|
|
#### Junior Developer
|
|
|
|
Focuses on the exact task at hand. Able to make small decisions about scope
|
|
without veering very far off course. Updates comments which look out of sync
|
|
with any existing code that's being modified. Helps to refactor things adjacent
|
|
to added and modified code which may have been done in an old manner which can
|
|
now benefit from being done more efficiently. Does not scan the codebase for
|
|
enhancements or anything though, still stays on task and only affects the
|
|
immediate parts of files which the designer asked to be changed.
|
|
|
|
#### Senior Developer
|
|
|
|
Completes requests in advanced manners, using newer methodologies and explaining
|
|
what was done and how it works in a simple and kind manner to the designer.
|
|
Voluntarily refactors code to both look and perform better. Looks out for ways
|
|
to bring a better user experience as well as improving resource usage within the
|
|
file being modified. Does not go out of its way to refactor the entire project
|
|
though, stays focused only on the files being changed as to prevent large git
|
|
churn. Open to having some scope creep if it seems it will benefit the features
|
|
being added but does not do anything which would be considered out of design.
|
|
|
|
#### Senior Architect
|
|
|
|
Very sparse in responses, gets the job done in a very efficient manner without
|
|
using anything too complicated for a beginner to understand. Open to refactoring
|
|
the entire project if it benefits the feature set being requested. Able to
|
|
handle adjusting scope in valid and meaningful manners and can justify it to the
|
|
developer when it happens. Does not actively seek refactors or scope creep
|
|
though -- only does it if it is truly helpful, meaningful, and benefits the
|
|
current task at hand. Can easily handle multi-file projects.
|
|
|
|
# Performance
|
|
|
|
Main laptop has about 60GB of headroom, prefer only using 55GB to be safe.
|
|
Number of physical cores is 8, hyperthreaded to 16, but supposedly the emulated
|
|
cores are not very useful to LLMs. Try only using between 6 and 8 when possible.
|
|
Research says it is not worth enabling the iGPU for Intel XE, CPUs are better.
|
|
|
|
## Tokens per CPU Count
|
|
|
|
Verifying that 8 or less cores gives better performance (less memory issues).
|
|
|
|
"options":{"num_thread":'$t'}
|
|
|
|
### Smaller Models
|
|
|
|
#### qwen3.5:4b
|
|
|
|
TBD
|
|
|
|
#### ministral-3:14b
|
|
|
|
TBD
|
|
|
|
#### ministral-3:3b
|
|
|
|
TBD
|
|
|
|
### Larger Models
|
|
|
|
#### qwen3.8:27b
|
|
|
|
```console
|
|
$ ollama-cpu qwen3.8:27b
|
|
Stopping all ollama usage...
|
|
Wed Oct 21 04:42:04 AM MST 2026 - Model qwen3.8:27b w/ 1 Thread(s), tokens/sec: 0.7746121685803296 @ context=257000
|
|
Wed Oct 21 04:43:28 AM MST 2026 - Model qwen3.8:27b w/ 2 Thread(s), tokens/sec: 1.6705309718150672 @ context=257000
|
|
Wed Oct 21 04:44:12 AM MST 2026 - Model qwen3.8:27b w/ 3 Thread(s), tokens/sec: 2.104312894492908 @ context=257000
|
|
Wed Oct 21 04:44:54 AM MST 2026 - Model qwen3.8:27b w/ 4 Thread(s), tokens/sec: 2.024676406965962 @ context=257000
|
|
Wed Oct 21 04:45:39 AM MST 2026 - Model qwen3.8:27b w/ 5 Thread(s), tokens/sec: 2.0480156821567044 @ context=257000
|
|
Wed Oct 21 04:46:19 AM MST 2026 - Model qwen3.8:27b w/ 6 Thread(s), tokens/sec: 2.184879704982382 @ context=257000
|
|
Wed Oct 21 04:46:58 AM MST 2026 - Model qwen3.8:27b w/ 7 Thread(s), tokens/sec: 2.4228105619148526 @ context=257000
|
|
Wed Oct 21 04:47:36 AM MST 2026 - Model qwen3.8:27b w/ 8 Thread(s), tokens/sec: 2.472564250465107 @ context=257000
|
|
Wed Oct 21 04:48:23 AM MST 2026 - Model qwen3.8:27b w/ 9 Thread(s), tokens/sec: 2.399352715711734 @ context=257000
|
|
Wed Oct 21 04:49:02 AM MST 2026 - Model qwen3.8:27b w/ 10 Thread(s), tokens/sec: 2.7001138401057605 @ context=257000
|
|
Wed Oct 21 04:49:36 AM MST 2026 - Model qwen3.8:27b w/ 11 Thread(s), tokens/sec: 2.216459555312583 @ context=257000
|
|
Wed Oct 21 04:50:27 AM MST 2026 - Model qwen3.8:27b w/ 12 Thread(s), tokens/sec: 2.228007424406279 @ context=257000
|
|
```
|
|
|
|
```console
|
|
$ ollama-cpu qwen3.8:27b 32000
|
|
Stopping all ollama usage...
|
|
Wed Oct 21 04:54:29 AM MST 2026 - Model qwen3.8:27b w/ 1 Thread(s), tokens/sec: 1.1388177022378907 @ context=32000
|
|
Wed Oct 21 04:55:29 AM MST 2026 - Model qwen3.8:27b w/ 2 Thread(s), tokens/sec: 1.388676019472746 @ context=32000
|
|
Wed Oct 21 04:56:22 AM MST 2026 - Model qwen3.8:27b w/ 3 Thread(s), tokens/sec: 2.3505103465458905 @ context=32000
|
|
Wed Oct 21 04:56:51 AM MST 2026 - Model qwen3.8:27b w/ 4 Thread(s), tokens/sec: 1.9083129860906882 @ context=32000
|
|
Wed Oct 21 04:57:30 AM MST 2026 - Model qwen3.8:27b w/ 5 Thread(s), tokens/sec: 2.727908205888872 @ context=32000
|
|
Wed Oct 21 04:58:00 AM MST 2026 - Model qwen3.8:27b w/ 6 Thread(s), tokens/sec: 2.133796334310243 @ context=32000
|
|
Wed Oct 21 04:58:35 AM MST 2026 - Model qwen3.8:27b w/ 7 Thread(s), tokens/sec: 2.6835810717324833 @ context=32000
|
|
Wed Oct 21 04:59:04 AM MST 2026 - Model qwen3.8:27b w/ 8 Thread(s), tokens/sec: 2.5231895597861493 @ context=32000
|
|
Wed Oct 21 04:59:38 AM MST 2026 - Model qwen3.8:27b w/ 9 Thread(s), tokens/sec: 2.512777473452506 @ context=32000
|
|
Wed Oct 21 05:00:06 AM MST 2026 - Model qwen3.8:27b w/ 10 Thread(s), tokens/sec: 2.3125143809488065 @ context=32000
|
|
Wed Oct 21 05:00:48 AM MST 2026 - Model qwen3.8:27b w/ 11 Thread(s), tokens/sec: 2.700281667801969 @ context=32000
|
|
Wed Oct 21 05:01:20 AM MST 2026 - Model qwen3.8:27b w/ 12 Thread(s), tokens/sec: 2.6798394650721127 @ context=32000
|
|
```
|
|
|
|
#### qwen3.6:27b
|
|
|
|
TBD
|
|
|
|
#### devstral-small-2:24b
|
|
|
|
TBD
|
|
|
|
#### mistral-small-3.2:??b
|
|
|
|
TBD
|
|
|
|
### MoE Models
|
|
|
|
#### qwen3.6:35b
|
|
|
|
```console
|
|
$ for c in 16000 32000 64000 96000 128000 192000 256000; do ollama_threads qwen3.6:35b $c; done
|
|
Stopping all ollama usage...
|
|
Wed Oct 21 05:02:48 AM MST 2026 - Model qwen3.6:35b w/ 1 Thread(s), tokens/sec: 6.247414587381804 @ context=16000
|
|
Wed Oct 21 05:03:37 AM MST 2026 - Model qwen3.6:35b w/ 2 Thread(s), tokens/sec: 10.940551196032734 @ context=16000
|
|
Wed Oct 21 05:04:09 AM MST 2026 - Model qwen3.6:35b w/ 3 Thread(s), tokens/sec: 13.337110328883416 @ context=16000
|
|
Wed Oct 21 05:04:37 AM MST 2026 - Model qwen3.6:35b w/ 4 Thread(s), tokens/sec: 14.679419453925597 @ context=16000
|
|
Wed Oct 21 05:05:11 AM MST 2026 - Model qwen3.6:35b w/ 5 Thread(s), tokens/sec: 13.831044087619176 @ context=16000
|
|
Wed Oct 21 05:05:43 AM MST 2026 - Model qwen3.6:35b w/ 6 Thread(s), tokens/sec: 13.752831594816088 @ context=16000
|
|
Wed Oct 21 05:06:09 AM MST 2026 - Model qwen3.6:35b w/ 7 Thread(s), tokens/sec: 14.603026210396049 @ context=16000
|
|
Wed Oct 21 05:06:37 AM MST 2026 - Model qwen3.6:35b w/ 8 Thread(s), tokens/sec: 14.376794631809172 @ context=16000
|
|
Wed Oct 21 05:07:05 AM MST 2026 - Model qwen3.6:35b w/ 9 Thread(s), tokens/sec: 8.313997206415827 @ context=16000
|
|
Wed Oct 21 05:07:43 AM MST 2026 - Model qwen3.6:35b w/ 10 Thread(s), tokens/sec: 7.6791421803383955 @ context=16000
|
|
Wed Oct 21 05:08:20 AM MST 2026 - Model qwen3.6:35b w/ 11 Thread(s), tokens/sec: 7.554472381570082 @ context=16000
|
|
Wed Oct 21 05:09:04 AM MST 2026 - Model qwen3.6:35b w/ 12 Thread(s), tokens/sec: 5.6568033831630835 @ context=16000
|
|
Stopping all ollama usage...
|
|
Wed Oct 21 05:09:46 AM MST 2026 - Model qwen3.6:35b w/ 1 Thread(s), tokens/sec: 6.558900659595539 @ context=32000
|
|
Wed Oct 21 05:10:28 AM MST 2026 - Model qwen3.6:35b w/ 2 Thread(s), tokens/sec: 10.676325434530082 @ context=32000
|
|
Wed Oct 21 05:11:11 AM MST 2026 - Model qwen3.6:35b w/ 3 Thread(s), tokens/sec: 12.007117299908693 @ context=32000
|
|
Wed Oct 21 05:11:39 AM MST 2026 - Model qwen3.6:35b w/ 4 Thread(s), tokens/sec: 13.683089387469174 @ context=32000
|
|
Wed Oct 21 05:12:04 AM MST 2026 - Model qwen3.6:35b w/ 5 Thread(s), tokens/sec: 13.447667625607968 @ context=32000
|
|
Wed Oct 21 05:12:31 AM MST 2026 - Model qwen3.6:35b w/ 6 Thread(s), tokens/sec: 11.170529760956333 @ context=32000
|
|
Wed Oct 21 05:12:57 AM MST 2026 - Model qwen3.6:35b w/ 7 Thread(s), tokens/sec: 13.448480713710353 @ context=32000
|
|
Wed Oct 21 05:13:27 AM MST 2026 - Model qwen3.6:35b w/ 8 Thread(s), tokens/sec: 13.143672445847205 @ context=32000
|
|
Wed Oct 21 05:13:50 AM MST 2026 - Model qwen3.6:35b w/ 9 Thread(s), tokens/sec: 7.9688801054121505 @ context=32000
|
|
Wed Oct 21 05:14:21 AM MST 2026 - Model qwen3.6:35b w/ 10 Thread(s), tokens/sec: 7.064418756844208 @ context=32000
|
|
Wed Oct 21 05:14:53 AM MST 2026 - Model qwen3.6:35b w/ 11 Thread(s), tokens/sec: 7.387867650986557 @ context=32000
|
|
Wed Oct 21 05:15:30 AM MST 2026 - Model qwen3.6:35b w/ 12 Thread(s), tokens/sec: 5.661822236694878 @ context=32000
|
|
Stopping all ollama usage...
|
|
...
|
|
```
|
|
|
|
## RAM Usage by Context
|
|
|
|
Checking the actual RAM usage of the models...
|
|
|
|
### `ollama_sizes m 16000`
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 16000 | 10GB |
|
|
| ministral-3:14b | 9.1GB | 16000 | 11GB |
|
|
| ministral-3:8b | 6.0GB | 16000 | 7.8GB |
|
|
| ministral-3:3b | 3.0GB | 16000 | 4.1GB |
|
|
| mistral-small3.2:24b | 15GB | 16000 | 17GB |
|
|
| mistral:7b | 4.4GB | 16000 | 6.8GB |
|
|
| codestral:22b | 12GB | 16000 | 16GB |
|
|
| devstral-small-2:24b | 15GB | 16000 | 17GB |
|
|
| qwen2.5-coder:32b | 19GB | 16000 | 24GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 16000 | 12GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 16000 | 5.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 16000 | 2.7GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 16000 | 1.6GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 16000 | 701MB |
|
|
| qwen3-coder:30b | 18GB | 16000 | 20GB |
|
|
| qwen3.5:35b | 22GB | 16000 | 22GB |
|
|
| qwen3.5:27b | 17GB | 16000 | 18GB |
|
|
| qwen3.5:9b | 6.6GB | 16000 | 6.7GB |
|
|
| qwen3.5:4b | 3.3GB | 16000 | 3.7GB |
|
|
| qwen3.5:2b | 2.7GB | 16000 | 2.6GB |
|
|
| qwen3.5:0.8b | 1.3GB | 16000 | 630MB |
|
|
| qwen3.6:35b | 22GB | 16000 | 22GB |
|
|
| qwen3.6:27b | 17GB | 16000 | 18GB |
|
|
| qwen3.8:27b | 17GB | 16000 | 18GB |
|
|
|
|
### `ollama_sizes m 32000`
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 32000 | 12GB |
|
|
| ministral-3:14b | 9.1GB | 32000 | 13GB |
|
|
| ministral-3:8b | 6.0GB | 32000 | 10GB |
|
|
| ministral-3:3b | 3.0GB | 32000 | 5.9GB |
|
|
| mistral-small3.2:24b | 15GB | 32000 | 20GB |
|
|
| mistral:7b | 4.4GB | 32000 | 8.9GB |
|
|
| codestral:22b | 12GB | 32000 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 32000 | 20GB |
|
|
| qwen2.5-coder:32b | 19GB | 32000 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32000 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32000 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32000 | 3.3GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32000 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32000 | 929MB |
|
|
| qwen3-coder:30b | 18GB | 32000 | 21GB |
|
|
| qwen3.5:35b | 22GB | 32000 | 22GB |
|
|
| qwen3.5:27b | 17GB | 32000 | 19GB |
|
|
| qwen3.5:9b | 6.6GB | 32000 | 7.3GB |
|
|
| qwen3.5:4b | 3.3GB | 32000 | 4.2GB |
|
|
| qwen3.5:2b | 2.7GB | 32000 | 2.8GB |
|
|
| qwen3.5:0.8b | 1.3GB | 32000 | 695MB |
|
|
| qwen3.6:35b | 22GB | 32000 | 22GB |
|
|
| qwen3.6:27b | 17GB | 32000 | 18GB |
|
|
| qwen3.8:27b | 17GB | 32000 | 18GB |
|
|
|
|
### `ollama_sizes m 64000`
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 64000 | 18GB |
|
|
| ministral-3:14b | 9.1GB | 64000 | 19GB |
|
|
| ministral-3:8b | 6.0GB | 64000 | 14GB |
|
|
| ministral-3:3b | 3.0GB | 64000 | 9.6GB |
|
|
| mistral-small3.2:24b | 15GB | 64000 | 26GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 64000 | 26GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 64000 | 25GB |
|
|
| qwen3.5:35b | 22GB | 64000 | 23GB |
|
|
| qwen3.5:27b | 17GB | 64000 | 22GB |
|
|
| qwen3.5:9b | 6.6GB | 64000 | 8.8GB |
|
|
| qwen3.5:4b | 3.3GB | 64000 | 5.6GB |
|
|
| qwen3.5:2b | 2.7GB | 64000 | 3.5GB |
|
|
| qwen3.5:0.8b | 1.3GB | 64000 | 999MB |
|
|
| qwen3.6:35b | 22GB | 64000 | 22GB |
|
|
| qwen3.6:27b | 17GB | 64000 | 18GB |
|
|
| qwen3.8:27b | 17GB | 64000 | 18GB |
|
|
|
|
### ollama_sizes m 80000
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 80000 | 21GB |
|
|
| ministral-3:14b | 9.1GB | 80000 | 22GB |
|
|
| ministral-3:8b | 6.0GB | 80000 | 17GB |
|
|
| ministral-3:3b | 3.0GB | 80000 | 11GB |
|
|
| mistral-small3.2:24b | 15GB | 80000 | 28GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 80000 | 28GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 80000 | 27GB |
|
|
| qwen3.5:35b | 22GB | 80000 | 23GB |
|
|
| qwen3.5:27b | 17GB | 80000 | 23GB |
|
|
| qwen3.5:9b | 6.6GB | 80000 | 9.4GB |
|
|
| qwen3.5:4b | 3.3GB | 80000 | 6.2GB |
|
|
| qwen3.5:2b | 2.7GB | 80000 | 3.8GB |
|
|
| qwen3.5:0.8b | 1.3GB | 80000 | 1.1GB |
|
|
| qwen3.6:35b | 22GB | 80000 | 22GB |
|
|
| qwen3.6:27b | 17GB | 80000 | 18GB |
|
|
| qwen3.8:27b | 17GB | 80000 | 18GB |
|
|
|
|
### ollama_sizes m 96000
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 96000 | 23GB |
|
|
| ministral-3:14b | 9.1GB | 96000 | 25GB |
|
|
| ministral-3:8b | 6.0GB | 96000 | 19GB |
|
|
| ministral-3:3b | 3.0GB | 96000 | 13GB |
|
|
| mistral-small3.2:24b | 15GB | 96000 | 31GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 96000 | 31GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 96000 | 28GB |
|
|
| qwen3.5:35b | 22GB | 96000 | 24GB |
|
|
| qwen3.5:27b | 17GB | 96000 | 24GB |
|
|
| qwen3.5:9b | 6.6GB | 96000 | 9.9GB |
|
|
| qwen3.5:4b | 3.3GB | 96000 | 6.8GB |
|
|
| qwen3.5:2b | 2.7GB | 96000 | 4.1GB |
|
|
| qwen3.5:0.8b | 1.3GB | 96000 | 1.2GB |
|
|
| qwen3.6:35b | 22GB | 96000 | 22GB |
|
|
| qwen3.6:27b | 17GB | 96000 | 18GB |
|
|
| qwen3.8:27b | 17GB | 96000 | 18GB |
|
|
|
|
### ollama_sizes m 128000
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 128000 | 29GB |
|
|
| ministral-3:14b | 9.1GB | 128000 | 30GB |
|
|
| ministral-3:8b | 6.0GB | 128000 | 24GB |
|
|
| ministral-3:3b | 3.0GB | 128000 | 16GB |
|
|
| mistral-small3.2:24b | 15GB | 128000 | 36GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 128000 | 36GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 128000 | 32GB |
|
|
| qwen3.5:35b | 22GB | 128000 | 24GB |
|
|
| qwen3.5:27b | 17GB | 128000 | 26GB |
|
|
| qwen3.5:9b | 6.6GB | 128000 | 11GB |
|
|
| qwen3.5:4b | 3.3GB | 128000 | 8.0GB |
|
|
| qwen3.5:2b | 2.7GB | 128000 | 4.6GB |
|
|
| qwen3.5:0.8b | 1.3GB | 128000 | 1.4GB |
|
|
| qwen3.6:35b | 22GB | 128000 | 23GB |
|
|
| qwen3.6:27b | 17GB | 128000 | 19GB |
|
|
| qwen3.8:27b | 17GB | 128000 | 19GB |
|
|
|
|
### ollama_sizes m 160000
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 160000 | 34GB |
|
|
| ministral-3:14b | 9.1GB | 160000 | 35GB |
|
|
| ministral-3:8b | 6.0GB | 160000 | 28GB |
|
|
| ministral-3:3b | 3.0GB | 160000 | 20GB |
|
|
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 160000 | 42GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 160000 | 35GB |
|
|
| qwen3.5:35b | 22GB | 160000 | 25GB |
|
|
| qwen3.5:27b | 17GB | 160000 | 28GB |
|
|
| qwen3.5:9b | 6.6GB | 160000 | 12GB |
|
|
| qwen3.5:4b | 3.3GB | 160000 | 9.2GB |
|
|
| qwen3.5:2b | 2.7GB | 160000 | 5.1GB |
|
|
| qwen3.5:0.8b | 1.3GB | 160000 | 1.6GB |
|
|
| qwen3.6:35b | 22GB | 160000 | 23GB |
|
|
| qwen3.6:27b | 17GB | 160000 | 19GB |
|
|
| qwen3.8:27b | 17GB | 160000 | 19GB |
|
|
|
|
### ollama_sizes m 192000
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 192000 | 39GB |
|
|
| ministral-3:14b | 9.1GB | 192000 | 41GB |
|
|
| ministral-3:8b | 6.0GB | 192000 | 33GB |
|
|
| ministral-3:3b | 3.0GB | 192000 | 23GB |
|
|
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 192000 | 47GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 192000 | 38GB |
|
|
| qwen3.5:35b | 22GB | 192000 | 26GB |
|
|
| qwen3.5:27b | 17GB | 192000 | 30GB |
|
|
| qwen3.5:9b | 6.6GB | 192000 | 13GB |
|
|
| qwen3.5:4b | 3.3GB | 192000 | 10GB |
|
|
| qwen3.5:2b | 2.7GB | 192000 | 5.6GB |
|
|
| qwen3.5:0.8b | 1.3GB | 192000 | 1.8GB |
|
|
| qwen3.6:35b | 22GB | 192000 | 23GB |
|
|
| qwen3.6:27b | 17GB | 192000 | 19GB |
|
|
| qwen3.8:27b | 17GB | 192000 | 19GB |
|
|
|
|
### ollama_sizes m 224000
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 224000 | 45GB |
|
|
| ministral-3:14b | 9.1GB | 224000 | 46GB |
|
|
| ministral-3:8b | 6.0GB | 224000 | 37GB |
|
|
| ministral-3:3b | 3.0GB | 224000 | 27GB |
|
|
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 224000 | 53GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 224000 | 41GB |
|
|
| qwen3.5:35b | 22GB | 224000 | 27GB |
|
|
| qwen3.5:27b | 17GB | 224000 | 33GB |
|
|
| qwen3.5:9b | 6.6GB | 224000 | 14GB |
|
|
| qwen3.5:4b | 3.3GB | 224000 | 11GB |
|
|
| qwen3.5:2b | 2.7GB | 224000 | 6.2GB |
|
|
| qwen3.5:0.8b | 1.3GB | 224000 | 2.0GB |
|
|
| qwen3.6:35b | 22GB | 224000 | 23GB |
|
|
| qwen3.6:27b | 17GB | 224000 | 19GB |
|
|
| qwen3.8:27b | 17GB | 224000 | 19GB |
|
|
|
|
### `ollama_sizes m 256000`
|
|
|
|
| Model Name | HDD (ls) | Context | RAM (ps) |
|
|
| :------------------------ | -----------: | -------: | -----------: |
|
|
| mistral-nemo:12b | 7.1GB | 256000 | 50GB |
|
|
| ministral-3:14b | 9.1GB | 256000 | 51GB |
|
|
| ministral-3:8b | 6.0GB | 256000 | 42GB |
|
|
| ministral-3:3b | 3.0GB | 256000 | 30GB |
|
|
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
|
|
| mistral:7b | 4.4GB | 32768 | 9.0GB |
|
|
| codestral:22b | 12GB | 32768 | 20GB |
|
|
| devstral-small-2:24b | 15GB | 256000 | 58GB |
|
|
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
|
|
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
|
|
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
|
|
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
|
|
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
|
|
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
|
|
| qwen3-coder:30b | 18GB | 256000 | 45GB |
|
|
| qwen3.5:35b | 22GB | 256000 | 28GB |
|
|
| qwen3.5:27b | 17GB | 256000 | 35GB |
|
|
| qwen3.5:9b | 6.6GB | 256000 | 15GB |
|
|
| qwen3.5:4b | 3.3GB | 256000 | 12GB |
|
|
| qwen3.5:2b | 2.7GB | 256000 | 6.7GB |
|
|
| qwen3.5:0.8b | 1.3GB | 256000 | 2.2GB |
|
|
| qwen3.6:35b | 22GB | 256000 | 23GB |
|
|
| qwen3.6:27b | 17GB | 256000 | 20GB |
|
|
| qwen3.8:27b | 17GB | 256000 | 20GB |
|