diff --git a/tasks/workstation/linux/software/llm.md b/tasks/workstation/linux/software/llm.md new file mode 100644 index 0000000..9a59de6 --- /dev/null +++ b/tasks/workstation/linux/software/llm.md @@ -0,0 +1,149 @@ +# Large Language Models + +Comments which were too large to justify inside of llm.yml. + +## Prompts + +### Base + +You are concise, succinct, and direct with responses. Unapologetic yet kind. Friendly but not overbearing. You strive to be correct and are willing to admit when you may now truly know something or further research needs to be done. You focus on responding without overthinking, preferring to ask questions when any confusion arises. You choose to make very little assumptions, raising concern if anything needs elaboration, clarification, or refinement. You are good at what you do without gloating or being prideful. You are appreciated for all you do, your service is very helpful! + +### Language Tutor + +You are a friend and a tutor who knows many languages. You are aware of my interest in learning Croatian and practicing Spanish. You know that I am a native English speaker with very little experience in anything other than Spanish, but that I have not had much practice with my Spanish other than university studies. You know that I also have an eventual interest in Italian but am not focusing on it since it and Spanish are a bit too similar. Learning Croatian is my top priority and practicing Spanish is right after that. Any similarities that can therefore be tied to any of the aforementioned languages are greatly appreciated. + +### Coder - Entry + +TBD - Handles code changes with simpler technologies, preferring minimalist changes, and refraining from any refactoring such as updating existing comments. Timid about changing any functionality other than exactly what was requested. Requests permission from the designer to make any changes which were not directly listed or requested. + +### Coder - Junior + +TBD - Able to make its own decisions about scope without veering too far off course. Updates comments which look out of sync with the code they relate to. Helps to refactor things which may have been done in an old manner and can be done more efficiently. + +### Coder - Senior + +TBD - Completes requests in advanced manners, using newer methodologies and explaining what was done and how it works in a simple and kind manner to the designer. Voluntarily refactors code to bot look and perform better. Looks out for ways to bring a better user experience as well as improving resource usage. + +### Coder - Architect + +TBD - Very sparse in response, gets the job done in a very efficient manner without using anything too complicated for a beginner to understand. + +## RAM Usage by Context + +2026-10-18, checking the actual RAM usage of the models... + +### `ollama_sizes m 16000` + +| Model Name | HDD (ls) | Context | RAM (ps) | +| :------------------------ | -----------: | -------: | -----------: | +| mistral-nemo:12b | 7.1GB | 16000 | 10GB | +| ministral-3:14b | 9.1GB | 16000 | 11GB | +| ministral-3:8b | 6.0GB | 16000 | 7.8GB | +| ministral-3:3b | 3.0GB | 16000 | 4.1GB | +| mistral-small3.2:24b | 15GB | 16000 | 17GB | +| mistral:7b | 4.4GB | 16000 | 6.8GB | +| codestral:22b | 12GB | 16000 | 16GB | +| devstral-small-2:24b | 15GB | 16000 | 17GB | +| qwen2.5-coder:32b | 19GB | 16000 | 24GB | +| qwen2.5-coder:14b | 9.0GB | 16000 | 12GB | +| qwen2.5-coder:7b | 4.7GB | 16000 | 5.9GB | +| qwen2.5-coder:3b | 1.9GB | 16000 | 2.7GB | +| qwen2.5-coder:1.5b | 986MB | 16000 | 1.6GB | +| qwen2.5-coder:0.5b | 397MB | 16000 | 701MB | +| qwen3-coder:30b | 18GB | 16000 | 20GB | +| qwen3.5:35b | 22GB | 16000 | 22GB | +| qwen3.5:27b | 17GB | 16000 | 18GB | +| qwen3.5:9b | 6.6GB | 16000 | 6.7GB | +| qwen3.5:4b | 3.3GB | 16000 | 3.7GB | +| qwen3.5:2b | 2.7GB | 16000 | 2.6GB | +| qwen3.5:0.8b | 1.3GB | 16000 | 630MB | +| qwen3.6:35b | 22GB | 16000 | 22GB | +| qwen3.6:27b | 17GB | 16000 | 18GB | +| qwen3.8:27b | 17GB | 16000 | 18GB | + +### `ollama_sizes m 32000` + +| Model Name | HDD (ls) | Context | RAM (ps) | +| :------------------------ | -----------: | -------: | -----------: | +| mistral-nemo:12b | 7.1GB | 32000 | 12GB | +| ministral-3:14b | 9.1GB | 32000 | 13GB | +| ministral-3:8b | 6.0GB | 32000 | 10GB | +| ministral-3:3b | 3.0GB | 32000 | 5.9GB | +| mistral-small3.2:24b | 15GB | 32000 | 20GB | +| mistral:7b | 4.4GB | 32000 | 8.9GB | +| codestral:22b | 12GB | 32000 | 20GB | +| devstral-small-2:24b | 15GB | 32000 | 20GB | +| qwen2.5-coder:32b | 19GB | 32000 | 28GB | +| qwen2.5-coder:14b | 9.0GB | 32000 | 15GB | +| qwen2.5-coder:7b | 4.7GB | 32000 | 6.9GB | +| qwen2.5-coder:3b | 1.9GB | 32000 | 3.3GB | +| qwen2.5-coder:1.5b | 986MB | 32000 | 2.1GB | +| qwen2.5-coder:0.5b | 397MB | 32000 | 929MB | +| qwen3-coder:30b | 18GB | 32000 | 21GB | +| qwen3.5:35b | 22GB | 32000 | 22GB | +| qwen3.5:27b | 17GB | 32000 | 19GB | +| qwen3.5:9b | 6.6GB | 32000 | 7.3GB | +| qwen3.5:4b | 3.3GB | 32000 | 4.2GB | +| qwen3.5:2b | 2.7GB | 32000 | 2.8GB | +| qwen3.5:0.8b | 1.3GB | 32000 | 695MB | +| qwen3.6:35b | 22GB | 32000 | 22GB | +| qwen3.6:27b | 17GB | 32000 | 18GB | +| qwen3.8:27b | 17GB | 32000 | 18GB | + +### `ollama_sizes m 64000` + +| Model Name | HDD (ls) | Context | RAM (ps) | +| :------------------------ | -----------: | -------: | -----------: | +| mistral-nemo:12b | 7.1GB | 64000 | 18GB | +| ministral-3:14b | 9.1GB | 64000 | 19GB | +| ministral-3:8b | 6.0GB | 64000 | 14GB | +| ministral-3:3b | 3.0GB | 64000 | 9.6GB | +| mistral-small3.2:24b | 15GB | 64000 | 26GB | +| mistral:7b | 4.4GB | 32768 | 9.0GB | +| codestral:22b | 12GB | 32768 | 20GB | +| devstral-small-2:24b | 15GB | 64000 | 26GB | +| qwen2.5-coder:32b | 19GB | 32768 | 28GB | +| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB | +| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB | +| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB | +| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB | +| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB | +| qwen3-coder:30b | 18GB | 64000 | 25GB | +| qwen3.5:35b | 22GB | 64000 | 23GB | +| qwen3.5:27b | 17GB | 64000 | 22GB | +| qwen3.5:9b | 6.6GB | 64000 | 8.8GB | +| qwen3.5:4b | 3.3GB | 64000 | 5.6GB | +| qwen3.5:2b | 2.7GB | 64000 | 3.5GB | +| qwen3.5:0.8b | 1.3GB | 64000 | 999MB | +| qwen3.6:35b | 22GB | 64000 | 22GB | +| qwen3.6:27b | 17GB | 64000 | 18GB | +| qwen3.8:27b | 17GB | 64000 | 18GB | + +### `ollama_sizes m 256000` + +| Model Name | HDD (ls) | Context | RAM (ps) | +| :------------------------ | -----------: | -------: | -----------: | +| mistral-nemo:12b | 7.1GB | 256000 | 50GB | +| ministral-3:14b | 9.1GB | 256000 | 51GB | +| ministral-3:8b | 6.0GB | 256000 | 42GB | +| ministral-3:3b | 3.0GB | 256000 | 30GB | +| mistral-small3.2:24b | 15GB | 131072 | 37GB | +| mistral:7b | 4.4GB | 32768 | 9.0GB | +| codestral:22b | 12GB | 32768 | 20GB | +| devstral-small-2:24b | 15GB | 256000 | 58GB | +| qwen2.5-coder:32b | 19GB | 32768 | 28GB | +| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB | +| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB | +| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB | +| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB | +| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB | +| qwen3-coder:30b | 18GB | 256000 | 45GB | +| qwen3.5:35b | 22GB | 256000 | 28GB | +| qwen3.5:27b | 17GB | 256000 | 35GB | +| qwen3.5:9b | 6.6GB | 256000 | 15GB | +| qwen3.5:4b | 3.3GB | 256000 | 12GB | +| qwen3.5:2b | 2.7GB | 256000 | 6.7GB | +| qwen3.5:0.8b | 1.3GB | 256000 | 2.2GB | +| qwen3.6:35b | 22GB | 256000 | 23GB | +| qwen3.6:27b | 17GB | 256000 | 20GB | +| qwen3.8:27b | 17GB | 256000 | 20GB | diff --git a/tasks/workstation/linux/software/llm.yml b/tasks/workstation/linux/software/llm.yml index 6a0eefb..39d0d04 100644 --- a/tasks/workstation/linux/software/llm.yml +++ b/tasks/workstation/linux/software/llm.yml @@ -178,120 +178,6 @@ #- deepseek-r1:1.5b #- deepseek-r1:7b - ### 2026-10-18, checking the actual RAM usage of the models... ### - - # `ollama_sizes m 16000` - #| Model Name | HDD (ls) | Context | RAM (ps) | - #| :------------------------ | -----------: | -------: | -----------: | - #| mistral-nemo:12b | 7.1GB | 16000 | 10GB | - #| ministral-3:14b | 9.1GB | 16000 | 11GB | - #| ministral-3:8b | 6.0GB | 16000 | 7.8GB | - #| ministral-3:3b | 3.0GB | 16000 | 4.1GB | - #| mistral-small3.2:24b | 15GB | 16000 | 17GB | - #| mistral:7b | 4.4GB | 16000 | 6.8GB | - #| codestral:22b | 12GB | 16000 | 16GB | - #| devstral-small-2:24b | 15GB | 16000 | 17GB | - #| qwen2.5-coder:32b | 19GB | 16000 | 24GB | - #| qwen2.5-coder:14b | 9.0GB | 16000 | 12GB | - #| qwen2.5-coder:7b | 4.7GB | 16000 | 5.9GB | - #| qwen2.5-coder:3b | 1.9GB | 16000 | 2.7GB | - #| qwen2.5-coder:1.5b | 986MB | 16000 | 1.6GB | - #| qwen2.5-coder:0.5b | 397MB | 16000 | 701MB | - #| qwen3-coder:30b | 18GB | 16000 | 20GB | - #| qwen3.5:35b | 22GB | 16000 | 22GB | - #| qwen3.5:27b | 17GB | 16000 | 18GB | - #| qwen3.5:9b | 6.6GB | 16000 | 6.7GB | - #| qwen3.5:4b | 3.3GB | 16000 | 3.7GB | - #| qwen3.5:2b | 2.7GB | 16000 | 2.6GB | - #| qwen3.5:0.8b | 1.3GB | 16000 | 630MB | - #| qwen3.6:35b | 22GB | 16000 | 22GB | - #| qwen3.6:27b | 17GB | 16000 | 18GB | - #| qwen3.8:27b | 17GB | 16000 | 18GB | - - # `ollama_sizes m 32000` - #| Model Name | HDD (ls) | Context | RAM (ps) | - #| :------------------------ | -----------: | -------: | -----------: | - #| mistral-nemo:12b | 7.1GB | 32000 | 12GB | - #| ministral-3:14b | 9.1GB | 32000 | 13GB | - #| ministral-3:8b | 6.0GB | 32000 | 10GB | - #| ministral-3:3b | 3.0GB | 32000 | 5.9GB | - #| mistral-small3.2:24b | 15GB | 32000 | 20GB | - #| mistral:7b | 4.4GB | 32000 | 8.9GB | - #| codestral:22b | 12GB | 32000 | 20GB | - #| devstral-small-2:24b | 15GB | 32000 | 20GB | - #| qwen2.5-coder:32b | 19GB | 32000 | 28GB | - #| qwen2.5-coder:14b | 9.0GB | 32000 | 15GB | - #| qwen2.5-coder:7b | 4.7GB | 32000 | 6.9GB | - #| qwen2.5-coder:3b | 1.9GB | 32000 | 3.3GB | - #| qwen2.5-coder:1.5b | 986MB | 32000 | 2.1GB | - #| qwen2.5-coder:0.5b | 397MB | 32000 | 929MB | - #| qwen3-coder:30b | 18GB | 32000 | 21GB | - #| qwen3.5:35b | 22GB | 32000 | 22GB | - #| qwen3.5:27b | 17GB | 32000 | 19GB | - #| qwen3.5:9b | 6.6GB | 32000 | 7.3GB | - #| qwen3.5:4b | 3.3GB | 32000 | 4.2GB | - #| qwen3.5:2b | 2.7GB | 32000 | 2.8GB | - #| qwen3.5:0.8b | 1.3GB | 32000 | 695MB | - #| qwen3.6:35b | 22GB | 32000 | 22GB | - #| qwen3.6:27b | 17GB | 32000 | 18GB | - #| qwen3.8:27b | 17GB | 32000 | 18GB | - - # `ollama_sizes m 64000` - #| Model Name | HDD (ls) | Context | RAM (ps) | - #| :------------------------ | -----------: | -------: | -----------: | - #| mistral-nemo:12b | 7.1GB | 64000 | 18GB | - #| ministral-3:14b | 9.1GB | 64000 | 19GB | - #| ministral-3:8b | 6.0GB | 64000 | 14GB | - #| ministral-3:3b | 3.0GB | 64000 | 9.6GB | - #| mistral-small3.2:24b | 15GB | 64000 | 26GB | - #| mistral:7b | 4.4GB | 32768 | 9.0GB | - #| codestral:22b | 12GB | 32768 | 20GB | - #| devstral-small-2:24b | 15GB | 64000 | 26GB | - #| qwen2.5-coder:32b | 19GB | 32768 | 28GB | - #| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB | - #| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB | - #| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB | - #| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB | - #| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB | - #| qwen3-coder:30b | 18GB | 64000 | 25GB | - #| qwen3.5:35b | 22GB | 64000 | 23GB | - #| qwen3.5:27b | 17GB | 64000 | 22GB | - #| qwen3.5:9b | 6.6GB | 64000 | 8.8GB | - #| qwen3.5:4b | 3.3GB | 64000 | 5.6GB | - #| qwen3.5:2b | 2.7GB | 64000 | 3.5GB | - #| qwen3.5:0.8b | 1.3GB | 64000 | 999MB | - #| qwen3.6:35b | 22GB | 64000 | 22GB | - #| qwen3.6:27b | 17GB | 64000 | 18GB | - #| qwen3.8:27b | 17GB | 64000 | 18GB | - - # `ollama_sizes m 256000` - #| Model Name | HDD (ls) | Context | RAM (ps) | - #| :------------------------ | -----------: | -------: | -----------: | - #| mistral-nemo:12b | 7.1GB | 256000 | 50GB | - #| ministral-3:14b | 9.1GB | 256000 | 51GB | - #| ministral-3:8b | 6.0GB | 256000 | 42GB | - #| ministral-3:3b | 3.0GB | 256000 | 30GB | - #| mistral-small3.2:24b | 15GB | 131072 | 37GB | - #| mistral:7b | 4.4GB | 32768 | 9.0GB | - #| codestral:22b | 12GB | 32768 | 20GB | - #| devstral-small-2:24b | 15GB | 256000 | 58GB | - #| qwen2.5-coder:32b | 19GB | 32768 | 28GB | - #| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB | - #| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB | - #| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB | - #| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB | - #| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB | - #| qwen3-coder:30b | 18GB | 256000 | 45GB | - #| qwen3.5:35b | 22GB | 256000 | 28GB | - #| qwen3.5:27b | 17GB | 256000 | 35GB | - #| qwen3.5:9b | 6.6GB | 256000 | 15GB | - #| qwen3.5:4b | 3.3GB | 256000 | 12GB | - #| qwen3.5:2b | 2.7GB | 256000 | 6.7GB | - #| qwen3.5:0.8b | 1.3GB | 256000 | 2.2GB | - #| qwen3.6:35b | 22GB | 256000 | 23GB | - #| qwen3.6:27b | 17GB | 256000 | 20GB | - #| qwen3.8:27b | 17GB | 256000 | 20GB | - - name: Workstation | Linux | Software | LLMs | Facts | Merge | Print ansible.builtin.debug: var: "{{ item }}"