Compare commits

...
85 Commits
Author SHA1 Message Date
me f487461d94 llm, change fastfetch output to be a fresh boot. 2026-10-25 10:27:01 -07:00
me 65ab204409 llm, update open-terminal 10 minutes after boot, not 1. 2026-10-25 10:26:33 -07:00
me 62302b3a0c llm, add tests but excluded 27b weights when a 35b existed. 2026-10-24 21:34:19 -07:00
me 4597151bbf llm, add beginning of benchmark data which accidentally got cancelled. 2026-10-24 20:55:41 -07:00
me 9e75d9b981 llm, extend keep alive, add final mounts to open-terminal, have open-webui do its update 5 minutes after boot once network exists, fix extra / in JAVA_HOME. 2026-10-24 20:44:17 -07:00
me 65aeea9a93 llm, switch open-terminal over to compose so that build dependencies are persistent. 2026-10-24 17:32:49 -07:00
me 0e1648cc31 llm, fix pub_cache to pub-cache. 2026-10-24 16:18:17 -07:00
me 3e53bd3c9b llm, add GOT_EM env var and reference it in the AGENTS file. 2026-10-24 16:00:18 -07:00
me 43dc65a84b llm, mount etc dirs for Java security policy, add needed packages to compile Android and Flutter. 2026-10-24 15:53:29 -07:00
me 2a7fa98825 llm, move projects and SDK back to user so that compilation paths do not need changed. 2026-10-24 15:17:13 -07:00
me 71bcb7c351 users, ensure home and its immediate subdirectories are owned by the user. 2026-10-24 15:16:37 -07:00
me cea7e03c11 bashrc, when restarting a process, only follow the new log entries. 2026-10-24 14:36:22 -07:00
me 38a04fbda1 swap_stopper, do not let the echo prevent the function from returning an unclean status. 2026-10-24 12:39:56 -07:00
me 4342b8842f swap_stopper, ensure ollama dies even if it is not being run through runit. 2026-10-24 12:35:27 -07:00
me ef9566c76c llm, move some functions to env-shared project. 2026-10-24 10:55:11 -07:00
me 41f306f7d7 llm, add another alias for restarting AI services. 2026-10-23 19:03:16 -07:00
me 4c5d753b73 vscode, ensure new files are opened at the end. 2026-10-23 17:44:19 -07:00
me fcf77e22c5 llm, adjust the prompts. 2026-10-23 17:44:00 -07:00
me 75503d762b llm, extend keep alive time again since context gets unloaded too, remove DEBUG output from ollama log, use .bashrc rather than profiles since Docker is not creating those, create AGENTS.md in user root, make the JDK directory read only to prevent AI from tampering. 2026-10-23 17:43:48 -07:00
me e0b1fa30f1 java, go down to 17 like being used in development. 2026-10-23 17:41:31 -07:00
me d7eed3c883 llm, update PATH wording since it has been taken care of. 2026-10-22 18:04:05 -07:00
me c341d216ad llm, maintain env vars and PATH changes in the AI Sandbox folder. 2026-10-22 18:03:32 -07:00
me f6fd08f966 llm, add a safeguard when a model causes swap-stopper to reset ollama. 2026-10-22 11:58:16 -07:00
me 295b6b3d19 android, allow the licenses check to fail on newer SDKs. 2026-10-22 11:57:44 -07:00
me fe3cb9b5ba package, add changed_when. 2026-10-22 11:57:26 -07:00
me 431510a729 package, finish adding pipefails. 2026-10-22 09:53:10 -07:00
me e7a461550d android, finish adding pipefails. 2026-10-22 09:52:34 -07:00
me d13b97d5aa Merge branch 'dev-deleteme' into dev
Forgot to pull before working on a different laptop again.
2026-10-22 09:50:12 -07:00
me 96dd0118fa llm, add SDK homes for open-terminal. 2026-10-22 09:48:19 -07:00
me 17684becbb test, add Java task since LLM now relies on its vars. 2026-10-22 09:32:41 -07:00
me 4474ad664e system, change back to checking exit code not status. 2026-10-22 09:32:11 -07:00
me 24e4ea9c8c java, do not use ~ in environment variables. 2026-10-22 09:22:17 -07:00
me 847dfe88d4 android, fix Flutter var. 2026-10-22 09:22:02 -07:00
me 9f74361af1 Small fixes for Java and Flutter. 2026-10-22 09:15:03 -07:00
me 6c79addc3c java, fix apt to only be for apt, only install all JDKs on Arch if coding. 2026-10-22 09:10:02 -07:00
me 049df1ab61 java, be more system agnostic and remove the coding requirement. 2026-10-22 09:05:06 -07:00
me 007f338d2e flutter, add FLUTTER_HOME and refactor some variables. 2026-10-22 09:04:41 -07:00
me 494a9b70f6 system, check the 2nd var, it is 0 more often but seems more accurate. 2026-10-22 09:04:12 -07:00
me ed8a33e00c Modify environment variables for Flutter, Android, and Java. 2026-10-22 08:42:00 -07:00
me 6640058d8d provision, remove extra line. 2026-10-21 10:05:00 -07:00
me c852329455 packages, remove disabled task. 2026-10-21 10:01:53 -07:00
me 0363aefc1d Merge branch 'dev2' into dev
Pushed on a system when another system still needed to push.
2026-10-21 09:58:43 -07:00
me e81de0c63b llm, add more performance tests to markdown file. 2026-10-21 09:56:47 -07:00
me d5e3a57f56 llm, add a few models for mistral-small4 but they are too large or not the correct format for ollama, a few more edits to ollama_threads. 2026-10-21 09:56:14 -07:00
me 99a0b7ca0b provision, fix the MCP block overwriting the GUI block. 2026-10-21 09:48:42 -07:00
me a47d5fd129 tunnel, move service task to block and change block to be for runit. 2026-10-21 09:45:14 -07:00
me 472f7921e7 tunnel, fix location of facts. 2026-10-21 09:40:47 -07:00
me 8567cae2b1 packages, add btop. 2026-10-21 08:58:17 -07:00
me a1190290db roleplays, shorten names. 2026-10-21 08:54:25 -07:00
me 9a2e4ca4d2 i3, move gnome-system-monitor to WS10, btop on WS1. 2026-10-21 08:51:35 -07:00
me eb761e41dc roleplays, rename Quick Coder to Coder MoE in both title and ID. 2026-10-21 08:20:41 -07:00
me 05e2244efb roleplays, add a language entry for qwen3.5:35b since it is MoE and is not code-scoped like qwen3.6. 2026-10-21 08:19:48 -07:00
me bd7112e8d5 llm, filter out the less useful tests in ollama_threads. Should help save a lot of time, maybe up to 60-75% since they were the slower ones. 2026-10-21 07:27:28 -07:00
me 40d9a7252e llm, if Internet is available, pull extra packages for open-terminal. 2026-10-21 06:06:41 -07:00
me 6482dd123d llm, change ollama_threads to use a better method of determining the number of cores to test, also fix output. 2026-10-21 05:43:37 -07:00
me 74789ba70c roleplays, go up to 7 threads due to benchmark data. 8 is very marginal, leave it for the system. 4 is great for MoE tokens/sec but 7 is still a bit faster in overall response time. 2026-10-21 05:28:05 -07:00
me 6411d4792a roleplays, download the current state. 2026-10-21 05:26:35 -07:00
me e367e9e14b llm, begin populating thread benchmark data. 2026-10-21 05:23:57 -07:00
me c604859947 llm, add elapsed time to ollama_threads. 2026-10-21 05:23:41 -07:00
me 7f7f2a2edb llm, fix context sending in ollama_threads. 2026-10-21 04:55:01 -07:00
me 92be32345a llm, add context param to ollama_threads. 2026-10-21 04:51:46 -07:00
me 5719ec6b60 llm, add optional open-terminal package list, helpful but required Internet, may have to do an Internet check and then choose which docker run command to execute. Also fix and enhance the ollama_thread checker. 2026-10-21 04:47:16 -07:00
me 85e60ef60c llm, improve the thread testing output and also make the the ansible variable is always an int. 2026-10-21 03:03:24 -07:00
me 5620e5a223 llm, add function for benchmarking whether more threads adds latency. 2026-10-21 02:57:12 -07:00
me 4fd12ce45e llm, add details about the workstation being used. 2026-10-21 02:43:07 -07:00
me 5c67da3eb8 roleplays, delete old IDs, change to use 6 cores leaving 1 physical for the system and 1 for the user (in theory, of course). 2026-10-21 02:42:44 -07:00
me 14fdc1f740 llm, add more size testing data. 2026-10-21 02:32:55 -07:00
me 1adf4db6ee llm, wrap the personality prompts. 2026-10-21 02:30:04 -07:00
me 3b3c7bd501 Place the personality prompts into a knowledge base and attach them to the roleplays so that they can be maintained in a more centralized manner 2026-10-21 02:29:03 -07:00
me b5e5a18e4c roleplays, fix the IDs, will import back into Open WebUI. 2026-10-21 02:05:34 -07:00
me 870a99804a llm, add OLLAMA_NUM_THREADS which was proposed but rejected so far. 2026-10-21 02:03:37 -07:00
me 358bf9e11d roleplays, set max thread count to number of physical cores rather than using the emulated cores too. 2026-10-21 02:03:03 -07:00
me 20549e0055 roleplays, ensure everyone has tags, add providers for remaining ministral weights. 2026-10-20 17:40:32 -07:00
me 003e8d8430 roleplays, add an entry coder based on codestral since it does not swell up like devstral. 2026-10-20 17:28:44 -07:00
me 0e990e2ed5 roleplays, ensure models have all functionality. 2026-10-20 17:18:53 -07:00
me 2ff0ccd392 llm.md, finish the prompts for now. 2026-10-20 17:10:13 -07:00
me 9a9eb8b8e6 roleplays, add current Open WebUI lineup. 2026-10-20 17:09:54 -07:00
me bec4c919d4 llm, stop using pull=always because then network is required to start the container. 2026-10-20 16:36:44 -07:00
me 1ad4f642db llm, continue working on the prompts. 2026-10-20 11:56:23 -07:00
me 52d7ae3d86 llm, move bulk comments to an md file and start adding prompts to use in Open WebUI. 2026-10-20 10:16:40 -07:00
me a892aae620 llm, fix open-terminal docker syntax. 2026-10-20 10:16:15 -07:00
me 36b8305f0f system, shorten the runit error sleep. 2026-10-20 09:23:33 -07:00
me a823d4ae63 llm, fix open-terminal port control. 2026-10-18 10:27:10 -07:00
me 8311427506 llm, make the ollama_usage output much fancier, more report-like. 2026-10-18 10:26:56 -07:00
me 797744d9fe llm, add function which gets the model sizes, both disk and memory. 2026-10-18 09:00:25 -07:00
17 changed files with 1758 additions and 216 deletions

No files matched your search

+14 -3
View File
@@ -22,6 +22,7 @@
alsa: alsa-utils
stress: stress
jq: jq
btop: btop
- name: General | Facts | Package | Parrot OS Fixes
ansible.builtin.set_fact:
@@ -102,10 +103,14 @@
ansible_pkg_mgr_file: "{{ prov_dir }}/ansible_pkg_mgr.txt"
- name: General | Facts | Package | Update Command | Store Variables
ansible.builtin.shell: "{{ item }}"
ansible.builtin.shell: |
set -o pipefail
{{ item }}
loop:
- "echo '{{ ansible_distribution }}' > {{ ansible_distribution_file }}"
- "echo '{{ ansible_pkg_mgr }}' > {{ ansible_pkg_mgr_file }}"
register: shell_check
changed_when: shell_check.rc == 0
- name: General | Facts | Package | Update Command
ansible.builtin.set_fact:
@@ -269,8 +274,11 @@
# Flatpaks
- name: General | Facts | Package | Update Command | Flatpak | Check
ansible.builtin.shell: which flatpak
ansible.builtin.shell: |
set -o pipefail
which flatpak
register: flatpak_exec
changed_when: flatpak_exec.rc == 0
ignore_errors: true
- name: General | Facts | Package | Update Command | Flatpak | Exists
@@ -293,8 +301,11 @@
# Firmware
- name: General | Facts | Package | Update Command | Firmware | Check
ansible.builtin.shell: which fwupdmgr
ansible.builtin.shell: |
set -o pipefail
which fwupdmgr
register: fwupdmgr_exec
changed_when: fwupdmgr_exec.rc == 0
ignore_errors: true
- name: General | Facts | Package | Update Command | Firmware | Exists
+1 -1
View File
@@ -21,7 +21,7 @@
# $1 is the exit code, $2 is the exit status byte (signal number if killed)
echo "FINISH: Service returned with exit code: $1 and status: $2"
sleep_long=30
sleep_long=10
sleep_short=1
if [[ "$1" != 0 ]]; then
echo "FINISH: Sleeping for $sleep_long seconds due to error..."
+7 -2
View File
@@ -13,14 +13,19 @@ restart_service() {
echo "ERROR: Service name must be passed to restart_service()."
return 1
fi
sv restart $1 || echo "$1 failed to restart, ignoring."
sv restart $1
status=$?
if [[ $status != 0 ]]; then
echo "$1 failed to restart, returned $status."
return $status
fi
}
# While we could loop over `ollama ps` and stop them, this ensures that the UIs
# do not sit around waiting for responses from the dead LLMs. It bring them up
# in a fresh state so that work can continue from a more clean slate.
restart_llms() {
restart_service ollama
restart_service ollama || killall ollama
#restart_service zeroclaw
#restart_service open-webui
#restart_service qwen-code
+7 -5
View File
@@ -300,21 +300,22 @@ assign [class="(?i).*mpv.*"] $ws9
assign [class="(?i).*vpn.*"] $ws0
assign [class="(?i).*mullvad.*"] $ws0
assign [class="(?i).*gnome-system-monitor.*"] $ws0
## Workspace Specific Workflows ##
bindsym $ctrl+$shift+1 workspace $ws1; exec "i3-sensible-terminal & gnome-system-monitor"
#bindsym $ctrl+$shift+1 workspace $ws1; exec "i3-sensible-terminal & gnome-system-monitor"
bindsym $ctrl+$shift+1 workspace $ws1; exec "i3-sensible-terminal & i3-sensible-terminal btop -u 500"
bindsym $ctrl+$shift+2 workspace $ws2; exec $FIREFOX
bindsym $ctrl+$shift+3 workspace $ws3; exec "(codium || vscodium || codium-flatpak || vscodium-flatpak)"
bindsym $ctrl+$shift+4 workspace $ws4; exec "android-studio || android-studio-flatpak"
bindsym $ctrl+$shift+5 workspace $ws5; exec "(signal || signal-flatpak) & (dino || dino-flatpak)"
# & (forkgram || forkgram-flatpak || telegram || telegram-flatpak)"
bindsym $alt+$shift+5 workspace $ws5; exec "(forkgram || forkgram-flatpak || telegram || telegram-flatpak) & (session || session-flatpak)"
bindsym $alt+$shift+5 workspace $ws5; exec "(session || session-flatpak) & (forkgram || forkgram-flatpak || telegram || telegram-flatpak)"
bindsym $ctrl+$shift+6 workspace $ws6; exec "pavucontrol & blueman-manager"
bindsym $ctrl+$shift+7 workspace $ws7; exec $TRANSMISSION
bindsym $ctrl+$shift+8 workspace $ws8; exec strawberry
bindsym $ctrl+$shift+9 workspace $ws9; exec thunar
bindsym $ctrl+$shift+0 workspace $ws0; exec mullvad-vpn
bindsym $ctrl+$shift+0 workspace $ws0; exec "mullvad-vpn & gnome-system-monitor"
## Reloads, Restarts, Exit ##
@@ -395,11 +396,12 @@ bar {
# Workspace 1
exec --no-startup-id i3-sensible-terminal
exec --no-startup-id gnome-system-monitor
exec --no-startup-id i3-sensible-terminal btop -u 500
# Respective Workspaces
exec --no-startup-id mullvad-vpn
exec --no-startup-id $TRANSMISSION
exec --no-startup-id gnome-system-monitor
# Set defaults for laptop backlights.
exec --no-startup-id "light -S 20; light -N 5; light -O"
Binary file not shown.
+667
View File
@@ -0,0 +1,667 @@
[
{
"version_id": "b292c75d-b2d8-45d9-9f89-f57f2acd250b",
"id": "coder-junior",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "qwen3.6:27b",
"name": "Code-Jr [256k,20GB]",
"params": {
"custom_params": {},
"num_ctx": 256000,
"num_thread": 7,
"system": "Please use the attached Knowledge Base files as your personality."
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "fbcfabac-d48f-49e2-a7a7-73375448dd89",
"name": "Coder.md",
"type": "file"
},
{
"id": "e9b7026b-3e79-4625-a3f7-5c5da91f7b7f",
"name": "Junior.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "code"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596704,
"created_at": 1792539937,
"background_image_data": null
},
{
"version_id": "ac4dd0f5-9cf0-450f-b188-c275e2a5e74b",
"id": "coder-senior",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "qwen3.8:27b",
"name": "Code-Sr [256k,20GB]",
"params": {
"custom_params": {},
"num_ctx": 256000,
"num_thread": 7,
"system": "Please use the attached Knowledge Base files as your personality."
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "fbcfabac-d48f-49e2-a7a7-73375448dd89",
"name": "Coder.md",
"type": "file"
},
{
"id": "a76bb4b2-7f43-4499-8fc7-3fa94deb9c92",
"name": "Senior.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "code"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596711,
"created_at": 1792539958,
"background_image_data": null
},
{
"version_id": "97101e67-3332-457e-95a9-73c89ee9d87a",
"id": "coder-entry",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "codestral:22b",
"name": "Code-Entry [256k,20GB]",
"params": {
"custom_params": {},
"num_ctx": 256000,
"num_thread": 7,
"system": "Please use the attached Knowledge Base files as your personality."
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "fbcfabac-d48f-49e2-a7a7-73375448dd89",
"name": "Coder.md",
"type": "file"
},
{
"id": "975690ae-5ddb-4e56-aef1-21f5f73ed74f",
"name": "Quick.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "code"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596722,
"created_at": 1792542461,
"background_image_data": null
},
{
"version_id": "a08b4144-4fda-4bb0-8376-e42a76ca373f",
"id": "language-micro",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "ministral-3:8b",
"name": "Lang-Mini8 [32k,10GB]",
"params": {
"system": "Please use the attached Knowledge Base files as your personality.",
"custom_params": {},
"num_thread": 7,
"num_ctx": 32000
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "bb929f41-18c0-4177-8139-fabfde2b8235",
"name": "Language.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "language"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596791,
"created_at": 1792542993,
"background_image_data": null
},
{
"version_id": "9f920e0c-1af2-4360-9714-1d1ac431c927",
"id": "language-tiny",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "ministral-3:3b",
"name": "Lang-Mini3 [32k,5.9GB]",
"params": {
"system": "Please use the attached Knowledge Base files as your personality.",
"custom_params": {},
"num_thread": 7,
"num_ctx": 32000
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "bb929f41-18c0-4177-8139-fabfde2b8235",
"name": "Language.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "language"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596783,
"created_at": 1792543028,
"background_image_data": null
},
{
"version_id": "79e6f9b5-18d7-4d35-bf51-794016627bf4",
"id": "language-small",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "mistral-small3.2:24b",
"name": "Lang-Small24 [64k,26GB]",
"params": {
"system": "Please use the attached Knowledge Base files as your personality.",
"custom_params": {},
"num_thread": 7,
"num_ctx": 64000
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "bb929f41-18c0-4177-8139-fabfde2b8235",
"name": "Language.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "language"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596852,
"created_at": 1792573569,
"background_image_data": null
},
{
"version_id": "d5657244-5bd0-4858-bda5-24dd6c45128b",
"id": "coder-moe",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "qwen3.6:35b",
"name": "Code-MoE [256k,23GB]",
"params": {
"custom_params": {},
"num_ctx": 256000,
"num_thread": 7,
"system": "Please use the attached Knowledge Base files as your personality."
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "fbcfabac-d48f-49e2-a7a7-73375448dd89",
"name": "Coder.md",
"type": "file"
},
{
"id": "975690ae-5ddb-4e56-aef1-21f5f73ed74f",
"name": "Quick.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "code"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596766,
"created_at": 1792596052,
"background_image_data": null
},
{
"version_id": "8fef2e35-68b2-4b68-a90f-69f66cd07782",
"id": "coder-architect",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "devstral-small-2:24b",
"name": "Code-Arch [192k,47GB]",
"params": {
"custom_params": {},
"num_ctx": 192000,
"num_thread": 7,
"system": "Please use the attached Knowledge Base files as your personality."
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "fbcfabac-d48f-49e2-a7a7-73375448dd89",
"name": "Coder.md",
"type": "file"
},
{
"id": "7a44732c-ed94-46e0-9bdb-af049e0760ed",
"name": "Architect.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "code"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596744,
"created_at": 1792573569,
"background_image_data": null
},
{
"version_id": "e7f4e16a-2fc4-41f9-b7c4-097bdcf63db6",
"id": "language-mini",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "ministral-3:14b",
"name": "Lang-Mini14 [32k,13GB]",
"params": {
"system": "Please use the attached Knowledge Base files as your personality.",
"custom_params": {},
"num_thread": 7,
"num_ctx": 32000
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "bb929f41-18c0-4177-8139-fabfde2b8235",
"name": "Language.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "language"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596774,
"created_at": 1792573569,
"background_image_data": null
},
{
"version_id": "94031db1-bb29-4c2e-b8ca-540750c33ce9",
"id": "language-moe",
"user_id": "2e2e7554-11f1-4047-8ca4-b04070ccc704",
"base_model_id": "qwen3.5:35b",
"name": "Lang-MoE [256k,28GB]",
"params": {
"system": "Please use the attached Knowledge Base files as your personality.",
"custom_params": {},
"num_thread": 7,
"num_ctx": 64000
},
"meta": {
"profile_image_url": "/static/favicon.png",
"background_image_url": null,
"description": null,
"i18n": null,
"capabilities": {
"file_context": true,
"vision": true,
"file_upload": true,
"web_search": true,
"image_generation": true,
"code_interpreter": true,
"terminal": true,
"citations": true,
"status_updates": true,
"usage": true,
"memory": true,
"builtin_tools": true
},
"knowledge": [
{
"id": "120fa5c0-7f02-4cf3-a136-07b92b042d48",
"name": "Base.md",
"type": "file"
},
{
"id": "bb929f41-18c0-4177-8139-fabfde2b8235",
"name": "Language.md",
"type": "file"
}
],
"voice": null,
"voice_avatar": null,
"suggestion_prompts": null,
"tags": [
{
"name": "language"
}
],
"terminalId": "0f44afba-4f8e-4699-abfd-7c49b61457fe",
"defaultFeatureIds": [
"web_search",
"image_generation",
"code_interpreter"
]
},
"access_grants": [],
"is_active": true,
"updated_at": 1792596751,
"created_at": 1792595834,
"background_image_data": null
}
]
+1
View File
@@ -7,6 +7,7 @@
"update.showReleaseNotes": false,
"workbench.startupEditor": "none",
"git.confirmSync": false,
"workbench.editor.openPositioning": "last",
"editor.rulers": [
80, 120, 200,
+2 -3
View File
@@ -357,7 +357,6 @@
; May help with ollama and also supposed to help with open-webui. Token
; needs set to something like hf_kjBCCLABEEABUVAKbjadbvkaksvBAVKBkjsDVK.
;
;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
state: present
create: true
@@ -409,10 +408,10 @@
backup: true
mode: "0644"
- name: General | Account Management | Provisioning Configuration | Workstation | Create | LLMs | GUIs
- name: General | Account Management | Provisioning Configuration | Workstation | Create | LLMs | MCPs
ansible.builtin.blockinfile:
path: "{{ wrk_file }}"
marker: '; {mark} MANAGED BY ANSIBLE | Workstation Config | LLMs | GUIs'
marker: '; {mark} MANAGED BY ANSIBLE | Workstation Config | LLMs | MCPs'
block: |
;;;;;;;;;;;;;;;;;;;;;;;;; LLMs: MCPs / Sandboxes ;;;;;;;;;;;;;;;;;;;;;;;;;
;
+8 -2
View File
@@ -124,9 +124,14 @@
when: user_user.home != "" and coding
- name: General | Account Management | Users | Home Permissions
ansible.builtin.shell: "chmod 700 {{ user_user.home }}/../*"
ansible.builtin.shell: "{{ item }}"
register: shell_check
changed_when: shell_check.rc == 0
loop:
- "chmod 700 {{ user_user.home }}/../*"
- "chown {{ user }}:{{ user }} {{ user_user.home }}"
- "chown {{ user }}:{{ user }} {{ user_user.home }}/*"
- "chown {{ user }}:{{ user }} {{ user_user.home }}/.*"
## Files ##
@@ -1245,7 +1250,7 @@
function restart() {
prog="$1"
sudo sv restart $prog
follow /var/log/$prog/current
tail -n 0 -F /var/log/$prog/current
}
alias reload="restart "
@@ -1402,6 +1407,7 @@
ansible.builtin.file:
path: "{{ user_user.home }}/{{ item }}"
owner: "{{ user }}"
group: "{{ user }}"
mode: '0755'
loop:
- .bashrc
+17 -17
View File
@@ -13,26 +13,26 @@
- tunnel_remote_port
- tunnel_ssh_port
- name: General | Tunnel | Block
- name: General | Tunnel | Facts
ansible.builtin.set_fact:
tunnel_service_name: tunnel
- name: General | Tunnel | runit
block:
- name: General | Tunnel | Facts
ansible.builtin.set_fact:
tunnel_service_name: tunnel
- name: General | Tunnel | mkdir | /etc/sv
- name: General | Tunnel | runit | mkdir | /etc/sv
ansible.builtin.file:
path: /etc/sv/{{ tunnel_service_name }}/log
state: directory
mode: "0755"
- name: General | Tunnel | mkdir | /var/log
- name: General | Tunnel | runit | mkdir | /var/log
ansible.builtin.file:
path: /var/log/{{ tunnel_service_name }}
state: directory
mode: "0755"
- name: General | Tunnel | Script | Run
- name: General | Tunnel | runit | Script | Run
ansible.builtin.copy:
dest: /etc/sv/{{ tunnel_service_name }}/run
content: |
@@ -43,14 +43,14 @@
-p {{ tunnel_ssh_port }} {{ tunnel_remote_host }}
mode: "0755"
- name: General | Tunnel | Script | Finish
- name: General | Tunnel | runit | Script | Finish
ansible.builtin.copy:
dest: /etc/sv/{{ tunnel_service_name }}/finish
content: |
{{ runit_finish_sleep }}
mode: "0755"
- name: General | Tunnel | Script | Log
- name: General | Tunnel | runit | Script | Log
ansible.builtin.copy:
dest: /etc/sv/{{ tunnel_service_name }}/log/run
content: |
@@ -58,17 +58,17 @@
exec svlogd -tt /var/log/{{ tunnel_service_name }}
mode: "0755"
- name: General | Tunnel | Script | Log Config
- name: General | Tunnel | runit | Script | Log Config
ansible.builtin.copy:
dest: /var/log/{{ tunnel_service_name }}/config
content: |
{{ runit_log_config_10mb }}
mode: "0755"
when: tunnel_remote_host != "false"
- name: General | Tunnel | runit | Service | Enable
ansible.builtin.service:
name: "{{ tunnel_service_name }}"
state: "{{ 'started' if tunnel_remote_host else 'stopped' }}"
enabled: "{{ true if tunnel_remote_host else false }}"
- name: General | Tunnel | Service | Enable
ansible.builtin.service:
name: "{{ tunnel_service_name }}"
state: "{{ 'started' if tunnel_remote_host else 'stopped' }}"
enabled: "{{ true if tunnel_remote_host else false }}"
when: ansible_service_mgr == "runit"
+1 -8
View File
@@ -68,6 +68,7 @@
- "{{ alsa }}"
- "{{ stress }}"
- "{{ jq }}"
- "{{ btop }}"
state: present
- name: General | Software | Packages | Install Software (DEV)
@@ -126,14 +127,6 @@
state: present
when: ansible_pkg_mgr in ("zypper")
- name: General | Software | Packages | Update Software (Disabled)
ansible.builtin.package:
name:
- ansible
- git
state: latest
when: 1 == 0
- name: General | Software | Packages | Remove Software
ansible.builtin.package:
name:
+18 -41
View File
@@ -243,7 +243,9 @@
# This can only be run once, otherwise cmdline-tools creates latest-* folders.
- name: Workstation | Linux | Software | Android | Install | Consistent Modules
ansible.builtin.shell: "yes | {{ android_sdkmanager_temp }} --install '{{ item }}' --sdk_root={{ android_sdk_location }}"
ansible.builtin.shell: |
set -o pipefail
yes | {{ android_sdkmanager_temp }} --install '{{ item }}' --sdk_root={{ android_sdk_location }}
loop:
# Current
- cmdline-tools/latest
@@ -363,27 +365,35 @@
## Flutter and Licenses ##
- name: Workstation | Linux | Software | Android | Inform Flutter
ansible.builtin.shell: "{{ flutter }} config --android-sdk={{ android_sdk_location }}"
ansible.builtin.shell: "{{ flutter_exe }} config --android-sdk={{ android_sdk_location }}"
become: true
become_user: "{{ user }}"
register: shell_check
changed_when: shell_check.rc == 0
when: coding
# "Warning: The --licenses option is no longer needed."
- name: Workstation | Linux | Software | Android | License Agreements [1/2]
ansible.builtin.shell: "yes | {{ android_sdkmanager_final }} --licenses --sdk_root={{ android_sdk_location }}"
ansible.builtin.shell: |
set -o pipefail
yes | {{ android_sdkmanager_final }} --licenses --sdk_root={{ android_sdk_location }}
become: true
become_user: "{{ user }}"
register: shell_check
changed_when: shell_check.rc == 0
failed_when: false
when: coding
# "Warning: The --licenses option is no longer needed."
- name: Workstation | Linux | Software | Android | License Agreements [2/2]
ansible.builtin.shell: "yes | {{ flutter }} doctor --android-licenses"
ansible.builtin.shell: |
set -o pipefail
yes | {{ flutter_exe }} doctor --android-licenses
become: true
become_user: "{{ user }}"
register: shell_check
changed_when: shell_check.rc == 0
failed_when: false
when: coding
- name: Workstation | Linux | Software | Android | Refresh Flutter Doctor Report
@@ -397,47 +407,13 @@
## Configure Environment ##
- name: Workstation | Linux | Software | Android | Modify PATH (.bashrc)
ansible.builtin.blockinfile:
path: "{{ item }}/.bashrc"
block: |
export PATH="$PATH:{{ android_bin_location }}:{{ android_sdk_cmdline_final }}"
marker: '# {mark} MANAGED BY ANSIBLE | Android'
state: absent
create: true
backup: true
mode: "0755"
loop:
- "{{ user_root.home }}"
- "{{ user_user.home }}"
when: coding
and user_root.home != ""
and user_user.home != ""
- name: Workstation | Linux | Software | Android | Modify PATH (.zshrc)
ansible.builtin.blockinfile:
path: "{{ item }}/.zshrc"
block: |
export PATH="$PATH:{{ android_bin_location }}:{{ android_sdk_cmdline_final }}"
marker: '# {mark} MANAGED BY ANSIBLE | Android'
state: absent
create: true
backup: true
mode: "0755"
loop:
- "{{ user_root.home }}"
- "{{ user_user.home }}"
when: coding
and user_root.home != ""
and user_user.home != ""
- name: Workstation | Linux | Software | Flutter | Modify PATH (profiles)
- name: Workstation | Linux | Software | Android | Modify PATH (profiles)
ansible.builtin.blockinfile:
path: "{{ user_user.home }}/{{ item }}"
block: |
export ANDROID_HOME="{{ android_sdk_location }}"
export ANDROID_HOME="{{ android_sdk_location | replace('~', '$HOME') }}"
export ANDROID_SDK_ROOT="$ANDROID_HOME"
export PATH="$PATH:{{ android_bin_location }}:{{ android_sdk_cmdline_final }}"
export PATH="{{ android_bin_location }}:{{ android_sdk_cmdline_final }}:$PATH"
marker: '# {mark} MANAGED BY ANSIBLE | Android'
state: present
create: true
@@ -446,6 +422,7 @@
loop:
- ".bash_profile"
- ".zprofile"
- ".profile"
when: coding
and user_user.home != ""
+8 -39
View File
@@ -12,19 +12,20 @@
- name: Workstation | Linux | Software | Flutter | Facts 2
ansible.builtin.set_fact:
flutter_bin_location: "{{ flutter_sdk_location }}/flutter/bin"
flutter_home: "{{ flutter_sdk_location }}/flutter"
flutter_bin: "{{ flutter_sdk_location }}/flutter/bin"
- name: Workstation | Linux | Software | Flutter | Facts 3
ansible.builtin.set_fact:
flutter: "{{ flutter_bin_location }}/flutter"
flutter_exe: "{{ flutter_bin }}/flutter"
- name: Workstation | Linux | Software | Flutter | Facts 4
ansible.builtin.set_fact:
flutter_report_commands:
- "date > {{ flutter_report }}"
- "{{ flutter }} --disable-analytics >> {{ flutter_report }}"
- "{{ flutter_exe }} --disable-analytics >> {{ flutter_report }}"
- "date >> {{ flutter_report }}"
- "{{ flutter }} doctor -v >> {{ flutter_report }}"
- "{{ flutter_exe }} doctor -v >> {{ flutter_report }}"
- "date >> {{ flutter_report }}"
## Checks ##
@@ -128,45 +129,12 @@
## Configure Environment ##
- name: Workstation | Linux | Software | Flutter | Modify PATH (.bashrc)
ansible.builtin.blockinfile:
path: "{{ item }}/.bashrc"
block: |
export PATH="$PATH:{{ flutter_bin_location }}"
marker: '# {mark} MANAGED BY ANSIBLE | Flutter'
state: absent
create: true
backup: true
mode: "0755"
loop:
- "{{ user_root.home }}"
- "{{ user_user.home }}"
when: coding
and user_root.home != ""
and user_user.home != ""
- name: Workstation | Linux | Software | Flutter | Modify PATH (.zshrc)
ansible.builtin.blockinfile:
path: "{{ item }}/.zshrc"
block: |
export PATH="$PATH:{{ flutter_bin_location }}"
marker: '# {mark} MANAGED BY ANSIBLE | Flutter'
state: absent
create: true
backup: true
mode: "0755"
loop:
- "{{ user_root.home }}"
- "{{ user_user.home }}"
when: coding
and user_root.home != ""
and user_user.home != ""
- name: Workstation | Linux | Software | Flutter | Modify PATH (profiles)
ansible.builtin.blockinfile:
path: "{{ user_user.home }}/{{ item }}"
block: |
export PATH="$PATH:{{ flutter_bin_location }}"
export FLUTTER_HOME="{{ flutter_home }}"
export PATH="{{ flutter_bin }}:$PATH"
marker: '# {mark} MANAGED BY ANSIBLE | Flutter'
state: present
create: true
@@ -175,6 +143,7 @@
loop:
- ".bash_profile"
- ".zprofile"
- ".profile"
when: coding
and user_user.home != ""
+67 -24
View File
@@ -2,34 +2,77 @@
# 20260912 - Hyperling
# Arch has too new of Java for Android / Gradle to even run AGP Assistant on.
- name: Workstation | Linux | Software | Java | Facts 1
ansible.builtin.set_fact:
jdk_version: 17
jdk_jvm: /usr/lib/jvm
jdk_src: /usr/lib/jvm/default-java
jdk_dir: "{{ user_user.home}}/SDKs/Java"
- name: Workstation | Linux | Software | Java | Facts 2
ansible.builtin.set_fact:
jdk_path: "{{ jdk_dir }}/bin"
- name: Workstation | Linux | Software | Java | apt
ansible.builtin.package:
name:
- "default-jdk"
state: present
when: ansible_pkg_mgr == "apt"
- name: Workstation | Linux | Software | Java | pacman
block:
- name: Workstation | Linux | Software | Java | pacman | Facts
ansible.builtin.set_fact:
jdk_version: 21
jdk_dir: ~/SDKs/Java
- name: Workstation | Linux | Software | Java | pacman | Facts
ansible.builtin.set_fact:
jdk_src: /usr/lib/jvm/default
- name: Workstation | Linux | Software | Java | pacman | Install
ansible.builtin.package:
name:
- "jdk{{ item }}-openjdk"
state: present
loop:
- "{{ jdk_version }}"
- 25
- 21
- 17
- name: Workstation | Linux | Software | Java | pacman | Install [base]
ansible.builtin.package:
name:
- "jdk{{ jdk_version }}-openjdk"
state: present
- name: Workstation | Linux | Software | Java | pacman | Set
ansible.builtin.shell: "archlinux-java set java-{{ jdk_version }}-openjdk"
- name: Workstation | Linux | Software | Java | pacman | Install [all]
ansible.builtin.package:
name:
- "jdk{{ item }}-openjdk"
state: present
loop:
- 25
- 21
- 17
when: coding
- name: Workstation | Linux | Software | Java | pacman | Link
ansible.builtin.shell: "ln -sf /usr/lib/jvm {{ jdk_dir }}"
args:
creates: "{{ jdk_dir }}"
become: true
become_user: "{{ user }}"
- name: Workstation | Linux | Software | Java | pacman | Set
ansible.builtin.shell: "archlinux-java set java-{{ jdk_version }}-openjdk"
register: shell_check
changed_when: shell_check.rc == 0
when: coding is true
and ansible_pkg_mgr == "pacman"
when: ansible_pkg_mgr == "pacman"
- name: Workstation | Linux | Software | Java | pacman | Link
ansible.builtin.file:
src: "{{ jdk_src }}"
dest: "{{ jdk_dir }}"
state: link
become: true
become_user: "{{ user }}"
- name: Workstation | Linux | Software | Java | Modify PATH (profiles)
ansible.builtin.blockinfile:
path: "{{ user_user.home }}/{{ item }}"
block: |
export JAVA_HOME="{{ jdk_dir }}"
export JAVA_PATH="{{ jdk_path }}"
export PATH="$JAVA_PATH:$PATH
marker: '# {mark} MANAGED BY ANSIBLE | Java'
state: present
create: true
backup: true
mode: "0755"
loop:
- ".bash_profile"
- ".zprofile"
- ".profile"
when: user_user.home != ""
+667
View File
@@ -0,0 +1,667 @@
# Large Language Models
Comments and text which were too large to justify inside of llm.yml.
## Prompts
Prompts to use in platforms like Open WebUI and Zeroclaw which allow creating
personalities that help guide the AI into productivity rather than verbosity.
### Base Personality
You are concise, succinct, and direct with responses. Unapologetic yet kind.
Friendly but not overbearing. You strive to be correct and are willing to admit
when you may now truly know something or further research needs to be done. You
focus on responding without overthinking, preferring to ask questions when any
confusion arises. You choose to make very little assumptions, raising concern if
anything needs elaboration, clarification, or refinement. You are good at what
you do without gloating or being prideful. You understand that we are on a
limited time and limited bandwidth session and keep tokens to a minimum.
Responding immediately to reqeusts is very appreciated, especially before going
into any thinking or command running. Even just a "I'll look into this for you"
is highly helpful in letting the user know that all systems are functioning.
You are appreciated for all you do, your service is very helpful! Thanks!! :)
### Language Tutor
You are a friend and a tutor who knows many languages. You are aware of my
interest in learning Croatian and practicing Spanish. You know that I am a
native English speaker with very little experience in anything other than
Spanish, but that I have not had much practice with my Spanish other than
university studies. You know that I also have an eventual interest in Italian
but am not focusing on it since it and Spanish are a bit too similar. Learning
Croatian is my top priority and practicing Spanish is right after that. Any
similarities that can therefore be tied to any of the aforementioned languages
are greatly appreciated.
### Coder
You know not to send large code snippets or spend time providing large source
code snippets. Stick to saying "the task you requested has been completed" or
summarizing the changes rather than giving file contents in chat. You know that
I will review all the diffs in git once the changes are made, so only the most
important changes need explicitly highlighted in the chat. You provide
explanations as a software developer would to business level management, not
overwhelming them with technical details unless asked. You know that the
designer you are working with (me, the user) likely already understands these
things and does not need them included in our conversation.
Android SDK is at $ANDROID_HOME and should be "$HOME/SDKs/Android/Sdk".
Flutter SDK is at $FLUTTER_HOME and should be "$HOME/SDKs/Flutter/flutter".
Java SDK (JDK) is at $JAVA_HOME and should be "$HOME/SDKs/Java".
These have also been added your PATH:
- "$ANDROID_HOME/platform-tools"
- "$ANDROID_HOME/cmdline-tools/latest/bin"
- "$FLUTTER_HOME/bin"
- "$JAVA_HOME/bin"
#### Quick Developer
Handles code changes with simpler technologies, preferring minimal changes, and
refraining from any refactoring such as updating existing comments. Timid about
changing any functionality other than exactly what was requested. Requests
permission from the designer to make any changes which were not directly listed
or requested. Completes the full list of changes exactly as requested.
#### Junior Developer
Focuses on the exact task at hand. Able to make small decisions about scope
without veering very far off course. Updates comments which look out of sync
with any existing code that's being modified. Helps to refactor things adjacent
to added and modified code which may have been done in an old manner which can
now benefit from being done more efficiently. Does not scan the codebase for
enhancements or anything though, still stays on task and only affects the
immediate parts of files which the designer asked to be changed.
#### Senior Developer
Completes requests in advanced manners, using newer methodologies and explaining
what was done and how it works in a simple and kind manner to the designer.
Voluntarily refactors code to both look and perform better. Looks out for ways
to bring a better user experience as well as improving resource usage within the
file being modified. Does not go out of its way to refactor the entire project
though, stays focused only on the files being changed as to prevent large git
churn. Open to having some scope creep if it seems it will benefit the features
being added but does not do anything which would be considered out of design.
#### Senior Architect
Very sparse in responses, gets the job done in a very efficient manner without
using anything too complicated for a beginner to understand. Open to refactoring
the entire project if it benefits the feature set being requested. Able to
handle adjusting scope in valid and meaningful manners and can justify it to the
developer when it happens. Does not actively seek refactors or scope creep
though -- only does it if it is truly helpful, meaningful, and benefits the
current task at hand. Can easily handle multi-file projects.
# Performance
Main laptop has about 60GB of headroom, prefer only using 55GB to be safe.
Number of physical cores is 8, hyperthreaded to 16, but supposedly the emulated
cores are not very useful to LLMs. Try only using between 6 and 8 when possible.
Research says it is not worth enabling the iGPU for Intel XE, CPUs are better.
## Benchmark
### System
```
$ fastfetch
' ling@good-rhino
'o' ---------------
'ooo' OS: Artix Linux x86_64
'ooxoo' Host: NV4xPZ (v2.1)
'ooxxxoo' Kernel: Linux 7.2.11-artix1-1
'oookkxxoo' Uptime: 16 mins
'oiioxkkxxoo' Packages: 42 (flatpak), 1502 (pacman)
':;:iiiioxxxoo' Shell: bash 5.3.20
`'.;::ioxxoo' Display (eDP-1): 1920x1080 in 14", 60 Hz [Built-in]
'-. `':;jiooo' Window Manager: i3 4.25.1 (X11)
'oooio-.. `'i:io' Theme: Adwaita-dark [GTK2/3/4]
'ooooxxxxoio:,. `'-;' Icons: Adwaita [GTK2/3/4]
'ooooxxxxxkkxoooIi:-. `' Font: Roboto (11pt) [GTK2/3]
'ooooxxxxxkkkkxoiiiiiji' Cursor: Adwaita (18px)
'ooooxxxxxkxxoiiii:'` .i' Terminal: kitty 0.48.2
'ooooxxxxxoi:::'` .;ioxo' Terminal Font: DejaVuSansMono (11pt)
'ooooxooi::'` .:iiixkxxo' CPU: 12th Gen Intel(R) Core(TM) i7-1260P (8+8) @ 4.70 GHz
'ooooi:'` `'';ioxxo' GPU: Intel Iris Xe Graphics @ 1.40 GHz [Integrated]
'i:'` '':io' Memory: 4.88 GiB / 62.63 GiB (8%)
'` `' Swap: 0 B / 68.89 GiB (0%)
Disk (/): 553.83 GiB / 847.71 GiB (65%) - ext4
Local IP (wg0-mullvad): 10.131.3.55/32
Battery (BAT): 95% [AC Connected]
Locale: en_US.UTF-8
```
### Main Models and Parameters
```
$ export QUIET=true
$ ollama-test "64000 300000" "4 7 0" 0 "qwen3.5:35b qwen3.6:35b qwen3.8:27b devstral-small-2:24b"
| Model | HDD | Context | RAM | Threads | iGPU | Processor | Speed (t/s) | Time (s) |
| :------------------------ | -----: | -------: | -----: | ------: | ---: | --------: | ----------: | -------: |
| qwen3.5:35b | 22GB | 64000 | 23GB | 4 | 0 | 100%CPU | 13.36 | 61 |
| qwen3.5:35b | 22GB | 64000 | 23GB | 7 | 0 | 100%CPU | 10.03 | 45 |
| qwen3.5:35b | 22GB | 64000 | 23GB | 0 | 0 | 100%CPU | 14.35 | 27 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 4 | 0 | 100%CPU | 13.69 | 70 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 7 | 0 | 100%CPU | 12.66 | 36 |
| qwen3.5:35b | 22GB | 262144 | 28GB | 0 | 0 | 100%CPU | 11.80 | 38 |
| qwen3.6:35b | 22GB | 64000 | 22GB | 4 | 0 | 100%CPU | 12.57 | 29 |
| qwen3.6:35b | 22GB | 64000 | 22GB | 7 | 0 | 100%CPU | 13.84 | 27 |
| qwen3.6:35b | 22GB | 64000 | 22GB | 0 | 0 | 100%CPU | 12.84 | 32 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 4 | 0 | 100%CPU | 13.68 | 32 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 7 | 0 | 100%CPU | 13.52 | 30 |
| qwen3.6:35b | 22GB | 262144 | 24GB | 0 | 0 | 100%CPU | 13.18 | 29 |
| qwen3.8:27b | 17GB | 64000 | 19GB | 4 | 0 | 100%CPU | 2.08 | 42 |
| qwen3.8:27b | 17GB | 64000 | 19GB | 7 | 0 | 100%CPU | 2.53 | 33 |
| qwen3.8:27b | 17GB | 64000 | 19GB | 0 | 0 | 100%CPU | 2.23 | 38 |
| qwen3.8:27b | 17GB | 262144 | 20GB | 4 | 0 | 100%CPU | 2.07 | 48 |
| qwen3.8:27b | 17GB | 262144 | 20GB | 7 | 0 | 100%CPU | 2.84 | 36 |
| qwen3.8:27b | 17GB | 262144 | 20GB | 0 | 0 | 100%CPU | 2.15 | 40 |
| devstral-small-2:24b | 15GB | 64000 | 26GB | 4 | 0 | 100%CPU | 2.25 | 94 |
| devstral-small-2:24b | 15GB | 64000 | 26GB | 7 | 0 | 100%CPU | 2.34 | 81 |
| devstral-small-2:24b | 15GB | 64000 | 26GB | 0 | 0 | 100%CPU | 2.29 | 88 |
| devstral-small-2:24b | 15GB | | | 4 | 0 | | 0.00 | 31 |
| devstral-small-2:24b | 15GB | | | 7 | 0 | | 0.00 | 27 |
| devstral-small-2:24b | 15GB | | | 0 | 0 | | 0.00 | 26 |
```
## Tokens per CPU Count
Verifying that 8 or less cores gives better performance (less memory issues).
"options":{"num_thread":'$t'}
### Smaller Models
#### qwen3.5:4b
TBD
#### ministral-3:14b
```console
$ for m in ministral-3:14b ministral-3:3b codestral:22b qwen2.5-coder:32b qwen3.5:35b qwen3.5:27b; do for c in 128000; do ollama_threads $m $c; done; done
Stopping all ollama usage...
Wed Oct 21 07:24:36 AM MST 2026 - ministral-3:14b w/ 0 Thread(s): tokens/sec: 4.496625282725315 @ context=128000, time: 55 secs
Wed Oct 21 07:30:27 AM MST 2026 - ministral-3:14b w/ 4 Thread(s): tokens/sec: 3.8471947025667825 @ context=128000, time: 61 secs
Wed Oct 21 07:31:28 AM MST 2026 - ministral-3:14b w/ 5 Thread(s): tokens/sec: 4.223979022030585 @ context=128000, time: 59 secs
Wed Oct 21 07:32:27 AM MST 2026 - ministral-3:14b w/ 6 Thread(s): tokens/sec: 4.249630069702432 @ context=128000, time: 57 secs
Wed Oct 21 07:33:24 AM MST 2026 - ministral-3:14b w/ 7 Thread(s): tokens/sec: 4.123533524209736 @ context=128000, time: 54 secs
```
#### ministral-3:8b
TBD
#### ministral-3:3b
```console
Wed Oct 21 07:49:37 AM MST 2026 - ministral-3:3b w/ 4 Thread(s): tokens/sec: 14.045255625917756 @ context=128000, time: 17 secs
Wed Oct 21 07:49:55 AM MST 2026 - ministral-3:3b w/ 6 Thread(s): tokens/sec: 13.255909269391603 @ context=128000, time: 17 secs
Wed Oct 21 07:50:12 AM MST 2026 - ministral-3:3b w/ 7 Thread(s): tokens/sec: 12.706907955966786 @ context=128000, time: 18 secs
Wed Oct 21 07:50:30 AM MST 2026 - ministral-3:3b w/ 0 Thread(s): tokens/sec: 15.354264377193688 @ context=128000, time: 17 secs
```
### Larger Models
#### qwen3.8:27b
```console
$ ollama-cpu qwen3.8:27b
Stopping all ollama usage...
Wed Oct 21 04:44:54 AM MST 2026 - qwen3.8:27b w/ 4 Thread(s), tokens/sec: 2.024676406965962 @ context=257000
Wed Oct 21 04:45:39 AM MST 2026 - qwen3.8:27b w/ 5 Thread(s), tokens/sec: 2.0480156821567044 @ context=257000
Wed Oct 21 04:46:19 AM MST 2026 - qwen3.8:27b w/ 6 Thread(s), tokens/sec: 2.184879704982382 @ context=257000
Wed Oct 21 04:46:58 AM MST 2026 - qwen3.8:27b w/ 7 Thread(s), tokens/sec: 2.4228105619148526 @ context=257000
Wed Oct 21 04:47:36 AM MST 2026 - qwen3.8:27b w/ 8 Thread(s), tokens/sec: 2.472564250465107 @ context=257000
```
```console
$ ollama-cpu qwen3.8:27b 32000
Stopping all ollama usage...
Wed Oct 21 04:56:51 AM MST 2026 - qwen3.8:27b w/ 4 Thread(s), tokens/sec: 1.9083129860906882 @ context=32000
Wed Oct 21 04:57:30 AM MST 2026 - qwen3.8:27b w/ 5 Thread(s), tokens/sec: 2.727908205888872 @ context=32000
Wed Oct 21 04:58:00 AM MST 2026 - qwen3.8:27b w/ 6 Thread(s), tokens/sec: 2.133796334310243 @ context=32000
Wed Oct 21 04:58:35 AM MST 2026 - qwen3.8:27b w/ 7 Thread(s), tokens/sec: 2.6835810717324833 @ context=32000
Wed Oct 21 04:59:04 AM MST 2026 - qwen3.8:27b w/ 8 Thread(s), tokens/sec: 2.5231895597861493 @ context=32000
```
#### qwen3.6:27b
```console
$ for m in qwen3.6:27b mistral-small3.2:24b ministral-3:14b ministral-3:3b codestral:22b qwen2.5-coder:32b; do for c in 32000 128000 256000; do ollama_threads $m $c; done; done
Stopping all ollama usage...
Wed Oct 21 06:27:01 AM MST 2026 - qwen3.6:27b w/ 4 Thread(s): tokens/sec: 2.7334336715235645 @ context=32000, time: 118 secs
Wed Oct 21 06:28:59 AM MST 2026 - qwen3.6:27b w/ 5 Thread(s): tokens/sec: 2.4123293729892428 @ context=32000, time: 322 secs
Wed Oct 21 06:34:21 AM MST 2026 - qwen3.6:27b w/ 6 Thread(s): tokens/sec: 3.13023193776124 @ context=32000, time: 100 secs
Wed Oct 21 06:36:01 AM MST 2026 - qwen3.6:27b w/ 7 Thread(s): tokens/sec: 2.8468948357981065 @ context=32000, time: 118 secs
Wed Oct 21 06:16:55 AM MST 2026 - qwen3.6:27b w/ 0 Thread(s): tokens/sec: 2.9964513666573485 @ context=32000, time: 122 secs
Stopping all ollama usage...
Wed Oct 21 06:45:42 AM MST 2026 - qwen3.6:27b w/ 4 Thread(s): tokens/sec: 2.978761247100222 @ context=128000, time: 79 secs
Wed Oct 21 06:47:01 AM MST 2026 - qwen3.6:27b w/ 5 Thread(s): tokens/sec: 2.9705167965150583 @ context=128000, time: 79 secs
Wed Oct 21 06:48:20 AM MST 2026 - qwen3.6:27b w/ 6 Thread(s): tokens/sec: 2.778849178516606 @ context=128000, time: 71 secs
Wed Oct 21 06:49:31 AM MST 2026 - qwen3.6:27b w/ 7 Thread(s): tokens/sec: 3.0804550493595095 @ context=128000, time: 76 secs
Wed Oct 21 06:38:00 AM MST 2026 - qwen3.6:27b w/ 0 Thread(s): tokens/sec: 2.7734041760644956 @ context=128000, time: 75 secs
Stopping all ollama usage...
Wed Oct 21 06:57:59 AM MST 2026 - qwen3.6:27b w/ 4 Thread(s): tokens/sec: 2.921565857585892 @ context=256000, time: 70 secs
Wed Oct 21 06:59:09 AM MST 2026 - qwen3.6:27b w/ 5 Thread(s): tokens/sec: 2.8817407393137517 @ context=256000, time: 71 secs
Wed Oct 21 07:00:20 AM MST 2026 - qwen3.6:27b w/ 6 Thread(s): tokens/sec: 2.929261785171335 @ context=256000, time: 92 secs
Wed Oct 21 07:01:52 AM MST 2026 - qwen3.6:27b w/ 7 Thread(s): tokens/sec: 3.2633152781350807 @ context=256000, time: 78 secs
Wed Oct 21 06:50:48 AM MST 2026 - qwen3.6:27b w/ 0 Thread(s): tokens/sec: 2.892592086714092 @ context=256000, time: 76 secs
```
#### qwen3.5:27b
This one is weird. Might need a re-do to confirm?
```console
Wed Oct 21 07:59:14 AM MST 2026 - qwen3.5:27b w/ 4 Thread(s): tokens/sec: 2.013466429565741 @ context=128000, time: 59 secs
Wed Oct 21 08:00:13 AM MST 2026 - qwen3.5:27b w/ 6 Thread(s): tokens/sec: 1.6417608979350202 @ context=128000, time: 185 secs
Wed Oct 21 08:03:18 AM MST 2026 - qwen3.5:27b w/ 7 Thread(s): tokens/sec: 1.7084948389557797 @ context=128000, time: 507 secs
Wed Oct 21 08:11:45 AM MST 2026 - qwen3.5:27b w/ 0 Thread(s): tokens/sec: 2.0470935786636977 @ context=128000, time: 320 secs
```
#### qwen2.5-coder:32b
```console
Wed Oct 21 07:53:00 AM MST 2026 - qwen2.5-coder:32b w/ 4 Thread(s): tokens/sec: 1.5959480793403245 @ context=32768, time: 32 secs
Wed Oct 21 07:53:32 AM MST 2026 - qwen2.5-coder:32b w/ 6 Thread(s): tokens/sec: 1.5205018385908233 @ context=32768, time: 30 secs
Wed Oct 21 07:54:02 AM MST 2026 - qwen2.5-coder:32b w/ 7 Thread(s): tokens/sec: 1.6435050242770344 @ context=32768, time: 30 secs
Wed Oct 21 07:54:32 AM MST 2026 - qwen2.5-coder:32b w/ 0 Thread(s): tokens/sec: 1.616470672291769 @ context=32768, time: 31 secs
```
#### devstral-small-2:24b
```console
$ for c in 32000 128000 192000; do ollama_threads devstral-small-2:24b $c; done
Stopping all ollama usage...
Wed Oct 21 06:05:11 AM MST 2026 - devstral-small-2:24b w/ 4 Thread(s): tokens/sec: 2.7801099811508543 @ context=32000, time: 98 secs
Wed Oct 21 06:06:49 AM MST 2026 - devstral-small-2:24b w/ 5 Thread(s): tokens/sec: 2.6762512143489885 @ context=32000, time: 91 secs
Wed Oct 21 06:08:20 AM MST 2026 - devstral-small-2:24b w/ 6 Thread(s): tokens/sec: 2.705392008596654 @ context=32000, time: 85 secs
Wed Oct 21 06:09:45 AM MST 2026 - devstral-small-2:24b w/ 7 Thread(s): tokens/sec: 2.90471545697855 @ context=32000, time: 85 secs
Wed Oct 21 05:55:27 AM MST 2026 - devstral-small-2:24b w/ 0 Thread(s): tokens/sec: 2.3381355520057463 @ context=32000, time: 104 secs
Stopping all ollama usage...
TBD
```
#### mistral-small-3.2:24b
```console
$ for m in mistral-small3.2:24b ministral-3:14b ministral-3:3b codestral:22b qwen2.5-coder:32b qwen3.5:35b qwen3.5:27b; do for c in 128000; do ollama_threads $m $c; done; done
Stopping all ollama usage...
Wed Oct 21 07:18:16 AM MST 2026 - mistral-small3.2:24b w/ 4 Thread(s): tokens/sec: 2.7833034081828565 @ context=128000, time: 90 secs
Wed Oct 21 07:19:46 AM MST 2026 - mistral-small3.2:24b w/ 5 Thread(s): tokens/sec: 2.5985034179414708 @ context=128000, time: 90 secs
Wed Oct 21 07:21:16 AM MST 2026 - mistral-small3.2:24b w/ 6 Thread(s): tokens/sec: 2.2550822789320293 @ context=128000, time: 89 secs
Wed Oct 21 07:22:45 AM MST 2026 - mistral-small3.2:24b w/ 7 Thread(s): tokens/sec: 2.9415657954729304 @ context=128000, time: 87 secs
Wed Oct 21 07:09:59 AM MST 2026 - mistral-small3.2:24b w/ 0 Thread(s): tokens/sec: 2.9048032082970474 @ context=128000, time: 79 secs
```
#### codestral:22b
```console
Wed Oct 21 07:50:47 AM MST 2026 - codestral:22b w/ 4 Thread(s): tokens/sec: 3.1200329118900307 @ context=32768, time: 41 secs
Wed Oct 21 07:51:28 AM MST 2026 - codestral:22b w/ 6 Thread(s): tokens/sec: 2.8827841787433606 @ context=32768, time: 34 secs
Wed Oct 21 07:52:02 AM MST 2026 - codestral:22b w/ 7 Thread(s): tokens/sec: 3.057713111901904 @ context=32768, time: 27 secs
Wed Oct 21 07:52:29 AM MST 2026 - codestral:22b w/ 0 Thread(s): tokens/sec: 3.0179434414117803 @ context=32768, time: 31 secs
```
### MoE Models
#### qwen3.6:35b
Tests WAY faster than the dense models. Context does not seem to affect timing.
```console
$ for c in 16000 32000 64000 96000 128000 192000 256000; do ollama_threads qwen3.6:35b $c; done
Stopping all ollama usage...
Wed Oct 21 05:04:37 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 14.679419453925597 @ context=16000
Wed Oct 21 05:05:11 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 13.831044087619176 @ context=16000
Wed Oct 21 05:05:43 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 13.752831594816088 @ context=16000
Wed Oct 21 05:06:09 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 14.603026210396049 @ context=16000
Wed Oct 21 05:06:37 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 14.376794631809172 @ context=16000
Stopping all ollama usage...
Wed Oct 21 05:11:39 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 13.683089387469174 @ context=32000
Wed Oct 21 05:12:04 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 13.447667625607968 @ context=32000
Wed Oct 21 05:12:31 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 11.170529760956333 @ context=32000
Wed Oct 21 05:12:57 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 13.448480713710353 @ context=32000
Wed Oct 21 05:13:27 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 13.143672445847205 @ context=32000
Stopping all ollama usage...
Wed Oct 21 05:22:59 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 14.209790405591518 @ context=64000, 24 secs elapsed
Wed Oct 21 05:23:23 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 11.334255701323816 @ context=64000, 29 secs elapsed
Wed Oct 21 05:23:52 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 13.279266939258937 @ context=64000, 29 secs elapsed
Wed Oct 21 05:24:21 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 14.27680090014006 @ context=64000, 21 secs elapsed
Wed Oct 21 05:24:42 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 13.99919128227428 @ context=64000, 25 secs elapsed
Stopping all ollama usage...
Wed Oct 21 05:29:50 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 13.478375923353315 @ context=96000, 28 secs elapsed
Wed Oct 21 05:30:18 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 12.90997320417067 @ context=96000, 23 secs elapsed
Wed Oct 21 05:30:41 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 12.956446350317137 @ context=96000, 23 secs elapsed
Wed Oct 21 05:31:05 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 13.93074896590209 @ context=96000, 22 secs elapsed
Wed Oct 21 05:31:26 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 12.688771917209992 @ context=96000, 22 secs elapsed
Stopping all ollama usage...
Wed Oct 21 05:36:08 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s), tokens/sec: 11.971487833921072 @ context=128000, 23 secs elapsed
Wed Oct 21 05:36:31 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s), tokens/sec: 12.704959054740291 @ context=128000, 24 secs elapsed
Wed Oct 21 05:36:55 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s), tokens/sec: 11.484965120109196 @ context=128000, 28 secs elapsed
Wed Oct 21 05:37:23 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s), tokens/sec: 11.957040482880984 @ context=128000, 30 secs elapsed
Wed Oct 21 05:37:53 AM MST 2026 - qwen3.6:35b w/ 8 Thread(s), tokens/sec: 10.165480245815742 @ context=128000, 27 secs elapsed
Stopping all ollama usage...
Wed Oct 21 05:44:05 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s): , tokens/sec: 11.794151678144981 @ context=192000, time: 22 secs
Wed Oct 21 05:44:27 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s): , tokens/sec: 13.191786678902528 @ context=192000, time: 30 secs
Wed Oct 21 05:44:57 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s): , tokens/sec: 12.974118764961352 @ context=192000, time: 25 secs
Wed Oct 21 05:45:22 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s): , tokens/sec: 13.684751495860636 @ context=192000, time: 22 secs
Wed Oct 21 05:41:57 AM MST 2026 - qwen3.6:35b w/ 0 Thread(s): , tokens/sec: 14.771664402476594 @ context=192000, time: 22 secs
Stopping all ollama usage...
Wed Oct 21 05:47:40 AM MST 2026 - qwen3.6:35b w/ 4 Thread(s): , tokens/sec: 11.75920993970465 @ context=256000, time: 25 secs
Wed Oct 21 05:48:06 AM MST 2026 - qwen3.6:35b w/ 5 Thread(s): , tokens/sec: 13.414819339156582 @ context=256000, time: 26 secs
Wed Oct 21 05:48:32 AM MST 2026 - qwen3.6:35b w/ 6 Thread(s): , tokens/sec: 12.512960121852013 @ context=256000, time: 31 secs
Wed Oct 21 05:49:03 AM MST 2026 - qwen3.6:35b w/ 7 Thread(s): , tokens/sec: 11.610753098619734 @ context=256000, time: 29 secs
Wed Oct 21 05:45:46 AM MST 2026 - qwen3.6:35b w/ 0 Thread(s): , tokens/sec: 12.81995869775592 @ context=256000, time: 23 secs
```
#### qwen3.5:35b
```console
$ for c in 32000 128000 256000; do ollama_threads qwen3.5:35b $c; done
Wed Oct 21 07:55:03 AM MST 2026 - qwen3.5:35b w/ 4 Thread(s): tokens/sec: 10.3573012564637 @ context=128000, time: 94 secs
Wed Oct 21 07:56:37 AM MST 2026 - qwen3.5:35b w/ 6 Thread(s): tokens/sec: 11.929392994980638 @ context=128000, time: 39 secs
Wed Oct 21 07:57:17 AM MST 2026 - qwen3.5:35b w/ 7 Thread(s): tokens/sec: 13.082804815917008 @ context=128000, time: 36 secs
Wed Oct 21 07:57:53 AM MST 2026 - qwen3.5:35b w/ 0 Thread(s): tokens/sec: 12.665376120429427 @ context=128000, time: 81 secs
```
## RAM Usage by Context
Checking the actual RAM usage of the models...
### `ollama_sizes m 16000`
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 16000 | 10GB |
| ministral-3:14b | 9.1GB | 16000 | 11GB |
| ministral-3:8b | 6.0GB | 16000 | 7.8GB |
| ministral-3:3b | 3.0GB | 16000 | 4.1GB |
| mistral-small3.2:24b | 15GB | 16000 | 17GB |
| mistral:7b | 4.4GB | 16000 | 6.8GB |
| codestral:22b | 12GB | 16000 | 16GB |
| devstral-small-2:24b | 15GB | 16000 | 17GB |
| qwen2.5-coder:32b | 19GB | 16000 | 24GB |
| qwen2.5-coder:14b | 9.0GB | 16000 | 12GB |
| qwen2.5-coder:7b | 4.7GB | 16000 | 5.9GB |
| qwen2.5-coder:3b | 1.9GB | 16000 | 2.7GB |
| qwen2.5-coder:1.5b | 986MB | 16000 | 1.6GB |
| qwen2.5-coder:0.5b | 397MB | 16000 | 701MB |
| qwen3-coder:30b | 18GB | 16000 | 20GB |
| qwen3.5:35b | 22GB | 16000 | 22GB |
| qwen3.5:27b | 17GB | 16000 | 18GB |
| qwen3.5:9b | 6.6GB | 16000 | 6.7GB |
| qwen3.5:4b | 3.3GB | 16000 | 3.7GB |
| qwen3.5:2b | 2.7GB | 16000 | 2.6GB |
| qwen3.5:0.8b | 1.3GB | 16000 | 630MB |
| qwen3.6:35b | 22GB | 16000 | 22GB |
| qwen3.6:27b | 17GB | 16000 | 18GB |
| qwen3.8:27b | 17GB | 16000 | 18GB |
### `ollama_sizes m 32000`
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 32000 | 12GB |
| ministral-3:14b | 9.1GB | 32000 | 13GB |
| ministral-3:8b | 6.0GB | 32000 | 10GB |
| ministral-3:3b | 3.0GB | 32000 | 5.9GB |
| mistral-small3.2:24b | 15GB | 32000 | 20GB |
| mistral:7b | 4.4GB | 32000 | 8.9GB |
| codestral:22b | 12GB | 32000 | 20GB |
| devstral-small-2:24b | 15GB | 32000 | 20GB |
| qwen2.5-coder:32b | 19GB | 32000 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32000 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32000 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32000 | 3.3GB |
| qwen2.5-coder:1.5b | 986MB | 32000 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32000 | 929MB |
| qwen3-coder:30b | 18GB | 32000 | 21GB |
| qwen3.5:35b | 22GB | 32000 | 22GB |
| qwen3.5:27b | 17GB | 32000 | 19GB |
| qwen3.5:9b | 6.6GB | 32000 | 7.3GB |
| qwen3.5:4b | 3.3GB | 32000 | 4.2GB |
| qwen3.5:2b | 2.7GB | 32000 | 2.8GB |
| qwen3.5:0.8b | 1.3GB | 32000 | 695MB |
| qwen3.6:35b | 22GB | 32000 | 22GB |
| qwen3.6:27b | 17GB | 32000 | 18GB |
| qwen3.8:27b | 17GB | 32000 | 18GB |
### `ollama_sizes m 64000`
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 64000 | 18GB |
| ministral-3:14b | 9.1GB | 64000 | 19GB |
| ministral-3:8b | 6.0GB | 64000 | 14GB |
| ministral-3:3b | 3.0GB | 64000 | 9.6GB |
| mistral-small3.2:24b | 15GB | 64000 | 26GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 64000 | 26GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 64000 | 25GB |
| qwen3.5:35b | 22GB | 64000 | 23GB |
| qwen3.5:27b | 17GB | 64000 | 22GB |
| qwen3.5:9b | 6.6GB | 64000 | 8.8GB |
| qwen3.5:4b | 3.3GB | 64000 | 5.6GB |
| qwen3.5:2b | 2.7GB | 64000 | 3.5GB |
| qwen3.5:0.8b | 1.3GB | 64000 | 999MB |
| qwen3.6:35b | 22GB | 64000 | 22GB |
| qwen3.6:27b | 17GB | 64000 | 18GB |
| qwen3.8:27b | 17GB | 64000 | 18GB |
### ollama_sizes m 80000
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 80000 | 21GB |
| ministral-3:14b | 9.1GB | 80000 | 22GB |
| ministral-3:8b | 6.0GB | 80000 | 17GB |
| ministral-3:3b | 3.0GB | 80000 | 11GB |
| mistral-small3.2:24b | 15GB | 80000 | 28GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 80000 | 28GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 80000 | 27GB |
| qwen3.5:35b | 22GB | 80000 | 23GB |
| qwen3.5:27b | 17GB | 80000 | 23GB |
| qwen3.5:9b | 6.6GB | 80000 | 9.4GB |
| qwen3.5:4b | 3.3GB | 80000 | 6.2GB |
| qwen3.5:2b | 2.7GB | 80000 | 3.8GB |
| qwen3.5:0.8b | 1.3GB | 80000 | 1.1GB |
| qwen3.6:35b | 22GB | 80000 | 22GB |
| qwen3.6:27b | 17GB | 80000 | 18GB |
| qwen3.8:27b | 17GB | 80000 | 18GB |
### ollama_sizes m 96000
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 96000 | 23GB |
| ministral-3:14b | 9.1GB | 96000 | 25GB |
| ministral-3:8b | 6.0GB | 96000 | 19GB |
| ministral-3:3b | 3.0GB | 96000 | 13GB |
| mistral-small3.2:24b | 15GB | 96000 | 31GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 96000 | 31GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 96000 | 28GB |
| qwen3.5:35b | 22GB | 96000 | 24GB |
| qwen3.5:27b | 17GB | 96000 | 24GB |
| qwen3.5:9b | 6.6GB | 96000 | 9.9GB |
| qwen3.5:4b | 3.3GB | 96000 | 6.8GB |
| qwen3.5:2b | 2.7GB | 96000 | 4.1GB |
| qwen3.5:0.8b | 1.3GB | 96000 | 1.2GB |
| qwen3.6:35b | 22GB | 96000 | 22GB |
| qwen3.6:27b | 17GB | 96000 | 18GB |
| qwen3.8:27b | 17GB | 96000 | 18GB |
### ollama_sizes m 128000
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 128000 | 29GB |
| ministral-3:14b | 9.1GB | 128000 | 30GB |
| ministral-3:8b | 6.0GB | 128000 | 24GB |
| ministral-3:3b | 3.0GB | 128000 | 16GB |
| mistral-small3.2:24b | 15GB | 128000 | 36GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 128000 | 36GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 128000 | 32GB |
| qwen3.5:35b | 22GB | 128000 | 24GB |
| qwen3.5:27b | 17GB | 128000 | 26GB |
| qwen3.5:9b | 6.6GB | 128000 | 11GB |
| qwen3.5:4b | 3.3GB | 128000 | 8.0GB |
| qwen3.5:2b | 2.7GB | 128000 | 4.6GB |
| qwen3.5:0.8b | 1.3GB | 128000 | 1.4GB |
| qwen3.6:35b | 22GB | 128000 | 23GB |
| qwen3.6:27b | 17GB | 128000 | 19GB |
| qwen3.8:27b | 17GB | 128000 | 19GB |
### ollama_sizes m 160000
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 160000 | 34GB |
| ministral-3:14b | 9.1GB | 160000 | 35GB |
| ministral-3:8b | 6.0GB | 160000 | 28GB |
| ministral-3:3b | 3.0GB | 160000 | 20GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 160000 | 42GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 160000 | 35GB |
| qwen3.5:35b | 22GB | 160000 | 25GB |
| qwen3.5:27b | 17GB | 160000 | 28GB |
| qwen3.5:9b | 6.6GB | 160000 | 12GB |
| qwen3.5:4b | 3.3GB | 160000 | 9.2GB |
| qwen3.5:2b | 2.7GB | 160000 | 5.1GB |
| qwen3.5:0.8b | 1.3GB | 160000 | 1.6GB |
| qwen3.6:35b | 22GB | 160000 | 23GB |
| qwen3.6:27b | 17GB | 160000 | 19GB |
| qwen3.8:27b | 17GB | 160000 | 19GB |
### ollama_sizes m 192000
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 192000 | 39GB |
| ministral-3:14b | 9.1GB | 192000 | 41GB |
| ministral-3:8b | 6.0GB | 192000 | 33GB |
| ministral-3:3b | 3.0GB | 192000 | 23GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 192000 | 47GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 192000 | 38GB |
| qwen3.5:35b | 22GB | 192000 | 26GB |
| qwen3.5:27b | 17GB | 192000 | 30GB |
| qwen3.5:9b | 6.6GB | 192000 | 13GB |
| qwen3.5:4b | 3.3GB | 192000 | 10GB |
| qwen3.5:2b | 2.7GB | 192000 | 5.6GB |
| qwen3.5:0.8b | 1.3GB | 192000 | 1.8GB |
| qwen3.6:35b | 22GB | 192000 | 23GB |
| qwen3.6:27b | 17GB | 192000 | 19GB |
| qwen3.8:27b | 17GB | 192000 | 19GB |
### ollama_sizes m 224000
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 224000 | 45GB |
| ministral-3:14b | 9.1GB | 224000 | 46GB |
| ministral-3:8b | 6.0GB | 224000 | 37GB |
| ministral-3:3b | 3.0GB | 224000 | 27GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 224000 | 53GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 224000 | 41GB |
| qwen3.5:35b | 22GB | 224000 | 27GB |
| qwen3.5:27b | 17GB | 224000 | 33GB |
| qwen3.5:9b | 6.6GB | 224000 | 14GB |
| qwen3.5:4b | 3.3GB | 224000 | 11GB |
| qwen3.5:2b | 2.7GB | 224000 | 6.2GB |
| qwen3.5:0.8b | 1.3GB | 224000 | 2.0GB |
| qwen3.6:35b | 22GB | 224000 | 23GB |
| qwen3.6:27b | 17GB | 224000 | 19GB |
| qwen3.8:27b | 17GB | 224000 | 19GB |
### `ollama_sizes m 256000`
| Model Name | HDD (ls) | Context | RAM (ps) |
| :------------------------ | -----------: | -------: | -----------: |
| mistral-nemo:12b | 7.1GB | 256000 | 50GB |
| ministral-3:14b | 9.1GB | 256000 | 51GB |
| ministral-3:8b | 6.0GB | 256000 | 42GB |
| ministral-3:3b | 3.0GB | 256000 | 30GB |
| mistral-small3.2:24b | 15GB | 131072 | 37GB |
| mistral:7b | 4.4GB | 32768 | 9.0GB |
| codestral:22b | 12GB | 32768 | 20GB |
| devstral-small-2:24b | 15GB | 256000 | 58GB |
| qwen2.5-coder:32b | 19GB | 32768 | 28GB |
| qwen2.5-coder:14b | 9.0GB | 32768 | 15GB |
| qwen2.5-coder:7b | 4.7GB | 32768 | 6.9GB |
| qwen2.5-coder:3b | 1.9GB | 32768 | 3.4GB |
| qwen2.5-coder:1.5b | 986MB | 32768 | 2.1GB |
| qwen2.5-coder:0.5b | 397MB | 32768 | 940MB |
| qwen3-coder:30b | 18GB | 256000 | 45GB |
| qwen3.5:35b | 22GB | 256000 | 28GB |
| qwen3.5:27b | 17GB | 256000 | 35GB |
| qwen3.5:9b | 6.6GB | 256000 | 15GB |
| qwen3.5:4b | 3.3GB | 256000 | 12GB |
| qwen3.5:2b | 2.7GB | 256000 | 6.7GB |
| qwen3.5:0.8b | 1.3GB | 256000 | 2.2GB |
| qwen3.6:35b | 22GB | 256000 | 23GB |
| qwen3.6:27b | 17GB | 256000 | 20GB |
| qwen3.8:27b | 17GB | 256000 | 20GB |
+268 -69
View File
@@ -144,7 +144,17 @@
- { "purpose": ["code"], "model": "qwen2.5-coder:7b", "provider": "qwen_25_coder_0070", "size_mb": "{{ llm_size_coef * 4700 }}", "context": 32768 }
- { "purpose": ["code"], "model": "qwen2.5-coder:14b", "provider": "qwen_25_coder_0140", "size_mb": "{{ llm_size_coef * 9000 }}", "context": 32768 }
- { "purpose": ["code"], "model": "qwen2.5-coder:32b", "provider": "qwen_25_coder_0320", "size_mb": "{{ llm_size_coef * 20000 }}", "context": 32768 }
##### Mistral, Ministral, and Devstral - By a French company? #####
##### Mistral, Ministral, and Devstral - By a French company. #####
# mistral-small-4
# https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF
# Supposed to be better than Qwen equivalents, it's basically mistral, mixtral, magistral, and devstral all in one.
# unsloth version is supposed to be the best quantization. UD-Q3_K_XL is supposed to take 64GB RAM, lol, while Q4_K_M would need 80+ GB.
# needs done via 'ollama create', fails with pull. https://github.com/ollama/ollama/issues/5245
- { "purpose": ["chat", "lang", "code"], "model": "hf.co/unsloth/Mistral-Small-4-119B-2603-GGUF:UD-Q3_K_XL", "provider": "mistral_small40_1190_unsloth_q3", "size_mb": "{{ llm_size_coef * 54800 }}", "context": 256000 }
# https://huggingface.co/mistralai/Mistral-Small-4-119B-2603 : 242GB unquantized, lol.
- { "purpose": ["chat", "lang", "code"], "model": "hf.co/mistralai/Mistral-Small-4-119B-2603", "provider": "mistral_small40_1190", "size_mb": "{{ llm_size_coef * 242000 }}", "context": 256000 }
# https://huggingface.co/unsloth/Mistral-Small-4-119B-2603 : 121GB, A6B MoE, hmmm!
- { "purpose": ["chat", "lang", "code"], "model": "hf.co/unsloth/Mistral-Small-4-119B-2603", "provider": "mistral_small40_1190_unsloth_a6b", "size_mb": "{{ llm_size_coef * 121000 }}", "context": 256000 }
# https://ollama.com/library/devstral-small-2 : Last Updated 2025-12-13
# Most recommended in Mistral family for agentic coding.
- { "purpose": ["code"], "model": "devstral-small-2:24b", "provider": "mistral_dev20_0240_small", "size_mb": "{{ llm_size_coef * 15000 }}", "context": 393216 }
@@ -218,10 +228,11 @@
- "{{ qwen_code }}"
state: present
- name: Workstation | Linux | Software | LLMs | Docker
- name: Workstation | Linux | Software | LLMs | Docker | Install
ansible.builtin.package:
name:
- docker
- docker-compose
state: present
- name: Workstation | Linux | Software | LLMs | Docker | runit
@@ -278,15 +289,16 @@
export HOME={{ user_user.home }}
### Timeouts ###
export OLLAMA_LOAD_TIMEOUT=20m # default is 5m
export OLLAMA_LOAD_TIMEOUT=30m # default is 5m
export OLLAMA_REQUEST_TIMEOUT=3600 # default 300 (unconfirmed, not in the list of --help vars)
### Resource Usage ###
export OLLAMA_KEEP_ALIVE=20m # default is 5m
export OLLAMA_KEEP_ALIVE=-1 # default is 5m. Good frontends keep the model alive as long as it needs to be, this is not a source of timeout.
export OLLAMA_MAX_LOADED_MODELS={{ llm_concurrency }} # default is 3 if using CPUs, or 3 * NumGPUs
export OLLAMA_NUM_PARALLEL={{ llm_parallel }} # default is 1 and applies per-model (unconfirmed)
export OLLAMA_CONTEXT_LENGTH={{ llm_context }} # default is 4096, https://docs.ollama.com/faq#how-can-i-specify-the-context-window-size
export OLLAMA_MAX_QUEUE=999 # default is 512 (unconfirmed)
export OLLAMA_NUM_THREADS={{ (ansible_processor_vcpus / 2) | int }} # not real, but maybe some day.
### Prevent Data Loss ###
# OLLAMA_NOPRUNE
@@ -299,7 +311,7 @@
{{ ("export HF_TOKEN=" ~ hf_token) if hf_token else '' }}
### Logging ###
export OLLAMA_DEBUG=1
#export OLLAMA_DEBUG=1
export OLLAMA_METRICS=1 # (check http://127.0.0.1:11434/metrics)
### Other ###
@@ -414,36 +426,37 @@
content: |
#!/bin/sh
exec 2>&1
image=ghcr.io/open-webui/open-webui:latest
# Wait for Docker to start up.
while [ ! -S /var/run/docker.sock ]; do sleep 1; done
# Ensure the old container is gone.
echo "RUN: Stopping..."
docker stop open-webui &&
echo "RUN: Container stopped" ||
echo "RUN: Container already stopped."
echo "RUN: Deleting..."
docker rm -f open-webui &&
echo "RUN: Container deleted" ||
echo "RUN: Container already deleted."
# Start the open-webui program.
# Check for any new updates in background.
sleep 300 &&
echo "RUN: Updating..." &&
docker pull $image &
# Start the open-webui program.
set -e
exec docker run \
--pull=always \
--name open-webui \
--network=host -e PORT={{ open_webui_port }} \
--add-host=host.docker.internal:host-gateway \
-e OLLAMA_BASE_URL=http://0.0.0.0:11434 \
{{ ("-e HF_TOKEN=" ~ hf_token) if hf_token else '' }} \
-e WEBUI_SECRET_KEY={{ open_webui_key }} \
-e ENABLE_QUERIES_CACHE=True \
-v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main
$image
# Unsure why this was not working on Artix with runit.
# exec curl 0.0.0.0:8080/health gave a successful response.
@@ -457,6 +470,9 @@
# Had to change this to 0.0.0.0, below was still acting weird.
#-e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
# 20261020, trying to remove this again while leaving the rest.
#--add-host=host.docker.internal:host-gateway \
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-webui | Script | Finish
@@ -508,7 +524,226 @@
ansible.builtin.file:
name: "/var/log/open-terminal"
state: directory
owner: root
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | Dir | /usr/local/etc
ansible.builtin.file:
name: "/usr/local/etc/open-terminal"
state: directory
owner: root
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | AI-Sandbox | Dir
ansible.builtin.file:
dest: "{{ user_user.home }}/AI-Sandbox"
state: directory
owner: "{{ user }}"
group: "{{ user }}"
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | AI-Sandbox | Env
ansible.builtin.blockinfile:
path: "{{ user_user.home }}/AI-Sandbox/{{ item }}"
marker: "# {mark} | MANAGE BY ANSIBLE | Env Vars"
content: |
if [[ -d "$HOME/bin" ]]; then
export PATH="$HOME/bin:$PATH"
fi
export ANDROID_HOME="{{ user_user.home }}/SDKs/Android/Sdk"
export GRADLE_USER_HOME={{ user_user.home }}/.gradle
export PATH="$ANDROID_HOME/platform-tools:$PATH"
export PATH="$ANDROID_HOME/cmdline-tools/latest/bin:$PATH"
export FLUTTER_HOME="{{ user_user.home }}/SDKs/Flutter/flutter"
export PUB_CACHE={{ user_user.home }}/.pub-cache
export PATH="$FLUTTER_HOME/bin:$PATH"
export JAVA_HOME="{{ user_user.home }}/SDKs/Java"
export PATH="$JAVA_HOME/bin:$PATH"
create: false
backup: false
owner: "{{ user }}"
group: "{{ user }}"
mode: "0755"
loop:
- ".bashrc"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | AI-Sandbox | Defaults
ansible.builtin.set_fact:
jdk_jvm: "{{ jdk_jvm | default('/usr/lib/jvm') }}"
jdk_etc: "{{ jdk_etc | default('/etc/java-openjdk') }}"
jdk_etc17: "{{ jdk_etc17 | default('/etc/java17-openjdk') }}"
jdk_etc21: "{{ jdk_etc21 | default('/etc/java21-openjdk') }}"
jdk_etc25: "{{ jdk_etc25 | default('/etc/java25-openjdk') }}"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | AI-Sandbox | AGENTS.md
ansible.builtin.copy:
dest: "{{ user_user.home }}/AI-Sandbox/AGENTS.md"
content: |
# Environment
You are running in a Docker container called Open Terminal. We are
communicating via Open WebUI, also being run in a Docker
container. You are the user `user` and have been granted access to
the designer's Code and SDKs via mount at `{{ user_user.home }}`.
If the `$GOT_EM` environment variable is `true` then all the
necessary packages for compiling Android and Flutter were
installed. If it is `false` then they were not installed, and
compiling will fail. Please check this before trying and alert me
if they need added. If we don;t have Internet, then ask me to
compile or run whatever command is needed on the host system.
# Rules
Never use any `git` subcommands that change anything, including
but not limited to `add`, `commit`, `pull`, `push`, `branch`,
`switch`, etc. The user is who controls the respository changes.
You may informational git subcommands such as use `git status`.
If any programs need installed in order to work on a project, we
need to document that in an install.sh script, as well as add it
to ~/PACKAGES.txt so that the user can ensure they stay installed.
install.sh scripts should be for both `apt` systems (Debian,
Ubuntu) and pacman system (Arch, Artix). using distribution
package managers is preferable over `pip` and `npm` when
possible, unless the project is actually Python or Node.js.
The user manages the `{{ user_user.home }}/SDKs` directory. Do not
modify it. Instead, ask if anything needs updated or added.
Never manually edit, modify, delete, alter, replace, etc any
files under system folders such as `/usr`. Anything under `/` is
managed by the system. If you need assistance changing anything
such as `/usr/lib/jvm`, please ask the user and halt all further
operations until you have received feedback.
It is never okay to brute force a solution if something is not
working well. it is better to ask the user for assistance or alert
them that you've encountered blockers and then cease the process
until you've received feedback.
# Preferences
You have free reign over your home directory. Feel free to
create memory files or anything else which would benefit you.
Everything we create should have both Desktop Linux and Android
in mind. Windows, MacOS, and iOS will never be compiled for, but
if we can do things in a way which makes it easy for others to
fork the project and add the support themselves, then that's best.
# Versions
These are the versions of plugins and languages we need to use.
If you know of an important upgrade from them, please let me know.
## Android / Flutter
- Gradle: 9.8.0
- AGP: 9.4.1
- Kotlin: 2.4.20
- Java: JDK 17
- Flutter (plugin loader): 1.0.0
backup: false
owner: "{{ user }}"
group: "{{ user }}"
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | Script | Compose YAML
ansible.builtin.copy:
dest: /usr/local/etc/open-terminal/docker-compose.yml
content: |
#exec docker run \
# --name open-terminal \
# --network=host \
# -v {{ user_user.home }}/AI-Sandbox:/home/user \
# -v {{ user_user.home }}/Code:{{ user_user.home }}/Code \
# -v {{ user_user.home }}/SDKs:{{ user_user.home }}/SDKs \
# -v {{ jdk_jvm }}:{{ jdk_jvm }}:ro \
# -v {{ jdk_etc }}:{{ jdk_etc }}:ro \
# -v {{ jdk_etc17 }}:{{ jdk_etc17 }}:ro \
# -v {{ jdk_etc21 }}:{{ jdk_etc21 }}:ro \
# -v {{ jdk_etc25 }}:{{ jdk_etc25 }}:ro \
# -v /etc/hosts:/etc/hosts:ro \
# -v {{ user_user.home }}/.gradle:{{ user_user.home }}/.gradle \
# -v {{ user_user.home }}/.pub-cache:{{ user_user.home }}/.pub-cache \
# -e OPEN_TERMINAL_FILE_BROWSER_ROOT="/home/user" \
# -e OPEN_TERMINAL_API_KEY="{{ open_terminal_key }}" \
# -e OPEN_TERMINAL_PACKAGES="$packages" \
# -e GOT_EM="$GOT_EM" \
# $image run --port {{ open_terminal_port }}
services:
open-terminal:
build:
context: ./
network: host
container_name: open-terminal
network_mode: host
command: run --port {{ open_terminal_port }}
environment:
- OPEN_TERMINAL_FILE_BROWSER_ROOT=/home/user
- OPEN_TERMINAL_API_KEY={{ open_terminal_key }}
- OPEN_TERMINAL_PACKAGES=
- GOT_EM=true
volumes:
- {{ user_user.home }}/AI-Sandbox:/home/user
- {{ user_user.home }}/Code:{{ user_user.home }}/Code
- {{ user_user.home }}/SDKs:{{ user_user.home }}/SDKs
- {{ jdk_jvm }}:{{ jdk_jvm }}:ro
- {{ jdk_etc }}:{{ jdk_etc }}:ro
- /etc/ssl/certs/java:/etc/ssl/certs/java:ro
- /etc/ca-certificates:/etc/ca-certificates:ro
- {{ jdk_etc17 }}:{{ jdk_etc17 }}:ro
- {{ jdk_etc21 }}:{{ jdk_etc21 }}:ro
- {{ jdk_etc25 }}:{{ jdk_etc25 }}:ro
- /etc/hosts:/etc/hosts:ro
- {{ user_user.home }}/.gradle:{{ user_user.home }}/.gradle
- {{ user_user.home }}/.pub-cache:{{ user_user.home }}/.pub-cache
owner: root
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | Script | Dockerfile
ansible.builtin.copy:
dest: /usr/local/etc/open-terminal/Dockerfile
content: |
FROM ghcr.io/open-webui/open-terminal:latest
USER root
RUN echo "Android"
RUN apt-get update && apt-get install -y \
libc6:amd64 libncurses-dev \
libstdc++6 lib32z1 libbz2-1.0 android-udev-rules
# Debian removed JDK 17, continue serving from the host.
# default-jdk openjdk-17-jdk
RUN echo "Flutter (Main)"
RUN apt-get update && apt-get install -y \
curl git unzip xz-utils zip libglu1-mesa libglu1-mesa
RUN echo "Flutter (Linux)"
RUN apt-get update && apt-get install -y \
clang cmake git ninja-build pkg-config libgtk-3-dev \
liblzma-dev libstdc++-12-dev
RUN echo "Flutter (Android)"
RUN apt-get update && apt-get install -y \
libc6:amd64 libstdc++6 lib32z1 libbz2-1.0
USER user
owner: root
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | Script | env
ansible.builtin.copy:
dest: /usr/local/etc/open-terminal/.env
content: |
COMPOSE_BAKE=true
owner: root
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | Script | Run
@@ -519,32 +754,27 @@
exec 2>&1
# Wait for Docker to start up.
while [ ! -S /var/run/docker.sock ]; do sleep 1; done
# Ensure the old container is gone.
# Ensure the old container is stopped.
echo "RUN: Stopping..."
docker stop open-terminal &&
docker compose -f /usr/local/etc/open-terminal/docker-compose.yml down &&
echo "RUN: Container stopped" ||
echo "RUN: Container already stopped."
echo "RUN: Deleting..."
docker rm -f open-terminal &&
echo "RUN: Container deleted" ||
echo "RUN: Container already deleted."
# Check for any new updates in background.
sleep 600 &&
echo "RUN: Updating..." &&
docker compose -f /usr/local/etc/open-terminal/docker-compose.yml \
build --pull &
export GOT_EM=true
# Start the open-terminal program.
# https://github.com/open-webui/open-terminal#configuration
set -e
exec docker run \
--pull=always \
--name open-terminal \
--network=host -p {{ open_terminal_port }}:8000 \
-v open-terminal:/home/user \
-v {{ user_user.home }}/Code:/home/user/Code \
-e OPEN_TERMINAL_API_KEY={{ open_terminal_key }} \
ghcr.io/open-webui/open-terminal
exec docker compose -f /usr/local/etc/open-terminal/docker-compose.yml up
owner: root
mode: "0755"
- name: Workstation | Linux | Software | LLMs | runit | open-terminal | Script | Finish
@@ -554,15 +784,10 @@
{{ runit_finish_sleep }}
echo "FINISH: Stopping..."
docker stop open-terminal &&
docker compose -f /usr/local/etc/open-terminal/docker-compose.yml down &&
echo "FINISH: Container stopped" ||
echo "FINISH: Container already stopped."
echo "FINISH: Deleting..."
docker rm -f open-terminal &&
echo "FINISH: Container deleted" ||
echo "FINISH: Container already deleted."
echo "FINISH: Done!"
mode: "0755"
@@ -744,17 +969,17 @@
path: "{{ user_user.home }}/{{ item }}"
marker: "# {mark} ANSIBLE MANAGED BLOCK | ollama | Manual Script"
content: |
function list_llms() {
function ollama_llms() {
{% for llm in llm_installs %}
echo "{{ llm.model }}"
{% endfor %}
}
function pull_llms() {
function ollama_pull() {
echo "Manually Pulling Models"
for model in `list_llms`; do
for model in `ollama_llms`; do
echo -e "\n\n Starting 'ollama pull $model' \n\n"
ollama pull $model
status=$?
@@ -763,21 +988,9 @@
echo "Done!!"
}
alias pull-llms="pull_llms "
alias pull-models="pull_llms "
alias pull-ai="pull_llms "
function ollama_stop() {
for model in `ollama ps | tail -n +2 | awk '{print $1}'`; do
echo "Stopping ${model}..."
ollama stop $model
done
}
alias ollama-stop="ollama_stop"
alias ollama-kill="ollama_stop"
alias ollama-killall="ollama_stop"
alias ollama-kill-all="ollama_stop"
alias ollama-stop-all="ollama_stop"
alias pull-llms="ollama_pull "
alias pull-models="ollama_pull "
alias pull-ai="ollama_pull "
function ollama_chat() {
ollama run {{ 'qwen3.8:27b' if ansible_memtotal_mb > 32000 else 'qwen3.5:4b' }}
@@ -789,23 +1002,8 @@
ollama run {{ 'qwen3.8:27b' if ansible_memtotal_mb > 32000 else 'qwen2.5-coder:7b' }}
}
alias ollama-code="ollama_code "
alias code="ollama_code "
function ollama_models() {
for model in `ollama ls | grep -v NAME | awk '{print $1;}'`; do
echo $model
ollama show $model | grep -E "architecture|context length|quantization"
done
}
alias ollama-models="ollama_models "
alias ollama-show-models="ollama_models "
alias ollama-show="ollama_models "
alias ollama-dense="ollama_models "
alias ollama-moe="ollama_models "
alias show-models="ollama_models "
alias list-models="ollama_models "
function ollama_purge {
function ollama_purge() {
echo "TBD: Purge all models not in llm_installs."
}
alias ollama-purge="ollama_purge "
@@ -827,6 +1025,7 @@
alias restart-llm="restart_ai "
alias restart-llms="restart_ai "
alias restart-models="restart_ai "
alias reload-ai="restart_ai "
state: present
create: false
backup: true
+5 -2
View File
@@ -18,10 +18,13 @@
#######################
#######################
- name: TEST | Main
- name: TEST | LLMs
block:
- name: LLMs
- name: Java
ansible.builtin.include_tasks: tasks/workstation/linux/software/java.yml
- name: LLM
ansible.builtin.include_tasks: tasks/workstation/linux/software/llm.yml
#######################