Compare commits

..

93 Commits

Author SHA1 Message Date
7b9bbcbc26 Ignore disabled files as well. 2025-10-15 04:43:42 -07:00
cbaf052ea2 Do not track Prod and Dev copies of configurations. 2025-10-15 04:40:27 -07:00
c4c3cb6434 Put the repo location into a variable. 2025-10-14 15:28:43 -07:00
e0f846164e Merge pull request 'Nextcloud, Hyperling.com, LibreTranslate, and backup.sh Enhancements' (#11) from dev into main
Reviewed-on: #11
2025-10-14 14:58:58 -07:00
9314bb8bb4 Have the git command give us the branch instead. 2025-10-14 14:32:26 -07:00
b9d9771523 Add echo of $BRANCH contents. 2025-10-14 14:30:40 -07:00
d2aa3c195b Fix typo in variable name. 2025-10-14 14:23:40 -07:00
dd60d5a721 Fix typos and wording in comments. 2025-10-14 14:23:18 -07:00
c04eb08f78 Comment the micro server requirements. 2025-10-14 14:22:27 -07:00
418983c490 Add env variable for which branch to use. Move CPU and RAM requirements to env so that only 1 docker-compose.yml example is needed. 2025-10-14 14:21:59 -07:00
95060838e2 Remove using the dev branch on Production! Should only be on the Stage version of the Dockerfile. 2025-10-14 14:14:04 -07:00
6973640b4e Explicitly mention where the output is coming from. 2025-10-14 14:12:02 -07:00
a49d0129e7 Enhance output when starting Hyperling.com. 2025-10-14 14:10:00 -07:00
394acfa06d Critical changes for upcoming Hyperling.com 2.0! Automatic page updates after a pull request within 30 seconds, no reload necessary! 2025-10-14 14:08:13 -07:00
04089744e6 Change label for starting cronjob. 2025-10-14 12:22:41 -07:00
75711c6c20 Ensure backup's TRASH folder gets created and used properly. 2025-10-14 08:35:23 -07:00
c4032b5b94 Remove directory from basename variable. 2025-10-14 08:14:49 -07:00
d7820ed373 Add safeties to backup script. 2025-10-14 08:12:57 -07:00
d9f8318e66 Add recommendation to use /opt/ for LibreTranslate models. 2025-10-14 07:12:59 -07:00
a5a80276ac Folder name should not be plural. 2025-10-14 07:07:07 -07:00
c070d7cb92 Use a dedicated backup directory rather than /tmp. 2025-10-14 07:02:13 -07:00
c138fe67d6 No need for the if, just remove any files that match the pattern. 2025-10-14 06:55:42 -07:00
2697645abe Properly check if other backups exist. 2025-10-14 06:43:54 -07:00
ce111d01f3 Chnage order of distros to alphabetical. 2025-10-13 05:32:14 -07:00
7abe909cdf Move packages to env file and allow extras per distro. 2025-10-13 05:31:57 -07:00
f57a0d322d Add which to install commands. 2025-10-10 16:08:58 -07:00
9381d61d99 Progress on testing provisioning on multiple distros. So far Debian and Ubuntu are working. Others need packages added. 2025-10-10 16:07:17 -07:00
bde78a1489 Create files for testing Ansible script on other distros. 2025-10-10 14:28:01 -07:00
4acbd47388 Change separating symbols to match System backup formatting. 2025-10-09 07:50:08 -07:00
51d6d32790 Change order of backup wording. Add timestamp. 2025-10-09 07:47:03 -07:00
1e665a40b7 Add date and hostname to backup file. Delete previous dates. 2025-10-09 07:37:14 -07:00
675ac90c31 Stop trying the mysql_upgrade. Add changing row formats to DYNAMIC. 2025-10-04 12:51:30 -07:00
f0679bd05f Merge branch 'dev' of ssh://git.hyperling.com:13731/me/env-docker into dev
This is to bring prod into alignment with repo while working on local changes due to Trixie upgrades.
2025-10-04 14:07:28 +00:00
897493dd1f Fix version of ImageMagickCore for Trixie. 2025-10-04 14:02:17 +00:00
5a5035fd33 Add Docker to Docker backup file. 2025-10-04 06:34:11 -07:00
e178141357 Add extra backup locations. Did this a few days ago but it seems it did not get pushed and the commit disappeared. 2025-08-22 13:44:48 -07:00
6062818475 Add the version of Express as being done in package.json. 2025-08-21 13:24:53 -07:00
f77557151a Add size to container check. 2025-08-21 11:22:24 -07:00
fb7e31ed69 Merge pull request 'Allow Alternative Hugo Website Branches' (#10) from dev into main
Reviewed-on: #10
2025-08-19 15:11:39 -07:00
0478e8e108 Allow specifying the branch of the Hugo project. 2025-08-19 15:05:15 -07:00
2663bb5351 Allow specifying the branch of the Hugo project. 2025-08-19 15:03:12 -07:00
4aa7fa4b00 Merge pull request 'Improve Hugo Configuration' (#9) from dev into main
Reviewed-on: #9
2025-08-17 17:06:11 -07:00
d7a1e6f58a Remove hugo user reference from files. 2025-08-17 11:49:05 -07:00
f622cba366 Add the mail files to the container's log output. 2025-08-17 11:44:16 -07:00
e86660d34e Using the crontab command adds the job to root's jobs. Allow cron to pick up the file automatically by not running any explicit calls. 2025-08-17 11:43:54 -07:00
cfb07a6e93 Use the www-data user rather than creating a hugo user. 2025-08-17 11:26:47 -07:00
4dd0661e1e Move permission commands to a single command after the cronjob file is copied. 2025-08-17 11:02:13 -07:00
d6e1186d86 Modify script to remove NGINX default files and check HTTP codes rather than curl response status. 2025-08-17 11:01:10 -07:00
57ade9f67d Give the cronjob some safety so that it churns less. 2025-08-17 11:00:34 -07:00
ddeadcf723 Add safeguards to cron job so that website does not get deleted without having replacement files. 2025-08-17 09:12:23 -07:00
dbb54b6f81 Do not continue if the git repo does not exist. 2025-08-17 09:01:24 -07:00
5afc3bfaee Merge pull request 'Productionize Hugo Configuration' (#8) from dev into main
Reviewed-on: #8
2025-08-15 12:13:15 -07:00
5323b6647f Change restart loop timers. 2025-08-15 11:49:00 -07:00
6dac535211 Remove running service commands since they don't actually keep the programs enabled. 2025-08-15 11:46:23 -07:00
d035f9d8e7 Refactor how the project is run, using nginx instead of hugo server, a cron job to pull git changes, and a start script to ensure the correct services are started each run and are monitored to stay up. 2025-08-15 11:45:23 -07:00
dfb9a306c5 Allow copies of Hugo configuration to exist outside of the Git project. 2025-08-15 11:43:46 -07:00
1ba2739665 Merge pull request 'Add Example Hugo Configuration' (#7) from dev into main
Reviewed-on: #7
2025-08-14 12:36:40 -07:00
0ca793423b Add working configuration for how to run a slim Hugo container. 2025-08-14 12:30:25 -07:00
1284fc3946 Merge pull request 'LibreTranslate Reboot Fix' (#6) from dev into main
Reviewed-on: #6
2025-07-26 13:57:57 -07:00
a3d6cdcee1 Number list rather than relying on autoincrement. 2025-07-26 13:55:04 -07:00
a868f874d7 Add file paths. 2025-07-26 13:54:37 -07:00
7046699ed6 Reapply changes from previous commits. Unsure why the synced branch did not have them if it was up to date. 2025-07-26 13:42:12 -07:00
eb2dda9d66 Allow container to reboot properly if using /tmp/ for models. Fixes newly created folder having permissions issues. 2025-07-26 13:35:32 -07:00
15f01061ba Move request characters to the env file. 2025-07-26 12:36:29 -07:00
4aebf1e21d Fix typo in header and set ordered list to number itself. 2025-07-26 12:17:18 -07:00
5af5c13f43 Merge pull request 'Add LibreTranslate' (#5) from dev into main
Reviewed-on: #5
2025-07-26 12:02:50 -07:00
27689e25d8 Be more specific about the docker home location, rather than accidentally promoting the use of a variable which would not work. 2025-07-26 12:01:10 -07:00
29d1affcd8 Be more specific about the docker home location, rather than accidentally promoting the use of a variable which would not work. 2025-07-26 12:00:43 -07:00
43ee8de7ef Add comments and an explicit exit. 2025-07-26 11:59:45 -07:00
83fdb54768 Move the LT_LOAD_ONLY to the value so that it can be omitted if wanting to download all languages. 2025-07-26 11:28:04 -07:00
16df857f08 Shrink down to just 1 yml file, putting the CPU and RAM into the env file, since the only docker-compose differences were the resource configuration. 2025-07-23 06:05:29 -07:00
620f3f81b2 Shrink down to just 1 yml file, putting the CPU and RAM into the env file, since the only docker-compose differences were the resource configuration. 2025-07-23 06:05:21 -07:00
593b1fbaf2 Add an initial working configuration for LibreTranslate. 2025-07-23 05:55:40 -07:00
e661b67b57 Merge pull request 'Add Nextcloud Dockerfile' (#4) from dev into main
Reviewed-on: #4
2025-06-16 16:41:44 -07:00
8aaa8e97b9 Update the BAKE performance data/ 2025-06-16 18:19:00 -06:00
8c6b258f48 Add BAKE to Nextcloud subproject. 2025-06-16 17:32:33 -06:00
8b7f94d51f Add note as to what type of system the testing was done on. 2025-06-16 17:32:15 -06:00
ac60f54142 Move the most important fixes to a Docker build file. fixes.sh is now just a list of maintenance jobs, name it so. Hooray!! 2025-06-16 17:17:47 -06:00
7265823f58 Add curl to the container so that the run script doesn't need to install it each time. 2025-06-16 15:43:55 -06:00
811442cd94 Add an .env file for the website. 2025-06-16 15:32:47 -06:00
a90bf8bf28 Provide a better explanation of what's happening in the log. 2025-06-16 15:20:00 -06:00
82547068e3 Add sudo to the website's container. 2025-06-16 15:09:48 -06:00
167d813599 Merge pull request 'Nextcloud Improvements' (#3) from dev into main
Reviewed-on: #3
2025-06-16 12:30:19 -07:00
494d0c04bd Allow long-form parameters. 2025-06-16 14:28:04 -06:00
4469754097 Remove TODOs, going to keep cron at the Docker server level, container packages too volatile. 2025-06-16 14:24:38 -06:00
4ca8a8ae02 Update comments, rearrange file, fix case statement and handle empty parameter correctly. 2025-06-16 14:23:57 -06:00
a3ce3a7ee1 Add usage and verbose output option for what the cron file is getting accomplished. 2025-06-16 14:01:25 -06:00
0e761c5112 Add memory limit setting to php command. 2025-06-16 13:18:40 -06:00
cf1e23e91c Call on www-data using sudo within the container, gives better output. 2025-06-16 12:53:43 -06:00
93be789790 Fix the "the input device is not a TTY" cron log message. 2025-06-16 12:19:01 -06:00
6611665770 Add TODO for getting cron.php working properly. 2025-03-28 17:15:41 -07:00
f8d4c4654e Add extra flag to enable doing things like migrating MIME types. May increase the amount of time by quite a bit on large systems, but mine is small. :) 2025-03-28 17:03:23 -07:00
2e7efbc6eb Allow the stage keyword to be at the beginning too. 2025-01-04 10:05:23 -07:00
36 changed files with 878 additions and 69 deletions

15
.gitignore vendored
View File

@@ -25,6 +25,8 @@ Config/DNS/config/*
# Ignore anything in disabled folders.
disabled
*.disabled
disabled.*
# Ignore any .env files which are not explicitly committed to the project.
*.env
@@ -35,5 +37,16 @@ docker-compose.yml
# 2024-01-24 Hide static files for Hyperling.com.
Config/Hyperling.com/files/*
# Ignore things like Config/Hyperling.com-Stage/
# Ignore things like "Config/Hyperling.com-Stage/"
*-Stage
Stage-*
# Ignore copies of the Hugo configuration, such as "Config/Hugo-MyWebsite".
Hugo-*
*-Hugo
# Ignore Prod and Dev copies as well.
*-Prod
Prod-*
*-Dev
Dev-*

View File

@@ -0,0 +1,35 @@
# 2025-08-13 Hyperling
## Image ##
FROM debian:bookworm-slim
## Setup ##
# System Dependencies
RUN apt-get update && apt-get install -y git hugo nginx cron curl bash sudo htop
# User and Group
#RUN groupadd -r hugo && useradd -r -g hugo hugo
# Hugo Directory Tree
RUN mkdir -pv /var/www/hugo/
# NGINX Directory Tree
RUN mkdir -pv /var/www/html/
# Copy Cron Job to Update Git Repo
COPY files/crontab /etc/crontab
COPY files/cronjob.sh /var/www/hugo/cronjob.sh
# Hugo User Permissions
RUN chown -Rv www-data:www-data /var/www/
RUN chmod +x /var/www/hugo/cronjob.sh
# Copy Start Script
COPY files/main.sh /root/main.sh
RUN chmod +x /root/main.sh
## Main ##
# Install + Run Website
WORKDIR /var/www/
USER root
CMD /root/main.sh "$REPO" "$BRANCH" "$PROD" "$DEV"

View File

@@ -0,0 +1,30 @@
# 2025-08-13
# Configuration for running a Hugo website pulled from Git.
services:
app:
container_name: hugo
build:
context: ./
network: host
restart: always
ports:
- 8013:80 # Production minified files served using NGINX.
- 1380:1380 # Development files with drafts served by Hugo Server.
environment:
- REPO=$REPO
- BRANCH=$BRANCH
- PROD=$PROD
- DEV=$DEV
healthcheck:
test: curl -sS http://localhost:80 || curl -sS http://localhost:1380 || exit 1
interval: 1m
timeout: 10s
retries: 2
start_period: 30s
deploy:
mode: global
resources:
limits:
cpus: '0.10'
memory: 64M

View File

@@ -0,0 +1,22 @@
# 2025-08-13 Hyperling
#
## Docker ##
#
COMPOSE_BAKE=true
#
## Git Website Repository
#
REPO=https://git.hyperling.com/me/hugo-jackanope
BRANCH=main
#
## Web Environments
# Please use values YES/TRUE and NO/FALSE.
# Whether to start NGINX
PROD=YES
# Whether to start Hugo Server
DEV=NO

View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
cd /var/www/hugo/site
echo "*** Running cronjob @ `date` ***"
# Pull any updates, and if the project is already up to date, exit successfully.
git pull | grep -v "up to date"
status="$?"
echo "* Pull status is '$status'."
if [[ $status != 0 && -e /var/www/html/index.html ]]; then
echo "* Site is already up to date and copied, exiting."
exit 0
fi
# If the project was not already up to date, build the new files for NGINX.
hugo --gc --minify
status="$?"
echo "* Hugo status is '$status'."
if [[ $status == 0 && -e public/index.html ]]; then
echo "* Copying files..."
rm -rfv /var/www/html/*
mv -v public/* /var/www/html/
fi
exit 0

View File

@@ -0,0 +1 @@
* * * * * www-data /var/www/hugo/cronjob.sh

View File

@@ -0,0 +1,92 @@
#!/usr/bin/env bash
REPO="$1"
echo "REPO='$REPO'"
BRANCH="$2"
if [[ -n $BRANCH ]]; then
BRANCH="--branch $BRANCH"
fi
echo "BRANCH='$BRANCH'"
PROD="$3"
typeset -u PROD
echo "PROD='$PROD'"
DEV="$4"
typeset -u DEV
echo "DEV='$DEV'"
SITE="/var/www/hugo/site"
echo "*** Creating Git Repo ***"
if [[ -d $SITE ]]; then
rm -rfv $SITE
fi
sudo -u www-data git clone --recurse-submodules $BRANCH $REPO $SITE
status="$?"
echo "*** Validating Git Repo ***"
if [[ $status != 0 || ! -d $SITE/.git ]]; then
echo "ERROR: Hugo project may not have cloned correctly. status='$status'"
echo "Aborting."
exit 1
fi
echo "* Site exists!"
echo "*** Copying Static Files to NGINX ***"
rm -rfv /var/www/html/*
sudo -u www-data /var/www/hugo/cronjob.sh
echo "*** Starting Cron ***"
service cron start
service cron status
if [[ "$PROD" == "Y"* || "$PROD" == "T"* ]]; then
echo "*** Starting Production Server Loop ***"
while true; do
http_code="`curl -sS http://localhost:80 -o /dev/null -w "%{http_code}"`"
if [[ $http_code != 200 ]]; then
echo "* Prod server not detected, starting..."
service nginx status
service nginx start
service nginx status
fi
sleep 15
done &
cd /var/log/nginx
tail -f access.log error.log &
fi
if [[ "$DEV" == "Y"* || "$DEV" == "T"* ]]; then
echo "*** Starting Development Server Loop ***"
while true; do
http_code="`curl -sS http://localhost:1380 -o /dev/null -w "%{http_code}"`"
if [[ $http_code != 200 ]]; then
echo "* Dev server not detected, starting..."
cd $SITE
killall hugo 2>/dev/null
sudo -u www-data hugo server -D --noBuildLock --bind 0.0.0.0 -p 1380 &
fi
sleep 30
done &
fi
echo "*** Following Mail Files ***"
cd /var/mail
touch mail www-data
chown -v mail:mail mail
chown -v www-data:mail www-data
chmod -v 660 mail www-data
tail -f mail www-data &
cd
echo "*** Finished $0 @ `date` ***"
wait -n
exit $?

View File

@@ -6,20 +6,28 @@
FROM node:lts-slim
# Cache System Dependencies
RUN apt-get update && apt-get install -y git php-cli
RUN apt-get update && apt-get install -y git php-cli sudo curl procps
# Cache Node Dependencies
RUN mkdir -p /var/www/api
WORKDIR /var/www/api
RUN echo '{ "dependencies": { "express": "" } }' > package.json
RUN echo '{ "dependencies": { "express": ">=4.18.1 < 5.0.0" } }' > package.json
RUN npm install
# Install + Run Website
CMD cd /var/www/api && \
rm -rfv pages main.js run.sh && \
git clone https://git.hyperling.com/me/nodejs-website website && \
rm -rfv website/files && \
mv -v website/* ./ && \
echo "Dockerfile: Ensure symlink is properly deleted if it exists..." && \
ls -l website/files; rm -v website/files; sleep 0 && \
echo "Dockerfile: Clone the website's Git repo..." && \
rm -rfv website && \
echo "Starting Website" && \
git clone https://git.hyperling.com/me/nodejs-website $BRANCH website && \
echo "Dockerfile: Remove dummy files and replace with symlink..." && \
rm -rfv website/files && \
cd website && \
echo -n "Dockerfile: Using branch " && \
git branch && \
ln -sv ../files ./files && \
echo "Dockerfile: Allow files under files/ to be ignored by Git..." && \
find ./files -exec git update-index --assume-unchanged {} \; && \
echo "Dockerfile: Start website!" && \
./run.sh

View File

@@ -10,11 +10,15 @@ services:
restart: always
ports:
- 8317:8080
environment:
- BRANCH=$BRANCH
- CPU=$CPU
- MEM=$MEM
volumes:
- ./files:/var/www/api/files
deploy:
mode: global
resources:
limits:
cpus: '0.25'
memory: 256M
cpus: $CPU
memory: $MEM

View File

@@ -1,21 +0,0 @@
# 2022-10-29
# Hyperling.com configuration.
# 2023-01-24 container needs 128M to get started fully then settles to 25M.
services:
app:
container_name: www-app
build:
context: ./
network: host
restart: always
ports:
- 8317:8080
volumes:
- ./files:/var/www/api/files
deploy:
mode: global
resources:
limits:
cpus: '0.10'
memory: 128M

View File

@@ -0,0 +1,24 @@
# This file should be renamed '.env' and have any private values modified.
## 2025-06-16
## Performance Notes for Enabling BAKE
# Compose by itself takes about 35s to build this project.
# Enabling this setting first cause the build to take 80s.
# Subsequent builds consistently take less than 2 seconds.
# Testing was done on the micro server, not a workstation.
COMPOSE_BAKE=true
## Branch ##
# Allow choosing the branch. By leaving it blank, the main branch will be used.
# Should be in the full "--branch my-branch" syntax.
BRANCH=
#BRANCH="--branch dev"
## Performance ##
# Standard Server
CPU=0.25
MEM=0.25G
# Micro Server
# 2023-01-24 container needs 128M to get started fully then settles to 25M.
#CPU=0.10
#MEM=128M

View File

@@ -0,0 +1,3 @@
# Install and run LibreTranslate
FROM libretranslate/libretranslate:latest

View File

@@ -0,0 +1,31 @@
# LibreTranslate Configuration
## Install
1. Copy `env.example` to `.env` and adjust the values as necessary.
```
cd $DOCKER_HOME/Config/LibreTranslate
cp env.example .env
vi .env
```
2. Run the `prep.sh` file to create the volume mounts.
```
./prep.sh
```
3. If using a temporary location for `models/` such as `/tmp/`, ensure
that the reboot script is in `root`'s crontab to set the folders back up.
```
@reboot /opt/Docker/Config/LibreTranslate/cron-reboot.sh
```
4. Start the container up as normal.
```
cp docker-compose.main.yml docker-compose.yml
docker compose up -d
```

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# 2025-07-26 Hyperling
# Ensure the LibreTranslate container starts correctly after a reboot, especially if placing models/ under /tmp/.
## Setup ##
DIR="$(dirname -- "${BASH_SOURCE[0]}")"
cd $DIR
source ./.env
## Main ##
# Shutdown the container.
docker compose down
# Create folders.
./prep.sh
# Start the container.
docker compose up -d
# Finish successfully.
exit 0

View File

@@ -0,0 +1,55 @@
# LibreTranslate configuration.
# Example docker-compose.yml:
# https://github.com/LibreTranslate/LibreTranslate/blob/main/docker-compose.yml
# Named volumes ensure that the container user gets the correct permissions.
volumes:
libretranslate_api_keys:
driver: local
driver_opts:
type: none
device: $API_KEYS_DIR
o: bind
libretranslate_models:
driver: local
driver_opts:
type: none
device: $MODELS_DIR
o: bind
services:
## Main ##
lt-app:
container_name: lt-app
build:
context: ./
network: host
restart: always
ports:
- "5000:5000"
# Uncomment this for logging in docker compose logs
tty: true
healthcheck:
test: ['CMD-SHELL', './venv/bin/python scripts/healthcheck.py']
command: --req-limit $REQUESTS --char-limit $CHARACTERS
environment:
# Allow the saving of API Keys. Requires volume libretranslate_api_keys.
- LT_API_KEYS=true
- LT_API_KEYS_DB_PATH=/app/db/api_keys.db
# Optimize loading time. Requires volume libretranslate_models.
- LT_UPDATE_MODELS=$UPDATE_MODELS
- $LANGUAGES
volumes:
# Store the API keys.
- libretranslate_api_keys:/app/db
# Avoid re-downloading language models every reboot.
- libretranslate_models:/home/libretranslate/.local:rw
deploy:
mode: global
resources:
limits:
cpus: $CPU
memory: $RAM

View File

@@ -0,0 +1,53 @@
# This file should be renamed '.env' and have any private values modified.
COMPOSE_BAKE=true
### Container Specs ###
## For a small-scale server (such as 1 CPU, 1GB RAM).
# CPU=0.25
# RAM=640M
## For a fully capable server.
CPU=2.00
RAM=4.0G
# How much the users are able to utilize the server.
REQUESTS=100
CHARACTERS=2000
### Languages ###
## Check for language model updates on each run.
UPDATE_MODELS=true
## Use default behavior of downloading all models.
# LANGUAGES=ALL
## Otherwise set to a comma-separated list to enable only the desired models.
LANGUAGES='LT_LOAD_ONLY=en,es,fr,de'
### API Keys ###
API_KEYS=true
API_KEYS_DB_PATH=/app/db/api_keys.db
API_KEYS_DIR=../../Volumes/LibreTranslate/api_keys
### Models ###
## Storing models prevents them from needing downloaded each run.
## Placing them under Volumes means they will increase backup sizes.
MODELS_DIR=../../Volumes/LibreTranslate/models
## Not providing any location puts them in the default Docker /var/ location.
# MODELS_DIR=
## Storing them in /tmp/ will cause them to get reset after reboots, but hides
## them from backup type locations such as Volumes/LibreTranslate/ and /var/.
## Please see the README for the CRON command related to this setting.
# MODELS_DIR=/tmp/LibreTranslate/models
## Can also place them under /opt/ which may be recommended by FHS standards.
## This keeps them from taking up precious RAM / swap if using tmpfs or similar.
# MODELS_DIR=/opt/LibreTranslate/models

25
Config/LibreTranslate/prep.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# 2025-07-22 Hyperling
# Create the necessary folders for LibreTranslate's volumes to work.
# This must be run before the container will start properly.
## Setup ##
DIR="$(dirname -- "${BASH_SOURCE[0]}")"
source $DIR/.env
## Main ##
# Create folders.
if [[ $API_KEYS_DIR == "../*" ]]; then
API_KEYS_DIR="$DIR/$API_KEYS_DIR"
fi
mkdir -pv $API_KEYS_DIR
if [[ $MODELS_DIR == "../*" ]]; then
MODELS_DIR="$DIR/$MODELS_DIR"
fi
mkdir -pv $MODELS_DIR
# Finish successfully.
exit 0

View File

@@ -0,0 +1,11 @@
# 2025-06-16 Hyperling
# Tired of running fixes.sh to install extra packages. Bake them in!
FROM nextcloud:stable
# Cache System Dependencies
RUN apt-get update && apt-get install -y sudo libmagickcore-7.q16-10-extra htop \
iputils-ping dnsutils vim bzip2 libbz2-dev
# Configure PHP Dependency
RUN docker-php-ext-install bz2

View File

@@ -4,19 +4,64 @@
# This should be added to root's crontab with the full path, such as:
# */5 * * * * /opt/Docker/Config/Nextcloud/cron.ksh
# Check if a job is already going.
DIR="$(dirname -- "${BASH_SOURCE[0]}")"
PROG="$(basename -- "${BASH_SOURCE[0]}")"
# Check if a job is already going.
RUNNING=`ps -ef | grep $PROG | grep -v grep | grep -v $$ | grep -v "sh -c" | wc -l`
if (( $RUNNING > 0 )); then
exit $RUNNING
fi
# 2023-08-25 From crontab.
sh -c "docker exec -u www-data nc-app php cron.php --define apc.enable_cli=1"
# Usage function for when -h or bad parameters are passed.
function usage() {
cat <<- EOF
Script to help with scheduling Nextcloud's cron requirements.
Usage: $PROG [-h|-v]
-h ) Display the usage and help text.
| --help
-v) Pass a verbose request to cron.php.
| --verbose
EOF
exit $1
}
# 2023-08-25 From fixes.sh, keep ownership correct and apps up to date.
sh -c "docker exec -it nc-app chown -Rc www-data:www-data ."
# Check for any parameters.
verbose=""
case "$1" in
"") ;;
"-h"|"--help")
usage 0
;;
"-v"|"--verbose")
verbose="-v"
;;
*)
echo -e "ERROR: Unknown parameter '$1'. Exiting.\n"
usage 1
;;
esac
# Keep ownership correct and apps up to date. Also exists in fixes.sh.
sh -c "docker exec -i nc-app chown -Rc www-data:www-data ."
# No longer update apps in advance of NC updates, allow the upgrade process to do it.
#sh -c "docker exec -itu www-data nc-app ./occ app:update --all"
# Prepare the variables being passed to the execution command.
if [[ -f $DIR/.env ]]; then
source $DIR/.env
else
PHP_MEMORY_LIMIT=256M
fi
# Main part of what would go in the crontab.
sh -c "
docker exec nc-app \
sudo -u www-data \
php \
-d apc.enable_cli=1 \
-d memory_limit=$PHP_MEMORY_LIMIT \
-f cron.php $verbose \
"
exit 0

View File

@@ -72,7 +72,8 @@ services:
## Nextcloud ##
nc-app:
container_name: nc-app
image: nextcloud:stable
build:
context: ./
restart: always
ports:
- 8080:80

View File

@@ -73,7 +73,8 @@ services:
## Nextcloud ##
nc-app:
container_name: nc-app
image: nextcloud:stable
build:
context: ./
restart: always
ports:
- 8080:80

View File

@@ -1,5 +1,6 @@
# Example environment file for Nextcloud stack, should be copied as `.env`. The
# variables here only apply to the compose file. If you need it passed to a
# Example environment file for Nextcloud stack, should be copied as `.env`.
# The variables here only apply to the compose file. If you need it passed to a
# container then it also needs specified in its `environment:` operator.
#
# ** All usernames and passwords need changed before running in production! **
@@ -7,6 +8,17 @@
# Full guide on the Nextcloud parameters which may be supplied:
# https://github.com/docker-library/docs/blob/master/nextcloud/README.md#auto-configuration-via-environment-variables
#
## Docker ##
#
# 20250616 - Performance testing was done on an old workstation.
# Only Compose: 3.8, 2.5, 1.8, 1.8, 2.1, 2.0, 1.7 seconds
# Compose Bake: 3.7, 0.9, 2.3, 2.0, 3.1, 2.1, 1.8 seconds
# Not really needed but also not harmful and will eliminate the warning.
# Timings do not include the initial pull for either method.
COMPOSE_BAKE=true
#
## Nextcloud ##
#

View File

@@ -1,5 +1,6 @@
# Example environment file for Nextcloud stack, should be copied as `.env`. The
# variables here only apply to the compose file. If you need it passed to a
# Example environment file for Nextcloud stack, should be copied as `.env`.
# The variables here only apply to the compose file. If you need it passed to a
# container then it also needs specified in its `environment:` operator.
#
# ** All usernames and passwords need changed before running in production! **
@@ -7,6 +8,17 @@
# Full guide on the Nextcloud parameters which may be supplied:
# https://github.com/docker-library/docs/blob/master/nextcloud/README.md#auto-configuration-via-environment-variables
#
## Docker ##
#
# 20250616 - Performance testing was done on an old workstation.
# Only Compose: 3.8, 2.5, 1.8, 1.8, 2.1, 2.0, 1.7 seconds
# Compose Bake: 3.7, 0.9, 2.3, 2.0, 3.1, 2.1, 1.8 seconds
# Not really needed but also not harmful and will eliminate the warning.
# Timings do not include the initial pull for either method.
COMPOSE_BAKE=true
#
## Nextcloud ##
#

View File

@@ -1,6 +1,6 @@
#!/bin/bash
# 2022-09-25 Hyperling
# Put fixes in a file so they do not need remembered.
# Put maintenance needs into a file so the commands do not need remembered.
## Setup ##
@@ -11,19 +11,20 @@ source $DIR/../../source.env
## Main ##
echo -e "\n*** APT ***"
echo -e "\n`date` - Update Apt Cache"
docker exec -it nc-app apt update -y
echo -e "\n`date` - Install Additional Software"
docker exec -it nc-app apt install -y sudo libmagickcore-6.q16-6-extra htop \
iputils-ping dnsutils vim bzip2 libbz2-dev # php-bz2
# 20240130
# https://help.nextcloud.com/t/docker-image-setup-warning-missing-bz2-after-update-to-nc-28-0-0/176605
echo -e "\n`date` - Compile PHP Modules"
docker exec -it nc-app docker-php-ext-install bz2
##### 20250616 - Apt and PHP commands now baked into the Dockerfile!! :D #####
###echo -e "\n*** APT ***"
###
###echo -e "\n`date` - Update Apt Cache"
###docker exec -it nc-app apt update -y
###
###echo -e "\n`date` - Install Additional Software"
###docker exec -it nc-app apt install -y sudo libmagickcore-6.q16-6-extra htop \
### iputils-ping dnsutils vim bzip2 libbz2-dev # php-bz2
###
#### 20240130
#### https://help.nextcloud.com/t/docker-image-setup-warning-missing-bz2-after-update-to-nc-28-0-0/176605
###echo -e "\n`date` - Compile PHP Modules"
###docker exec -it nc-app docker-php-ext-install bz2
# 2023-12-04 Make sure cron and chmod commands get run.
echo -e "\n*** CRON ***"
@@ -34,12 +35,27 @@ $DOCKER_HOME/Config/Nextcloud/cron.sh && echo "Success!"
# 2022-10-30 More additions after moving to Nextcloud version 25.
echo -e "\n*** DATABASE ***"
echo -e "\n`date` - Enable Maintenance Mode"
docker exec -itu www-data nc-app ./occ maintenance:mode --on
# 20251004: Disabled since it has not been working.
# Clear DB errors by allowing the DB to perform any incremental upgrades.
# https://jira.mariadb.org/browse/MDEV-26342
echo -e "\n`date` - mysql_upgrade"
docker exec -itu www-data nc-app ./occ maintenance:mode --on
time docker exec -it nc-db mysql_upgrade \
-user="root" --password="$MYSQL_ROOT_PASSWORD"
#echo -e "\n`date` - mysql_upgrade"
#time docker exec -it nc-db mysql_upgrade \
# -user="root" --password="$MYSQL_ROOT_PASSWORD"
echo -e "\n`date` - Enable Dynamic Row Formats"
docker exec -it nc-db mysql -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" \
-e "SELECT CONCAT('ALTER TABLE ', TABLE_NAME,' ROW_FORMAT=DYNAMIC; ') FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = '$MYSQL_DATABASE' AND ENGINE = 'InnoDB' AND ROW_FORMAT != 'DYNAMIC'; " \
-B -N |
while read -r sql; do
echo "SQL being executed: $sql "
docker exec nc-db mysql -u "$MYSQL_USER" -p"$MYSQL_PASSWORD" -e "$sql" "$MYSQL_DATABASE";
echo " - Done!\ "
done
echo -e "\n`date` - Disable Maintenance Mode"
docker exec -itu www-data nc-app ./occ maintenance:mode --off
echo -e "\n`date` - Add Missing Columns"
@@ -69,7 +85,7 @@ docker exec -itu www-data nc-app ./occ files:scan-app-data
echo -e "\n`date` - Theme Update"
docker exec -itu www-data nc-app ./occ maintenance:theme:update
echo -e "\n`date` - Repair"
docker exec -itu www-data nc-app ./occ maintenance:repair
docker exec -itu www-data nc-app ./occ maintenance:repair --include-expensive
# May also be useful but do not have much experience with them.
echo -e "\n`date` - Clean Versions"

View File

@@ -0,0 +1,12 @@
# Create a VM for testing Ansible provisioning.
FROM archlinux:base
RUN pacman -Syu --noconfirm && pacman -Sy --noconfirm $PACKAGES $EXTRA_ARCH
COPY bin/main.sh /root/main.sh
RUN chmod +x /root/main.sh
WORKDIR /root/
USER root
CMD ./main.sh

View File

@@ -0,0 +1,12 @@
# Create a VM for testing Ansible provisioning.
FROM debian:trixie
RUN apt update && apt dist-upgrade -y && apt install -y $PACKAGES $EXTRA_DEBIAN
COPY bin/main.sh /root/main.sh
RUN chmod +x /root/main.sh
WORKDIR /root/
USER root
CMD ./main.sh

View File

@@ -0,0 +1,12 @@
# Create a VM for testing Ansible provisioning.
FROM fedora:latest
RUN dnf upgrade --refresh -y && dnf install -y $PACKAGES $EXTRA_FEDORA
COPY bin/main.sh /root/main.sh
RUN chmod +x /root/main.sh
WORKDIR /root/
USER root
CMD ./main.sh

View File

@@ -0,0 +1,14 @@
# Create a VM for testing Ansible provisioning.
FROM opensuse/tumbleweed
RUN zypper -n refresh \
&& zypper -n dist-upgrade -y \
&& zypper -n install -y $PACKAGES $EXTRA_OPENSUSE
COPY bin/main.sh /root/main.sh
RUN chmod +x /root/main.sh
WORKDIR /root/
USER root
CMD ./main.sh

View File

@@ -0,0 +1,12 @@
# Create a VM for testing Ansible provisioning.
FROM ubuntu:rolling
RUN apt update && apt dist-upgrade -y && apt install -y $PACKAGES $EXTRA_UBUNTU
COPY bin/main.sh /root/main.sh
RUN chmod +x /root/main.sh
WORKDIR /root/
USER root
CMD ./main.sh

View File

@@ -0,0 +1,3 @@
# Provision Test Images
Create containers of popular Linux distributions in order to test Ansible.

View File

@@ -0,0 +1,21 @@
#!/usr/bin/env bash
sh -c "rm -rfv prov-test"
git clone $REPO --branch=$BRANCH prov-test
cd prov-test
echo "Starting Executable Script"
tail -F /var/mail/mail &
tail -F /var/mail/ansible &
logfile="/root/test.log"
$EXEC > $logfile &
tail -F $logfile
echo "*** Finished $0 @ `date` ***"
wait -n
exit $?

View File

@@ -0,0 +1,142 @@
# Create containers which each execute a provisioning script and exit.
volumes:
pt-storage:
driver: local
driver_opts:
type: none
device: $PT_STORAGE_DIR
o: bind
services:
pt-arch:
container_name: pt-arch
build:
context: ./
dockerfile: Dockerfiles/arch
network: host
restart: no
environment:
- PACKAGES=$PACKAGES
- EXTRA_ARCH=$EXTRA_ARCH
- REPO=$REPO
- BRANCH=$BRANCH
- EXEC=$EXEC
volumes:
- type: volume
source: pt-storage
target: /root
volume:
subpath: arch
deploy:
mode: global
resources:
limits:
cpus: $CPU
memory: $RAM
pt-debian:
container_name: pt-debian
build:
context: ./
dockerfile: Dockerfiles/debian
network: host
restart: no
environment:
- PACKAGES=$PACKAGES
- EXTRA_DEBIAN=$EXTRA_DEBIAN
- REPO=$REPO
- BRANCH=$BRANCH
- EXEC=$EXEC
volumes:
- type: volume
source: pt-storage
target: /root
volume:
subpath: debian
deploy:
mode: global
resources:
limits:
cpus: $CPU
memory: $RAM
pt-fedora:
container_name: pt-fedora
build:
context: ./
dockerfile: Dockerfiles/fedora
network: host
restart: no
environment:
- PACKAGES=$PACKAGES
- EXTRA_FEDORA=$EXTRA_FEDORA
- REPO=$REPO
- BRANCH=$BRANCH
- EXEC=$EXEC
volumes:
- type: volume
source: pt-storage
target: /root
volume:
subpath: fedora
deploy:
mode: global
resources:
limits:
cpus: $CPU
memory: $RAM
pt-opensuse:
container_name: pt-opensuse
build:
context: ./
dockerfile: Dockerfiles/opensuse
network: host
restart: no
environment:
- PACKAGES=$PACKAGES
- EXTRA_OPENSUSE=$EXTRA_OPENSUSE
- REPO=$REPO
- BRANCH=$BRANCH
- EXEC=$EXEC
volumes:
- type: volume
source: pt-storage
target: /root
volume:
subpath: opensuse
deploy:
mode: global
resources:
limits:
cpus: $CPU
memory: $RAM
pt-ubuntu:
container_name: pt-ubuntu
build:
context: ./
dockerfile: Dockerfiles/ubuntu
network: host
restart: no
environment:
- PACKAGES=$PACKAGES
- EXTRA_UBUNTU=$EXTRA_UBUNTU
- REPO=$REPO
- BRANCH=$BRANCH
- EXEC=$EXEC
volumes:
- type: volume
source: pt-storage
target: /root
volume:
subpath: ubuntu
deploy:
mode: global
resources:
limits:
cpus: $CPU
memory: $RAM

View File

@@ -0,0 +1,24 @@
# This file should be renamed '.env' and have any private values modified.
COMPOSE_BAKE=true
## Performance
CPU=0.2
RAM=0.2G
## Storage
#STORAGE_DIR=../../Volumes/ProvisionTests
PT_STORAGE_DIR=/tmp/ProvisionTests
## Packages
PACKAGES="git bash curl sudo which"
EXTRA_ARCH=""
EXTRA_DEBIAN=""
EXTRA_FEDORA=""
EXTRA_OPENSUSE=""
EXTRA_UBUNTU=""
## Script
REPO=https://git.hyperling.com/me/env-ansible
BRANCH=dev
EXEC="./setup.sh -l"

23
Config/ProvisionTests/prep.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# 2025-07-22 Hyperling
# Create the necessary folders for LibreTranslate's volumes to work.
# This must be run before the container will start properly.
## Setup ##
DIR="$(dirname -- "${BASH_SOURCE[0]}")"
source $DIR/.env
source ../../source.env
## Main ##
# Create folders.
mkdir -pv "$PT_STORAGE_DIR/arch"
mkdir -pv "$PT_STORAGE_DIR/debian"
mkdir -pv "$PT_STORAGE_DIR/fedora"
mkdir -pv "$PT_STORAGE_DIR/opensuse"
mkdir -pv "$PT_STORAGE_DIR/ubuntu"
# Finish successfully.
exit 0

View File

@@ -9,23 +9,35 @@ source $DIR/../source.env
## Variables ##
file="/tmp/Backup.zip"
TAG="Docker"
DATE="`date "+%Y%m%d-%H%M%S"`"
BACKUP_DIR="/srv/backup"
BASENAME="Backup"
file="$BACKUP_DIR/$BASENAME.$DATE.$HOSTNAME.$TAG.zip"
time="`which time`"
## Main ##
# Ensure backup directory exists with correct permissions.
mkdir -pv "$BACKUP_DIR"
mkdir -pv "$BACKUP_DIR/TRASH"
chmod -Rv 775 "$BACKUP_DIR"
chown -Rv root:root "$BACKUP_DIR"
# Remove the last backup.
if [[ -e $file ]]; then
echo "`date` - Removing existing file."
rm -fv $file
fi
echo "`date` - Removing existing files."
cd "$BACKUP_DIR"
mv -v "$BASENAME"*"$TAG"* TRASH/
rm -v TRASH/*
echo -e "\n`date` - Take down services for a cold backup."
manage.sh -d
echo -e "\n`date` - Create the backup for '$DOCKER_HOME'."
cd $DOCKER_HOME
$time zip -r $file.tmp . 1>/dev/null
$time zip -r $file.tmp \
$DOCKER_HOME \
/etc/crontab /etc/cron.d /var/spool/cron \
/var/{log,mail} 1>/dev/null
mv -v $file.tmp $file
echo -e "\n`date` - Done with zipping, check size."

View File

@@ -130,7 +130,7 @@ fi
if [[ -z $up && -z $down && -z $build && -z $pull && -z $clean
&& -z $interact && -z $logs && -z $stats
]]; then
docker ps
docker ps --size
exit 0
fi