silverbullet-notes/docs/servers/hetzner/01-databases.md

# Hetzner Server (192.168.12.3) - Database Documentation

_Last updated: 2026-01-05_

This document provides comprehensive documentation of all database systems running on the Hetzner server.

---

## Table of Contents

- [MariaDB/MySQL Databases](#mariadbmysql-databases)
- [PostgreSQL Databases](#postgresql-databases)
- [Redis Instances](#redis-instances)
- [Database Access Methods](#database-access-methods)
- [Backup Information](#backup-information)

---

## MariaDB/MySQL Databases

### Primary MariaDB Instance (mariadb-secondary)

**Instance Information:**
- **Container:** mariadb-secondary
- **Image:** mariadb:latest
- **Host:** 192.168.12.3
- **Port:** 3306 (exposed on all interfaces)
- **Type:** MariaDB (latest)
- **Role:** Read-only replica
- **Server ID:** 2
- **Replication Source:** 192.168.1.251 (primary server)

**Configuration:**
- **Config File:** `/volume1/docker/mariadb/custom.cnf` mounted as `/etc/mysql/my.cnf`
- **Data Directory:** `/volume1/docker/mariadb/databases` → `/var/lib/mysql`
- **Log Directory:** `/volume1/docker/mariadb/log/mysql` → `/var/log/mysql`
- **Backup Directory:** `/volume1/docker/backup` → `/backup`
- **Character Set:** utf8mb4
- **Collation:** utf8mb4_general_ci
- **Timezone:** America/New_York

**Resource Limits:**
- **Memory Reservation:** 60M
- **CPU Limit:** 0.4

**Database List:**
The following databases are expected based on configuration:

- **`node-staging`**: Used for transaction staging by financial bot
  - **Purpose:** Isolated testing environment for financial automation
  - **Replication:** Does NOT replicate from primary
  - **Used by:** Node-RED financial automation flows
  - **Notes:** Intentionally excluded from replication for isolation

- **`traefik_config`**: Traefik configuration storage
  - **User:** traefik_user
  - **Used by:** traefik-mod container
  - **Purpose:** Database-backed Traefik configuration with history/audit trail

- **Other databases:** May include replicated databases from primary server at 192.168.1.251

**Replication Configuration:**
- Configured as read-only replica (read_only=1)
- Server ID: 2
- Replicates all databases except `node-staging`
- Custom configuration controls replication filters

**Access Methods:**
```bash
# Access MariaDB shell (requires root password)
docker exec -it mariadb-secondary mysql -u root -p

# Check specific database
docker exec -it mariadb-secondary mysql -u root -p -e "USE node_staging; SHOW TABLES;"

# Check replication status
docker exec -it mariadb-secondary mysql -u root -p -e "SHOW REPLICA STATUS\G"

# List all databases
docker exec -it mariadb-secondary mysql -u root -p -e "SHOW DATABASES;"
```

**phpMyAdmin Access:**
User prefers phpMyAdmin for database management. MariaDB can be accessed via phpMyAdmin on the primary server (192.168.1.251) or local phpMyAdmin instance if configured.

**Health Monitoring:**
- **Healthcheck:** TCP connection to localhost:3306
- **Interval:** 72 seconds
- **Timeout:** 3 seconds
- **Retries:** 2
- **Start Period:** 60 seconds
- **Autoheal:** Enabled

---

### Firefly MariaDB Instance (Firefly-DB)

**Instance Information:**
- **Container:** Firefly-DB
- **Image:** mariadb:11.3-jammy
- **Host:** firefly-db (internal hostname)
- **Port:** 3306 (internal only, not exposed)
- **Type:** MariaDB 11.3
- **Role:** Application database (standalone)
- **Network:** firefly_default (isolated)

**Configuration:**
- **Data Directory:** `/volume1/docker/firefly/db` → `/var/lib/mysql`
- **Timezone:** America/New_York
- **InnoDB Buffer Pool Size:** 128M

**Database:**
- **Database Name:** firefly
- **Database User:** fireflyuser
- **Root User:** root
- **Used by:** Firefly III personal finance application

**Resource Limits:**
- **Memory Limit:** 384M
- **Memory Reservation:** 128M
- **CPU Shares:** 768

**Access Methods:**
```bash
# Access Firefly database
docker exec -it Firefly-DB mysql -u fireflyuser -p firefly

# Access as root
docker exec -it Firefly-DB mysql -u root -p

# Quick query
docker exec -it Firefly-DB mysql -u fireflyuser -p -e "USE firefly; SHOW TABLES;"
```

**Health Monitoring:**
- **Healthcheck:** TCP connection to localhost:3306
- **Interval:** 30 seconds
- **Timeout:** 10 seconds
- **Retries:** 3
- **Autoheal:** Enabled

**Notes:**
- Standalone database, not part of replication
- Optimized for Firefly III workload
- Reduced buffer pool size for memory efficiency
- Isolated on firefly_default network

---

## PostgreSQL Databases

### Authentik PostgreSQL Instance

**Instance Information:**
- **Container:** authentik-postgres
- **Image:** postgres:16-alpine
- **Host:** postgresql (internal hostname)
- **Port:** 5432 (internal only, not exposed)
- **Type:** PostgreSQL 16
- **Role:** Application database (standalone)
- **Network:** authentik-internal (isolated)

**Configuration:**
- **Data Directory:** `${AUTHENTIK_DATA_PATH}/postgres` → `/var/lib/postgresql/data`
- **Database Name:** authentik (default)
- **Database User:** authentik (default)

**Database:**
- **Database Name:** authentik
- **Database User:** authentik
- **Used by:** Authentik identity provider (server + worker)

**Access Methods:**
```bash
# Access PostgreSQL shell
docker exec -it authentik-postgres psql -U authentik -d authentik

# List databases
docker exec -it authentik-postgres psql -U authentik -c "\l"

# List tables in authentik database
docker exec -it authentik-postgres psql -U authentik -d authentik -c "\dt"
```

**Health Monitoring:**
- **Healthcheck:** `pg_isready -d authentik -U authentik`
- **Start Period:** 20 seconds
- **Interval:** 30 seconds
- **Timeout:** 5 seconds
- **Retries:** 5
- **Autoheal:** Enabled

**Notes:**
- Alpine-based image for smaller footprint
- Isolated on authentik-internal network
- Required by both authentik-server and authentik-worker
- Dependency healthcheck ensures database is ready before starting dependent services

---

### Matrix PostgreSQL Instance

**Instance Information:**
- **Container:** matrix-postgres
- **Image:** postgres:17.7-alpine
- **Host:** (Matrix network)
- **Port:** 5432 (internal only)
- **Type:** PostgreSQL 17.7
- **Role:** Matrix Synapse database
- **Network:** matrix-postgres (isolated)

**Notes:**
- **Managed separately:** This database is part of the Matrix/Synapse stack
- **Not documented in detail:** Per instructions, Matrix components are listed but not deeply documented
- **Used by:** matrix-synapse, matrix-bridges, matrix-bots
- **Backup container:** matrix-postgres-backup (prodrigestivill/postgres-backup-local:18-alpine)

---

## Redis Instances

### Authentik Redis

**Instance Information:**
- **Container:** authentik-redis
- **Image:** redis:alpine
- **Host:** redis (internal hostname)
- **Port:** 6379 (internal only)
- **Network:** authentik-internal (isolated)

**Configuration:**
- **Data Directory:** `${AUTHENTIK_DATA_PATH}/redis` → `/data`
- **Persistence:** Enabled
- **Save Interval:** 60 seconds (1 change)
- **Log Level:** warning

**Purpose:**
- Session storage for Authentik
- Cache for Authentik server and worker
- Used by: authentik-server, authentik-worker

**Access Methods:**
```bash
# Access Redis CLI
docker exec -it authentik-redis redis-cli

# Check connection
docker exec -it authentik-redis redis-cli ping

# Get info
docker exec -it authentik-redis redis-cli info

# Monitor commands
docker exec -it authentik-redis redis-cli monitor
```

**Health Monitoring:**
- **Healthcheck:** `redis-cli ping | grep PONG`
- **Start Period:** 20 seconds
- **Interval:** 30 seconds
- **Timeout:** 3 seconds
- **Retries:** 5
- **Autoheal:** Enabled

---

### Authelia Redis (NOT CURRENTLY RUNNING)

**Instance Information:**
- **Container:** authelia_redis
- **Image:** redis:alpine
- **Network:** traefik

**Configuration:**
- **Data Directory:** `./redis_data` → `/data`
- **Save Interval:** 60 seconds (1 change)
- **Log Level:** warning

**Resource Limits:**
- **CPU:** 0.20
- **Memory Limit:** 30M
- **Memory Reservation:** 10M

**Notes:**
- Container is configured but not currently running
- Would be used for Authelia session storage if Authelia were active

---

### Firefly Redis

**Instance Information:**
- **Container:** Firefly-REDIS
- **Image:** redis:latest
- **Host:** firefly-redis (internal hostname)
- **Port:** 6379 (internal only)
- **Network:** firefly_default (isolated)

**Configuration:**
- **Data Directory:** `/volume1/docker/firefly/redis` → `/data`
- **User:** 1000:1000
- **Filesystem:** Read-only (tmpfs for writes)

**Purpose:**
- Cache for Firefly III application
- Session storage
- Queue backend

**Resource Limits:**
- **Memory Limit:** 128M
- **Memory Reservation:** 50M
- **CPU Shares:** 512

**Access Methods:**
```bash
# Access Redis CLI
docker exec -it Firefly-REDIS redis-cli

# Check connection
docker exec -it Firefly-REDIS redis-cli ping

# Monitor cache usage
docker exec -it Firefly-REDIS redis-cli info memory
```

**Health Monitoring:**
- **Healthcheck:** `redis-cli ping || exit 1`
- **Interval:** 30 seconds
- **Timeout:** 5 seconds
- **Retries:** 3
- **Autoheal:** Enabled

**Security:**
- Read-only filesystem
- Security opt: no-new-privileges
- Runs as non-root user (1000:1000)

---

## Database Access Methods

### Recommended Access Method

**User Preference:** phpMyAdmin for MariaDB/MySQL management

For MariaDB databases (mariadb-secondary, Firefly-DB):
1. Access phpMyAdmin on primary server (192.168.1.251)
2. Connect to 192.168.12.3:3306 for mariadb-secondary
3. Use web interface for all database operations

### Command-Line Access (Alternative)

**MariaDB (mariadb-secondary):**
```bash
# Interactive shell
docker exec -it mariadb-secondary mysql -u root -p

# Single query
docker exec -it mariadb-secondary mysql -u root -p -e "QUERY"

# Dump database
docker exec mariadb-secondary mysqldump -u root -p database_name > backup.sql
```

**Firefly MariaDB:**
```bash
# Interactive shell
docker exec -it Firefly-DB mysql -u fireflyuser -p firefly

# As root
docker exec -it Firefly-DB mysql -u root -p
```

**Authentik PostgreSQL:**
```bash
# Interactive shell
docker exec -it authentik-postgres psql -U authentik -d authentik

# Single query
docker exec -it authentik-postgres psql -U authentik -d authentik -c "QUERY"
```

**Redis Instances:**
```bash
# Authentik Redis
docker exec -it authentik-redis redis-cli

# Firefly Redis
docker exec -it Firefly-REDIS redis-cli
```

---

## Backup Information

### MariaDB Backup (mariadb-secondary)

**Backup Directory:** `/volume1/docker/backup` (mounted as `/backup` in container)

**Replication as Backup:**
- Acts as live replica of primary server at 192.168.1.251
- All databases (except node-staging) are real-time copies
- Provides disaster recovery capability for primary server

**Manual Backup:**
```bash
# Backup specific database
docker exec mariadb-secondary mysqldump -u root -p database_name > /volume1/docker/backup/database_name_$(date +%Y%m%d).sql

# Backup all databases
docker exec mariadb-secondary mysqldump -u root -p --all-databases > /volume1/docker/backup/all_databases_$(date +%Y%m%d).sql
```

### Firefly Database Backup

**Backup Location:** `/volume1/docker/firefly/db` (database data directory)

**Manual Backup:**
```bash
# Dump Firefly database
docker exec Firefly-DB mysqldump -u fireflyuser -p firefly > /volume1/docker/backup/firefly_$(date +%Y%m%d).sql
```

### Authentik PostgreSQL Backup

**Backup Location:** `${AUTHENTIK_DATA_PATH}/postgres`

**Manual Backup:**
```bash
# Dump authentik database
docker exec authentik-postgres pg_dump -U authentik authentik > /volume1/docker/backup/authentik_$(date +%Y%m%d).sql
```

### Matrix PostgreSQL Backup

**Automated Backup:**
- **Container:** matrix-postgres-backup
- **Image:** prodrigestivill/postgres-backup-local:18-alpine
- **Status:** Running (healthy)
- Automated PostgreSQL backups for Matrix Synapse database

---

## Database Summary

**Total Database Instances:** 5 active, 1 configured (not running)

**Active Databases:**
1. **mariadb-secondary** - MariaDB latest (read-only replica + node-staging)
2. **Firefly-DB** - MariaDB 11.3 (Firefly application database)
3. **authentik-postgres** - PostgreSQL 16 (Authentik identity provider)
4. **matrix-postgres** - PostgreSQL 17.7 (Matrix/Synapse, managed separately)
5. **Firefly-REDIS** - Redis (Firefly cache)
6. **authentik-redis** - Redis (Authentik sessions)

**Configured but Not Running:**
1. **authelia_redis** - Redis (Authelia sessions)

**Key Databases by Purpose:**

| Database | Type | Purpose | Used By | Replication |
|----------|------|---------|---------|-------------|
| node-staging | MariaDB | Financial bot testing | Node-RED | No (isolated) |
| traefik_config | MariaDB | Traefik config storage | traefik-mod | Yes (from primary) |
| firefly | MariaDB | Personal finance data | Firefly III | No (standalone) |
| authentik | PostgreSQL | Identity/SSO data | Authentik | No (standalone) |

**Notes:**
- All databases have healthchecks enabled
- All active databases monitored by autoheal
- mariadb-secondary serves dual role: replica + local databases
- node-staging database intentionally isolated (no replication)