all repos — weimar @ 4725091cf8486591001604eeabcb1c4632f7ab7f

Unnamed repository; edit this file 'description' to name the repository.

docs/ARCHITECTURE.md (view raw)

  1# weimar Architecture
  2
  3> A single-binary media repository for home servers — memes, group interest images, and videos with simple tag-based discovery.
  4> Inspired by Chevereto V4 (`docs/CHEVERE.md`), scaled down for a small group on a single home server.
  5
  6---
  7
  8## 1. Design Principles
  9
 101. **Single binary** — Go backend with SvelteKit SPA (adapter-static) embedded via `embed.FS`. No Docker, no Node process, no external runtime.
 112. **Zero external dependencies** at runtime (except optional `ffmpeg` for video thumbnails).
 123. **Simple trumps scalable** — SQLite, filesystem storage, no queues, no caches.
 134. **Authentication-first** — users are real accounts, not anonymous.
 145. **CLI admin** — no web dashboard; admin operations are CLI commands.
 15
 16---
 17
 18## 2. Tech Stack
 19
 20| Layer | Technology |
 21|-------|------------|
 22| **Backend** | Go (net/http, standard library + chi or similar router) |
 23| **Frontend** | SvelteKit (adapter-static, pre-rendered SPA) |
 24| **Database** | SQLite (via modernc.org/sqlite or mattn/go-sqlite3) |
 25| **Session store** | SQLite (same database) |
 26| **Image processing** | Go native (image/jpeg, image/png, image/gif, golang.org/x/image) |
 27| **Video thumbnails** | `ffmpeg` binary — optional, skipped if missing |
 28| **EXIF stripping** | Go native (exif library) |
 29
 30---
 31
 32## 3. Directory Structure
 33
 34```
 35weimar/
 36├── main.go                       # Entry point: CLI flag parsing, serve
 37├── go.mod / go.sum
 38├── weimar.toml                   # User-provided config file
 39 40├── cmd/
 41│   ├── serve.go                  # weimar serve (HTTP server)
 42│   ├── users.go                  # weimar users (list/create/delete/password-reset)
 43│   └── image.go                  # weimar image delete
 44 45├── internal/
 46│   ├── server/
 47│   │   ├── server.go             # HTTP server, routes, embedded SPA
 48│   │   └── middleware.go         # Auth middleware, CORS, session handling
 49│   ├── auth/
 50│   │   ├── session.go            # Server-side session (cookie-based, SQLite-backed)
 51│   │   └── register.go           # Registration logic
 52│   ├── api/
 53│   │   ├── upload.go             # POST /api/upload
 54│   │   ├── images.go             # GET /api/images, GET /api/images/{id}
 55│   │   ├── download.go           # GET /api/images/{id}/download
 56│   │   ├── thumbnail.go          # GET /api/images/{id}/thumbnail
 57│   │   ├── auth.go               # POST /api/auth/login, /register
 58│   │   └── tags.go               # GET /api/tags/suggest
 59│   ├── db/
 60│   │   ├── sqlite.go             # Database connection and migrations
 61│   │   ├── models.go             # Structs: User, Image, Session, Tag
 62│   │   └── queries.go            # SQL queries
 63│   ├── storage/
 64│   │   ├── disk.go               # File read/write/delete, hash-partitioned paths
 65│   │   └── processing.go         # EXIF stripping, thumbnail generation
 66│   └── config/
 67│       └── config.go             # TOML config parsing
 68 69├── web/                          # SvelteKit SPA (built, embedded)
 70│   ├── src/
 71│   │   ├── routes/               # SvelteKit pages (login, browse, upload, image view)
 72│   │   ├── lib/
 73│   │   │   ├── api.ts            # Fetch wrapper for API calls
 74│   │   │   └── types.ts          # TypeScript interfaces
 75│   │   └── app.html
 76│   ├── svelte.config.js          # adapter-static
 77│   └── package.json
 78 79├── data/
 80│   ├── images/                   # Hash-partitioned image storage
 81│   │   └── ab/cd/ef/<hash>.<ext>
 82│   └── weimar.db                 # SQLite database
 83 84└── docs/
 85    ├── CHEVERE.md                # Chevereto V4 reference architecture
 86    └── ARCHITECTURE.md           # This file
 87```
 88
 89---
 90
 91## 4. Data Model
 92
 93### SQLite Tables
 94
 95**users**
 96| Column | Type | Notes |
 97|--------|------|-------|
 98| id | INTEGER PRIMARY KEY | |
 99| username | TEXT UNIQUE NOT NULL | |
100| password_hash | TEXT NOT NULL | bcrypt |
101| is_admin | INTEGER DEFAULT 0 | |
102| created_at | TEXT (ISO 8601) | |
103| updated_at | TEXT (ISO 8601) | |
104
105**sessions**
106| Column | Type | Notes |
107|--------|------|-------|
108| id | INTEGER PRIMARY KEY | |
109| user_id | INTEGER REFERENCES users(id) | |
110| token | TEXT UNIQUE NOT NULL | Random 64-char hex |
111| expires_at | TEXT (ISO 8601) | |
112| created_at | TEXT (ISO 8601) | |
113
114**images**
115| Column | Type | Notes |
116|--------|------|-------|
117| id | INTEGER PRIMARY KEY | |
118| filename | TEXT NOT NULL | Original filename |
119| storage_path | TEXT NOT NULL | Relative path in storage dir |
120| thumbnail_path | TEXT | Relative path, nullable (no ffmpeg) |
121| uploaded_by | INTEGER REFERENCES users(id) | |
122| file_size | INTEGER | Bytes |
123| width | INTEGER | |
124| height | INTEGER | |
125| mime_type | TEXT | e.g. image/jpeg, video/mp4 |
126| created_at | TEXT (ISO 8601) | |
127| updated_at | TEXT (ISO 8601) | |
128
129**image_tags**
130| Column | Type | Notes |
131|--------|------|-------|
132| image_id | INTEGER REFERENCES images(id) | ON DELETE CASCADE |
133| tag | TEXT NOT NULL | Lowercase, trimmed |
134
135**tags** (materialized for autocomplete)
136| Column | Type | Notes |
137|--------|------|-------|
138| tag | TEXT PRIMARY KEY | Lowercase, unique |
139| usage_count | INTEGER DEFAULT 0 | Denormalized for sort |
140
141---
142
143## 5. API Endpoints
144
145| Method | Path | Auth | Description |
146|--------|------|------|-------------|
147| POST | `/api/auth/login` | No | Login, returns session cookie |
148| POST | `/api/auth/register` | No* | Create account (*if registration open) |
149| POST | `/api/upload` | Yes | Upload image/video + tags |
150| GET | `/api/images` | Yes | Paginated listing, ?tags filter, newest-first |
151| GET | `/api/images/{id}` | Yes | Single image metadata |
152| GET | `/api/images/{id}/download` | Yes | Serve original file |
153| GET | `/api/images/{id}/thumbnail` | Yes | Serve thumbnail |
154| GET | `/api/tags/suggest?q=` | Yes | Tag autocomplete (LIKE query) |
155
156All authenticated endpoints use the session cookie (not Bearer tokens).
157
158---
159
160## 6. Request Flow
161
162```
163Browser → SvelteKit SPA
164           ↓ fetch()
165         Go HTTP server
166           ↓ middleware
167         Auth check (session cookie validation)
168169         Route handler
170           ├─ db queries (SQLite)
171           ├─ storage ops (disk read/write)
172           └─ image processing (EXIF strip, thumbnail)
173174         JSON response → SPA renders
175```
176
177---
178
179## 7. Admin CLI Commands
180
181```
182weimar serve --config /path/to/weimar.toml --admin-email user@example.com --admin-password s3cur3
183weimar users list --config /path/to/weimar.toml
184weimar users create --config /path/to/weimar.toml <username> <password>
185weimar users delete --config /path/to/weimar.toml <username>
186weimar users password-reset --config /path/to/weimar.toml <username>
187weimar image delete --config /path/to/weimar.toml <id>
188```
189
190All commands require `--config` (or `WEIMAR_CONFIG` env var).
191
192---
193
194## 8. Image Storage
195
196### Path Structure
197```
198{storage_path}/ab/cd/ef/<sha256_hex_prefix>.<ext>
199```
200- First 6 hex chars of SHA-256 of file content → 3 levels of 2 chars
201- Prevents thousands of files in a single directory
202
203### Processing Pipeline
2041. Validate file type (magic bytes, not extension)
2052. Check file size against config limit
2063. Generate SHA-256 hash for storage path
2074. Strip EXIF metadata (images only)
2085. Generate thumbnail:
209   - Images: resize to configurable max dimension (default 300px)
210   - Video: `ffmpeg -i <input> -vframes 1 <output.jpg>` (skip if ffmpeg not found)
2116. Write original + thumbnail to disk (hash-partitioned paths)
2127. Insert metadata + tags into SQLite
2138. Upsert tags into tag_usage table for autocomplete
214
215---
216
217## 9. Configuration (TOML)
218
219```toml
220[server]
221host = "0.0.0.0"
222port = 8080
223
224[database]
225path = "./data/weimar.db"
226
227[storage]
228path = "./data/images"
229
230[upload]
231max_size_mb = 50
232rate_limit_per_minute = 0  # 0 = unlimited
233
234[auth]
235allow_registration = true  # false = gated, admin creates users only
236```
237
238---
239
240## 10. Session Management
241
242- **Login**: verify password → generate random 64-char hex token → store in `sessions` table → set `httpOnly` cookie named `weimar_session`
243- **Auth check**: read cookie → look up token in SQLite → check `expires_at` → attach user context to request
244- **Expiry**: sessions expire after 30 days of inactivity (reset on each request)
245- **Logout**: delete session from DB + clear cookie
246
247---
248
249## 11. Thumbnail Strategy
250
251- **Images**: server-side resize via Go `image` package (nearest-neighbour or bilinear). Store as JPEG.
252- **Videos**: shell out to `ffmpeg -i {input} -vframes 1 -s 300x300 {output.jpg}`. If ffmpeg not found on `$PATH`, store `null` in `thumbnail_path`. SvelteKit handles missing thumbnail gracefully.
253- **Format**: Thumbnails always stored as JPEG regardless of source format.
254
255---
256
257## 12. Tag System
258
259- **Input**: user types comma-separated tag string on upload
260- **Normalization**: lowercase, trimmed, whitespace-collapsed
261- **Autocomplete**: `GET /api/tags/suggest?q=fun``SELECT tag FROM tags WHERE tag LIKE 'fun%' ORDER BY usage_count DESC LIMIT 10`
262- **No collision resolution**: `funny-cat` and `funny_cat` and `funnycat` are distinct tags
263- **Deletion**: when an image is deleted, decrement `usage_count` for its tags; remove tag row if count reaches 0
264
265---
266
267## 13. Key Design Decisions (vs Chevereto)
268
269| Concern | Chevereto V4 | weimar |
270|---------|-------------|--------|
271| Database | MySQL/MariaDB | SQLite |
272| Deployment | Docker / VPS / cPanel | Single binary |
273| Admin UI | Full web dashboard | CLI only |
274| Multi-tenancy | Yes (Traefik-based SaaS) | No |
275| Storage backends | Local + S3 + S3-compatible | Local filesystem only |
276| Image sizes | Original + thumb + medium | Original + thumb only |
277| Cache | Redis + file-based | None |
278| Job queue | Yes (cron-driven) | No |
279| Moderation | Flag-based + external services | None |
280| Video thumbnails | php-ffmpeg (required) | ffmpeg (optional) |
281| Mail | Symfony Mailer (15+ transports) | None |
282| 2FA | Google2FA | None |
283| Rate limiting | Flood protection | Configurable, off by default |
284| API auth | JWT + API Keys + signed requests | Session cookie only |
285| Language | 30+ languages | Single language (English) |
286
287---
288
289## 14. Out of Scope (Future Considerations)
290
291- Search beyond tag filtering (full-text search)
292- Albums / collections
293- User profiles
294- Comment system
295- Federation (ActivityPub)
296- S3/remote storage backends
297- Multi-architecture builds (x86_64 only is fine)
298- Prometheus metrics
299- OAuth / social login
300- Webhook notifications