docs/ARCHITECTURE.md (view raw)
1# weimar Architecture
2
3> A single-binary media repository for home servers — memes, group interest images, and videos with simple tag-based discovery.
4> Inspired by Chevereto V4 (`docs/CHEVERE.md`), scaled down for a small group on a single home server.
5
6---
7
8## 1. Design Principles
9
101. **Single binary** — Go backend with SvelteKit SPA (adapter-static) embedded via `embed.FS`. No Docker, no Node process, no external runtime.
112. **Zero external dependencies** at runtime (except optional `ffmpeg` for video thumbnails).
123. **Simple trumps scalable** — SQLite, filesystem storage, no queues, no caches.
134. **Authentication-first** — users are real accounts, not anonymous.
145. **CLI admin** — no web dashboard; admin operations are CLI commands.
15
16---
17
18## 2. Tech Stack
19
20| Layer | Technology |
21|-------|------------|
22| **Backend** | Go 1.22+ (net/http, http.ServeMux with method+pattern routing) |
23| **Frontend** | SvelteKit (adapter-static, pre-rendered SPA) |
24| **Database** | SQLite (via modernc.org/sqlite or mattn/go-sqlite3) |
25| **Session store** | SQLite (same database) |
26| **Image processing** | Go native (image/jpeg, image/png, image/gif, golang.org/x/image) |
27| **Avatar processing** | Go native (golang.org/x/image/draw), resize to 128px JPEG |
28| **Video thumbnails** | `ffmpeg` binary — optional, skipped if missing |
29| **EXIF stripping** | Go native (exif library) |
30
31---
32
33## 3. Directory Structure
34
35```
36weimar/
37├── go.mod / go.sum
38├── weimar.toml # User-provided config file
39├── AGENTS.md # Agent guide (commands, patterns, gotchas)
40│
41├── cmd/
42│ └── weimar/
43│ ├── main.go # Entry point: os.Exit on Execute() error
44│ ├── root.go # Persistent flags, viper config init, defaults
45│ ├── serve.go # weimar serve (HTTP server + //go:embed web/build/*)
46│ ├── users.go # weimar users (list/create/delete/password-reset)
47│ ├── image.go # weimar image delete
48│ └── version.go # weimar version
49│
50├── internal/
51│ ├── server/
52│ │ ├── server.go # HTTP server, route registration, embedded SPA
53│ │ └── middleware.go # requireAuth wrapper, writeJSONError
54│ ├── auth/
55│ │ ├── session.go # Server-side session (cookie-based, SQLite-backed)
56│ │ └── register.go # Registration logic + sentinel errors
57│ ├── api/
58│ │ ├── api.go # API struct (dependency holder), writeJSON/writeError
59│ │ ├── auth.go # POST /api/auth/login, /register, /logout
60│ │ ├── upload.go # POST /api/upload (file + tags + processing)
61│ │ ├── images.go # GET /api/images, GET /api/images/{id}
62│ │ ├── download.go # GET /api/images/{id}/download
63│ │ ├── thumbnail.go # GET /api/images/{id}/thumbnail
64│ │ ├── tags.go # GET /api/tags/suggest
65│ │ ├── users.go # GET /api/users/me
66│ │ └── avatar.go # POST /api/users/me/avatar, GET /api/users/{id}/avatar
67│ ├── db/
68│ │ ├── sqlite.go # Database connection and migrations
69│ │ ├── models.go # Structs: User, Image, Session, Tag
70│ │ └── queries.go # All SQL CRUD operations
71│ ├── storage/
72│ │ ├── disk.go # File read/write/delete, hash-partitioned paths
73│ │ └── processing.go # MIME detection, EXIF stripping, thumbnail generation
74│ └── config/
75│ └── config.go # FromViper() reads typed Config from viper
76│
77├── web/ # SvelteKit SPA (adapter-static, CSR-only)
78│ ├── src/
79│ │ ├── routes/
80│ │ │ ├── +layout.svelte # Nav bar, auth check (getMe), logout
81│ │ │ ├── +layout.js # ssr=false, prerender=false
82│ │ │ ├── +page.svelte # Home page (constructivist design)
83│ │ │ ├── browse/+page.svelte # Gallery grid (6-col, tags, pagination, viewer)
84│ │ │ ├── login/+page.svelte # Login/Register form
85│ │ │ ├── upload/+page.svelte # File upload + tag autocomplete
86│ │ │ ├── settings/+page.svelte # Avatar upload, user info
87│ │ │ └── image/[id]/+page.svelte # Image detail + download
88│ │ └── lib/
89│ │ ├── api.ts # Fetch wrapper, typed API functions
90│ │ ├── Avatar.svelte # Avatar component (image or SVG fallback)
91│ │ └── ImageViewer.svelte # Full-screen overlay viewer (keyboard nav, idle fade)
92│ ├── svelte.config.js # adapter-static
93│ └── package.json
94│
95├── data/
96│ ├── images/ # Hash-partitioned image storage
97│ │ └── ab/cd/ef/<hash>.<ext>
98│ └── weimar.db # SQLite database
99│
100└── docs/
101 ├── CHEVERE.md # Chevereto V4 reference architecture
102 └── ARCHITECTURE.md # This file
103```
104
105---
106
107## 4. Data Model
108
109### SQLite Tables
110
111**users**
112| Column | Type | Notes |
113|--------|------|-------|
114| id | INTEGER PRIMARY KEY | |
115| username | TEXT UNIQUE NOT NULL | |
116| password_hash | TEXT NOT NULL | bcrypt |
117| is_admin | INTEGER DEFAULT 0 | |
118| avatar_path | TEXT | Relative path in storage dir, nullable |
119| created_at | TEXT (ISO 8601) | |
120| updated_at | TEXT (ISO 8601) | |
121
122**sessions**
123| Column | Type | Notes |
124|--------|------|-------|
125| id | INTEGER PRIMARY KEY | |
126| user_id | INTEGER REFERENCES users(id) | |
127| token | TEXT UNIQUE NOT NULL | Random 64-char hex |
128| expires_at | TEXT (ISO 8601) | |
129| created_at | TEXT (ISO 8601) | |
130
131**images**
132| Column | Type | Notes |
133|--------|------|-------|
134| id | INTEGER PRIMARY KEY | |
135| filename | TEXT NOT NULL | Original filename |
136| storage_path | TEXT NOT NULL | Relative path in storage dir |
137| thumbnail_path | TEXT | Relative path, nullable (no ffmpeg) |
138| uploaded_by | INTEGER REFERENCES users(id) | |
139| file_size | INTEGER | Bytes |
140| width | INTEGER | |
141| height | INTEGER | |
142| mime_type | TEXT | e.g. image/jpeg, video/mp4 |
143| created_at | TEXT (ISO 8601) | |
144| updated_at | TEXT (ISO 8601) | |
145
146**image_tags**
147| Column | Type | Notes |
148|--------|------|-------|
149| image_id | INTEGER REFERENCES images(id) | ON DELETE CASCADE |
150| tag | TEXT NOT NULL | Lowercase, trimmed |
151
152**tags** (materialized for autocomplete)
153| Column | Type | Notes |
154|--------|------|-------|
155| tag | TEXT PRIMARY KEY | Lowercase, unique |
156| usage_count | INTEGER DEFAULT 0 | Denormalized for sort |
157
158---
159
160## 5. API Endpoints
161
162| Method | Path | Auth | Description |
163|--------|------|------|-------------|
164| POST | `/api/auth/login` | No | Login, returns session cookie |
165| POST | `/api/auth/register` | No* | Create account (*if registration open) |
166| POST | `/api/auth/logout` | Yes | Clear session |
167| POST | `/api/upload` | Yes | Upload image/video + tags |
168| GET | `/api/images` | No* | Paginated listing, ?tags filter, newest-first *public unless require_auth_for_browse=true |
169| GET | `/api/images/{id}` | No* | Single image metadata (incl. tags_detail, uploader) |
170| GET | `/api/images/{id}/download` | No* | Serve original file |
171| GET | `/api/images/{id}/thumbnail` | No* | Serve thumbnail (JPEG, 300px max) |
172| GET | `/api/tags/suggest?q=` | Yes | Tag autocomplete (LIKE query by usage) |
173| POST | `/api/images/{id}/tags` | Yes | Add tags (owner only) |
174| DELETE | `/api/images/{id}/tags` | Yes | Remove tags (owner only) |
175| GET | `/api/users/me` | Yes | Current user info (username, avatar_url, is_admin) |
176| POST | `/api/users/me/avatar` | Yes | Upload avatar (resized to 128px JPEG) |
177| GET | `/api/users/{id}/avatar` | No | Serve avatar (Cache-Control: public, 86400s) |
178
179Browse endpoints (GET `/api/images*`) use `optionalAuth` middleware: if `auth.require_auth_for_browse` is false (default), unauthenticated requests pass through without a user context instead of returning 401. Authenticated users still get `is_owner` and other user-specific fields.
180
181All authenticated endpoints use the session cookie (not Bearer tokens).
182
183---
184
185## 6. Request Flow
186
187```
188Browser → SvelteKit SPA
189 ↓ fetch()
190 Go HTTP server
191 ↓ middleware
192 Auth check (session cookie validation — requireAuth or optionalAuth)
193 ↓
194 Route handler
195 ├─ db queries (SQLite)
196 ├─ storage ops (disk read/write)
197 └─ image processing (EXIF strip, thumbnail)
198 ↓
199 JSON response → SPA renders
200```
201
202Browse endpoints use `optionalAuth` middleware: tries to authenticate from the session cookie but passes through without a user context on failure (unless `auth.require_auth_for_browse` is true, in which case it rejects like `requireAuth`). Mutating endpoints (upload, tag edit, logout, settings) always use `requireAuth` which rejects unauthenticated requests with 401.
203
204---
205
206## 7. Admin CLI Commands
207
208```
209weimar serve --config /path/to/weimar.toml --admin-email user@example.com --admin-password s3cur3
210weimar users list --config /path/to/weimar.toml
211weimar users create --config /path/to/weimar.toml <username> <password>
212weimar users delete --config /path/to/weimar.toml <username>
213weimar users password-reset --config /path/to/weimar.toml <username>
214weimar image delete --config /path/to/weimar.toml <id>
215```
216
217All commands require `--config` (or `WEIMAR_CONFIG` env var).
218
219---
220
221## 8. Image Storage
222
223### Path Structure
224```
225{storage_path}/ab/cd/ef/<sha256_hex_prefix>.<ext>
226```
227- First 6 hex chars of SHA-256 of file content → 3 levels of 2 chars
228- Prevents thousands of files in a single directory
229
230### Processing Pipeline
2311. Validate file type (magic bytes, not extension)
2322. Check file size against config limit
2333. Generate SHA-256 hash for storage path
2344. Strip EXIF metadata (images only)
2355. Generate thumbnail:
236 - Images: resize to configurable max dimension (default 300px)
237 - Video: `ffmpeg -i <input> -vframes 1 <output.jpg>` (skip if ffmpeg not found)
2386. Write original + thumbnail to disk (hash-partitioned paths)
2397. Insert metadata + tags into SQLite
2408. Upsert tags into tag_usage table for autocomplete
241
242---
243
244## 9. Configuration (TOML)
245
246```toml
247[server]
248host = "0.0.0.0"
249port = 8080
250
251[database]
252path = "./data/weimar.db"
253
254[storage]
255path = "./data/images"
256
257[upload]
258max_size_mb = 50
259rate_limit_per_minute = 0 # 0 = unlimited
260
261[auth]
262allow_registration = true # false = gated, admin creates users only
263require_auth_for_browse = false # true = unauth users blocked from browsing
264```
265
266---
267
268## 10. Session Management
269
270- **Login**: verify password → generate random 64-char hex token → store in `sessions` table → set `httpOnly` cookie named `weimar_session`
271- **Auth check**: read cookie → look up token in SQLite → check `expires_at` → attach user context to request
272- **Expiry**: sessions expire after 30 days of inactivity (reset on each request)
273- **Logout**: delete session from DB + clear cookie
274
275---
276
277## 11. Thumbnail Strategy
278
279- **Images**: server-side resize via Go `image` package (nearest-neighbour or bilinear). Store as JPEG.
280- **Videos**: shell out to `ffmpeg -i {input} -vframes 1 -s 300x300 {output.jpg}`. If ffmpeg not found on `$PATH`, store `null` in `thumbnail_path`. SvelteKit handles missing thumbnail gracefully.
281- **Format**: Thumbnails always stored as JPEG regardless of source format.
282
283---
284
285## 12. Tag System
286
287- **Input**: user types comma-separated tag string on upload
288- **Normalization**: lowercase, trimmed, whitespace-collapsed
289- **Autocomplete**: `GET /api/tags/suggest?q=fun` → `SELECT tag FROM tags WHERE tag LIKE 'fun%' ORDER BY usage_count DESC LIMIT 10`
290- **No collision resolution**: `funny-cat` and `funny_cat` and `funnycat` are distinct tags
291- **Deletion**: when an image is deleted, decrement `usage_count` for its tags; remove tag row if count reaches 0
292
293---
294
295## 13. Key Design Decisions (vs Chevereto)
296
297| Concern | Chevereto V4 | weimar |
298|---------|-------------|--------|
299| Database | MySQL/MariaDB | SQLite |
300| Deployment | Docker / VPS / cPanel | Single binary |
301| Admin UI | Full web dashboard | CLI only |
302| Multi-tenancy | Yes (Traefik-based SaaS) | No |
303| Storage backends | Local + S3 + S3-compatible | Local filesystem only |
304| Image sizes | Original + thumb + medium | Original + thumb only |
305| Cache | Redis + file-based | None |
306| Job queue | Yes (cron-driven) | No |
307| Moderation | Flag-based + external services | None |
308| Video thumbnails | php-ffmpeg (required) | ffmpeg (optional) |
309| Mail | Symfony Mailer (15+ transports) | None |
310| 2FA | Google2FA | None |
311| Rate limiting | Flood protection | Configurable, off by default |
312| API auth | JWT + API Keys + signed requests | Session cookie only |
313| Language | 30+ languages | Single language (English) |
314
315---
316
317## 14. Out of Scope (Future Considerations)
318
319- Search beyond tag filtering (full-text search)
320- Albums / collections
321- User profiles
322- Comment system
323- Federation (ActivityPub)
324- S3/remote storage backends
325- Multi-architecture builds (x86_64 only is fine)
326- Prometheus metrics
327- OAuth / social login
328- Webhook notifications