all repos — llm_aggregator @ 2e7ca2fff0eca5f948de3dec07502b00d7f03039

A CLI tool to aggregate RSS feeds and summarise them with LLMs

CHANGELOG.md (view raw)

  1# Changelog
  2
  3All notable changes to this project will be documented in this file.
  4
  5The format is based on [Keep a
  6Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
  7[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  8
  9## [1.0.2] - 2026-05-04
 10
 11### Changed
 12
 13- Module path updated to `codeberg.org/maxwelljensen/llm_aggregator` — all
 14  internal imports updated accordingly.
 15
 16## [1.0.0] - 2026-04-27
 17
 18### Added
 19
 20- Linting and code quality tooling via `golangci-lint`.
 21
 22### Fixed
 23
 24- Tight CPU-spinning polling loop in signal handler (added `time.Sleep` to
 25  avoid busy-waiting).
 26- `--temperature 0` was silently overwritten by the default (0.7); temperature
 27  is now a pointer so `nil` means "not provided" and a non-nil value — including
 28  `0.0` — means "explicitly set".
 29- `WasInterrupted` godoc comment capitalised to satisfy `golint`.
 30
 31## [0.15.0] - 2026-04-27
 32
 33### Added
 34
 35- **`--timeout N` flag**: LLM request timeout in seconds (default: 300). If the
 36  LLM API call takes longer than this, the request is aborted and a descriptive
 37  error is returned. The timeout is implemented as a `context.WithTimeout`
 38  derived from the signal-handling context, so both signal interrupts and
 39  timeouts abort the call.
 40- **`DefaultLLMTimeout` constant**: Centralises the default timeout value
 41  (300 seconds) in `internal/defaults`.
 42- **LLM client unit tests**: Three tests for the constructor: timeout default,
 43  all-parameter defaults, and API key validation.
 44
 45### Changed
 46
 47- **`LLMClient.llmTimeout` field**: Renamed from `timeoutSeconds` for clarity.
 48  The field stores the timeout in seconds; 0 means no timeout.
 49
 50## [0.14.0] - 2026-04-27
 51
 52### Added
 53
 54- **Signal handling for graceful shutdown**: `llm_aggregator` now handles
 55  `SIGINT`, `SIGTERM`, and `SIGHUP`. When a signal arrives during execution:
 56  - The context is cancelled, stopping in-progress operations as soon as possible
 57  - Any partial LLM summary already received is written to stdout (or the output file)
 58  - The process exits with code 130 (128 + signal number) instead of a generic error
 59  - In TUI mode, `q`/`Ctrl+C` also triggers clean shutdown via `RequestExit()`
 60- **`Runtime.Interrupted` and `Runtime.WasInterrupted()`**: New state field and
 61  query method allow callers to detect signal-caused termination.
 62
 63## [0.13.0] - 2026-04-26
 64
 65### Added
 66
 67- **`--stdin` flag**: Read raw RSS/Atom XML directly from standard input,
 68  composable with `-f/--feeds-file` to collate articles from both sources into
 69  a single LLM summary. Stdin articles appear before feeds-file articles in the
 70  collated output.
 71  - `curl -s $URL | llm_aggregator --stdin -p "Summarise"`
 72  - `llm_aggregator -f feeds.txt --stdin -p "Summarise all"`
 73  - Environment variable: `LLM_AGGREGATOR_STDIN`
 74- **`--feeds-file` is no longer required**: Either `--feeds-file` or `--stdin`
 75  (or both) must be provided. A clear error is returned if neither is specified.
 76
 77### Changed
 78
 79- Refactored `ParseFeedsFromFile` to delegate to a new `ParseFeedFromReader`
 80  method that accepts any `io.Reader`, enabling both stdin and unit test usage
 81  with `strings.Reader`.
 82
 83## [0.12.0] - 2026-04-26
 84
 85### Added
 86
 87- **`-P/--plain` flag**: Output the raw LLM summary without any formatting or
 88  metadata. Useful when the summary is to be consumed by downstream scripts.
 89
 90## [0.11.0] - 2026-04-23
 91
 92- Progress bar now tracks estimated token count (via tiktoken) before sending
 93  to LLM, then updates with actual completion tokens from the API response.
 94  Gives a more accurate picture of how much content is being sent.
 95
 96## [0.10.0] - 2026-04-23
 97
 98### Added
 99
100- **Short command-line flags**: Single-character shortcuts for the most
101  commonly used options:
102  - `-f` / `--feeds-file` — feeds file path
103  - `-p` / `--prompt` — summarisation prompt
104  - `-n` / `--max-articles-per-feed` — articles per feed limit
105  - `-d` / `--max-days-old` — article age filter
106  - `-i` / `--include-keywords` — keyword include filter
107  - `-e` / `--exclude-keywords` — keyword exclude filter
108  - `-m` / `--model` — LLM model selection
109  - `-o` / `--output` — output format
110  - `-t` / `--tui` — enable TUI
111  - `-D` / `--dry-run` — dry run mode
112- **Styled help output**: Custom lipgloss-styled help text at
113  `internal/cli/help.go` replaces go-arg's default plain-text output. Flags are
114  rendered in cyan, section headers in bold, with consistent column alignment.
115  Respects `$NO_COLOR`.
116
117## [0.9.0] - 2026-04-23
118
119### Added
120
121- **Markdown rendering in TUI**: Implemented `charmbracelet/glamour` to
122  render LLM summary output with proper markdown styling. Headers, bold,
123  italic, code blocks, lists, and other markdown elements are now styled
124  for better readability in the terminal.
125- **Formalised infobar keybinds**: TUI keybinds are now defined as
126  dedicated style variables at the top of the model for easier maintenance.
127  Keybinds are bold while separators and action text are subtle but
128  distinguishable.
129- **Thinking indicator as separate element**: The `[💭 Thinking: ON/OFF]`
130  toggle indicator is now on its own line above the keybinds, using a
131  distinct style (magenta+bold when ON, subtle+italic when OFF).
132- **Dynamic text wrapping on resize**: TUI summary text now re-wraps when
133  the terminal window size or font changes, with proper markdown re-rendering.
134
135## [0.8.0] - 2026-04-23
136
137### Added
138
139- **Thinking tags toggle in TUI**: LLMs that output `<think>`/`</think>`
140  thinking tags now have those sections automatically removed from the summary
141  display by default. Users can press `t` to toggle the thinking section back
142  on, displayed in gray at the head of the output with XML tags stripped for
143  cleaner presentation. A status indicator shows whether thinking is currently
144  visible.
145
146## [0.7.1] - 2026-04-22
147
148### Fixed
149
150- Environment variables were correctly overriding config file values, but CLI
151  arguments without explicit flags were also overriding config file values
152  unintentionally.
153  - Root cause: `go-arg` was populating struct fields with default values
154    from `default:` tags even when those CLI flags weren't provided
155  - Fix: Removed `default:` tags from Args struct fields for LLM options
156    (`model`, `base-url`, `api-key`, `max-tokens`, `temperature`);
157    Viper's own default handling is sufficient
158  - Fix: Changed Args struct fields from value types to pointer types
159    (`*string`, `*int`, `*float64`) so that "not provided on CLI" can be
160    distinguished from "provided but zero/empty"
161  - Fix: Updated `BindCLIArgs()` and `isZero()` to properly handle pointer
162    types and distinguish nil pointers from zero values
163- `isZero()` with compound type switch cases: Compound type cases like `case
164  *int, *int8, *int16, *int32, *int64` don't properly handle nil comparison when
165  stored in an interface variable. Fixed by splitting into individual type cases.
166- `isZero()` missing nil interface handling: When a nil pointer is stored
167  in an `any` interface, `value != nil` returns `true` (the interface itself
168  is not nil). Added early `if v == nil { return true }` check.
169
170## [0.7.0] - 2026-04-22
171
172### Added
173
174- **`--dry-run` option**: Validate configuration and preview what would happen
175  without making any LLM API calls. Useful for validating feeds file and
176  configuration, checking article counts and filtering results and anything
177  else that can be done "offline".
178- **Styled terminal output using `lipgloss`**: All CLI output now uses ANSI
179  colour codes for better readability:
180- **`$NO_COLOR` environment variable support**: Respects the ["no
181color"](https://no-color.org/) standard. When `NO_COLOR` is set, all styling is
182disabled and plain text is output instead.
183
184### Changed
185
186- API key validation now only occurs when not using `--dry-run` mode
187
188## [0.6.0] - 2026-04-22
189
190### Added
191
192- **Comprehensive unit test suite**, including CLI argument tests
193  (`--feeds-file`, `--prompt`, `--api-key`, `--model`, `--base-url`),
194  configuration tests, aggregator tests, output formatter tests, processor
195  tests and defaults tests.
196- **`--base-url` CLI option**: configure custom API endpoints for different LLM
197  providers. Should support any OpenAI-compatible provider.
198  - Environment variable: `LLM_AGGREGATOR_BASE_URL`
199- `internal/defaults` package
200- `docs/TESTING.md` which contains a comprehensive testing guide with test
201  descriptions
202
203### Changed
204
205- **Configuration defaults**: All packages now use `defaults.Default*` constants
206  - `internal/config`: Uses `defaults.Default*` for default values
207  - `internal/runtime`: Uses `defaults.Default*` for runtime defaults
208  - `internal/llm`: Uses `defaults.Default*` for LLM client defaults
209
210### Fixed
211
212- `nil` pointer safety in aggregator
213  - Added nil checks for `progressCtx` across all log statements
214  - `FeedAggregator` now works correctly when progress context is nil
215  - Tests use `NewFeedAggregatorWithProgress` with explicit progress context
216
217## [0.5.0] - 2026-04-22
218
219### Added
220
221- **Accurate token counting with tiktoken**
222  - New `internal/tokeniser` package using OpenAI's tiktoken-go library
223  - BPE tokenisation for accurate token counting vs rough character estimation
224  - Encoding caching to avoid repeated initialisation overhead
225  - Model-aware encoding selection (cl100k_base, o200k_base, etc.)
226  - `CountTokens()` and `CountMessagesTokens()` functions
227  - Fallback to rough estimation when tiktoken initialisation fails
228- **Concurrent feed fetching**: Feeds are now fetched concurrently using
229  `golang.org/x/sync/errgroup`
230  - Semaphore limits concurrent requests to 10 to avoid server overload
231  - Mutex-protected shared state for thread-safe article collection
232  - Partial failures don't stop other feeds from being processed
233- **Scrollable summary in TUI**
234  - Bubble Tea viewport component for browsing long summaries
235  - Keyboard navigation: j/k or arrows (scroll), Space/B (page), g/G
236  (start/end)
237  - Mouse wheel scrolling support
238  - Scroll progress indicator showing position and total lines
239
240### Changed
241
242- **Streamlined configuration management**: Viper now uses global instance with
243automatic precedence handling. Simplified configuration flow: Parse → GetViper
244→ BindCLIArgs → ViperToRuntime
245- **TUI colour scheme**: All hex colour codes replaced with ANSI 256-colour
246  palette
247
248### Fixed
249
250- Viewport now properly handles scroll events via `Update()`. Summary content
251  pre-wrapped to viewport width before display
252- Fixed string literal colour names to use actual variables
253
254## [0.4.0] - 2026-04-21
255
256### Changed 
257
258- **TUI is now fully functional and production-ready**: Replaced the
259work-in-progress TUI implementation with a robust Bubble Tea command pattern:
260  - Runtime execution now uses native `tea.Cmd` instead of manual goroutine
261  orchestration.
262  - Added real-time progress updates via `progress.Progress` interface with
263  stage, substage, and article count messages.
264  - TUI displays a spinner, elapsed time, and final summary inline.
265  - Removed signal handling and channel synchronisation in favour of Bubble
266  Tea's built-in lifecycle.
267  - `TUIProgress` now only sends messages to an existing program, simplifying
268  integration.
269  - Both verbose mode (`SimpleLogger`) and TUI mode share the same
270  `progress.Progress` interface.
271
272## [0.3.0] - 2026-04-21
273
274### Added
275
276- **Configuration file support via TOML** (XDG-compliant and cross-platform)
277  - Load settings from `~/.config/llm_aggregator/config.toml`
278  - Supports all aggregation, API, and output options
279  - Example configuration file at `configs/config.example.toml`
280- **Environment variable support with `LLM_AGGREGATOR_` prefix**
281  - All configuration options can be set via environment variables
282  - Precedence order: CLI arguments > environment variables > config file >
283  built‑in defaults
284- New `internal/config` package with Viper integration
285  - `Load()` and `Save()` methods for configuration management
286  - `GetConfigPath()` for XDG‑compliant config file location
287  - `ConfigExists()` to check for existing configuration
288- Comprehensive unit tests for configuration loading, saving, and environment
289  variable precedence (`internal/config/config_test.go`)
290
291### Changed
292
293- **Environment variable renamed:** `DEEPSEEK_API_KEY`294  `LLM_AGGREGATOR_API_KEY`
295- **CLI help text** now reflects the new API key environment variable and
296  general configuration options
297- **Command‑line argument precedence** implemented in `cmd/llm_aggregator/main.go`
298via `applyConfiguration()`
299- Updated all help text, documentation, and code references
300- `README.md` completely revised with a new "Configuration" section
301
302### Removed
303
304- Global `QuietMode` and `VerboseMode` variables from
305`internal/config/config.go`. Replaced by runtime‑specific configuration in
306`runtime.Runtime` and CLI arguments
307- Hardcoded system prompt in `runtime.NewRuntime()`. Now left empty, allowing
308configuration or client default to be used
309- CLI flags now correctly override config file and environment variables
310
311## [0.2.0] - 2026-04-21
312
313### Changed
314
315- Moved all progress and status messages behind `-v/--verbose` flag
316- Default CLI mode is now silent except for final output and errors
317- Components (aggregator, processor, LLM client) use logger interface
318controlled by verbose flag
319
320## [0.1.0] - 2026-04-21
321
322### Added
323
324- Complete translation from Python prototype to Go implementation
325- RSS/Atom feed parsing using `gofeed` library
326- DeepSeek API integration via `openai-go` client, configured for `/chat/completions` endpoint
327- Content filtering and processing with keyword‑based include/exclude, date filtering, and sorting
328- Command‑line interface using `go‑arg` with automatic help and version flags
329- Multiple output formats: plain text, GitHub‑flavoured markdown, and JSON
330- Terminal user interface (TUI) built with `bubbletea` featuring:
331  - Animated progress bar with gradient colours (`#FF7CCB` to `#FDFF8C`)
332  - Live article counters (aggregated/processed)
333  - Elapsed time display
334  - Coloured status indicators using `lipgloss` styling
335  - Keyboard controls (q/Ctrl+C to quit)
336- Web content extraction fallback using `goquery` when feed descriptions are minimal
337- Configurable limits: articles per feed, maximum age, total articles
338- Token estimation and API usage logging
339- Environment variable support (`LLM_AGGREGATOR_API_KEY`) for authentication
340- Example feeds file with technology, programming, and free software sources
341
342### Changed
343
344- Program structure organised into standard Go layout: `cmd/`, `internal/`, `pkg/`
345- English spelling conventions maintained throughout (colour, initialise, summarise)
346- Error handling improved with specific messages for common API failures (invalid key, rate limits, etc.)
347- Progress reporting unified through a `ProgressContext` interface for both TUI and CLI modes
348
349### Fixed
350
351- Initial API endpoint mismatch (using `/responses` instead of `/chat/completions`)
352- Nil pointer dereferences when handling optional feed metadata (author, dates)
353- String repetition syntax errors in Go (replaced `"="*80` with `strings.Repeat`)
354- CLI help/version flag handling to show information without requiring other arguments
355- Type compatibility issues with `openai-go` v3 API (message parameters, token usage fields)