all repos — llm_aggregator @ 29a4710e981512a82abba77ec8d7b43e413453e7

A CLI tool to aggregate RSS feeds and summarise them with LLMs

CHANGELOG.md (view raw)

  1# Changelog
  2
  3All notable changes to this project will be documented in this file.
  4
  5The format is based on [Keep a
  6Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
  7[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  8
  9## [0.15.0] - 2026-04-27
 10
 11### Added
 12
 13- **`--timeout N` flag**: LLM request timeout in seconds (default: 300). If the
 14  LLM API call takes longer than this, the request is aborted and a descriptive
 15  error is returned. The timeout is implemented as a `context.WithTimeout`
 16  derived from the signal-handling context, so both signal interrupts and
 17  timeouts abort the call.
 18- **`DefaultLLMTimeout` constant**: Centralises the default timeout value
 19  (300 seconds) in `internal/defaults`.
 20- **LLM client unit tests**: Three tests for the constructor: timeout default,
 21  all-parameter defaults, and API key validation.
 22
 23### Changed
 24
 25- **`LLMClient.llmTimeout` field**: Renamed from `timeoutSeconds` for clarity.
 26  The field stores the timeout in seconds; 0 means no timeout.
 27
 28## [0.14.0] - 2026-04-27
 29
 30### Added
 31
 32- **Signal handling for graceful shutdown**: `llm_aggregator` now handles
 33  `SIGINT`, `SIGTERM`, and `SIGHUP`. When a signal arrives during execution:
 34  - The context is cancelled, stopping in-progress operations as soon as possible
 35  - Any partial LLM summary already received is written to stdout (or the output file)
 36  - The process exits with code 130 (128 + signal number) instead of a generic error
 37  - In TUI mode, `q`/`Ctrl+C` also triggers clean shutdown via `RequestExit()`
 38- **`Runtime.Interrupted` and `Runtime.WasInterrupted()`**: New state field and
 39  query method allow callers to detect signal-caused termination.
 40
 41## [0.13.0] - 2026-04-26
 42
 43### Added
 44
 45- **`--stdin` flag**: Read raw RSS/Atom XML directly from standard input,
 46  composable with `-f/--feeds-file` to collate articles from both sources into
 47  a single LLM summary. Stdin articles appear before feeds-file articles in the
 48  collated output.
 49  - `curl -s $URL | llm_aggregator --stdin -p "Summarise"`
 50  - `llm_aggregator -f feeds.txt --stdin -p "Summarise all"`
 51  - Environment variable: `LLM_AGGREGATOR_STDIN`
 52- **`--feeds-file` is no longer required**: Either `--feeds-file` or `--stdin`
 53  (or both) must be provided. A clear error is returned if neither is specified.
 54
 55### Changed
 56
 57- Refactored `ParseFeedsFromFile` to delegate to a new `ParseFeedFromReader`
 58  method that accepts any `io.Reader`, enabling both stdin and unit test usage
 59  with `strings.Reader`.
 60
 61## [0.12.0] - 2026-04-26
 62
 63### Added
 64
 65- **`-P/--plain` flag**: Output the raw LLM summary without any formatting or
 66  metadata. Useful when the summary is to be consumed by downstream scripts.
 67
 68## [0.11.0] - 2026-04-23
 69
 70- Progress bar now tracks estimated token count (via tiktoken) before sending
 71  to LLM, then updates with actual completion tokens from the API response.
 72  Gives a more accurate picture of how much content is being sent.
 73
 74## [0.10.0] - 2026-04-23
 75
 76### Added
 77
 78- **Short command-line flags**: Single-character shortcuts for the most
 79  commonly used options:
 80  - `-f` / `--feeds-file` — feeds file path
 81  - `-p` / `--prompt` — summarisation prompt
 82  - `-n` / `--max-articles-per-feed` — articles per feed limit
 83  - `-d` / `--max-days-old` — article age filter
 84  - `-i` / `--include-keywords` — keyword include filter
 85  - `-e` / `--exclude-keywords` — keyword exclude filter
 86  - `-m` / `--model` — LLM model selection
 87  - `-o` / `--output` — output format
 88  - `-t` / `--tui` — enable TUI
 89  - `-D` / `--dry-run` — dry run mode
 90- **Styled help output**: Custom lipgloss-styled help text at
 91  `internal/cli/help.go` replaces go-arg's default plain-text output. Flags are
 92  rendered in cyan, section headers in bold, with consistent column alignment.
 93  Respects `$NO_COLOR`.
 94
 95## [0.9.0] - 2026-04-23
 96
 97### Added
 98
 99- **Markdown rendering in TUI**: Implemented `charmbracelet/glamour` to
100  render LLM summary output with proper markdown styling. Headers, bold,
101  italic, code blocks, lists, and other markdown elements are now styled
102  for better readability in the terminal.
103- **Formalised infobar keybinds**: TUI keybinds are now defined as
104  dedicated style variables at the top of the model for easier maintenance.
105  Keybinds are bold while separators and action text are subtle but
106  distinguishable.
107- **Thinking indicator as separate element**: The `[💭 Thinking: ON/OFF]`
108  toggle indicator is now on its own line above the keybinds, using a
109  distinct style (magenta+bold when ON, subtle+italic when OFF).
110- **Dynamic text wrapping on resize**: TUI summary text now re-wraps when
111  the terminal window size or font changes, with proper markdown re-rendering.
112
113## [0.8.0] - 2026-04-23
114
115### Added
116
117- **Thinking tags toggle in TUI**: LLMs that output `<think>`/`</think>`
118  thinking tags now have those sections automatically removed from the summary
119  display by default. Users can press `t` to toggle the thinking section back
120  on, displayed in gray at the head of the output with XML tags stripped for
121  cleaner presentation. A status indicator shows whether thinking is currently
122  visible.
123
124## [0.7.1] - 2026-04-22
125
126### Fixed
127
128- Environment variables were correctly overriding config file values, but CLI
129  arguments without explicit flags were also overriding config file values
130  unintentionally.
131  - Root cause: `go-arg` was populating struct fields with default values
132    from `default:` tags even when those CLI flags weren't provided
133  - Fix: Removed `default:` tags from Args struct fields for LLM options
134    (`model`, `base-url`, `api-key`, `max-tokens`, `temperature`);
135    Viper's own default handling is sufficient
136  - Fix: Changed Args struct fields from value types to pointer types
137    (`*string`, `*int`, `*float64`) so that "not provided on CLI" can be
138    distinguished from "provided but zero/empty"
139  - Fix: Updated `BindCLIArgs()` and `isZero()` to properly handle pointer
140    types and distinguish nil pointers from zero values
141- `isZero()` with compound type switch cases: Compound type cases like `case
142  *int, *int8, *int16, *int32, *int64` don't properly handle nil comparison when
143  stored in an interface variable. Fixed by splitting into individual type cases.
144- `isZero()` missing nil interface handling: When a nil pointer is stored
145  in an `any` interface, `value != nil` returns `true` (the interface itself
146  is not nil). Added early `if v == nil { return true }` check.
147
148## [0.7.0] - 2026-04-22
149
150### Added
151
152- **`--dry-run` option**: Validate configuration and preview what would happen
153  without making any LLM API calls. Useful for validating feeds file and
154  configuration, checking article counts and filtering results and anything
155  else that can be done "offline".
156- **Styled terminal output using `lipgloss`**: All CLI output now uses ANSI
157  colour codes for better readability:
158- **`$NO_COLOR` environment variable support**: Respects the ["no
159color"](https://no-color.org/) standard. When `NO_COLOR` is set, all styling is
160disabled and plain text is output instead.
161
162### Changed
163
164- API key validation now only occurs when not using `--dry-run` mode
165
166## [0.6.0] - 2026-04-22
167
168### Added
169
170- **Comprehensive unit test suite**, including CLI argument tests
171  (`--feeds-file`, `--prompt`, `--api-key`, `--model`, `--base-url`),
172  configuration tests, aggregator tests, output formatter tests, processor
173  tests and defaults tests.
174- **`--base-url` CLI option**: configure custom API endpoints for different LLM
175  providers. Should support any OpenAI-compatible provider.
176  - Environment variable: `LLM_AGGREGATOR_BASE_URL`
177- `internal/defaults` package
178- `docs/TESTING.md` which contains a comprehensive testing guide with test
179  descriptions
180
181### Changed
182
183- **Configuration defaults**: All packages now use `defaults.Default*` constants
184  - `internal/config`: Uses `defaults.Default*` for default values
185  - `internal/runtime`: Uses `defaults.Default*` for runtime defaults
186  - `internal/llm`: Uses `defaults.Default*` for LLM client defaults
187
188### Fixed
189
190- `nil` pointer safety in aggregator
191  - Added nil checks for `progressCtx` across all log statements
192  - `FeedAggregator` now works correctly when progress context is nil
193  - Tests use `NewFeedAggregatorWithProgress` with explicit progress context
194
195## [0.5.0] - 2026-04-22
196
197### Added
198
199- **Accurate token counting with tiktoken**
200  - New `internal/tokeniser` package using OpenAI's tiktoken-go library
201  - BPE tokenisation for accurate token counting vs rough character estimation
202  - Encoding caching to avoid repeated initialisation overhead
203  - Model-aware encoding selection (cl100k_base, o200k_base, etc.)
204  - `CountTokens()` and `CountMessagesTokens()` functions
205  - Fallback to rough estimation when tiktoken initialisation fails
206- **Concurrent feed fetching**: Feeds are now fetched concurrently using
207  `golang.org/x/sync/errgroup`
208  - Semaphore limits concurrent requests to 10 to avoid server overload
209  - Mutex-protected shared state for thread-safe article collection
210  - Partial failures don't stop other feeds from being processed
211- **Scrollable summary in TUI**
212  - Bubble Tea viewport component for browsing long summaries
213  - Keyboard navigation: j/k or arrows (scroll), Space/B (page), g/G
214  (start/end)
215  - Mouse wheel scrolling support
216  - Scroll progress indicator showing position and total lines
217
218### Changed
219
220- **Streamlined configuration management**: Viper now uses global instance with
221automatic precedence handling. Simplified configuration flow: Parse → GetViper
222→ BindCLIArgs → ViperToRuntime
223- **TUI colour scheme**: All hex colour codes replaced with ANSI 256-colour
224  palette
225
226### Fixed
227
228- Viewport now properly handles scroll events via `Update()`. Summary content
229  pre-wrapped to viewport width before display
230- Fixed string literal colour names to use actual variables
231
232## [0.4.0] - 2026-04-21
233
234### Changed 
235
236- **TUI is now fully functional and production-ready**: Replaced the
237work-in-progress TUI implementation with a robust Bubble Tea command pattern:
238  - Runtime execution now uses native `tea.Cmd` instead of manual goroutine
239  orchestration.
240  - Added real-time progress updates via `progress.Progress` interface with
241  stage, substage, and article count messages.
242  - TUI displays a spinner, elapsed time, and final summary inline.
243  - Removed signal handling and channel synchronisation in favour of Bubble
244  Tea's built-in lifecycle.
245  - `TUIProgress` now only sends messages to an existing program, simplifying
246  integration.
247  - Both verbose mode (`SimpleLogger`) and TUI mode share the same
248  `progress.Progress` interface.
249
250## [0.3.0] - 2026-04-21
251
252### Added
253
254- **Configuration file support via TOML** (XDG-compliant and cross-platform)
255  - Load settings from `~/.config/llm_aggregator/config.toml`
256  - Supports all aggregation, API, and output options
257  - Example configuration file at `configs/config.example.toml`
258- **Environment variable support with `LLM_AGGREGATOR_` prefix**
259  - All configuration options can be set via environment variables
260  - Precedence order: CLI arguments > environment variables > config file >
261  built‑in defaults
262- New `internal/config` package with Viper integration
263  - `Load()` and `Save()` methods for configuration management
264  - `GetConfigPath()` for XDG‑compliant config file location
265  - `ConfigExists()` to check for existing configuration
266- Comprehensive unit tests for configuration loading, saving, and environment
267  variable precedence (`internal/config/config_test.go`)
268
269### Changed
270
271- **Environment variable renamed:** `DEEPSEEK_API_KEY`272  `LLM_AGGREGATOR_API_KEY`
273- **CLI help text** now reflects the new API key environment variable and
274  general configuration options
275- **Command‑line argument precedence** implemented in `cmd/llm_aggregator.go`
276via `applyConfiguration()`
277- Updated all help text, documentation, and code references
278- `README.md` completely revised with a new "Configuration" section
279
280### Removed
281
282- Global `QuietMode` and `VerboseMode` variables from
283`internal/config/config.go`. Replaced by runtime‑specific configuration in
284`runtime.Runtime` and CLI arguments
285- Hardcoded system prompt in `runtime.NewRuntime()`. Now left empty, allowing
286configuration or client default to be used
287- CLI flags now correctly override config file and environment variables
288
289## [0.2.0] - 2026-04-21
290
291### Changed
292
293- Moved all progress and status messages behind `-v/--verbose` flag
294- Default CLI mode is now silent except for final output and errors
295- Components (aggregator, processor, LLM client) use logger interface
296controlled by verbose flag
297
298## [0.1.0] - 2026-04-21
299
300### Added
301
302- Complete translation from Python prototype to Go implementation
303- RSS/Atom feed parsing using `gofeed` library
304- DeepSeek API integration via `openai-go` client, configured for `/chat/completions` endpoint
305- Content filtering and processing with keyword‑based include/exclude, date filtering, and sorting
306- Command‑line interface using `go‑arg` with automatic help and version flags
307- Multiple output formats: plain text, GitHub‑flavoured markdown, and JSON
308- Terminal user interface (TUI) built with `bubbletea` featuring:
309  - Animated progress bar with gradient colours (`#FF7CCB` to `#FDFF8C`)
310  - Live article counters (aggregated/processed)
311  - Elapsed time display
312  - Coloured status indicators using `lipgloss` styling
313  - Keyboard controls (q/Ctrl+C to quit)
314- Web content extraction fallback using `goquery` when feed descriptions are minimal
315- Configurable limits: articles per feed, maximum age, total articles
316- Token estimation and API usage logging
317- Environment variable support (`LLM_AGGREGATOR_API_KEY`) for authentication
318- Example feeds file with technology, programming, and free software sources
319
320### Changed
321
322- Program structure organised into standard Go layout: `cmd/`, `internal/`, `pkg/`
323- English spelling conventions maintained throughout (colour, initialise, summarise)
324- Error handling improved with specific messages for common API failures (invalid key, rate limits, etc.)
325- Progress reporting unified through a `ProgressContext` interface for both TUI and CLI modes
326
327### Fixed
328
329- Initial API endpoint mismatch (using `/responses` instead of `/chat/completions`)
330- Nil pointer dereferences when handling optional feed metadata (author, dates)
331- String repetition syntax errors in Go (replaced `"="*80` with `strings.Repeat`)
332- CLI help/version flag handling to show information without requiring other arguments
333- Type compatibility issues with `openai-go` v3 API (message parameters, token usage fields)