all repos — llm_aggregator @ d3fad3374aee5ee549219c1387a0c5d87ce1a99b

A CLI tool to aggregate RSS feeds and summarise them with LLMs

CHANGELOG.md (view raw)

  1# Changelog
  2
  3All notable changes to this project will be documented in this file.
  4
  5The format is based on [Keep a
  6Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
  7[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  8
  9## [Unreleased]
 10
 11### Added
 12
 13- Linting using `golangci-lint`, mostly  for readability and maintainability.
 14
 15### Fixed
 16
 17- Fixed all lint issues across the codebase, including unchecked error returns,
 18  string formatting performance, dead code, and documentation style.
 19
 20## [0.15.0] - 2026-04-27
 21
 22### Added
 23
 24- **`--timeout N` flag**: LLM request timeout in seconds (default: 300). If the
 25  LLM API call takes longer than this, the request is aborted and a descriptive
 26  error is returned. The timeout is implemented as a `context.WithTimeout`
 27  derived from the signal-handling context, so both signal interrupts and
 28  timeouts abort the call.
 29- **`DefaultLLMTimeout` constant**: Centralises the default timeout value
 30  (300 seconds) in `internal/defaults`.
 31- **LLM client unit tests**: Three tests for the constructor: timeout default,
 32  all-parameter defaults, and API key validation.
 33
 34### Changed
 35
 36- **`LLMClient.llmTimeout` field**: Renamed from `timeoutSeconds` for clarity.
 37  The field stores the timeout in seconds; 0 means no timeout.
 38
 39## [0.14.0] - 2026-04-27
 40
 41### Added
 42
 43- **Signal handling for graceful shutdown**: `llm_aggregator` now handles
 44  `SIGINT`, `SIGTERM`, and `SIGHUP`. When a signal arrives during execution:
 45  - The context is cancelled, stopping in-progress operations as soon as possible
 46  - Any partial LLM summary already received is written to stdout (or the output file)
 47  - The process exits with code 130 (128 + signal number) instead of a generic error
 48  - In TUI mode, `q`/`Ctrl+C` also triggers clean shutdown via `RequestExit()`
 49- **`Runtime.Interrupted` and `Runtime.WasInterrupted()`**: New state field and
 50  query method allow callers to detect signal-caused termination.
 51
 52## [0.13.0] - 2026-04-26
 53
 54### Added
 55
 56- **`--stdin` flag**: Read raw RSS/Atom XML directly from standard input,
 57  composable with `-f/--feeds-file` to collate articles from both sources into
 58  a single LLM summary. Stdin articles appear before feeds-file articles in the
 59  collated output.
 60  - `curl -s $URL | llm_aggregator --stdin -p "Summarise"`
 61  - `llm_aggregator -f feeds.txt --stdin -p "Summarise all"`
 62  - Environment variable: `LLM_AGGREGATOR_STDIN`
 63- **`--feeds-file` is no longer required**: Either `--feeds-file` or `--stdin`
 64  (or both) must be provided. A clear error is returned if neither is specified.
 65
 66### Changed
 67
 68- Refactored `ParseFeedsFromFile` to delegate to a new `ParseFeedFromReader`
 69  method that accepts any `io.Reader`, enabling both stdin and unit test usage
 70  with `strings.Reader`.
 71
 72## [0.12.0] - 2026-04-26
 73
 74### Added
 75
 76- **`-P/--plain` flag**: Output the raw LLM summary without any formatting or
 77  metadata. Useful when the summary is to be consumed by downstream scripts.
 78
 79## [0.11.0] - 2026-04-23
 80
 81- Progress bar now tracks estimated token count (via tiktoken) before sending
 82  to LLM, then updates with actual completion tokens from the API response.
 83  Gives a more accurate picture of how much content is being sent.
 84
 85## [0.10.0] - 2026-04-23
 86
 87### Added
 88
 89- **Short command-line flags**: Single-character shortcuts for the most
 90  commonly used options:
 91  - `-f` / `--feeds-file` — feeds file path
 92  - `-p` / `--prompt` — summarisation prompt
 93  - `-n` / `--max-articles-per-feed` — articles per feed limit
 94  - `-d` / `--max-days-old` — article age filter
 95  - `-i` / `--include-keywords` — keyword include filter
 96  - `-e` / `--exclude-keywords` — keyword exclude filter
 97  - `-m` / `--model` — LLM model selection
 98  - `-o` / `--output` — output format
 99  - `-t` / `--tui` — enable TUI
100  - `-D` / `--dry-run` — dry run mode
101- **Styled help output**: Custom lipgloss-styled help text at
102  `internal/cli/help.go` replaces go-arg's default plain-text output. Flags are
103  rendered in cyan, section headers in bold, with consistent column alignment.
104  Respects `$NO_COLOR`.
105
106## [0.9.0] - 2026-04-23
107
108### Added
109
110- **Markdown rendering in TUI**: Implemented `charmbracelet/glamour` to
111  render LLM summary output with proper markdown styling. Headers, bold,
112  italic, code blocks, lists, and other markdown elements are now styled
113  for better readability in the terminal.
114- **Formalised infobar keybinds**: TUI keybinds are now defined as
115  dedicated style variables at the top of the model for easier maintenance.
116  Keybinds are bold while separators and action text are subtle but
117  distinguishable.
118- **Thinking indicator as separate element**: The `[💭 Thinking: ON/OFF]`
119  toggle indicator is now on its own line above the keybinds, using a
120  distinct style (magenta+bold when ON, subtle+italic when OFF).
121- **Dynamic text wrapping on resize**: TUI summary text now re-wraps when
122  the terminal window size or font changes, with proper markdown re-rendering.
123
124## [0.8.0] - 2026-04-23
125
126### Added
127
128- **Thinking tags toggle in TUI**: LLMs that output `<think>`/`</think>`
129  thinking tags now have those sections automatically removed from the summary
130  display by default. Users can press `t` to toggle the thinking section back
131  on, displayed in gray at the head of the output with XML tags stripped for
132  cleaner presentation. A status indicator shows whether thinking is currently
133  visible.
134
135## [0.7.1] - 2026-04-22
136
137### Fixed
138
139- Environment variables were correctly overriding config file values, but CLI
140  arguments without explicit flags were also overriding config file values
141  unintentionally.
142  - Root cause: `go-arg` was populating struct fields with default values
143    from `default:` tags even when those CLI flags weren't provided
144  - Fix: Removed `default:` tags from Args struct fields for LLM options
145    (`model`, `base-url`, `api-key`, `max-tokens`, `temperature`);
146    Viper's own default handling is sufficient
147  - Fix: Changed Args struct fields from value types to pointer types
148    (`*string`, `*int`, `*float64`) so that "not provided on CLI" can be
149    distinguished from "provided but zero/empty"
150  - Fix: Updated `BindCLIArgs()` and `isZero()` to properly handle pointer
151    types and distinguish nil pointers from zero values
152- `isZero()` with compound type switch cases: Compound type cases like `case
153  *int, *int8, *int16, *int32, *int64` don't properly handle nil comparison when
154  stored in an interface variable. Fixed by splitting into individual type cases.
155- `isZero()` missing nil interface handling: When a nil pointer is stored
156  in an `any` interface, `value != nil` returns `true` (the interface itself
157  is not nil). Added early `if v == nil { return true }` check.
158
159## [0.7.0] - 2026-04-22
160
161### Added
162
163- **`--dry-run` option**: Validate configuration and preview what would happen
164  without making any LLM API calls. Useful for validating feeds file and
165  configuration, checking article counts and filtering results and anything
166  else that can be done "offline".
167- **Styled terminal output using `lipgloss`**: All CLI output now uses ANSI
168  colour codes for better readability:
169- **`$NO_COLOR` environment variable support**: Respects the ["no
170color"](https://no-color.org/) standard. When `NO_COLOR` is set, all styling is
171disabled and plain text is output instead.
172
173### Changed
174
175- API key validation now only occurs when not using `--dry-run` mode
176
177## [0.6.0] - 2026-04-22
178
179### Added
180
181- **Comprehensive unit test suite**, including CLI argument tests
182  (`--feeds-file`, `--prompt`, `--api-key`, `--model`, `--base-url`),
183  configuration tests, aggregator tests, output formatter tests, processor
184  tests and defaults tests.
185- **`--base-url` CLI option**: configure custom API endpoints for different LLM
186  providers. Should support any OpenAI-compatible provider.
187  - Environment variable: `LLM_AGGREGATOR_BASE_URL`
188- `internal/defaults` package
189- `docs/TESTING.md` which contains a comprehensive testing guide with test
190  descriptions
191
192### Changed
193
194- **Configuration defaults**: All packages now use `defaults.Default*` constants
195  - `internal/config`: Uses `defaults.Default*` for default values
196  - `internal/runtime`: Uses `defaults.Default*` for runtime defaults
197  - `internal/llm`: Uses `defaults.Default*` for LLM client defaults
198
199### Fixed
200
201- `nil` pointer safety in aggregator
202  - Added nil checks for `progressCtx` across all log statements
203  - `FeedAggregator` now works correctly when progress context is nil
204  - Tests use `NewFeedAggregatorWithProgress` with explicit progress context
205
206## [0.5.0] - 2026-04-22
207
208### Added
209
210- **Accurate token counting with tiktoken**
211  - New `internal/tokeniser` package using OpenAI's tiktoken-go library
212  - BPE tokenisation for accurate token counting vs rough character estimation
213  - Encoding caching to avoid repeated initialisation overhead
214  - Model-aware encoding selection (cl100k_base, o200k_base, etc.)
215  - `CountTokens()` and `CountMessagesTokens()` functions
216  - Fallback to rough estimation when tiktoken initialisation fails
217- **Concurrent feed fetching**: Feeds are now fetched concurrently using
218  `golang.org/x/sync/errgroup`
219  - Semaphore limits concurrent requests to 10 to avoid server overload
220  - Mutex-protected shared state for thread-safe article collection
221  - Partial failures don't stop other feeds from being processed
222- **Scrollable summary in TUI**
223  - Bubble Tea viewport component for browsing long summaries
224  - Keyboard navigation: j/k or arrows (scroll), Space/B (page), g/G
225  (start/end)
226  - Mouse wheel scrolling support
227  - Scroll progress indicator showing position and total lines
228
229### Changed
230
231- **Streamlined configuration management**: Viper now uses global instance with
232automatic precedence handling. Simplified configuration flow: Parse → GetViper
233→ BindCLIArgs → ViperToRuntime
234- **TUI colour scheme**: All hex colour codes replaced with ANSI 256-colour
235  palette
236
237### Fixed
238
239- Viewport now properly handles scroll events via `Update()`. Summary content
240  pre-wrapped to viewport width before display
241- Fixed string literal colour names to use actual variables
242
243## [0.4.0] - 2026-04-21
244
245### Changed 
246
247- **TUI is now fully functional and production-ready**: Replaced the
248work-in-progress TUI implementation with a robust Bubble Tea command pattern:
249  - Runtime execution now uses native `tea.Cmd` instead of manual goroutine
250  orchestration.
251  - Added real-time progress updates via `progress.Progress` interface with
252  stage, substage, and article count messages.
253  - TUI displays a spinner, elapsed time, and final summary inline.
254  - Removed signal handling and channel synchronisation in favour of Bubble
255  Tea's built-in lifecycle.
256  - `TUIProgress` now only sends messages to an existing program, simplifying
257  integration.
258  - Both verbose mode (`SimpleLogger`) and TUI mode share the same
259  `progress.Progress` interface.
260
261## [0.3.0] - 2026-04-21
262
263### Added
264
265- **Configuration file support via TOML** (XDG-compliant and cross-platform)
266  - Load settings from `~/.config/llm_aggregator/config.toml`
267  - Supports all aggregation, API, and output options
268  - Example configuration file at `configs/config.example.toml`
269- **Environment variable support with `LLM_AGGREGATOR_` prefix**
270  - All configuration options can be set via environment variables
271  - Precedence order: CLI arguments > environment variables > config file >
272  built‑in defaults
273- New `internal/config` package with Viper integration
274  - `Load()` and `Save()` methods for configuration management
275  - `GetConfigPath()` for XDG‑compliant config file location
276  - `ConfigExists()` to check for existing configuration
277- Comprehensive unit tests for configuration loading, saving, and environment
278  variable precedence (`internal/config/config_test.go`)
279
280### Changed
281
282- **Environment variable renamed:** `DEEPSEEK_API_KEY`283  `LLM_AGGREGATOR_API_KEY`
284- **CLI help text** now reflects the new API key environment variable and
285  general configuration options
286- **Command‑line argument precedence** implemented in `cmd/llm_aggregator.go`
287via `applyConfiguration()`
288- Updated all help text, documentation, and code references
289- `README.md` completely revised with a new "Configuration" section
290
291### Removed
292
293- Global `QuietMode` and `VerboseMode` variables from
294`internal/config/config.go`. Replaced by runtime‑specific configuration in
295`runtime.Runtime` and CLI arguments
296- Hardcoded system prompt in `runtime.NewRuntime()`. Now left empty, allowing
297configuration or client default to be used
298- CLI flags now correctly override config file and environment variables
299
300## [0.2.0] - 2026-04-21
301
302### Changed
303
304- Moved all progress and status messages behind `-v/--verbose` flag
305- Default CLI mode is now silent except for final output and errors
306- Components (aggregator, processor, LLM client) use logger interface
307controlled by verbose flag
308
309## [0.1.0] - 2026-04-21
310
311### Added
312
313- Complete translation from Python prototype to Go implementation
314- RSS/Atom feed parsing using `gofeed` library
315- DeepSeek API integration via `openai-go` client, configured for `/chat/completions` endpoint
316- Content filtering and processing with keyword‑based include/exclude, date filtering, and sorting
317- Command‑line interface using `go‑arg` with automatic help and version flags
318- Multiple output formats: plain text, GitHub‑flavoured markdown, and JSON
319- Terminal user interface (TUI) built with `bubbletea` featuring:
320  - Animated progress bar with gradient colours (`#FF7CCB` to `#FDFF8C`)
321  - Live article counters (aggregated/processed)
322  - Elapsed time display
323  - Coloured status indicators using `lipgloss` styling
324  - Keyboard controls (q/Ctrl+C to quit)
325- Web content extraction fallback using `goquery` when feed descriptions are minimal
326- Configurable limits: articles per feed, maximum age, total articles
327- Token estimation and API usage logging
328- Environment variable support (`LLM_AGGREGATOR_API_KEY`) for authentication
329- Example feeds file with technology, programming, and free software sources
330
331### Changed
332
333- Program structure organised into standard Go layout: `cmd/`, `internal/`, `pkg/`
334- English spelling conventions maintained throughout (colour, initialise, summarise)
335- Error handling improved with specific messages for common API failures (invalid key, rate limits, etc.)
336- Progress reporting unified through a `ProgressContext` interface for both TUI and CLI modes
337
338### Fixed
339
340- Initial API endpoint mismatch (using `/responses` instead of `/chat/completions`)
341- Nil pointer dereferences when handling optional feed metadata (author, dates)
342- String repetition syntax errors in Go (replaced `"="*80` with `strings.Repeat`)
343- CLI help/version flag handling to show information without requiring other arguments
344- Type compatibility issues with `openai-go` v3 API (message parameters, token usage fields)