all repos — llm_aggregator @ a0e7d1a442a251ae94645cdddccf30ded92e97a1

A CLI tool to aggregate RSS feeds and summarise them with LLMs

CHANGELOG.md (view raw)

  1# Changelog
  2
  3All notable changes to this project will be documented in this file.
  4
  5The format is based on [Keep a
  6Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
  7[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  8
  9## [0.13.0] - 2026-04-26
 10
 11### Added
 12
 13- **`--stdin` flag**: Read raw RSS/Atom XML directly from standard input,
 14  composable with `-f/--feeds-file` to collate articles from both sources into
 15  a single LLM summary. Stdin articles appear before feeds-file articles in the
 16  collated output.
 17  - `curl -s $URL | llm_aggregator --stdin -p "Summarise"`
 18  - `llm_aggregator -f feeds.txt --stdin -p "Summarise all"`
 19  - Environment variable: `LLM_AGGREGATOR_STDIN`
 20- **`--feeds-file` is no longer required**: Either `--feeds-file` or `--stdin`
 21  (or both) must be provided. A clear error is returned if neither is specified.
 22
 23### Changed
 24
 25- Refactored `ParseFeedsFromFile` to delegate to a new `ParseFeedFromReader`
 26  method that accepts any `io.Reader`, enabling both stdin and unit test usage
 27  with `strings.Reader`.
 28
 29## [0.12.0] - 2026-04-26
 30
 31### Added
 32
 33- **`-P/--plain` flag**: Output the raw LLM summary without any formatting or
 34  metadata. Useful when the summary is to be consumed by downstream scripts.
 35
 36## [0.11.0] - 2026-04-23
 37
 38- Progress bar now tracks estimated token count (via tiktoken) before sending
 39  to LLM, then updates with actual completion tokens from the API response.
 40  Gives a more accurate picture of how much content is being sent.
 41
 42## [0.10.0] - 2026-04-23
 43
 44### Added
 45
 46- **Short command-line flags**: Single-character shortcuts for the most
 47  commonly used options:
 48  - `-f` / `--feeds-file` — feeds file path
 49  - `-p` / `--prompt` — summarisation prompt
 50  - `-n` / `--max-articles-per-feed` — articles per feed limit
 51  - `-d` / `--max-days-old` — article age filter
 52  - `-i` / `--include-keywords` — keyword include filter
 53  - `-e` / `--exclude-keywords` — keyword exclude filter
 54  - `-m` / `--model` — LLM model selection
 55  - `-o` / `--output` — output format
 56  - `-t` / `--tui` — enable TUI
 57  - `-D` / `--dry-run` — dry run mode
 58- **Styled help output**: Custom lipgloss-styled help text at
 59  `internal/cli/help.go` replaces go-arg's default plain-text output. Flags are
 60  rendered in cyan, section headers in bold, with consistent column alignment.
 61  Respects `$NO_COLOR`.
 62
 63## [0.9.0] - 2026-04-23
 64
 65### Added
 66
 67- **Markdown rendering in TUI**: Implemented `charmbracelet/glamour` to
 68  render LLM summary output with proper markdown styling. Headers, bold,
 69  italic, code blocks, lists, and other markdown elements are now styled
 70  for better readability in the terminal.
 71- **Formalised infobar keybinds**: TUI keybinds are now defined as
 72  dedicated style variables at the top of the model for easier maintenance.
 73  Keybinds are bold while separators and action text are subtle but
 74  distinguishable.
 75- **Thinking indicator as separate element**: The `[💭 Thinking: ON/OFF]`
 76  toggle indicator is now on its own line above the keybinds, using a
 77  distinct style (magenta+bold when ON, subtle+italic when OFF).
 78- **Dynamic text wrapping on resize**: TUI summary text now re-wraps when
 79  the terminal window size or font changes, with proper markdown re-rendering.
 80
 81## [0.8.0] - 2026-04-23
 82
 83### Added
 84
 85- **Thinking tags toggle in TUI**: LLMs that output `<think>`/`</think>`
 86  thinking tags now have those sections automatically removed from the summary
 87  display by default. Users can press `t` to toggle the thinking section back
 88  on, displayed in gray at the head of the output with XML tags stripped for
 89  cleaner presentation. A status indicator shows whether thinking is currently
 90  visible.
 91
 92## [0.7.1] - 2026-04-22
 93
 94### Fixed
 95
 96- Environment variables were correctly overriding config file values, but CLI
 97  arguments without explicit flags were also overriding config file values
 98  unintentionally.
 99  - Root cause: `go-arg` was populating struct fields with default values
100    from `default:` tags even when those CLI flags weren't provided
101  - Fix: Removed `default:` tags from Args struct fields for LLM options
102    (`model`, `base-url`, `api-key`, `max-tokens`, `temperature`);
103    Viper's own default handling is sufficient
104  - Fix: Changed Args struct fields from value types to pointer types
105    (`*string`, `*int`, `*float64`) so that "not provided on CLI" can be
106    distinguished from "provided but zero/empty"
107  - Fix: Updated `BindCLIArgs()` and `isZero()` to properly handle pointer
108    types and distinguish nil pointers from zero values
109- `isZero()` with compound type switch cases: Compound type cases like `case
110  *int, *int8, *int16, *int32, *int64` don't properly handle nil comparison when
111  stored in an interface variable. Fixed by splitting into individual type cases.
112- `isZero()` missing nil interface handling: When a nil pointer is stored
113  in an `any` interface, `value != nil` returns `true` (the interface itself
114  is not nil). Added early `if v == nil { return true }` check.
115
116## [0.7.0] - 2026-04-22
117
118### Added
119
120- **`--dry-run` option**: Validate configuration and preview what would happen
121  without making any LLM API calls. Useful for validating feeds file and
122  configuration, checking article counts and filtering results and anything
123  else that can be done "offline".
124- **Styled terminal output using `lipgloss`**: All CLI output now uses ANSI
125  colour codes for better readability:
126- **`$NO_COLOR` environment variable support**: Respects the ["no
127color"](https://no-color.org/) standard. When `NO_COLOR` is set, all styling is
128disabled and plain text is output instead.
129
130### Changed
131
132- API key validation now only occurs when not using `--dry-run` mode
133
134## [0.6.0] - 2026-04-22
135
136### Added
137
138- **Comprehensive unit test suite**, including CLI argument tests
139  (`--feeds-file`, `--prompt`, `--api-key`, `--model`, `--base-url`),
140  configuration tests, aggregator tests, output formatter tests, processor
141  tests and defaults tests.
142- **`--base-url` CLI option**: configure custom API endpoints for different LLM
143  providers. Should support any OpenAI-compatible provider.
144  - Environment variable: `LLM_AGGREGATOR_BASE_URL`
145- `internal/defaults` package
146- `docs/TESTING.md` which contains a comprehensive testing guide with test
147  descriptions
148
149### Changed
150
151- **Configuration defaults**: All packages now use `defaults.Default*` constants
152  - `internal/config`: Uses `defaults.Default*` for default values
153  - `internal/runtime`: Uses `defaults.Default*` for runtime defaults
154  - `internal/llm`: Uses `defaults.Default*` for LLM client defaults
155
156### Fixed
157
158- `nil` pointer safety in aggregator
159  - Added nil checks for `progressCtx` across all log statements
160  - `FeedAggregator` now works correctly when progress context is nil
161  - Tests use `NewFeedAggregatorWithProgress` with explicit progress context
162
163## [0.5.0] - 2026-04-22
164
165### Added
166
167- **Accurate token counting with tiktoken**
168  - New `internal/tokeniser` package using OpenAI's tiktoken-go library
169  - BPE tokenisation for accurate token counting vs rough character estimation
170  - Encoding caching to avoid repeated initialisation overhead
171  - Model-aware encoding selection (cl100k_base, o200k_base, etc.)
172  - `CountTokens()` and `CountMessagesTokens()` functions
173  - Fallback to rough estimation when tiktoken initialisation fails
174- **Concurrent feed fetching**: Feeds are now fetched concurrently using
175  `golang.org/x/sync/errgroup`
176  - Semaphore limits concurrent requests to 10 to avoid server overload
177  - Mutex-protected shared state for thread-safe article collection
178  - Partial failures don't stop other feeds from being processed
179- **Scrollable summary in TUI**
180  - Bubble Tea viewport component for browsing long summaries
181  - Keyboard navigation: j/k or arrows (scroll), Space/B (page), g/G
182  (start/end)
183  - Mouse wheel scrolling support
184  - Scroll progress indicator showing position and total lines
185
186### Changed
187
188- **Streamlined configuration management**: Viper now uses global instance with
189automatic precedence handling. Simplified configuration flow: Parse → GetViper
190→ BindCLIArgs → ViperToRuntime
191- **TUI colour scheme**: All hex colour codes replaced with ANSI 256-colour
192  palette
193
194### Fixed
195
196- Viewport now properly handles scroll events via `Update()`. Summary content
197  pre-wrapped to viewport width before display
198- Fixed string literal colour names to use actual variables
199
200## [0.4.0] - 2026-04-21
201
202### Changed 
203
204- **TUI is now fully functional and production-ready**: Replaced the
205work-in-progress TUI implementation with a robust Bubble Tea command pattern:
206  - Runtime execution now uses native `tea.Cmd` instead of manual goroutine
207  orchestration.
208  - Added real-time progress updates via `progress.Progress` interface with
209  stage, substage, and article count messages.
210  - TUI displays a spinner, elapsed time, and final summary inline.
211  - Removed signal handling and channel synchronisation in favour of Bubble
212  Tea's built-in lifecycle.
213  - `TUIProgress` now only sends messages to an existing program, simplifying
214  integration.
215  - Both verbose mode (`SimpleLogger`) and TUI mode share the same
216  `progress.Progress` interface.
217
218## [0.3.0] - 2026-04-21
219
220### Added
221
222- **Configuration file support via TOML** (XDG-compliant and cross-platform)
223  - Load settings from `~/.config/llm_aggregator/config.toml`
224  - Supports all aggregation, API, and output options
225  - Example configuration file at `configs/config.example.toml`
226- **Environment variable support with `LLM_AGGREGATOR_` prefix**
227  - All configuration options can be set via environment variables
228  - Precedence order: CLI arguments > environment variables > config file >
229  built‑in defaults
230- New `internal/config` package with Viper integration
231  - `Load()` and `Save()` methods for configuration management
232  - `GetConfigPath()` for XDG‑compliant config file location
233  - `ConfigExists()` to check for existing configuration
234- Comprehensive unit tests for configuration loading, saving, and environment
235  variable precedence (`internal/config/config_test.go`)
236
237### Changed
238
239- **Environment variable renamed:** `DEEPSEEK_API_KEY`240  `LLM_AGGREGATOR_API_KEY`
241- **CLI help text** now reflects the new API key environment variable and
242  general configuration options
243- **Command‑line argument precedence** implemented in `cmd/llm_aggregator.go`
244via `applyConfiguration()`
245- Updated all help text, documentation, and code references
246- `README.md` completely revised with a new "Configuration" section
247
248### Removed
249
250- Global `QuietMode` and `VerboseMode` variables from
251`internal/config/config.go`. Replaced by runtime‑specific configuration in
252`runtime.Runtime` and CLI arguments
253- Hardcoded system prompt in `runtime.NewRuntime()`. Now left empty, allowing
254configuration or client default to be used
255- CLI flags now correctly override config file and environment variables
256
257## [0.2.0] - 2026-04-21
258
259### Changed
260
261- Moved all progress and status messages behind `-v/--verbose` flag
262- Default CLI mode is now silent except for final output and errors
263- Components (aggregator, processor, LLM client) use logger interface
264controlled by verbose flag
265
266## [0.1.0] - 2026-04-21
267
268### Added
269
270- Complete translation from Python prototype to Go implementation
271- RSS/Atom feed parsing using `gofeed` library
272- DeepSeek API integration via `openai-go` client, configured for `/chat/completions` endpoint
273- Content filtering and processing with keyword‑based include/exclude, date filtering, and sorting
274- Command‑line interface using `go‑arg` with automatic help and version flags
275- Multiple output formats: plain text, GitHub‑flavoured markdown, and JSON
276- Terminal user interface (TUI) built with `bubbletea` featuring:
277  - Animated progress bar with gradient colours (`#FF7CCB` to `#FDFF8C`)
278  - Live article counters (aggregated/processed)
279  - Elapsed time display
280  - Coloured status indicators using `lipgloss` styling
281  - Keyboard controls (q/Ctrl+C to quit)
282- Web content extraction fallback using `goquery` when feed descriptions are minimal
283- Configurable limits: articles per feed, maximum age, total articles
284- Token estimation and API usage logging
285- Environment variable support (`LLM_AGGREGATOR_API_KEY`) for authentication
286- Example feeds file with technology, programming, and free software sources
287
288### Changed
289
290- Program structure organised into standard Go layout: `cmd/`, `internal/`, `pkg/`
291- English spelling conventions maintained throughout (colour, initialise, summarise)
292- Error handling improved with specific messages for common API failures (invalid key, rate limits, etc.)
293- Progress reporting unified through a `ProgressContext` interface for both TUI and CLI modes
294
295### Fixed
296
297- Initial API endpoint mismatch (using `/responses` instead of `/chat/completions`)
298- Nil pointer dereferences when handling optional feed metadata (author, dates)
299- String repetition syntax errors in Go (replaced `"="*80` with `strings.Repeat`)
300- CLI help/version flag handling to show information without requiring other arguments
301- Type compatibility issues with `openai-go` v3 API (message parameters, token usage fields)