all repos — llm_aggregator @ b7690ae41cc0c1076100d43a33ceb2aef502a2c0

A CLI tool to aggregate RSS feeds and summarise them with LLMs

CHANGELOG.md (view raw)

  1# Changelog
  2
  3All notable changes to this project will be documented in this file.
  4
  5The format is based on [Keep a
  6Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
  7[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
  8
  9## [0.11.0] - 2026-04-23
 10
 11- Progress bar now tracks estimated token count (via tiktoken) before sending
 12  to LLM, then updates with actual completion tokens from the API response.
 13  Gives a more accurate picture of how much content is being sent.
 14
 15## [0.10.0] - 2026-04-23
 16
 17### Added
 18
 19- **Short command-line flags**: Single-character shortcuts for the most
 20  commonly used options:
 21  - `-f` / `--feeds-file` — feeds file path
 22  - `-p` / `--prompt` — summarisation prompt
 23  - `-n` / `--max-articles-per-feed` — articles per feed limit
 24  - `-d` / `--max-days-old` — article age filter
 25  - `-i` / `--include-keywords` — keyword include filter
 26  - `-e` / `--exclude-keywords` — keyword exclude filter
 27  - `-m` / `--model` — LLM model selection
 28  - `-o` / `--output` — output format
 29  - `-t` / `--tui` — enable TUI
 30  - `-D` / `--dry-run` — dry run mode
 31- **Styled help output**: Custom lipgloss-styled help text at
 32  `internal/cli/help.go` replaces go-arg's default plain-text output. Flags are
 33  rendered in cyan, section headers in bold, with consistent column alignment.
 34  Respects `$NO_COLOR`.
 35
 36## [0.9.0] - 2026-04-23
 37
 38### Added
 39
 40- **Markdown rendering in TUI**: Implemented `charmbracelet/glamour` to
 41  render LLM summary output with proper markdown styling. Headers, bold,
 42  italic, code blocks, lists, and other markdown elements are now styled
 43  for better readability in the terminal.
 44- **Formalised infobar keybinds**: TUI keybinds are now defined as
 45  dedicated style variables at the top of the model for easier maintenance.
 46  Keybinds are bold while separators and action text are subtle but
 47  distinguishable.
 48- **Thinking indicator as separate element**: The `[💭 Thinking: ON/OFF]`
 49  toggle indicator is now on its own line above the keybinds, using a
 50  distinct style (magenta+bold when ON, subtle+italic when OFF).
 51- **Dynamic text wrapping on resize**: TUI summary text now re-wraps when
 52  the terminal window size or font changes, with proper markdown re-rendering.
 53
 54## [0.8.0] - 2026-04-23
 55
 56### Added
 57
 58- **Thinking tags toggle in TUI**: LLMs that output `<think>`/`</think>`
 59  thinking tags now have those sections automatically removed from the summary
 60  display by default. Users can press `t` to toggle the thinking section back
 61  on, displayed in gray at the head of the output with XML tags stripped for
 62  cleaner presentation. A status indicator shows whether thinking is currently
 63  visible.
 64
 65## [0.7.1] - 2026-04-22
 66
 67### Fixed
 68
 69- Environment variables were correctly overriding config file values, but CLI
 70  arguments without explicit flags were also overriding config file values
 71  unintentionally.
 72  - Root cause: `go-arg` was populating struct fields with default values
 73    from `default:` tags even when those CLI flags weren't provided
 74  - Fix: Removed `default:` tags from Args struct fields for LLM options
 75    (`model`, `base-url`, `api-key`, `max-tokens`, `temperature`);
 76    Viper's own default handling is sufficient
 77  - Fix: Changed Args struct fields from value types to pointer types
 78    (`*string`, `*int`, `*float64`) so that "not provided on CLI" can be
 79    distinguished from "provided but zero/empty"
 80  - Fix: Updated `BindCLIArgs()` and `isZero()` to properly handle pointer
 81    types and distinguish nil pointers from zero values
 82- `isZero()` with compound type switch cases: Compound type cases like `case
 83  *int, *int8, *int16, *int32, *int64` don't properly handle nil comparison when
 84  stored in an interface variable. Fixed by splitting into individual type cases.
 85- `isZero()` missing nil interface handling: When a nil pointer is stored
 86  in an `any` interface, `value != nil` returns `true` (the interface itself
 87  is not nil). Added early `if v == nil { return true }` check.
 88
 89## [0.7.0] - 2026-04-22
 90
 91### Added
 92
 93- **`--dry-run` option**: Validate configuration and preview what would happen
 94  without making any LLM API calls. Useful for validating feeds file and
 95  configuration, checking article counts and filtering results and anything
 96  else that can be done "offline".
 97- **Styled terminal output using `lipgloss`**: All CLI output now uses ANSI
 98  colour codes for better readability:
 99- **`$NO_COLOR` environment variable support**: Respects the ["no
100color"](https://no-color.org/) standard. When `NO_COLOR` is set, all styling is
101disabled and plain text is output instead.
102
103### Changed
104
105- API key validation now only occurs when not using `--dry-run` mode
106
107## [0.6.0] - 2026-04-22
108
109### Added
110
111- **Comprehensive unit test suite**, including CLI argument tests
112  (`--feeds-file`, `--prompt`, `--api-key`, `--model`, `--base-url`),
113  configuration tests, aggregator tests, output formatter tests, processor
114  tests and defaults tests.
115- **`--base-url` CLI option**: configure custom API endpoints for different LLM
116  providers. Should support any OpenAI-compatible provider.
117  - Environment variable: `LLM_AGGREGATOR_BASE_URL`
118- `internal/defaults` package
119- `docs/TESTING.md` which contains a comprehensive testing guide with test
120  descriptions
121
122### Changed
123
124- **Configuration defaults**: All packages now use `defaults.Default*` constants
125  - `internal/config`: Uses `defaults.Default*` for default values
126  - `internal/runtime`: Uses `defaults.Default*` for runtime defaults
127  - `internal/llm`: Uses `defaults.Default*` for LLM client defaults
128
129### Fixed
130
131- `nil` pointer safety in aggregator
132  - Added nil checks for `progressCtx` across all log statements
133  - `FeedAggregator` now works correctly when progress context is nil
134  - Tests use `NewFeedAggregatorWithProgress` with explicit progress context
135
136## [0.5.0] - 2026-04-22
137
138### Added
139
140- **Accurate token counting with tiktoken**
141  - New `internal/tokeniser` package using OpenAI's tiktoken-go library
142  - BPE tokenisation for accurate token counting vs rough character estimation
143  - Encoding caching to avoid repeated initialisation overhead
144  - Model-aware encoding selection (cl100k_base, o200k_base, etc.)
145  - `CountTokens()` and `CountMessagesTokens()` functions
146  - Fallback to rough estimation when tiktoken initialisation fails
147- **Concurrent feed fetching**: Feeds are now fetched concurrently using
148  `golang.org/x/sync/errgroup`
149  - Semaphore limits concurrent requests to 10 to avoid server overload
150  - Mutex-protected shared state for thread-safe article collection
151  - Partial failures don't stop other feeds from being processed
152- **Scrollable summary in TUI**
153  - Bubble Tea viewport component for browsing long summaries
154  - Keyboard navigation: j/k or arrows (scroll), Space/B (page), g/G
155  (start/end)
156  - Mouse wheel scrolling support
157  - Scroll progress indicator showing position and total lines
158
159### Changed
160
161- **Streamlined configuration management**: Viper now uses global instance with
162automatic precedence handling. Simplified configuration flow: Parse → GetViper
163→ BindCLIArgs → ViperToRuntime
164- **TUI colour scheme**: All hex colour codes replaced with ANSI 256-colour
165  palette
166
167### Fixed
168
169- Viewport now properly handles scroll events via `Update()`. Summary content
170  pre-wrapped to viewport width before display
171- Fixed string literal colour names to use actual variables
172
173## [0.4.0] - 2026-04-21
174
175### Changed 
176
177- **TUI is now fully functional and production-ready**: Replaced the
178work-in-progress TUI implementation with a robust Bubble Tea command pattern:
179  - Runtime execution now uses native `tea.Cmd` instead of manual goroutine
180  orchestration.
181  - Added real-time progress updates via `progress.Progress` interface with
182  stage, substage, and article count messages.
183  - TUI displays a spinner, elapsed time, and final summary inline.
184  - Removed signal handling and channel synchronisation in favour of Bubble
185  Tea's built-in lifecycle.
186  - `TUIProgress` now only sends messages to an existing program, simplifying
187  integration.
188  - Both verbose mode (`SimpleLogger`) and TUI mode share the same
189  `progress.Progress` interface.
190
191## [0.3.0] - 2026-04-21
192
193### Added
194
195- **Configuration file support via TOML** (XDG-compliant and cross-platform)
196  - Load settings from `~/.config/llm_aggregator/config.toml`
197  - Supports all aggregation, API, and output options
198  - Example configuration file at `configs/config.example.toml`
199- **Environment variable support with `LLM_AGGREGATOR_` prefix**
200  - All configuration options can be set via environment variables
201  - Precedence order: CLI arguments > environment variables > config file >
202  built‑in defaults
203- New `internal/config` package with Viper integration
204  - `Load()` and `Save()` methods for configuration management
205  - `GetConfigPath()` for XDG‑compliant config file location
206  - `ConfigExists()` to check for existing configuration
207- Comprehensive unit tests for configuration loading, saving, and environment
208  variable precedence (`internal/config/config_test.go`)
209
210### Changed
211
212- **Environment variable renamed:** `DEEPSEEK_API_KEY`213  `LLM_AGGREGATOR_API_KEY`
214- **CLI help text** now reflects the new API key environment variable and
215  general configuration options
216- **Command‑line argument precedence** implemented in `cmd/llm_aggregator.go`
217via `applyConfiguration()`
218- Updated all help text, documentation, and code references
219- `README.md` completely revised with a new "Configuration" section
220
221### Removed
222
223- Global `QuietMode` and `VerboseMode` variables from
224`internal/config/config.go`. Replaced by runtime‑specific configuration in
225`runtime.Runtime` and CLI arguments
226- Hardcoded system prompt in `runtime.NewRuntime()`. Now left empty, allowing
227configuration or client default to be used
228- CLI flags now correctly override config file and environment variables
229
230## [0.2.0] - 2026-04-21
231
232### Changed
233
234- Moved all progress and status messages behind `-v/--verbose` flag
235- Default CLI mode is now silent except for final output and errors
236- Components (aggregator, processor, LLM client) use logger interface
237controlled by verbose flag
238
239## [0.1.0] - 2026-04-21
240
241### Added
242
243- Complete translation from Python prototype to Go implementation
244- RSS/Atom feed parsing using `gofeed` library
245- DeepSeek API integration via `openai-go` client, configured for `/chat/completions` endpoint
246- Content filtering and processing with keyword‑based include/exclude, date filtering, and sorting
247- Command‑line interface using `go‑arg` with automatic help and version flags
248- Multiple output formats: plain text, GitHub‑flavoured markdown, and JSON
249- Terminal user interface (TUI) built with `bubbletea` featuring:
250  - Animated progress bar with gradient colours (`#FF7CCB` to `#FDFF8C`)
251  - Live article counters (aggregated/processed)
252  - Elapsed time display
253  - Coloured status indicators using `lipgloss` styling
254  - Keyboard controls (q/Ctrl+C to quit)
255- Web content extraction fallback using `goquery` when feed descriptions are minimal
256- Configurable limits: articles per feed, maximum age, total articles
257- Token estimation and API usage logging
258- Environment variable support (`LLM_AGGREGATOR_API_KEY`) for authentication
259- Example feeds file with technology, programming, and free software sources
260
261### Changed
262
263- Program structure organised into standard Go layout: `cmd/`, `internal/`, `pkg/`
264- English spelling conventions maintained throughout (colour, initialise, summarise)
265- Error handling improved with specific messages for common API failures (invalid key, rate limits, etc.)
266- Progress reporting unified through a `ProgressContext` interface for both TUI and CLI modes
267
268### Fixed
269
270- Initial API endpoint mismatch (using `/responses` instead of `/chat/completions`)
271- Nil pointer dereferences when handling optional feed metadata (author, dates)
272- String repetition syntax errors in Go (replaced `"="*80` with `strings.Repeat`)
273- CLI help/version flag handling to show information without requiring other arguments
274- Type compatibility issues with `openai-go` v3 API (message parameters, token usage fields)