all repos — mdget @ master

Unnamed repository; edit this file 'description' to name the repository.

509eee27
feat: add comprehensive test suite and GoReleaser release automation

- Add GoReleaser configuration (`.goreleaser.yaml`)
  - Build for Linux, Windows, macOS (arm64, amd64, 386)
  - Enable UPX compression for Linux and Darwin binaries
  - Use version 2 syntax with custom archive naming
- Add Makefile with test targets
  - Run all tests via `make test-all` or just `make`
  - Individual test categories: `test-args`, `test-http`,
    `test-conversion`, `test-output`, `test-errors`, `test-build`
  - Also supports `build`, `clean`, and `help` targets
- Add integration test suite in `test/` directory
  - `test_args.go`: command-line flag parsing, help/version output
  - `test_http.go`: HTTP fetching, custom/default User-Agent, timeouts
  - `test_conversion.go`: HTML to Markdown conversion (headings, lists,
    links, code blocks, blockquotes, etc.)
  - `test_output.go`: file output and stdout behavior
  - `test_errors.go`: invalid URLs, timeouts, HTTP 4xx/5xx errors
  - `test_build.go`: end-to-end tests with compiled binary
  - `test_helper.go`: shared utilities (run commands, temp files,
    assertions)
- Add `test/README.md` documenting how to run the test suite
- Add `CHANGELOG.md` with [Unreleased] and [1.0.0] sections
- Update `.gitignore`
  - Remove platform‑specific binary patterns (`mdget*linux`,
    `mdget*x86_64`, etc.)
  - Add `dist/` (GoReleaser output directory)
- Update `README.md`
Maxwell Jensen 85795372+maxwelljens@users.noreply.github.com
Mon, 20 Apr 2026 20:35:29 +0200
63f853a7
feat: initial commit of mdget

- Add core implementation (`cmd/mdget.go`)
  - Fetch webpages with configurable timeout and User-Agent
  - Convert HTML to Markdown using `html-to-markdown/v2`
  - Support output to stdout or file via `-o/--output`
  - Automatically prepend `https://` to bare domains
  - Provide `--version` and `--help` via `go-arg`

- Add project metadata
  - `go.mod` / `go.sum` with dependencies: `req/v3`,
    `html-to-markdown/v2`, `go-arg`
  - `LICENCE.txt` (EUPL 1.2) with full legal text
  - `README.md` with usage, examples, build instructions

- Add assets
  - `assets/logo.png` and `assets/screenshot.png`
  - `assets/eupl-12-badge.svg` for licence badge

- Configure `.gitignore` for Go binaries, test artifacts, coverage, and
  environment files
Maxwell Jensen 85795372+maxwelljens@users.noreply.github.com
Mon, 20 Apr 2026 13:42:49 +0200

mdget logo mdget screenshot

Codeberg Release Codeberg License

What is mdget?

mdget (“markdown get” or “midget”) is a simple command‑line utility that fetches a webpage and converts its HTML content to clean Markdown. It’s designed for quick, scriptable extraction of content from the web into a readable, portable format.

How do I use mdget?

mdget [OPTIONS]... [URL]

By default, mdget fetches the given URL and outputs the converted Markdown to standard output.

Options

-o, --output FILE      Write output to FILE instead of stdout
-t, --timeout SECONDS  Request timeout in seconds (default: 30)
-u, --user-agent AGENT Custom User‑Agent header (default: "mdget/0.1.0")
-h, --help             Show this help message and exit
-v, --version          Show version information and exit

Examples

# Basic usage: fetch a webpage and print Markdown to stdout
mdget https://example.com

# Save the output to a file
mdget --output page.md https://example.com

# Use a custom timeout and User‑Agent
mdget --timeout 10 --user-agent "MyBot/1.0" example.com

# Automatically add https:// prefix to bare domains
mdget github.com > github.md

# Show version information
mdget --version

# Show help message
mdget --help

How does mdget work?

mdget performs the following steps for each request:

  1. Parse command‑line arguments
  2. Validate and normalise the URL; if the provided URL does not start with http:// or https://, https:// is prepended.
  3. Fetch the HTML using the req HTTP client library, with configurable timeout and an optional User‑Agent.
  4. Convert HTML to Markdown using the html‑to‑markdown library, which preserves semantic structure and handles relative URLs correctly.
  5. Output the result to STDOUT or -o/--output if specified.

The program exits with a non‑zero status if the HTTP request fails, the server returns an error, or the HTML‑to‑Markdown conversion fails for whatever reason.

Dependencies

mdget is written in Go and uses the following libraries:

How do I build mdget?

mdget can be built with a standard Go toolchain:

go build ./cmd/mdget.go

Licence

This project is licensed under European Union Public Licence 1.2.

clone 
git clone https://maxwelljensen.eu/mdget.git