1<h1 align="center">
 2    <img src="assets/logo.png" alt="mdget logo">
 3    <img src="assets/screenshot.png" alt="mdget screenshot">
 4</h1>
 5
 6![Codeberg Release](https://img.shields.io/gitea/v/release/maxwelljensen/mdget?gitea_url=https%3A%2F%2Fcodeberg.org&style=for-the-badge)
 7![Codeberg License](assets/eupl-12-badge.svg)
 8
 9---
10
11## What is mdget?
12
13`mdget` ("markdown get" or "midget") is a simple command‑line utility that
14fetches a webpage and converts its HTML content to clean Markdown. It’s
15designed for quick, scriptable extraction of content from the web into a
16readable, portable format.
17
18## How do I use mdget?
19
20    mdget [OPTIONS]... [URL]
21
22By default, `mdget` fetches the given URL and outputs the converted Markdown to
23standard output.
24
25### Options
26
27    -o, --output FILE      Write output to FILE instead of stdout
28    -t, --timeout SECONDS  Request timeout in seconds (default: 30)
29    -u, --user-agent AGENT Custom User‑Agent header (default: "mdget/0.1.0")
30    -h, --help             Show this help message and exit
31    -v, --version          Show version information and exit
32
33### Examples
34
35    # Basic usage: fetch a webpage and print Markdown to stdout
36    mdget https://example.com
37
38    # Save the output to a file
39    mdget --output page.md https://example.com
40
41    # Use a custom timeout and User‑Agent
42    mdget --timeout 10 --user-agent "MyBot/1.0" example.com
43
44    # Automatically add https:// prefix to bare domains
45    mdget github.com > github.md
46
47    # Show version information
48    mdget --version
49
50    # Show help message
51    mdget --help
52
53## How does `mdget` work?
54
55`mdget` performs the following steps for each request:
56
571. Parse command‑line arguments
582. Validate and normalise the URL; if the provided URL does not start with
59   `http://` or `https://`, `https://` is prepended.
603. Fetch the HTML using the [`req` HTTP client library](https://req.cool/),
61   with configurable timeout and an optional User‑Agent.
624. Convert HTML to Markdown using the `html‑to‑markdown` library, which
63   preserves semantic structure and handles relative URLs correctly.
645. Output the result to `STDOUT` or `-o/--output` if specified.
65
66The program exits with a non‑zero status if the HTTP request fails, the server
67returns an error, or the HTML‑to‑Markdown conversion fails for whatever reason.
68
69## Dependencies
70
71`mdget` is written in Go and uses the following libraries:
72
73* [`req`](https://github.com/imroc/req) – a modern, feature‑rich HTTP client
74* [`html‑to‑markdown`](https://github.com/JohannesKaufmann/html-to-markdown) – a robust HTML‑to‑Markdown converter
75* [`go‑arg`](https://github.com/alexflint/go-arg) – struct‑based argument parsing with automatic help and version flags
76
77## How do I build mdget?
78
79`mdget` can be built with a standard Go toolchain:
80
81    go build ./cmd/mdget.go
82
83Or installed globally:
84
85    go install ./cmd/mdget.go
86
87## Licence
88
89This project is licensed under [European Union Public Licence
901.2](https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12).