ack/klammertext
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Markdown to PDF: the mdpdf command, and the fonts it needs

Browse Source 
This snapshot carries sks/tns/, the translation directory, into the
distribution for the first time, together with the two default font
families its stylesheet names.

sks/tns/ holds two converters in opposite directions.  md_to_sks.py
converts Markdown to Klammertext, recording what it cannot convert exactly
as "#[MD ... ]#" markers so a draft carries its own worklist.  md_to_pdf.py
renders Markdown straight to PDF through a headless Chromium driven over the
DevTools Protocol, bypassing Klammertext entirely -- the route for a
document that is not ready to convert, and a permanent one for Markdown that
Klammertext cannot represent well.  Neither is loaded by the SKS; md_to_pdf
needs markdown-it-py, which it keeps in a virtual environment of its own and
creates with --setup.

The everyday form of the second is the mdpdf command, a shell function in
sks/tns/mdpdf.sh that env/runtime.env sources, so anyone with the
Klammertext environment has it:

    mdpdf notes.md          # writes notes.pdf beside it

It supplies the house fonts, the size matching, and the code wrapping,
completes on *.md at the TAB key, and takes its defaults from MDPDF_*
variables so one can be changed in a shell profile without copying the
function.  It is POSIX shell rather than zsh, since runtime.env is sourced
from bash profiles too.

Two things the stylesheet does that a print stylesheet usually cannot.  Code
lines are wrapped to a column count MEASURED from the rendered page rather
than written down -- the browser is asked how many characters a code box
holds, over every box in the document, so the wrapping stays right when the
fonts, sizes or margins change.  And the page number is a CSS Paged Media
margin box, which current Chromium implements, so it is set in the
document's own face instead of the browser's generic sans.

fnt/ gains EB Garamond and Source Sans 3, the serif and sans the stylesheet
asks for by default.

 (from dev 97d4f244c737)
This commit is contained in:
Andy Kopra
2026-08-09 15:54:05 +02:00
parent 6e7596ab2e
commit 4a4c7404cc
16 changed files with 1654 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

934
sks/tns/md_to_pdf.py Normal file
View File

@@ -0,0 +1,934 @@
#!/usr/bin/env python3
"""Markdown -> PDF, via a headless Chromium browser driven over the DevTools
Protocol.
This reproduces, without a browser window or an extension, what Andy has been
doing by hand: open a Markdown file in Firefox with the Markdown Viewer
extension, apply a print stylesheet, and print to PDF. The stylesheet is the
valuable part and is supplied by --css; the default is the one saved beside
this script.
WHY THE DEVTOOLS PROTOCOL AND NOT --print-to-pdf. The command-line flag is
being withdrawn from Chromium. Measured 2026-08-08 on this machine: Chrome
136 still honours it, Brave (Chromium 151) does not -- it starts, loads the
page, and then simply idles until killed, for file:// and http:// alike.
Chromium's own guidance is to drive printing through the protocol, so that is
what this does. It will keep working as browsers advance; a script built on
the flag has a shelf life.
The protocol client here is hand-rolled on the standard library -- a WebSocket
handshake and frame codec in about eighty lines -- in keeping with the way
this project does the same for its language server and its .vsix builder. The
one dependency is the Markdown renderer, markdown-it-py, which is the library
family the Firefox extension itself uses. PEP 668 forbids installing it into
the system Python, so it lives in a virtual environment:
python3 -m venv ~/.venvs/klammertext-tns
~/.venvs/klammertext-tns/bin/pip install markdown-it-py mdit-py-plugins
and this script re-executes itself with that interpreter when it needs to.
Usage:
md_to_pdf.py <input.md> [output.pdf] [--css FILE]... [--browser PATH]
[--keep-html] [--paper A4|letter] [--margin INCHES]
"""
import argparse
import base64
import re
import json
import os
import shutil
import socket
import struct
import subprocess
import sys
import tempfile
import time
import urllib.request
from pathlib import Path
HERE = Path(__file__).resolve().parent
DEFAULT_CSS = HERE / "markdown.css"
VENV = Path.home() / ".venvs" / "klammertext-tns"
# Browsers that can serve as the renderer, most preferred first. Any
# Chromium will do: the protocol is the same. The .app paths are for macOS,
# where nothing lands on PATH -- without them this finds no browser on a Mac
# that has one installed.
BROWSERS = ["/opt/brave.com/brave/brave", "brave-browser", "google-chrome",
"chromium", "chromium-browser",
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser",
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Chromium.app/Contents/MacOS/Chromium"]
# --- the renderer ----------------------------------------------------------
def setup_venv():
"""Create the virtual environment this script needs.
Offered as --setup so that a new machine takes one command rather than
two remembered ones. PEP 668 marks the system Python as
externally-managed, so a venv is not a preference here; pip refuses to
install into the system otherwise.
"""
print(f"creating {VENV}")
subprocess.run([sys.executable, "-m", "venv", str(VENV)], check=True)
pip = VENV / ("Scripts" if os.name == "nt" else "bin") / "pip"
subprocess.run([str(pip), "install", "-q", "markdown-it-py", "mdit-py-plugins"],
check=True)
print(f"installed markdown-it-py and mdit-py-plugins into {VENV}")
def ensure_renderer():
"""Re-exec under the venv interpreter unless EVERY requirement is present.
Done by re-exec rather than by manipulating sys.path so that the script
stays runnable as itself: `python3 md_to_pdf.py ...` works whether or not
the caller knows about the virtual environment.
Both packages are checked, not just the first. This machine's system
Python carries a distro markdown_it but no mdit_py_plugins, so a check of
markdown_it alone was satisfied, the re-exec never happened, and the
plugins were then silently skipped -- producing a PDF whose 138 internal
links pointed at heading anchors that had never been generated. The
links were present and did nothing, which is the worst way to fail.
"""
if "--setup" in sys.argv:
setup_venv()
sys.exit(0)
try:
import markdown_it # noqa: F401
import mdit_py_plugins.anchors # noqa: F401
return
except ImportError:
pass
venv_python = VENV / "bin" / "python"
if venv_python.exists() and Path(sys.executable) != venv_python:
os.execv(str(venv_python), [str(venv_python), *sys.argv])
sys.exit(f"the Markdown renderer is not installed. Run:\n"
f" python3 {Path(__file__).name} --setup")
def github_slug(title):
"""A heading's anchor, by GitHub's rule.
Markdown documents write their internal links against this convention --
"## 2.9 Keystone correction" is linked as "(#29-keystone-correction)" --
so the slug has to match it exactly or every cross-reference in the
document lands nowhere. Lowercase, drop everything that is not
alphanumeric, space or hyphen, then spaces to hyphens.
"""
import re
slug = re.sub(r"[^\w\s-]", "", title.strip().lower(), flags=re.UNICODE)
return re.sub(r"[\s]+", "-", slug)
def render_markdown(text):
"""Markdown -> HTML, configured like the Firefox extension's renderer:
CommonMark plus the GitHub-flavoured additions people actually write.
Heading anchors are not optional. markdown-it's core emits <h2> with no
id, so a document full of "[see](#some-section)" produces link
annotations in the PDF that point at destinations which do not exist --
the links are there, and clicking them does nothing. Measured on the
Rectify guide before this was added: 148 link annotations, 138 of them
dead.
"""
# No try/except around these: the plugins are requirements, not
# improvements, and a missing one must stop the program rather than
# quietly produce a document that is wrong in a way nobody can see.
# ensure_renderer() has already checked they are importable.
from markdown_it import MarkdownIt
from mdit_py_plugins.footnote import footnote_plugin
from mdit_py_plugins.deflist import deflist_plugin
from mdit_py_plugins.anchors import anchors_plugin
md = (MarkdownIt("commonmark", {"html": True, "linkify": True,
"typographer": True})
.enable(["table", "strikethrough"])
.use(footnote_plugin).use(deflist_plugin)
.use(anchors_plugin, max_level=6, slug_func=github_slug))
return md.render(text)
# A section number -- "2", "2.9", "3.1.4", with or without a trailing period
# -- in the two places a document writes one: at the start of a heading, and
# at the start of a table-of-contents entry. A contents entry is recognised
# as a list item whose FIRST content is a link to a fragment: that is what a
# Markdown table of contents is ("- [2.9 Keystone correction](#29-...)"), and
# requiring the link to open the item leaves a numbered reference in running
# text alone, where a wide gap would read as a mistake. Measured on the
# 47-entry contents of the Rectify guide: every numbered fragment link in the
# document opens a list item, and none appears mid-sentence.
#
# The whitespace after the number is part of each match and is CONSUMED --
# see mark_section_numbers().
HEADING_NUMBER = re.compile(r"(<h[1-6]\b[^>]*>)\s*(\d+(?:\.\d+)*\.?)\s+")
TOC_NUMBER = re.compile(r'(<li>\s*<a href="#[^"]*"[^>]*>)\s*(\d+(?:\.\d+)*\.?)\s+')
def mark_section_numbers(html):
"""Wrap a leading section number in <span class="secnum">.
The number is ordinary text in the Markdown ("## 2.9 Keystone
correction"), so the gap between it and the title is one space character
and nothing in a stylesheet can reach it. Marking the number gives the
stylesheet something to hold: markdown.css sets the distance with
--secnum-gap, and heading and contents entry take it from the same
property, so the two cannot drift apart.
The space after the number is consumed rather than left in place, so the
whole gap is the one CSS value -- otherwise it would be that value plus a
space whose width varies with the font.
This runs after rendering, not before, so the anchor slugs are computed
from the heading text as written and internal links still resolve. A
heading that opens with a number which is not a section number ("2026 in
review") gets the gap too; that is the price of not asking the document
to mark its own numbers.
"""
wrap = lambda m: f'{m.group(1)}<span class="secnum">{m.group(2)}</span>'
return TOC_NUMBER.sub(wrap, HEADING_NUMBER.sub(wrap, html))
def font_store_dirs():
"""The Klammertext font store's search order: the KLAMMERTEXT_FONTS
directories, then the distribution's own fnt/. Same order the engine
uses, so an installed font shadows a distributed one of the same name."""
dirs = [Path(d) for d in
os.environ.get("KLAMMERTEXT_FONTS",
str(Path.home() / ".klammertext" / "fonts")).split(":") if d]
home = os.environ.get("KLAMMERTEXT_HOME")
if home:
dirs.append(Path(home) / "fnt")
# This script lives in <klammertext>/sks/tns/, so it can find the
# distributed fonts without being told where they are. It has to:
# KLAMMERTEXT_HOME is set by a shell profile, and a Makefile rule or a
# non-interactive ssh session has no profile -- the guide's build failed
# on the Mac with "no font 'eb-garamond' ... Available: (none)" for
# exactly that reason, on a machine where the font was present all along.
own = HERE.parent.parent / "fnt"
if own not in dirs:
dirs.append(own)
return dirs
def font_face_css(name):
"""The @font-face rules for one font in the store, with absolute URLs.
A font in the store is a <name>/ directory of .ttf files beside a
<name>.css declaring its variants -- already the browser's own format,
which is why this needs no conversion, only a path fix: the store writes
url('eb-garamond/Regular.ttf') relative to itself, and the generated HTML
carries a <base> pointing at the Markdown file's directory, so a relative
URL would resolve somewhere else entirely and the font would silently
fall back.
Returns (css, family). A font the browser cannot find is not an error it
reports -- it just uses something else -- so an unknown name must fail
here instead.
"""
import re
for d in font_store_dirs():
css_path = d / f"{name}.css"
if not css_path.exists():
continue
css = css_path.read_text(encoding="utf-8")
css = re.sub(r"url\(\s*['\"]?([^'\")]+)['\"]?\s*\)",
lambda m: f"url('{(d / m.group(1)).resolve().as_uri()}')", css)
family = re.search(r"font-family:\s*['\"]([^'\"]+)['\"]", css)
return css, (family.group(1) if family else name)
available = sorted({p.stem for d in font_store_dirs() if d.is_dir()
for p in d.glob("*.css")})
sys.exit(f"no font {name!r} in the font store. Available: "
+ (", ".join(available) or "(none)"))
def font_metrics(ttf_path):
"""(x-height, cap-height), each as a fraction of the em, from the OS/2
table -- the same source mac/font_store.cpp reads.
A sfnt file is a table directory: numTables at offset 4, then 16-byte
entries of tag/checksum/offset/length. unitsPerEm lives at offset 18 of
'head'; sxHeight and sCapHeight at 86 and 88 of 'OS/2', and only from
version 2 of that table -- an older font reports neither, which is why
this can return zeros and the caller must cope.
"""
data = ttf_path.read_bytes()
num_tables = struct.unpack(">H", data[4:6])[0]
tables = {}
for i in range(num_tables):
off = 12 + i * 16
tag, _, start, length = struct.unpack(">4sIII", data[off:off + 16])
tables[tag.decode("latin-1").strip()] = (start, length)
if "head" not in tables or "OS/2" not in tables:
return 0.0, 0.0
head = tables["head"][0]
units = struct.unpack(">H", data[head + 18:head + 20])[0] or 1000
os2 = tables["OS/2"][0]
version = struct.unpack(">H", data[os2:os2 + 2])[0]
if version < 2:
return 0.0, 0.0
x_height = struct.unpack(">h", data[os2 + 86:os2 + 88])[0]
cap_height = struct.unpack(">h", data[os2 + 88:os2 + 90])[0]
return x_height / units, cap_height / units
def regular_face(store_dir, name):
"""The Regular face of a font in the store, which the metrics come from."""
d = store_dir / name
for candidate in ("Regular.ttf", "Regular.otf"):
if (d / candidate).exists():
return d / candidate
faces = sorted(d.glob("*.ttf")) + sorted(d.glob("*.otf"))
return faces[0] if faces else None
def scale_factors(names, match="average"):
"""How much to scale each role so it looks the size of the serif font.
Three measures, the same three sks/document/document_html.cpp offers:
xheight serif_xh / other_xh equalises lowercase -- the letters
"e" and "x" come out the same height. fontspec calls this
MatchLowercase, and it is the usual answer when an old-style
serif meets a monospace.
capheight serif_ch / other_ch equalises capitals.
average the ratio of the MEANS the compromise, and the SKS default,
so the default here agrees with the other pipeline.
Which to use is not a fact about the fonts. It depends on what the eye
lands on: prose interleaved with lowercase identifiers wants xheight,
text full of CONSTANTS wants capheight. With a face whose x-height and
cap-height are far apart -- EB Garamond is 0.400 against 0.650 -- the
average visibly satisfies neither.
Returns {role: factor}; a role whose font reports no metrics is omitted
rather than guessed at.
"""
pick = {"xheight": lambda xh, ch: xh,
"capheight": lambda xh, ch: ch,
"average": lambda xh, ch: (xh + ch) / 2.0}[match]
metrics = {}
for role, name in names.items():
if not name:
continue
for d in font_store_dirs():
if (d / f"{name}.css").exists():
face = regular_face(d, name)
if face:
xh, ch = font_metrics(face)
if xh > 0 and ch > 0:
metrics[role] = pick(xh, ch)
break
if "serif" not in metrics:
return {}
serif = metrics["serif"]
return {role: serif / m for role, m in metrics.items() if m > 0}
def font_css(serif, sans, mono, match="average"):
"""@font-face blocks plus the SKS's own custom-property names.
--serif, --sans and --mono are what sks/font/css/font.css calls them, so a
stylesheet written for one pipeline reads the same in the other. Family
names are QUOTED: an unquoted digit-initial name ("Source Sans 3") is
invalid CSS, and a font-family using it via var() computes to inherit --
the font is lost with no error anywhere.
"""
faces, variables = [], []
for role, name, fallback in (("serif", serif, "serif"),
("sans", sans, "sans-serif"),
("mono", mono, "monospace")):
if not name:
continue
css, family = font_face_css(name)
faces.append(css)
variables.append(f' --{role}: "{family}", {fallback};')
if not variables:
return ""
# Scale factors, computed rather than guessed. Both spellings are
# emitted: the short ones this script has always used, and the ones
# sks/font/css/font.css defines, so a stylesheet written for either
# pipeline works with the other.
scales = scale_factors({"serif": serif, "sans": sans, "mono": mono}, match)
alias = {"sans": "sans-serif", "mono": "monospace"}
for role, factor in scales.items():
variables.append(f" --{role}-scale: {factor:.4f};")
if role in alias:
variables.append(f" --{alias[role]}-scale: {factor:.4f};")
return ("\n".join(faces) + "\n:root {\n" + "\n".join(variables) + "\n}\n")
# A language's line-continuation character, where continuing a line is
# legal at all. Absent from this table means DO NOT WRAP: PowerShell
# continues with a backtick and a backslash would corrupt it, an .ini or
# .desktop file has no continuation whatever, and an unlabelled block is as
# likely to be a directory tree as it is to be code. Wrapping those would
# turn a document that merely looks too wide into one that is wrong -- and
# wrong silently, since a broken .desktop file reports nothing.
CONTINUATION = {"bash": "\\", "sh": "\\", "shell": "\\", "zsh": "\\",
"console": "\\", "powershell": "`", "ps1": "`"}
def unquoted_hash(line):
"""The index of a comment's #, or -1. The quote tracking is crude, but
it only has to find a # that is not inside a string."""
quote = None
for i, c in enumerate(line):
if quote:
if c == quote:
quote = None
elif c in "\"'":
quote = c
elif c == "#":
return i
return -1
def wrap_code_line(line, width, cont, indent=" "):
"""A long line as a continued sequence, or None if it cannot be done.
Two content rules, both needed for bash alone, so neither is avoided by
assuming a language:
* break only at spaces OUTSIDE quotes, or a split lands inside a string
literal and changes what the command does;
* never break inside a comment. A backslash within a shell comment
does NOT continue it -- the comment ends at the newline regardless --
so the remainder would be read as a command. Breaking BEFORE the #
is safe, because the continuation puts the comment back into the same
logical line.
"""
if len(line) <= width or line.rstrip().endswith(cont):
return None
stripped = line.lstrip()
lead = line[:len(line) - len(stripped)]
hash_at = unquoted_hash(line)
def break_points(s, floor):
quote, points = None, []
for i, c in enumerate(s):
if quote:
if c == quote:
quote = None
elif c in "\"'":
quote = c
elif c == " " and i > floor and (hash_at < 0 or i < hash_at):
points.append(i)
return points
pieces, rest, prefix = [], line, lead
while len(rest) > width:
room = width - len(cont) - 1
points = [b for b in break_points(rest, len(prefix)) if b <= room]
if not points:
break # nothing splittable in range
b = points[-1]
pieces.append(rest[:b] + " " + cont)
prefix = lead + indent
rest = prefix + rest[b + 1:]
hash_at = unquoted_hash(rest)
if not pieces:
return None
pieces.append(rest)
return pieces
def wrap_fenced_code(text, width):
"""Wrap over-long lines in fenced blocks whose language permits it.
Returns (text, wrapped, skipped): what it did, and what it did not. A
line left long still overflows the page, and an overflowing page makes
the browser shrink the WHOLE document -- so the caller reports the
remainder rather than letting it pass unnoticed.
"""
out, inside, lang, wrapped, skipped = [], False, "", 0, 0
for line in text.split("\n"):
fence = re.match(r"\s*(?:```|~~~)(\w*)", line)
if fence:
if not inside:
lang = (fence.group(1) or "").lower()
inside = not inside
out.append(line)
continue
if inside and len(line) > width:
cont = CONTINUATION.get(lang)
pieces = wrap_code_line(line, width, cont) if cont else None
if pieces:
out.extend(pieces)
wrapped += 1
continue
skipped += 1
out.append(line)
return "\n".join(out), wrapped, skipped
def wrap_and_report(text, wrap):
"""Wrap the fenced blocks and say what happened, or did not."""
text, wrapped, skipped = wrap_fenced_code(text, wrap)
if wrapped or skipped:
note = f"wrapped {wrapped} code lines at {wrap} columns"
if skipped:
note += (f"; {skipped} left long -- no continuation character "
"exists for that block's language")
print(note)
return text
def build_html(md_path, css_paths, fonts_css="", wrap=0):
"""One self-contained HTML document.
The stylesheets are INLINED rather than linked: a headless browser fetches
a linked stylesheet asynchronously, and printing can begin before it
arrives -- an unstyled PDF that looks like a CSS bug. Inline text cannot
lose that race.
A <base> element points at the Markdown file's own directory so that
relative image paths resolve, which lets the generated HTML live in a
temporary directory instead of beside the source.
"""
text = md_path.read_text(encoding="utf-8")
if wrap:
text = wrap_and_report(text, wrap)
body = mark_section_numbers(render_markdown(text))
css = fonts_css + "\n".join(Path(p).read_text(encoding="utf-8") for p in css_paths)
base = md_path.resolve().parent.as_uri() + "/"
return (f'<!doctype html>\n<html><head><meta charset="utf-8">\n'
f'<base href="{base}">\n'
f'<title>{md_path.stem}</title>\n'
f'<style>\n{css}\n</style>\n</head>\n<body>\n{body}\n</body></html>\n')
def measure_code_columns(port, url, paper, margin):
"""How many monospace characters fit on one line inside a <pre>.
Asked of the browser rather than computed, because the answer depends on
things only it knows: the mono font's advance width at the size the
stylesheet computes for it (itself a metric-derived scale factor), the
pre's padding and border, and the printable width of the paper. Any of
those can change in the stylesheet, and a wrap column written down by
hand then quietly becomes wrong in one of two directions -- too large and
a line overflows, shrinking every page; too small and the code is broken
into continuations with the right half of the box left empty. The second
is what happened here: a hand-set 71 against a real capacity of 94, so
12 of the Rectify guide's 66 code lines were being continued for no
reason (2026-08-09).
Measured under PRINT conditions, like the overflow check in
print_to_pdf: on screen the viewport width and the @media print padding
are both wrong.
"""
width, height = (8.27, 11.69) if paper == "A4" else (8.5, 11.0)
available = (width - 2 * margin) * 96.0
ws = WebSocket(page_socket(port))
ws.call("Page.enable")
ws.call("Page.navigate", url=url)
ws.wait_for("Page.loadEventFired")
ws.call("Emulation.setEmulatedMedia", media="print")
ws.call("Emulation.setDeviceMetricsOverride", width=int(available), height=1200,
deviceScaleFactor=1, mobile=False)
# Every pre is measured, not just the first, and the NARROWEST answer
# wins: a block inside a list item is indented, and wrapping the document
# to what a full-width block holds would leave the indented ones to soft
# wrap -- the ragged break with no continuation character that the
# wrapping exists to replace. Measured on the Rectify guide: 624px at
# the margin, 584px inside a list.
#
# The font is taken from the element that actually carries the text, the
# <code> inside the <pre>, never the <pre> itself. Those are not the
# same size: the stylesheet's "code, pre" scale rule applies to both, so
# a <code> nested in a <pre> is scaled twice (14.00px -> 12.26px here).
# Measuring the pre's font therefore understates capacity by that factor
# -- 88 columns against the 104 the page really holds.
#
# A document with no code block at all returns 0, and the caller then
# leaves the text alone.
measure = ws.call("Runtime.evaluate", returnByValue=True, expression="""
(() => {
const pres = [...document.querySelectorAll('pre')];
if (!pres.length) return {columns: 0};
const probe = document.createElement('span');
probe.style.position = 'absolute';
probe.style.whiteSpace = 'pre';
probe.textContent = 'x'.repeat(100);
document.body.appendChild(probe);
let columns = Infinity;
for (const pre of pres) {
const cs = getComputedStyle(pre);
const content = pre.clientWidth
- parseFloat(cs.paddingLeft) - parseFloat(cs.paddingRight);
probe.style.font = getComputedStyle(pre.querySelector('code') || pre).font;
const charWidth = probe.getBoundingClientRect().width / 100;
if (content > 0 && charWidth > 0)
columns = Math.min(columns, content / charWidth);
}
probe.remove();
return {columns: columns === Infinity ? 0 : columns};
})()""")["result"]["value"]
ws.call("Emulation.clearDeviceMetricsOverride")
ws.call("Emulation.setEmulatedMedia", media="")
if not measure["columns"]:
return 0
# One column in hand: the browser's soft wrap and this arithmetic agree to
# within a rounding error, and a line that lands exactly on the edge would
# be broken by the browser anyway -- undoing the point of wrapping it.
return max(20, int(measure["columns"]) - 1)
# --- a WebSocket client, standard library only -----------------------------
class WebSocket:
"""The minimum client needed to talk CDP: RFC 6455 text frames, masked
outbound, reassembled inbound, with pings answered.
Inbound frames must handle the 64-bit length case: a printToPDF reply
carries the whole document as base64 and is routinely megabytes, far past
the 125-byte and 65535-byte forms.
"""
def __init__(self, url):
_, _, rest = url.partition("://")
hostport, _, path = rest.partition("/")
host, _, port = hostport.partition(":")
self.sock = socket.create_connection((host, int(port or 80)))
key = base64.b64encode(os.urandom(16)).decode()
self.sock.sendall(
f"GET /{path} HTTP/1.1\r\nHost: {hostport}\r\n"
f"Upgrade: websocket\r\nConnection: Upgrade\r\n"
f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n"
.encode())
# Read the handshake response one byte at a time: reading in blocks
# would consume the start of the first frame along with it.
head = b""
while not head.endswith(b"\r\n\r\n"):
b = self.sock.recv(1)
if not b:
raise RuntimeError("the browser closed the connection during the handshake")
head += b
if b"101" not in head.split(b"\r\n")[0]:
raise RuntimeError(f"websocket upgrade refused: {head.splitlines()[0]!r}")
self.next_id = 0
def _read(self, n):
buf = b""
while len(buf) < n:
chunk = self.sock.recv(n - len(buf))
if not chunk:
raise RuntimeError("the browser closed the connection")
buf += chunk
return buf
def send(self, method, **params):
self.next_id += 1
payload = json.dumps({"id": self.next_id, "method": method,
"params": params}).encode()
header = bytearray([0x81]) # FIN + text
n = len(payload)
if n < 126:
header.append(0x80 | n)
elif n < 65536:
header.append(0x80 | 126); header += struct.pack(">H", n)
else:
header.append(0x80 | 127); header += struct.pack(">Q", n)
mask = os.urandom(4)
header += mask
self.sock.sendall(bytes(header) +
bytes(b ^ mask[i % 4] for i, b in enumerate(payload)))
return self.next_id
def recv(self):
"""One complete message, reassembling continuation frames."""
message = b""
while True:
b0, b1 = self._read(2)
opcode, length = b0 & 0x0F, b1 & 0x7F
if length == 126:
length = struct.unpack(">H", self._read(2))[0]
elif length == 127:
length = struct.unpack(">Q", self._read(8))[0]
data = self._read(length) # server frames are never masked
if opcode == 0x9: # ping -> pong, then keep reading
self.sock.sendall(b"\x8a\x80" + os.urandom(4))
continue
if opcode == 0x8:
raise RuntimeError("the browser closed the websocket")
message += data
if b0 & 0x80: # FIN
return json.loads(message)
def call(self, method, **params):
"""Send a command and return its result, skipping the event stream."""
want = self.send(method, **params)
while True:
msg = self.recv()
if msg.get("id") == want:
if "error" in msg:
raise RuntimeError(f"{method}: {msg['error']}")
return msg.get("result", {})
def wait_for(self, event, timeout=60):
deadline = time.time() + timeout
while time.time() < deadline:
if self.recv().get("method") == event:
return True
raise RuntimeError(f"timed out waiting for {event}")
# --- the browser -----------------------------------------------------------
def find_browser(explicit):
for candidate in ([explicit] if explicit else BROWSERS):
path = shutil.which(candidate) or (candidate if Path(candidate).exists() else None)
if path:
return path
sys.exit("no Chromium-based browser found; pass --browser PATH")
def free_port():
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
def start_browser(browser, profile, port):
"""Headless, with its own profile so a running browser is untouched.
--user-data-dir matters for more than tidiness: without it the launch
would attach to the user's existing session, and killing it afterwards
would close their windows.
"""
proc = subprocess.Popen(
[browser, "--headless=new", "--disable-gpu", "--no-sandbox",
"--no-first-run", "--no-default-browser-check",
"--disable-component-update", "--disable-background-networking",
f"--user-data-dir={profile}", f"--remote-debugging-port={port}",
"about:blank"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
deadline = time.time() + 60
while time.time() < deadline:
if proc.poll() is not None:
sys.exit(f"{browser} exited before the protocol port opened")
try:
with urllib.request.urlopen(f"http://127.0.0.1:{port}/json/version",
timeout=1) as r:
json.load(r)
return proc
except Exception:
time.sleep(0.2)
proc.kill()
sys.exit("the browser never opened its DevTools port")
# Chromium's own page header and footer, requested by --page-numbers.
# printToPDF takes HTML templates, in which a handful of magic classes are
# substituted -- pageNumber, totalPages, title, url, date. The templates are
# rendered in their own context with a default font size of a few pixels, so
# they must carry their own inline style or they come out unreadably small,
# and that context is NOT the document's: the template cannot use the
# stylesheet's fonts or custom properties, which is why this one names a
# generic sans.
#
# The CSS alternative now exists, and this is kept anyway. Paged Media
# margin boxes (@page { @bottom-center { content: counter(page) } }) were
# implemented by no browser when this was written; measured 2026-08-09 on
# Brave 151 (Chromium 151), they work, count pages, and honour a
# font-family taken from a custom property. A stylesheet that wants the
# number in the document's own face should use them. Two cautions, both
# measured on the same day: a webfont the document body never uses is not
# loaded for a margin box -- var(--mono) in a box of a prose-only document
# fell back to the platform monospace, and in one variant the box rendered
# NOTHING at all -- and the feature is recent enough that an older Chromium
# silently prints no number. --page-numbers has neither hazard, so it stays
# the default answer and the templates stay here.
FOOTER = ('<div style="font-family:sans-serif; font-size:9px; color:#555; '
'width:100%; text-align:center; margin:0 0.5in;">'
'<span class="pageNumber"></span> of <span class="totalPages"></span>'
'</div>')
# displayHeaderFooter turns BOTH on, and an unset header template falls back
# to Chromium's default (title and date). An empty div suppresses it.
EMPTY = '<div></div>'
def page_socket(port):
"""The DevTools socket of the browser's existing page.
Attach to the about:blank page the browser was launched with, rather than
asking for a new target: /json/new has required PUT since a recent
Chromium (a GET or POST answers 405), and there is already a page.
"""
with urllib.request.urlopen(f"http://127.0.0.1:{port}/json/list", timeout=10) as r:
targets = json.load(r)
pages = [t for t in targets if t.get("type") == "page" and t.get("webSocketDebuggerUrl")]
if not pages:
raise RuntimeError("the browser exposed no page to print")
return pages[0]["webSocketDebuggerUrl"]
def print_to_pdf(port, url, pdf_path, paper, margin, page_numbers=False):
"""Drive one page through load and print."""
ws = WebSocket(page_socket(port))
ws.call("Page.enable")
ws.call("Page.navigate", url=url)
ws.wait_for("Page.loadEventFired")
width, height = (8.27, 11.69) if paper == "A4" else (8.5, 11.0)
# Warn if the document is wider than the page. This is the failure that
# cost the most to find: Chromium does not clip or paginate overflow when
# printing, it SHRINKS THE WHOLE DOCUMENT until the widest element fits.
# So one over-long code line silently rescales every page, by a factor
# that changes whenever that line does -- and every font size then looks
# wrong for a reason nothing in the CSS explains. Measured here rather
# than assumed, and reported rather than left to be discovered.
#
# The measurement must be made under PRINT conditions. Measuring the
# page as it stands reports the browser window's width, which has nothing
# to do with the paper, and the @media print rules -- which change the
# padding and sizes that decide whether anything overflows -- are not
# even in effect. So emulate print media, force the viewport to the
# printable width, measure, and put both back before printing.
available = (width - 2 * margin) * 96.0 # CSS pixels of printable width
ws.call("Emulation.setEmulatedMedia", media="print")
ws.call("Emulation.setDeviceMetricsOverride", width=int(available), height=1200,
deviceScaleFactor=1, mobile=False)
measure = ws.call("Runtime.evaluate", returnByValue=True, expression="""
(() => {
const pres = [...document.querySelectorAll('pre')];
let charWidth = 0;
if (pres.length) {
const probe = document.createElement('span');
probe.style.font = getComputedStyle(pres[0]).font;
probe.style.position = 'absolute';
probe.style.whiteSpace = 'pre';
probe.textContent = 'x'.repeat(100);
document.body.appendChild(probe);
charWidth = probe.getBoundingClientRect().width / 100;
probe.remove();
}
return {width: document.body.scrollWidth,
over: pres.filter(e => e.scrollWidth > e.clientWidth + 1).length,
charWidth: charWidth};
})()""")["result"]["value"]
ws.call("Emulation.clearDeviceMetricsOverride")
ws.call("Emulation.setEmulatedMedia", media="")
if measure["width"] > available + 1:
shrink = available / measure["width"]
fits = int(available / measure["charWidth"]) if measure["charWidth"] else 0
print(f"warning: the content is {measure['width']:.0f}px wide but the page "
f"holds {available:.0f}px, so the browser will shrink every page to "
f"{shrink * 100:.0f}%.")
if measure["over"]:
print(f" {measure['over']} code blocks overflow; about {fits} "
f"columns fit. Try --wrap-code {fits}, or add "
"'pre {{ white-space: pre-wrap }}' to the stylesheet."
.replace("{{", "{").replace("}}", "}"))
# A footer needs room to sit in: with too small a bottom margin Chromium
# renders it under the text or not at all.
bottom = max(margin, 0.6) if page_numbers else margin
result = ws.call(
"Page.printToPDF",
printBackground=True,
displayHeaderFooter=page_numbers,
headerTemplate=EMPTY,
footerTemplate=FOOTER if page_numbers else EMPTY,
# The stylesheet is the point of this program, so let an @page rule in
# it win over these defaults when it says anything.
preferCSSPageSize=True,
paperWidth=width, paperHeight=height,
marginTop=margin, marginBottom=bottom,
marginLeft=margin, marginRight=margin)
Path(pdf_path).write_bytes(base64.b64decode(result["data"]))
def main():
ap = argparse.ArgumentParser(description="Markdown -> PDF via headless Chromium (CDP)")
ap.add_argument("input", nargs="?")
ap.add_argument("output", nargs="?")
ap.add_argument("--css", action="append", default=None,
help=f"stylesheet to apply (repeatable; default {DEFAULT_CSS.name})")
ap.add_argument("--browser", help="path to a Chromium-based browser")
ap.add_argument("--paper", choices=["A4", "letter"], default="A4")
ap.add_argument("--margin", type=float, default=0.5, help="inches")
ap.add_argument("--setup", action="store_true",
help="create the virtual environment this script needs, and exit")
ap.add_argument("--page-numbers", action="store_true",
help='number the pages ("3 of 21") in the footer')
ap.add_argument("--keep-html", action="store_true",
help="keep the intermediate HTML beside the PDF")
ap.add_argument("--serif", help="serif font from the Klammertext font store")
ap.add_argument("--sans", help="sans font from the Klammertext font store")
ap.add_argument("--mono", help="monospace font from the Klammertext font store")
ap.add_argument("--wrap-code", default=0, metavar="COLUMNS",
help="break code lines longer than COLUMNS at a word "
"boundary, using the language's continuation "
"character and an indent; a block whose language has "
"no continuation is left alone and reported. "
"\"auto\" asks the browser how many characters fit "
"in a code box and uses that, which is right by "
"construction when the fonts, sizes or margins change")
ap.add_argument("--match", choices=["average", "xheight", "capheight"],
default="average",
help="which measure the sans and mono fonts are scaled to "
"match: xheight equalises lowercase (the letter \"e\"), "
"capheight equalises capitals, average is the "
"compromise and the SKS default")
args = ap.parse_args()
if not args.input:
ap.error("an input file is required")
md_path = Path(args.input)
if not md_path.exists():
sys.exit(f"no such file: {md_path}")
pdf_path = Path(args.output) if args.output else md_path.with_suffix(".pdf")
css_paths = args.css or ([str(DEFAULT_CSS)] if DEFAULT_CSS.exists() else [])
for c in css_paths:
if not Path(c).exists():
sys.exit(f"no such stylesheet: {c}")
fonts_css = font_css(args.serif, args.sans, args.mono, args.match)
auto_wrap = str(args.wrap_code).lower() == "auto"
if not auto_wrap and not str(args.wrap_code).isdigit():
ap.error(f"--wrap-code takes a column count or \"auto\", "
f"not {args.wrap_code!r}")
wrap = 0 if auto_wrap else int(args.wrap_code)
html = build_html(md_path, css_paths, fonts_css, wrap)
work = Path(tempfile.mkdtemp(prefix="md_to_pdf."))
try:
html_path = work / (md_path.stem + ".html")
html_path.write_text(html, encoding="utf-8")
browser = find_browser(args.browser)
port = free_port()
proc = start_browser(browser, work / "profile", port)
try:
# --wrap-code auto: the unwrapped document is already loaded, so
# ask it how wide a code line may be, then build the real one.
# Two loads of the same page cost about a second and remove the
# only number in this pipeline that had to be guessed.
if auto_wrap:
columns = measure_code_columns(port, html_path.as_uri(),
args.paper, args.margin)
if columns:
html = build_html(md_path, css_paths, fonts_css, columns)
html_path.write_text(html, encoding="utf-8")
print_to_pdf(port, html_path.as_uri(), pdf_path, args.paper,
args.margin, args.page_numbers)
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
if args.keep_html:
kept = pdf_path.with_suffix(".html")
kept.write_text(html, encoding="utf-8")
print(f"{kept}")
print(f"{pdf_path} ({pdf_path.stat().st_size} bytes, "
f"{Path(browser).name}, {len(css_paths)} stylesheet"
f"{'s' if len(css_paths) != 1 else ''})")
finally:
shutil.rmtree(work, ignore_errors=True)
if __name__ == "__main__":
ensure_renderer()
main()