ack/klammertext
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Markdown to PDF: the mdpdf command, and the fonts it needs

Browse Source 
This snapshot carries sks/tns/, the translation directory, into the
distribution for the first time, together with the two default font
families its stylesheet names.

sks/tns/ holds two converters in opposite directions.  md_to_sks.py
converts Markdown to Klammertext, recording what it cannot convert exactly
as "#[MD ... ]#" markers so a draft carries its own worklist.  md_to_pdf.py
renders Markdown straight to PDF through a headless Chromium driven over the
DevTools Protocol, bypassing Klammertext entirely -- the route for a
document that is not ready to convert, and a permanent one for Markdown that
Klammertext cannot represent well.  Neither is loaded by the SKS; md_to_pdf
needs markdown-it-py, which it keeps in a virtual environment of its own and
creates with --setup.

The everyday form of the second is the mdpdf command, a shell function in
sks/tns/mdpdf.sh that env/runtime.env sources, so anyone with the
Klammertext environment has it:

    mdpdf notes.md          # writes notes.pdf beside it

It supplies the house fonts, the size matching, and the code wrapping,
completes on *.md at the TAB key, and takes its defaults from MDPDF_*
variables so one can be changed in a shell profile without copying the
function.  It is POSIX shell rather than zsh, since runtime.env is sourced
from bash profiles too.

Two things the stylesheet does that a print stylesheet usually cannot.  Code
lines are wrapped to a column count MEASURED from the rendered page rather
than written down -- the browser is asked how many characters a code box
holds, over every box in the document, so the wrapping stays right when the
fonts, sizes or margins change.  And the page number is a CSS Paged Media
margin box, which current Chromium implements, so it is set in the
document's own face instead of the browser's generic sans.

fnt/ gains EB Garamond and Source Sans 3, the serif and sans the stylesheet
asks for by default.

 (from dev 97d4f244c737)
This commit is contained in:
Andy Kopra
2026-08-09 15:54:05 +02:00
parent 6e7596ab2e
commit 4a4c7404cc
16 changed files with 1654 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

233
sks/tns/markdown.css Normal file
View File

@@ -0,0 +1,233 @@
body {
font-family: var(--serif);
font-size: 100%;
line-height: 1.3;
}
/* The monospace font is scaled so its letters are the height of the serif
font's, rather than by eye: md_to_pdf.py computes --mono-scale from the
two fonts' x-height and cap-height (the same rule the SKS applies), so
this stays right if either font is changed. */
code, pre {
font-family: var(--mono);
font-size: calc(1em * var(--mono-scale, 1));
}
/* Once, not twice. A fenced block is <pre><code>, so both elements match
the rule above and the scale is applied to the scaled size: measured
before this rule, inline code came out at 14.00px and the same code in a
box at 12.26px -- a metric-matched scale silently defeating itself. The
code in a box inherits the pre's size instead. */
pre code {
font-size: 1em;
}
h1, h2, h3, h4, h5, h6 {
font-family: var(--sans);
font-weight: normal;
padding-top: .5rem;
padding-bottom: 0rem;
margin-top: .0rem;
margin-bottom: 0rem;
}
h1 {
font-size: 1.2rem;
}
h2 {
font-size: 1.1rem;
}
h3, h4, h5, h6 {
font-size: 1.0rem;
}
/* The gap between a section number and its title, in a heading and in a
table-of-contents entry alike. The number is plain text in the Markdown
("## 2.9 Keystone correction", "- [2.9 Keystone correction](#29-...)");
md_to_pdf.py wraps it in this span and eats the space that followed it, so
the whole distance is set here. Change --secnum-gap to widen or narrow
it, or give the contents its own gap with a "li .secnum" rule. To rule a
contents into columns rather than merely space it, add
"display: inline-block" and a "min-width" there: the titles then align
instead of each starting one gap after a number of its own width. */
:root {
--secnum-gap: 0.5em;
}
.secnum {
margin-right: var(--secnum-gap);
}
p, ol, ul {
margin-top: 0.5rem;
margin-bottom: 0.5rem;
}
pre {
/* A code line wider than the page does not overflow or clip when
printing: Chromium shrinks the WHOLE document until it fits, silently,
by a factor that changes with the longest line. Wrapping guarantees
that never happens. md_to_pdf.py --wrap-code breaks shell lines at a
word boundary with a "\" first, which is copy-pasteable; this catches
what it cannot wrap (PowerShell, .ini, ASCII diagrams). */
white-space: pre-wrap;
overflow-wrap: anywhere;
padding-left: 0rem;
border: 1px solid #888;
background-color: #EFE;
}
th {
border-bottom: 1px solid #000;
}
table, th, td {
border: 1px solid gray;
}
@media print {
/* A section title never sits alone at the foot of a page: the break is
forbidden between a heading and whatever follows it, so the heading
moves to the next page with its text. break-inside keeps a heading
that wraps from being split across the fold.
Chromium honours these when it paginates for printing (measured on a
16-page test document: three headings stranded at a page bottom
before, none after, and the page count unchanged). A heading it
cannot honour them for -- one taller than the page -- is laid out
as if they were absent rather than looped over, which is the
specified behaviour for an unsatisfiable avoid.
This does NOT govern how much of the following paragraph comes with
the heading: that is `orphans`, below. */
h1, h2, h3, h4, h5, h6 {
break-after: avoid;
break-inside: avoid;
}
/* A paragraph split by a page break leaves at least two lines on each
side of it: never one line stranded at the foot of a page (orphans),
never one line arriving alone at the top of the next (widows) -- and
with break-after above, that top-of-page line would be the one
directly under a heading the break has just moved. Both properties
are inherited, so declaring them on body covers list items and table
cells too.
Measured on a 16-page test document: Chromium honours both -- with
orphans: 1 it packs the text into 15 pages, and with widows: 1 it
moves a line back across a page boundary (same page count, different
pagination) -- and 2 is ALREADY its default for both, so rendering
with these declarations is byte-identical to rendering without them.
They are here to pin the values, not to change today's output: the CSS
initial value is 2 but a UA is free to differ, and a stylesheet added
after this one could relax it. */
body {
orphans: 2;
widows: 2;
}
pre {
padding: .75rem 1.25rem .75rem 1.25rem;
break-inside: avoid;
}
a[href]:after {
content: none;
}
th, td {
padding: .25rem .75rem;
}
table {
margin-left: 1rem;
margin-bottom: .5rem;
border-collapse: collapse;
}
th {
border-bottom: 1px solid #000;
text-align: left;
padding-right: 2rem;
padding-bottom: 0.2em;
}
tbody tr:first-child td {
padding-top: 0.0em;
}
blockquote {
margin-left: 1rem;
}
table, th, td {
border: 1px solid #BBB;
}
:root {
--link-blue: #042B8C; /* #03216C; */
}
a {
color: var(--link-blue);
}
a, a:link, a:visited {
outline: 0;
text-decoration: none;
}
a:hover {
text-decoration: underline;
text-underline-offset: 0.2em;
text-decoration-thickness: 1px;
outline: 0;
}
/* The page number, in the document's own face.
This is a CSS Paged Media margin box, and it is the reason to prefer
it over md_to_pdf.py's --page-numbers: Chromium's footer template is
rendered in a context of its own that cannot see this stylesheet, so
the number comes out in a generic sans, while var(--serif) here is the
EB Garamond the text is set in. Measured 2026-08-09 on Brave 151:
margin boxes work and count pages. They are recent, though -- an
older Chromium silently prints no number at all.
DO NOT ALSO PASS --page-numbers. The two mechanisms are independent
and both would draw, one over the other, and the flag additionally
widens the bottom margin.
HEIGHT. A margin box is drawn in the margin and cannot be moved out
of it, so where the number sits and how much text a page holds are
one question, not two. Chromium centres the box in the margin band
by default; these declarations pin it to the top of a slightly larger
band instead, which puts the number 27pt above the paper edge rather
than 12 -- a body line higher -- while the padding, all 1.5pt of it,
leaves the text block ending just above it. The text therefore keeps
the line it would otherwise have lost to the raised number: measured,
a document that grew from 3 pages to 4 when the number was raised
alone is back to 3.
THE TIGHT SPOT, and the number to change if it shows. Text and
number now share a narrow band, so on a page whose last line reaches
the bottom of the block they nearly touch: measured worst pages over
three documents, 7.6 / 3.1 / 2.4pt between the last line's box and
the number's. Nothing overlapped, but 2.4pt is close. Giving the
text back only HALF a line -- margin-bottom 0.60in, padding-top
4.4pt, the number within a point of where it is now -- measured
8.4 / 9.1 / 11.4pt instead, and cost no document a page.
var(--serif) is safe here because the body is set in it, so the
webfont is loaded. A face the document does not otherwise use is NOT
loaded for a margin box: var(--mono) in a prose-only document fell
back to the platform monospace, and in one variant the box rendered
nothing whatever -- silently, as ever with print CSS. */
@page {
margin-bottom: 0.56in; /* the number, and little else */
@bottom-center {
content: counter(page) " of " counter(pages);
font-family: var(--serif);
font-size: 9pt;
color: black;
vertical-align: top; /* not centred in the margin band */
padding-top: 1.5pt; /* its distance from the text */
}
}
}

934
sks/tns/md_to_pdf.py Normal file
View File

@@ -0,0 +1,934 @@
#!/usr/bin/env python3
"""Markdown -> PDF, via a headless Chromium browser driven over the DevTools
Protocol.
This reproduces, without a browser window or an extension, what Andy has been
doing by hand: open a Markdown file in Firefox with the Markdown Viewer
extension, apply a print stylesheet, and print to PDF. The stylesheet is the
valuable part and is supplied by --css; the default is the one saved beside
this script.
WHY THE DEVTOOLS PROTOCOL AND NOT --print-to-pdf. The command-line flag is
being withdrawn from Chromium. Measured 2026-08-08 on this machine: Chrome
136 still honours it, Brave (Chromium 151) does not -- it starts, loads the
page, and then simply idles until killed, for file:// and http:// alike.
Chromium's own guidance is to drive printing through the protocol, so that is
what this does. It will keep working as browsers advance; a script built on
the flag has a shelf life.
The protocol client here is hand-rolled on the standard library -- a WebSocket
handshake and frame codec in about eighty lines -- in keeping with the way
this project does the same for its language server and its .vsix builder. The
one dependency is the Markdown renderer, markdown-it-py, which is the library
family the Firefox extension itself uses. PEP 668 forbids installing it into
the system Python, so it lives in a virtual environment:
python3 -m venv ~/.venvs/klammertext-tns
~/.venvs/klammertext-tns/bin/pip install markdown-it-py mdit-py-plugins
and this script re-executes itself with that interpreter when it needs to.
Usage:
md_to_pdf.py <input.md> [output.pdf] [--css FILE]... [--browser PATH]
[--keep-html] [--paper A4|letter] [--margin INCHES]
"""
import argparse
import base64
import re
import json
import os
import shutil
import socket
import struct
import subprocess
import sys
import tempfile
import time
import urllib.request
from pathlib import Path
HERE = Path(__file__).resolve().parent
DEFAULT_CSS = HERE / "markdown.css"
VENV = Path.home() / ".venvs" / "klammertext-tns"
# Browsers that can serve as the renderer, most preferred first. Any
# Chromium will do: the protocol is the same. The .app paths are for macOS,
# where nothing lands on PATH -- without them this finds no browser on a Mac
# that has one installed.
BROWSERS = ["/opt/brave.com/brave/brave", "brave-browser", "google-chrome",
"chromium", "chromium-browser",
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser",
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Chromium.app/Contents/MacOS/Chromium"]
# --- the renderer ----------------------------------------------------------
def setup_venv():
"""Create the virtual environment this script needs.
Offered as --setup so that a new machine takes one command rather than
two remembered ones. PEP 668 marks the system Python as
externally-managed, so a venv is not a preference here; pip refuses to
install into the system otherwise.
"""
print(f"creating {VENV}")
subprocess.run([sys.executable, "-m", "venv", str(VENV)], check=True)
pip = VENV / ("Scripts" if os.name == "nt" else "bin") / "pip"
subprocess.run([str(pip), "install", "-q", "markdown-it-py", "mdit-py-plugins"],
check=True)
print(f"installed markdown-it-py and mdit-py-plugins into {VENV}")
def ensure_renderer():
"""Re-exec under the venv interpreter unless EVERY requirement is present.
Done by re-exec rather than by manipulating sys.path so that the script
stays runnable as itself: `python3 md_to_pdf.py ...` works whether or not
the caller knows about the virtual environment.
Both packages are checked, not just the first. This machine's system
Python carries a distro markdown_it but no mdit_py_plugins, so a check of
markdown_it alone was satisfied, the re-exec never happened, and the
plugins were then silently skipped -- producing a PDF whose 138 internal
links pointed at heading anchors that had never been generated. The
links were present and did nothing, which is the worst way to fail.
"""
if "--setup" in sys.argv:
setup_venv()
sys.exit(0)
try:
import markdown_it # noqa: F401
import mdit_py_plugins.anchors # noqa: F401
return
except ImportError:
pass
venv_python = VENV / "bin" / "python"
if venv_python.exists() and Path(sys.executable) != venv_python:
os.execv(str(venv_python), [str(venv_python), *sys.argv])
sys.exit(f"the Markdown renderer is not installed. Run:\n"
f" python3 {Path(__file__).name} --setup")
def github_slug(title):
"""A heading's anchor, by GitHub's rule.
Markdown documents write their internal links against this convention --
"## 2.9 Keystone correction" is linked as "(#29-keystone-correction)" --
so the slug has to match it exactly or every cross-reference in the
document lands nowhere. Lowercase, drop everything that is not
alphanumeric, space or hyphen, then spaces to hyphens.
"""
import re
slug = re.sub(r"[^\w\s-]", "", title.strip().lower(), flags=re.UNICODE)
return re.sub(r"[\s]+", "-", slug)
def render_markdown(text):
"""Markdown -> HTML, configured like the Firefox extension's renderer:
CommonMark plus the GitHub-flavoured additions people actually write.
Heading anchors are not optional. markdown-it's core emits <h2> with no
id, so a document full of "[see](#some-section)" produces link
annotations in the PDF that point at destinations which do not exist --
the links are there, and clicking them does nothing. Measured on the
Rectify guide before this was added: 148 link annotations, 138 of them
dead.
"""
# No try/except around these: the plugins are requirements, not
# improvements, and a missing one must stop the program rather than
# quietly produce a document that is wrong in a way nobody can see.
# ensure_renderer() has already checked they are importable.
from markdown_it import MarkdownIt
from mdit_py_plugins.footnote import footnote_plugin
from mdit_py_plugins.deflist import deflist_plugin
from mdit_py_plugins.anchors import anchors_plugin
md = (MarkdownIt("commonmark", {"html": True, "linkify": True,
"typographer": True})
.enable(["table", "strikethrough"])
.use(footnote_plugin).use(deflist_plugin)
.use(anchors_plugin, max_level=6, slug_func=github_slug))
return md.render(text)
# A section number -- "2", "2.9", "3.1.4", with or without a trailing period
# -- in the two places a document writes one: at the start of a heading, and
# at the start of a table-of-contents entry. A contents entry is recognised
# as a list item whose FIRST content is a link to a fragment: that is what a
# Markdown table of contents is ("- [2.9 Keystone correction](#29-...)"), and
# requiring the link to open the item leaves a numbered reference in running
# text alone, where a wide gap would read as a mistake. Measured on the
# 47-entry contents of the Rectify guide: every numbered fragment link in the
# document opens a list item, and none appears mid-sentence.
#
# The whitespace after the number is part of each match and is CONSUMED --
# see mark_section_numbers().
HEADING_NUMBER = re.compile(r"(<h[1-6]\b[^>]*>)\s*(\d+(?:\.\d+)*\.?)\s+")
TOC_NUMBER = re.compile(r'(<li>\s*<a href="#[^"]*"[^>]*>)\s*(\d+(?:\.\d+)*\.?)\s+')
def mark_section_numbers(html):
"""Wrap a leading section number in <span class="secnum">.
The number is ordinary text in the Markdown ("## 2.9 Keystone
correction"), so the gap between it and the title is one space character
and nothing in a stylesheet can reach it. Marking the number gives the
stylesheet something to hold: markdown.css sets the distance with
--secnum-gap, and heading and contents entry take it from the same
property, so the two cannot drift apart.
The space after the number is consumed rather than left in place, so the
whole gap is the one CSS value -- otherwise it would be that value plus a
space whose width varies with the font.
This runs after rendering, not before, so the anchor slugs are computed
from the heading text as written and internal links still resolve. A
heading that opens with a number which is not a section number ("2026 in
review") gets the gap too; that is the price of not asking the document
to mark its own numbers.
"""
wrap = lambda m: f'{m.group(1)}<span class="secnum">{m.group(2)}</span>'
return TOC_NUMBER.sub(wrap, HEADING_NUMBER.sub(wrap, html))
def font_store_dirs():
"""The Klammertext font store's search order: the KLAMMERTEXT_FONTS
directories, then the distribution's own fnt/. Same order the engine
uses, so an installed font shadows a distributed one of the same name."""
dirs = [Path(d) for d in
os.environ.get("KLAMMERTEXT_FONTS",
str(Path.home() / ".klammertext" / "fonts")).split(":") if d]
home = os.environ.get("KLAMMERTEXT_HOME")
if home:
dirs.append(Path(home) / "fnt")
# This script lives in <klammertext>/sks/tns/, so it can find the
# distributed fonts without being told where they are. It has to:
# KLAMMERTEXT_HOME is set by a shell profile, and a Makefile rule or a
# non-interactive ssh session has no profile -- the guide's build failed
# on the Mac with "no font 'eb-garamond' ... Available: (none)" for
# exactly that reason, on a machine where the font was present all along.
own = HERE.parent.parent / "fnt"
if own not in dirs:
dirs.append(own)
return dirs
def font_face_css(name):
"""The @font-face rules for one font in the store, with absolute URLs.
A font in the store is a <name>/ directory of .ttf files beside a
<name>.css declaring its variants -- already the browser's own format,
which is why this needs no conversion, only a path fix: the store writes
url('eb-garamond/Regular.ttf') relative to itself, and the generated HTML
carries a <base> pointing at the Markdown file's directory, so a relative
URL would resolve somewhere else entirely and the font would silently
fall back.
Returns (css, family). A font the browser cannot find is not an error it
reports -- it just uses something else -- so an unknown name must fail
here instead.
"""
import re
for d in font_store_dirs():
css_path = d / f"{name}.css"
if not css_path.exists():
continue
css = css_path.read_text(encoding="utf-8")
css = re.sub(r"url\(\s*['\"]?([^'\")]+)['\"]?\s*\)",
lambda m: f"url('{(d / m.group(1)).resolve().as_uri()}')", css)
family = re.search(r"font-family:\s*['\"]([^'\"]+)['\"]", css)
return css, (family.group(1) if family else name)
available = sorted({p.stem for d in font_store_dirs() if d.is_dir()
for p in d.glob("*.css")})
sys.exit(f"no font {name!r} in the font store. Available: "
+ (", ".join(available) or "(none)"))
def font_metrics(ttf_path):
"""(x-height, cap-height), each as a fraction of the em, from the OS/2
table -- the same source mac/font_store.cpp reads.
A sfnt file is a table directory: numTables at offset 4, then 16-byte
entries of tag/checksum/offset/length. unitsPerEm lives at offset 18 of
'head'; sxHeight and sCapHeight at 86 and 88 of 'OS/2', and only from
version 2 of that table -- an older font reports neither, which is why
this can return zeros and the caller must cope.
"""
data = ttf_path.read_bytes()
num_tables = struct.unpack(">H", data[4:6])[0]
tables = {}
for i in range(num_tables):
off = 12 + i * 16
tag, _, start, length = struct.unpack(">4sIII", data[off:off + 16])
tables[tag.decode("latin-1").strip()] = (start, length)
if "head" not in tables or "OS/2" not in tables:
return 0.0, 0.0
head = tables["head"][0]
units = struct.unpack(">H", data[head + 18:head + 20])[0] or 1000
os2 = tables["OS/2"][0]
version = struct.unpack(">H", data[os2:os2 + 2])[0]
if version < 2:
return 0.0, 0.0
x_height = struct.unpack(">h", data[os2 + 86:os2 + 88])[0]
cap_height = struct.unpack(">h", data[os2 + 88:os2 + 90])[0]
return x_height / units, cap_height / units
def regular_face(store_dir, name):
"""The Regular face of a font in the store, which the metrics come from."""
d = store_dir / name
for candidate in ("Regular.ttf", "Regular.otf"):
if (d / candidate).exists():
return d / candidate
faces = sorted(d.glob("*.ttf")) + sorted(d.glob("*.otf"))
return faces[0] if faces else None
def scale_factors(names, match="average"):
"""How much to scale each role so it looks the size of the serif font.
Three measures, the same three sks/document/document_html.cpp offers:
xheight serif_xh / other_xh equalises lowercase -- the letters
"e" and "x" come out the same height. fontspec calls this
MatchLowercase, and it is the usual answer when an old-style
serif meets a monospace.
capheight serif_ch / other_ch equalises capitals.
average the ratio of the MEANS the compromise, and the SKS default,
so the default here agrees with the other pipeline.
Which to use is not a fact about the fonts. It depends on what the eye
lands on: prose interleaved with lowercase identifiers wants xheight,
text full of CONSTANTS wants capheight. With a face whose x-height and
cap-height are far apart -- EB Garamond is 0.400 against 0.650 -- the
average visibly satisfies neither.
Returns {role: factor}; a role whose font reports no metrics is omitted
rather than guessed at.
"""
pick = {"xheight": lambda xh, ch: xh,
"capheight": lambda xh, ch: ch,
"average": lambda xh, ch: (xh + ch) / 2.0}[match]
metrics = {}
for role, name in names.items():
if not name:
continue
for d in font_store_dirs():
if (d / f"{name}.css").exists():
face = regular_face(d, name)
if face:
xh, ch = font_metrics(face)
if xh > 0 and ch > 0:
metrics[role] = pick(xh, ch)
break
if "serif" not in metrics:
return {}
serif = metrics["serif"]
return {role: serif / m for role, m in metrics.items() if m > 0}
def font_css(serif, sans, mono, match="average"):
"""@font-face blocks plus the SKS's own custom-property names.
--serif, --sans and --mono are what sks/font/css/font.css calls them, so a
stylesheet written for one pipeline reads the same in the other. Family
names are QUOTED: an unquoted digit-initial name ("Source Sans 3") is
invalid CSS, and a font-family using it via var() computes to inherit --
the font is lost with no error anywhere.
"""
faces, variables = [], []
for role, name, fallback in (("serif", serif, "serif"),
("sans", sans, "sans-serif"),
("mono", mono, "monospace")):
if not name:
continue
css, family = font_face_css(name)
faces.append(css)
variables.append(f' --{role}: "{family}", {fallback};')
if not variables:
return ""
# Scale factors, computed rather than guessed. Both spellings are
# emitted: the short ones this script has always used, and the ones
# sks/font/css/font.css defines, so a stylesheet written for either
# pipeline works with the other.
scales = scale_factors({"serif": serif, "sans": sans, "mono": mono}, match)
alias = {"sans": "sans-serif", "mono": "monospace"}
for role, factor in scales.items():
variables.append(f" --{role}-scale: {factor:.4f};")
if role in alias:
variables.append(f" --{alias[role]}-scale: {factor:.4f};")
return ("\n".join(faces) + "\n:root {\n" + "\n".join(variables) + "\n}\n")
# A language's line-continuation character, where continuing a line is
# legal at all. Absent from this table means DO NOT WRAP: PowerShell
# continues with a backtick and a backslash would corrupt it, an .ini or
# .desktop file has no continuation whatever, and an unlabelled block is as
# likely to be a directory tree as it is to be code. Wrapping those would
# turn a document that merely looks too wide into one that is wrong -- and
# wrong silently, since a broken .desktop file reports nothing.
CONTINUATION = {"bash": "\\", "sh": "\\", "shell": "\\", "zsh": "\\",
"console": "\\", "powershell": "`", "ps1": "`"}
def unquoted_hash(line):
"""The index of a comment's #, or -1. The quote tracking is crude, but
it only has to find a # that is not inside a string."""
quote = None
for i, c in enumerate(line):
if quote:
if c == quote:
quote = None
elif c in "\"'":
quote = c
elif c == "#":
return i
return -1
def wrap_code_line(line, width, cont, indent=" "):
"""A long line as a continued sequence, or None if it cannot be done.
Two content rules, both needed for bash alone, so neither is avoided by
assuming a language:
* break only at spaces OUTSIDE quotes, or a split lands inside a string
literal and changes what the command does;
* never break inside a comment. A backslash within a shell comment
does NOT continue it -- the comment ends at the newline regardless --
so the remainder would be read as a command. Breaking BEFORE the #
is safe, because the continuation puts the comment back into the same
logical line.
"""
if len(line) <= width or line.rstrip().endswith(cont):
return None
stripped = line.lstrip()
lead = line[:len(line) - len(stripped)]
hash_at = unquoted_hash(line)
def break_points(s, floor):
quote, points = None, []
for i, c in enumerate(s):
if quote:
if c == quote:
quote = None
elif c in "\"'":
quote = c
elif c == " " and i > floor and (hash_at < 0 or i < hash_at):
points.append(i)
return points
pieces, rest, prefix = [], line, lead
while len(rest) > width:
room = width - len(cont) - 1
points = [b for b in break_points(rest, len(prefix)) if b <= room]
if not points:
break # nothing splittable in range
b = points[-1]
pieces.append(rest[:b] + " " + cont)
prefix = lead + indent
rest = prefix + rest[b + 1:]
hash_at = unquoted_hash(rest)
if not pieces:
return None
pieces.append(rest)
return pieces
def wrap_fenced_code(text, width):
"""Wrap over-long lines in fenced blocks whose language permits it.
Returns (text, wrapped, skipped): what it did, and what it did not. A
line left long still overflows the page, and an overflowing page makes
the browser shrink the WHOLE document -- so the caller reports the
remainder rather than letting it pass unnoticed.
"""
out, inside, lang, wrapped, skipped = [], False, "", 0, 0
for line in text.split("\n"):
fence = re.match(r"\s*(?:```|~~~)(\w*)", line)
if fence:
if not inside:
lang = (fence.group(1) or "").lower()
inside = not inside
out.append(line)
continue
if inside and len(line) > width:
cont = CONTINUATION.get(lang)
pieces = wrap_code_line(line, width, cont) if cont else None
if pieces:
out.extend(pieces)
wrapped += 1
continue
skipped += 1
out.append(line)
return "\n".join(out), wrapped, skipped
def wrap_and_report(text, wrap):
"""Wrap the fenced blocks and say what happened, or did not."""
text, wrapped, skipped = wrap_fenced_code(text, wrap)
if wrapped or skipped:
note = f"wrapped {wrapped} code lines at {wrap} columns"
if skipped:
note += (f"; {skipped} left long -- no continuation character "
"exists for that block's language")
print(note)
return text
def build_html(md_path, css_paths, fonts_css="", wrap=0):
"""One self-contained HTML document.
The stylesheets are INLINED rather than linked: a headless browser fetches
a linked stylesheet asynchronously, and printing can begin before it
arrives -- an unstyled PDF that looks like a CSS bug. Inline text cannot
lose that race.
A <base> element points at the Markdown file's own directory so that
relative image paths resolve, which lets the generated HTML live in a
temporary directory instead of beside the source.
"""
text = md_path.read_text(encoding="utf-8")
if wrap:
text = wrap_and_report(text, wrap)
body = mark_section_numbers(render_markdown(text))
css = fonts_css + "\n".join(Path(p).read_text(encoding="utf-8") for p in css_paths)
base = md_path.resolve().parent.as_uri() + "/"
return (f'<!doctype html>\n<html><head><meta charset="utf-8">\n'
f'<base href="{base}">\n'
f'<title>{md_path.stem}</title>\n'
f'<style>\n{css}\n</style>\n</head>\n<body>\n{body}\n</body></html>\n')
def measure_code_columns(port, url, paper, margin):
"""How many monospace characters fit on one line inside a <pre>.
Asked of the browser rather than computed, because the answer depends on
things only it knows: the mono font's advance width at the size the
stylesheet computes for it (itself a metric-derived scale factor), the
pre's padding and border, and the printable width of the paper. Any of
those can change in the stylesheet, and a wrap column written down by
hand then quietly becomes wrong in one of two directions -- too large and
a line overflows, shrinking every page; too small and the code is broken
into continuations with the right half of the box left empty. The second
is what happened here: a hand-set 71 against a real capacity of 94, so
12 of the Rectify guide's 66 code lines were being continued for no
reason (2026-08-09).
Measured under PRINT conditions, like the overflow check in
print_to_pdf: on screen the viewport width and the @media print padding
are both wrong.
"""
width, height = (8.27, 11.69) if paper == "A4" else (8.5, 11.0)
available = (width - 2 * margin) * 96.0
ws = WebSocket(page_socket(port))
ws.call("Page.enable")
ws.call("Page.navigate", url=url)
ws.wait_for("Page.loadEventFired")
ws.call("Emulation.setEmulatedMedia", media="print")
ws.call("Emulation.setDeviceMetricsOverride", width=int(available), height=1200,
deviceScaleFactor=1, mobile=False)
# Every pre is measured, not just the first, and the NARROWEST answer
# wins: a block inside a list item is indented, and wrapping the document
# to what a full-width block holds would leave the indented ones to soft
# wrap -- the ragged break with no continuation character that the
# wrapping exists to replace. Measured on the Rectify guide: 624px at
# the margin, 584px inside a list.
#
# The font is taken from the element that actually carries the text, the
# <code> inside the <pre>, never the <pre> itself. Those are not the
# same size: the stylesheet's "code, pre" scale rule applies to both, so
# a <code> nested in a <pre> is scaled twice (14.00px -> 12.26px here).
# Measuring the pre's font therefore understates capacity by that factor
# -- 88 columns against the 104 the page really holds.
#
# A document with no code block at all returns 0, and the caller then
# leaves the text alone.
measure = ws.call("Runtime.evaluate", returnByValue=True, expression="""
(() => {
const pres = [...document.querySelectorAll('pre')];
if (!pres.length) return {columns: 0};
const probe = document.createElement('span');
probe.style.position = 'absolute';
probe.style.whiteSpace = 'pre';
probe.textContent = 'x'.repeat(100);
document.body.appendChild(probe);
let columns = Infinity;
for (const pre of pres) {
const cs = getComputedStyle(pre);
const content = pre.clientWidth
- parseFloat(cs.paddingLeft) - parseFloat(cs.paddingRight);
probe.style.font = getComputedStyle(pre.querySelector('code') || pre).font;
const charWidth = probe.getBoundingClientRect().width / 100;
if (content > 0 && charWidth > 0)
columns = Math.min(columns, content / charWidth);
}
probe.remove();
return {columns: columns === Infinity ? 0 : columns};
})()""")["result"]["value"]
ws.call("Emulation.clearDeviceMetricsOverride")
ws.call("Emulation.setEmulatedMedia", media="")
if not measure["columns"]:
return 0
# One column in hand: the browser's soft wrap and this arithmetic agree to
# within a rounding error, and a line that lands exactly on the edge would
# be broken by the browser anyway -- undoing the point of wrapping it.
return max(20, int(measure["columns"]) - 1)
# --- a WebSocket client, standard library only -----------------------------
class WebSocket:
"""The minimum client needed to talk CDP: RFC 6455 text frames, masked
outbound, reassembled inbound, with pings answered.
Inbound frames must handle the 64-bit length case: a printToPDF reply
carries the whole document as base64 and is routinely megabytes, far past
the 125-byte and 65535-byte forms.
"""
def __init__(self, url):
_, _, rest = url.partition("://")
hostport, _, path = rest.partition("/")
host, _, port = hostport.partition(":")
self.sock = socket.create_connection((host, int(port or 80)))
key = base64.b64encode(os.urandom(16)).decode()
self.sock.sendall(
f"GET /{path} HTTP/1.1\r\nHost: {hostport}\r\n"
f"Upgrade: websocket\r\nConnection: Upgrade\r\n"
f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n"
.encode())
# Read the handshake response one byte at a time: reading in blocks
# would consume the start of the first frame along with it.
head = b""
while not head.endswith(b"\r\n\r\n"):
b = self.sock.recv(1)
if not b:
raise RuntimeError("the browser closed the connection during the handshake")
head += b
if b"101" not in head.split(b"\r\n")[0]:
raise RuntimeError(f"websocket upgrade refused: {head.splitlines()[0]!r}")
self.next_id = 0
def _read(self, n):
buf = b""
while len(buf) < n:
chunk = self.sock.recv(n - len(buf))
if not chunk:
raise RuntimeError("the browser closed the connection")
buf += chunk
return buf
def send(self, method, **params):
self.next_id += 1
payload = json.dumps({"id": self.next_id, "method": method,
"params": params}).encode()
header = bytearray([0x81]) # FIN + text
n = len(payload)
if n < 126:
header.append(0x80 | n)
elif n < 65536:
header.append(0x80 | 126); header += struct.pack(">H", n)
else:
header.append(0x80 | 127); header += struct.pack(">Q", n)
mask = os.urandom(4)
header += mask
self.sock.sendall(bytes(header) +
bytes(b ^ mask[i % 4] for i, b in enumerate(payload)))
return self.next_id
def recv(self):
"""One complete message, reassembling continuation frames."""
message = b""
while True:
b0, b1 = self._read(2)
opcode, length = b0 & 0x0F, b1 & 0x7F
if length == 126:
length = struct.unpack(">H", self._read(2))[0]
elif length == 127:
length = struct.unpack(">Q", self._read(8))[0]
data = self._read(length) # server frames are never masked
if opcode == 0x9: # ping -> pong, then keep reading
self.sock.sendall(b"\x8a\x80" + os.urandom(4))
continue
if opcode == 0x8:
raise RuntimeError("the browser closed the websocket")
message += data
if b0 & 0x80: # FIN
return json.loads(message)
def call(self, method, **params):
"""Send a command and return its result, skipping the event stream."""
want = self.send(method, **params)
while True:
msg = self.recv()
if msg.get("id") == want:
if "error" in msg:
raise RuntimeError(f"{method}: {msg['error']}")
return msg.get("result", {})
def wait_for(self, event, timeout=60):
deadline = time.time() + timeout
while time.time() < deadline:
if self.recv().get("method") == event:
return True
raise RuntimeError(f"timed out waiting for {event}")
# --- the browser -----------------------------------------------------------
def find_browser(explicit):
for candidate in ([explicit] if explicit else BROWSERS):
path = shutil.which(candidate) or (candidate if Path(candidate).exists() else None)
if path:
return path
sys.exit("no Chromium-based browser found; pass --browser PATH")
def free_port():
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
def start_browser(browser, profile, port):
"""Headless, with its own profile so a running browser is untouched.
--user-data-dir matters for more than tidiness: without it the launch
would attach to the user's existing session, and killing it afterwards
would close their windows.
"""
proc = subprocess.Popen(
[browser, "--headless=new", "--disable-gpu", "--no-sandbox",
"--no-first-run", "--no-default-browser-check",
"--disable-component-update", "--disable-background-networking",
f"--user-data-dir={profile}", f"--remote-debugging-port={port}",
"about:blank"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
deadline = time.time() + 60
while time.time() < deadline:
if proc.poll() is not None:
sys.exit(f"{browser} exited before the protocol port opened")
try:
with urllib.request.urlopen(f"http://127.0.0.1:{port}/json/version",
timeout=1) as r:
json.load(r)
return proc
except Exception:
time.sleep(0.2)
proc.kill()
sys.exit("the browser never opened its DevTools port")
# Chromium's own page header and footer, requested by --page-numbers.
# printToPDF takes HTML templates, in which a handful of magic classes are
# substituted -- pageNumber, totalPages, title, url, date. The templates are
# rendered in their own context with a default font size of a few pixels, so
# they must carry their own inline style or they come out unreadably small,
# and that context is NOT the document's: the template cannot use the
# stylesheet's fonts or custom properties, which is why this one names a
# generic sans.
#
# The CSS alternative now exists, and this is kept anyway. Paged Media
# margin boxes (@page { @bottom-center { content: counter(page) } }) were
# implemented by no browser when this was written; measured 2026-08-09 on
# Brave 151 (Chromium 151), they work, count pages, and honour a
# font-family taken from a custom property. A stylesheet that wants the
# number in the document's own face should use them. Two cautions, both
# measured on the same day: a webfont the document body never uses is not
# loaded for a margin box -- var(--mono) in a box of a prose-only document
# fell back to the platform monospace, and in one variant the box rendered
# NOTHING at all -- and the feature is recent enough that an older Chromium
# silently prints no number. --page-numbers has neither hazard, so it stays
# the default answer and the templates stay here.
FOOTER = ('<div style="font-family:sans-serif; font-size:9px; color:#555; '
'width:100%; text-align:center; margin:0 0.5in;">'
'<span class="pageNumber"></span> of <span class="totalPages"></span>'
'</div>')
# displayHeaderFooter turns BOTH on, and an unset header template falls back
# to Chromium's default (title and date). An empty div suppresses it.
EMPTY = '<div></div>'
def page_socket(port):
"""The DevTools socket of the browser's existing page.
Attach to the about:blank page the browser was launched with, rather than
asking for a new target: /json/new has required PUT since a recent
Chromium (a GET or POST answers 405), and there is already a page.
"""
with urllib.request.urlopen(f"http://127.0.0.1:{port}/json/list", timeout=10) as r:
targets = json.load(r)
pages = [t for t in targets if t.get("type") == "page" and t.get("webSocketDebuggerUrl")]
if not pages:
raise RuntimeError("the browser exposed no page to print")
return pages[0]["webSocketDebuggerUrl"]
def print_to_pdf(port, url, pdf_path, paper, margin, page_numbers=False):
"""Drive one page through load and print."""
ws = WebSocket(page_socket(port))
ws.call("Page.enable")
ws.call("Page.navigate", url=url)
ws.wait_for("Page.loadEventFired")
width, height = (8.27, 11.69) if paper == "A4" else (8.5, 11.0)
# Warn if the document is wider than the page. This is the failure that
# cost the most to find: Chromium does not clip or paginate overflow when
# printing, it SHRINKS THE WHOLE DOCUMENT until the widest element fits.
# So one over-long code line silently rescales every page, by a factor
# that changes whenever that line does -- and every font size then looks
# wrong for a reason nothing in the CSS explains. Measured here rather
# than assumed, and reported rather than left to be discovered.
#
# The measurement must be made under PRINT conditions. Measuring the
# page as it stands reports the browser window's width, which has nothing
# to do with the paper, and the @media print rules -- which change the
# padding and sizes that decide whether anything overflows -- are not
# even in effect. So emulate print media, force the viewport to the
# printable width, measure, and put both back before printing.
available = (width - 2 * margin) * 96.0 # CSS pixels of printable width
ws.call("Emulation.setEmulatedMedia", media="print")
ws.call("Emulation.setDeviceMetricsOverride", width=int(available), height=1200,
deviceScaleFactor=1, mobile=False)
measure = ws.call("Runtime.evaluate", returnByValue=True, expression="""
(() => {
const pres = [...document.querySelectorAll('pre')];
let charWidth = 0;
if (pres.length) {
const probe = document.createElement('span');
probe.style.font = getComputedStyle(pres[0]).font;
probe.style.position = 'absolute';
probe.style.whiteSpace = 'pre';
probe.textContent = 'x'.repeat(100);
document.body.appendChild(probe);
charWidth = probe.getBoundingClientRect().width / 100;
probe.remove();
}
return {width: document.body.scrollWidth,
over: pres.filter(e => e.scrollWidth > e.clientWidth + 1).length,
charWidth: charWidth};
})()""")["result"]["value"]
ws.call("Emulation.clearDeviceMetricsOverride")
ws.call("Emulation.setEmulatedMedia", media="")
if measure["width"] > available + 1:
shrink = available / measure["width"]
fits = int(available / measure["charWidth"]) if measure["charWidth"] else 0
print(f"warning: the content is {measure['width']:.0f}px wide but the page "
f"holds {available:.0f}px, so the browser will shrink every page to "
f"{shrink * 100:.0f}%.")
if measure["over"]:
print(f" {measure['over']} code blocks overflow; about {fits} "
f"columns fit. Try --wrap-code {fits}, or add "
"'pre {{ white-space: pre-wrap }}' to the stylesheet."
.replace("{{", "{").replace("}}", "}"))
# A footer needs room to sit in: with too small a bottom margin Chromium
# renders it under the text or not at all.
bottom = max(margin, 0.6) if page_numbers else margin
result = ws.call(
"Page.printToPDF",
printBackground=True,
displayHeaderFooter=page_numbers,
headerTemplate=EMPTY,
footerTemplate=FOOTER if page_numbers else EMPTY,
# The stylesheet is the point of this program, so let an @page rule in
# it win over these defaults when it says anything.
preferCSSPageSize=True,
paperWidth=width, paperHeight=height,
marginTop=margin, marginBottom=bottom,
marginLeft=margin, marginRight=margin)
Path(pdf_path).write_bytes(base64.b64decode(result["data"]))
def main():
ap = argparse.ArgumentParser(description="Markdown -> PDF via headless Chromium (CDP)")
ap.add_argument("input", nargs="?")
ap.add_argument("output", nargs="?")
ap.add_argument("--css", action="append", default=None,
help=f"stylesheet to apply (repeatable; default {DEFAULT_CSS.name})")
ap.add_argument("--browser", help="path to a Chromium-based browser")
ap.add_argument("--paper", choices=["A4", "letter"], default="A4")
ap.add_argument("--margin", type=float, default=0.5, help="inches")
ap.add_argument("--setup", action="store_true",
help="create the virtual environment this script needs, and exit")
ap.add_argument("--page-numbers", action="store_true",
help='number the pages ("3 of 21") in the footer')
ap.add_argument("--keep-html", action="store_true",
help="keep the intermediate HTML beside the PDF")
ap.add_argument("--serif", help="serif font from the Klammertext font store")
ap.add_argument("--sans", help="sans font from the Klammertext font store")
ap.add_argument("--mono", help="monospace font from the Klammertext font store")
ap.add_argument("--wrap-code", default=0, metavar="COLUMNS",
help="break code lines longer than COLUMNS at a word "
"boundary, using the language's continuation "
"character and an indent; a block whose language has "
"no continuation is left alone and reported. "
"\"auto\" asks the browser how many characters fit "
"in a code box and uses that, which is right by "
"construction when the fonts, sizes or margins change")
ap.add_argument("--match", choices=["average", "xheight", "capheight"],
default="average",
help="which measure the sans and mono fonts are scaled to "
"match: xheight equalises lowercase (the letter \"e\"), "
"capheight equalises capitals, average is the "
"compromise and the SKS default")
args = ap.parse_args()
if not args.input:
ap.error("an input file is required")
md_path = Path(args.input)
if not md_path.exists():
sys.exit(f"no such file: {md_path}")
pdf_path = Path(args.output) if args.output else md_path.with_suffix(".pdf")
css_paths = args.css or ([str(DEFAULT_CSS)] if DEFAULT_CSS.exists() else [])
for c in css_paths:
if not Path(c).exists():
sys.exit(f"no such stylesheet: {c}")
fonts_css = font_css(args.serif, args.sans, args.mono, args.match)
auto_wrap = str(args.wrap_code).lower() == "auto"
if not auto_wrap and not str(args.wrap_code).isdigit():
ap.error(f"--wrap-code takes a column count or \"auto\", "
f"not {args.wrap_code!r}")
wrap = 0 if auto_wrap else int(args.wrap_code)
html = build_html(md_path, css_paths, fonts_css, wrap)
work = Path(tempfile.mkdtemp(prefix="md_to_pdf."))
try:
html_path = work / (md_path.stem + ".html")
html_path.write_text(html, encoding="utf-8")
browser = find_browser(args.browser)
port = free_port()
proc = start_browser(browser, work / "profile", port)
try:
# --wrap-code auto: the unwrapped document is already loaded, so
# ask it how wide a code line may be, then build the real one.
# Two loads of the same page cost about a second and remove the
# only number in this pipeline that had to be guessed.
if auto_wrap:
columns = measure_code_columns(port, html_path.as_uri(),
args.paper, args.margin)
if columns:
html = build_html(md_path, css_paths, fonts_css, columns)
html_path.write_text(html, encoding="utf-8")
print_to_pdf(port, html_path.as_uri(), pdf_path, args.paper,
args.margin, args.page_numbers)
finally:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
if args.keep_html:
kept = pdf_path.with_suffix(".html")
kept.write_text(html, encoding="utf-8")
print(f"{kept}")
print(f"{pdf_path} ({pdf_path.stat().st_size} bytes, "
f"{Path(browser).name}, {len(css_paths)} stylesheet"
f"{'s' if len(css_paths) != 1 else ''})")
finally:
shutil.rmtree(work, ignore_errors=True)
if __name__ == "__main__":
ensure_renderer()
main()

327
sks/tns/md_to_sks.py Normal file
View File

@@ -0,0 +1,327 @@
#!/usr/bin/env python3
"""Markdown -> Standard Klammer Set converter (first implementation).
SPECIFICATION: doc/markdown_to_klammertext.md. That document is authoritative
and is written to be sufficient to rebuild this program; this program is its
reference implementation, not the other way round. Its "The algorithm"
section states the pass order, which is forced, and the traps that make it so.
The conversion is to the SKS, not to Klammertext the language: the
correspondence between a Markdown construct and a klammer is a property of
the klammer set. That is why this lives under sks/ rather than in mac/.
What cannot be converted exactly is recorded in the output as a
#[MD <kind>: ... ]# marker -- lossy, gap, or judgment -- so a converted
document carries its own worklist and "how far from done is this file" is a
grep. A file with no markers is inside the convertible subset.
First run: the Rectify user guide (1,805 lines), 2026-08-07. It rendered to
html and pdf without error or warning, with 99 markers -- 91 of them internal
anchor links, the one gap that appears at volume.
Usage: python3 md_to_sks.py <input.md> <output.kt>
"""
import re, sys, os, datetime
from collections import Counter
SPECIALS = "^@#|*" # ^ first: the other quotings introduce ^
marks = Counter()
def mark(kind, text):
"""A #[MD ... ]# marker.
Two things have to be neutralised in the text, both of which cost a
session's debugging if they are not:
* "]#" would close the block early and "#[" would open one that must then
be balanced (the nesting hazard);
* a klammer name written with its "@" is READ before the block is
removed -- mark_literal_klammer_content() runs first in
process_katoms() -- so "@code has no :language" makes the engine demand
a "code@" close from inside removed text. Quoting the @ makes it a
special katom, which the literal scan does not see. (TODO #39.)
"""
marks[kind] += 1
text = text.replace("]#", "] #").replace("#[", "# [")
text = text.replace("@", "^@").replace("|", "^|")
return f"#[MD {kind}: {text} ]#"
def quote(s):
"""Quote every Klammertext special in writer text."""
for ch in SPECIALS:
s = s.replace(ch, "^" + ch)
return s
# ---- inline ---------------------------------------------------------------
CODE = re.compile(r'`([^`\n]+)`')
LINK = re.compile(r'(?<!\!)\[([^\]]*)\]\(([^)]+)\)')
BOLD = re.compile(r'\*\*([^*]+)\*\*|__([^_]+)__')
# Markdown's underscore emphasis does not open or close INSIDE a word, and
# technical prose is full of identifiers like *name*_rectified_1.*ext* where
# the underscores are part of the name. So the closing _ must not be
# followed by a word character -- without that, the converter invents
# emphasis in the middle of filenames.
ITAL = re.compile(r'(?<![\*\w])\*([^*\s][^*]*)\*(?!\*)|(?<![_\w])_([^_\s][^_]*)_(?![\w_])')
STRIKE = re.compile(r'~~([^~]+)~~')
def inline(text):
"""Convert one run of inline Markdown.
Order matters: code spans are protected first (their content must not be
read as emphasis), links and emphasis consume their markers next, and only
then are the remaining specials quoted -- quoting first would hide the
** and * that emphasis is recognised by.
"""
spans = []
def stash(m):
spans.append(m.group(1))
return f"\x00{len(spans)-1}\x00"
text = CODE.sub(stash, text)
# Markers are Klammertext, not writer text: they must survive the quoting
# pass below untouched, or the "#[" that makes them a removal block is
# itself quoted and the marker renders into the output instead of
# vanishing from it. Stash them like code spans.
verbatim = []
def keep(s):
verbatim.append(s)
return f"\x01{len(verbatim)-1}\x01"
# OPEN and CLOSE wrap the delimiters this function emits, so the
# whitespace fix below can find exactly its own delimiters. Matching
# them by regex on the finished text does not work: " @" is both the
# closing delimiter and the space before an opening "@i", and a rule
# keyed on it rewrites "@i" into "@#- i".
OPEN, CLOSE = "\x03", "\x02"
def opening(s): return keep(OPEN + s)
def closing(): return keep(" @" + CLOSE)
def link(m):
label, target = m.group(1), m.group(2)
if target.startswith("#"):
# An anchor into this document: no klammer gives an arbitrary
# heading an anchor, so keep the words and record the loss.
return emphasis(label) + " " + keep(mark("gap", f"link to {target}"))
return opening("@link " + quote(target) + " :text ") + emphasis(label) + closing()
def emphasis(t):
"""The marker-consuming transforms, sharing THIS call's stashes.
A link label or a struck-through run is itself inline Markdown and
must be converted -- but not by re-entering inline(), which would
start empty stashes while the text already holds this call's
placeholders. A code span inside a link label then restores against
the wrong list: IndexError, found by converting a second document.
"""
t = LINK.sub(link, t)
t = STRIKE.sub(lambda m: emphasis(m.group(1)) + " " +
keep(mark("gap", "strikethrough")), t)
t = BOLD.sub(lambda m: opening("@b ") + (m.group(1) or m.group(2)) + closing(), t)
return ITAL.sub(lambda m: opening("@i ") + (m.group(1) or m.group(2)) + closing(), t)
text = emphasis(text)
# Everything that is still writer text gets quoted; the stashed fragments
# are Klammertext already.
parts = re.split(r'(\x00\d+\x00|\x01\d+\x01)', text)
text = "".join(p if re.match(r'^[\x00\x01]', p) else quote(p) for p in parts)
text = re.sub(r'\x01(\d+)\x01', lambda m: verbatim[int(m.group(1))], text)
text = re.sub(r'\x00(\d+)\x00',
lambda m: OPEN + "@c " + quote(spans[int(m.group(1))]) + " @" + CLOSE, text)
# Klammertext delimiters need whitespace around them, but Markdown
# emphasis abuts its neighbours: "un**bold**ed" and "*name*_rectified"
# both put a word character hard against a delimiter, which the
# katomizer then reads as an unparsable word. A space makes it parse,
# and "#-" removes that space again from the OUTPUT, so intra-word
# emphasis converts exactly rather than approximately.
# Any non-space neighbour, not just a word character: "*name*_x" and
# "_1.*ext*" put an underscore or a period against the delimiter.
text = re.sub(r'(?<=\S)' + OPEN, ' #-', text).replace(OPEN, "")
text = re.sub(CLOSE + r'(?=[^\s])', '#- ', text).replace(CLOSE, "")
# A closing "@" abutting the next klammer's opening "@" spells "@@",
# which is a DEFINITION delimiter -- the engine then reports a span that
# ends without a beginning, pointing at text the writer never wrote.
# Adjacent klammers are ordinary in converted prose (*a*_b_ produces
# two), so separate them.
return re.sub(r'@(?=@)', '@ ', text)
# ---- blocks ---------------------------------------------------------------
HEADING = re.compile(r'^(#{1,6})\s+(.*?)\s*$')
MANUAL_NUMBER = re.compile(r'^\d+(\.\d+)*\.?\s+')
FENCE = re.compile(r'^\s*```(\w*)\s*$')
BULLET = re.compile(r'^(\s*)[-*+]\s+(.*)$')
ORDERED = re.compile(r'^(\s*)(\d+)[.)]\s+(.*)$')
TABLEROW = re.compile(r'^\s*\|(.*)\|\s*$')
ALIGNROW = re.compile(r'^\s*\|[\s:|-]+\|\s*$')
def cells(line):
return [c.strip() for c in TABLEROW.match(line).group(1).split("|")]
def convert(path):
src = open(path, encoding="utf-8").read().split("\n")
out, i, n = [], 0, len(src)
title = None
stripped_numbers = 0
dropped_toc = False
while i < n:
line = src[i]
m = FENCE.match(line)
if m: # fenced code
lang, body, i = m.group(1), [], i + 1
while i < n and not FENCE.match(src[i]):
body.append(src[i]); i += 1
i += 1
if lang:
out.append(mark("lossy", f'fenced language "{lang}" dropped '
"-- @code has no :language"))
out.append("@code |")
out.extend(body) # literal: nothing to quote
out.append("code@")
out.append("")
continue
m = HEADING.match(line)
if m:
level, text = len(m.group(1)), m.group(2)
if level == 1 and title is None:
title = inline(text); i += 1; continue
if MANUAL_NUMBER.match(text):
text = MANUAL_NUMBER.sub("", text); stripped_numbers += 1
if re.match(r'^contents$', text, re.I):
# Klammertext generates a table of contents; a hand-written
# one would duplicate it. Skip to the next heading.
j = i + 1
while j < n and not HEADING.match(src[j]):
j += 1
out.append(mark("judgment",
"a hand-written Contents section was dropped; "
":structure article generates one"))
out.append("")
dropped_toc = True
i = j
continue
out.append(f"@s{level-1} {inline(text)} @")
out.append("")
i += 1
continue
if TABLEROW.match(line): # table
rows, align = [], None
while i < n and TABLEROW.match(src[i]):
if ALIGNROW.match(src[i]):
align = [("r" if c.endswith(":") and c.startswith(":") is False
else "c" if c.startswith(":") and c.endswith(":")
else "l") for c in cells(src[i])]
else:
rows.append([inline(c) for c in cells(src[i])])
i += 1
body = " ||\n".join(" | ".join(r) for r in rows)
opts = f" :cell_hpos {' '.join(align)}" if align and set(align) != {"l"} else ""
# A Markdown table says nothing about column widths, and
# Klammertext's default "fit" never wraps -- a prose column then
# runs off the page (the pdf target warns). Give the widest
# column "fill", which takes the remaining width but no more than
# its own widest line.
widths = [max((len(r[c]) for r in rows if c < len(r)), default=0)
for c in range(max(len(r) for r in rows))]
if max(widths) > 40:
widest = widths.index(max(widths))
spec = " ".join("fill" if c == widest else "fit"
for c in range(len(widths)))
opts += f" :column_width {spec}"
out.append(mark("judgment",
"column widths are not in the Markdown; the "
f"widest column ({max(widths)} characters) was "
"given fill so the table wraps instead of "
"overflowing"))
out.append(f"@table{opts} |")
out.append(body)
out.append("table@")
out.append("")
continue
if BULLET.match(line) or ORDERED.match(line):
block, base = [], None
while i < n and (BULLET.match(src[i]) or ORDERED.match(src[i]) or
(src[i].strip() and src[i].startswith((" ", "\t")))):
block.append(src[i]); i += 1
out.append(convert_list(block))
out.append("")
continue
out.append(inline(line) if line.strip() else "")
i += 1
return out, title, stripped_numbers, dropped_toc
def convert_list(block):
"""One list, possibly nested, as @ul/@ol with bar-separated items."""
def emit(items, ordered, depth):
klammer = "@ol" if ordered else "@ul"
parts = []
for text, children in items:
body = inline(text)
if children:
body += " " + emit(children[0], children[1], depth + 1)
parts.append(body)
return klammer + " " + " | ".join(parts) + " @"
def parse(lines, indent):
items, i = [], 0
ordered = bool(ORDERED.match(lines[0])) if lines else False
while i < len(lines):
m = BULLET.match(lines[i]) or ORDERED.match(lines[i])
if not m or len(m.group(1)) < indent:
break
text = m.groups()[-1]
i += 1
child_lines = []
while i < len(lines):
m2 = BULLET.match(lines[i]) or ORDERED.match(lines[i])
if m2 and len(m2.group(1)) > indent:
child_lines.append(lines[i]); i += 1
elif not m2 and lines[i].strip():
text += " " + lines[i].strip(); i += 1
else:
break
children = None
if child_lines:
sub, sub_ordered = parse(child_lines, len(BULLET.match(child_lines[0]).group(1))
if BULLET.match(child_lines[0])
else len(ORDERED.match(child_lines[0]).group(1)))
children = (sub, sub_ordered)
items.append((text, children))
return items, ordered
items, ordered = parse([l for l in block if l.strip()], 0)
return emit(items, ordered, 0)
def main():
src_path, dst_path = sys.argv[1], sys.argv[2]
body, title, stripped, dropped_toc = convert(src_path)
today = datetime.date.today().isoformat()
head = [f"#[MD source: {os.path.abspath(src_path)}",
f" converted {today} ]#", ""]
if stripped:
head.append(mark("judgment",
f"{stripped} headings carried a manual number "
'("2.1 ..."); the numbers were removed because @s1/@s2 '
"number the headings themselves"))
head.append("")
head += ["@document", ":structure plain"]
if title:
head.append(f":title {title}")
head += [":text", ""]
text = "\n".join(head + body + ["", "@", ""])
text = re.sub(r'\n{3,}', "\n\n", text)
open(dst_path, "w", encoding="utf-8").write(text)
print(f"{dst_path}: {len(text.splitlines())} lines")
for kind in ("source", "lossy", "gap", "judgment"):
print(f" {kind:9} {marks[kind]}")
main()

94
sks/tns/mdpdf.sh Normal file
View File

@@ -0,0 +1,94 @@
# mdpdf -- Markdown to PDF in one word. A shell front end to md_to_pdf.py.
#
# mdpdf notes.md # writes notes.pdf beside it
# mdpdf notes # the .md may be left off
# mdpdf notes.md out.pdf --page-numbers
#
# This file is SOURCED, not run: `mdpdf` has to be a shell function so that
# TAB completion can be attached to it (compdef and complete both work on
# functions and commands, never on aliases). env/runtime.env sources it, so
# a user who has the Klammertext environment at all has the command; sourcing
# it by hand is only for a shell that does not load runtime.env:
#
# source "$KLAMMERTEXT_HOME/sks/tns/mdpdf.sh"
#
# It is plain POSIX shell rather than zsh, and works under bash, zsh and dash
# alike. Nothing here needs zsh -- and runtime.env is sourced from bash
# profiles too, so a zsh-only file would break for those users at the source,
# with a syntax error rather than a message.
#
# WHAT THE FUNCTION ADDS over calling md_to_pdf.py directly: the house font
# and wrapping defaults, tolerance of a bare basename, and completion. Each
# default is a variable, so a user overrides one in their profile without
# copying the function:
#
# MDPDF_SERIF, MDPDF_SANS, MDPDF_MONO font-store names
# MDPDF_MATCH average | xheight | capheight
# MDPDF_WRAP columns for --wrap-code;
# auto = measure, 0 = off
# MDPDF_BROWSER path to a Chromium-based browser
#
# They are read at CALL time, not here, so setting one after this file is
# sourced still takes effect. Any further md_to_pdf.py option may be given on
# the command line and wins over the default, since argparse takes the last
# occurrence of an option.
#
# No particular browser is required. md_to_pdf.py finds one from its own
# list -- Brave, then Chrome, then Chromium, on Linux and macOS -- because
# they all speak the same DevTools protocol; MDPDF_BROWSER is for a browser
# installed somewhere unusual, not for choosing a brand.
mdpdf() {
if [ -z "$KLAMMERTEXT_HOME" ]; then
printf 'mdpdf: KLAMMERTEXT_HOME is not set -- source env/runtime.env\n' >&2
return 1
fi
if [ "$#" -lt 1 ]; then
printf 'usage: mdpdf <file[.md]> [output.pdf] [md_to_pdf.py option ...]\n' >&2
return 1
fi
_mdpdf_in="$1"
shift
# A bare basename is accepted, but only as a fallback: a file that exists
# under the name given is never reinterpreted.
if [ ! -f "$_mdpdf_in" ] && [ -f "$_mdpdf_in.md" ]; then
_mdpdf_in="$_mdpdf_in.md"
fi
if [ ! -f "$_mdpdf_in" ]; then
printf 'mdpdf: no such file: %s\n' "$_mdpdf_in" >&2
unset _mdpdf_in
return 1
fi
# Prepended, so an explicit --browser on the command line still wins.
if [ -n "$MDPDF_BROWSER" ]; then
set -- --browser "$MDPDF_BROWSER" "$@"
fi
python3 "$KLAMMERTEXT_HOME/sks/tns/md_to_pdf.py" "$_mdpdf_in" \
--serif "${MDPDF_SERIF:-eb-garamond}" \
--sans "${MDPDF_SANS:-source-sans-3}" \
--mono "${MDPDF_MONO:-inconsolata}" \
--match "${MDPDF_MATCH:-xheight}" \
--wrap-code "${MDPDF_WRAP:-auto}" \
"$@"
_mdpdf_status=$?
unset _mdpdf_in
return $_mdpdf_status
}
# Completion, where the shell has it. In zsh compdef exists only after
# compinit has run, so its absence is not an error; in bash the form is the
# one doc/argument_completion.md gives for the other commands, plus -d so
# directories can still be descended into.
if [ -n "$ZSH_VERSION" ]; then
whence compdef >/dev/null 2>&1 && compdef '_files -g "*.md"' mdpdf
elif [ -n "$BASH_VERSION" ]; then
complete -f -d -X '!*.md' mdpdf
fi
# Sourced from runtime.env, whose own exit status must stay 0: without this,
# a shell where the completion test failed would report failure for
# `source runtime.env`.
true