From f855c5ccae938ed0121b136622c418c9d4d9fcab Mon Sep 17 00:00:00 2001 From: Andy Kopra Date: Mon, 27 Jul 2026 15:01:49 +0200 Subject: [PATCH] Editor support generalized: shared core, language server, Vim and VS Code (from dev eb5baf9cbe59) doc/edit/ now holds a shared Python implementation of the language's structural layer (klammertext_edit.py) and a dependency-free language server (klammertext_ls.py), with integrations for Emacs, Sublime Text, Vim, and Visual Studio Code. The editor test suite in tst/ covers the core's API and CLI, the language server protocol, the VS Code extension, headless Vim, and Emacs byte-equality. Co-Authored-By: Claude Fable 5 --- README.md | 8 +- doc/edit/README.md | 85 ++ doc/edit/emacs/klammertext-align.el | 9 +- doc/edit/emacs/klammertext-indent.el | 13 +- doc/edit/emacs/klammertext-mode.el | 17 +- doc/edit/shared/klammertext_edit.py | 1083 +++++++++++++++++ doc/edit/shared/klammertext_ls.py | 356 ++++++ doc/edit/sublime/Klammertext.py | 592 +++------ doc/edit/sublime/Klammertext.sublime-syntax | 18 +- doc/edit/sublime/Klammertext_align.py | 410 +------ doc/edit/sublime/Klammertext_indent.py | 260 +--- doc/edit/sublime/README.md | 65 +- doc/edit/vim/README.md | 117 ++ doc/edit/vim/autoload/klammertext.vim | 299 +++++ doc/edit/vim/ftdetect/klammertext.vim | 10 + doc/edit/vim/ftplugin/klammertext.vim | 63 + doc/edit/vim/syntax/klammertext.vim | 113 ++ doc/edit/vscode/README.md | 100 ++ doc/edit/vscode/extension.js | 309 +++++ doc/edit/vscode/language-configuration.json | 11 + doc/edit/vscode/package.json | 85 ++ .../syntaxes/klammertext.tmLanguage.json | 132 ++ tst/editor/editor_driver.py | 78 +- tst/editor/ls_test.py | 262 ++++ tst/editor/vim_feature_test.vim | 179 +++ tst/editor/vscode_ext_test.js | 225 ++++ tst/editor_test.sh | 189 ++- 27 files changed, 3918 insertions(+), 1170 deletions(-) create mode 100644 doc/edit/README.md create mode 100644 doc/edit/shared/klammertext_edit.py create mode 100644 doc/edit/shared/klammertext_ls.py create mode 100644 doc/edit/vim/README.md create mode 100644 doc/edit/vim/autoload/klammertext.vim create mode 100644 doc/edit/vim/ftdetect/klammertext.vim create mode 100644 doc/edit/vim/ftplugin/klammertext.vim create mode 100644 doc/edit/vim/syntax/klammertext.vim create mode 100644 doc/edit/vscode/README.md create mode 100644 doc/edit/vscode/extension.js create mode 100644 doc/edit/vscode/language-configuration.json create mode 100644 doc/edit/vscode/package.json create mode 100644 doc/edit/vscode/syntaxes/klammertext.tmLanguage.json create mode 100644 tst/editor/ls_test.py create mode 100644 tst/editor/vim_feature_test.vim create mode 100644 tst/editor/vscode_ext_test.js diff --git a/README.md b/README.md index 04c7bc1..5dee23b 100644 --- a/README.md +++ b/README.md @@ -27,8 +27,10 @@ on). ## Editor support -Syntax highlighting and editing support for Emacs and Sublime Text are in -[`doc/edit/`](doc/edit/). +Editing support for Emacs, Sublime Text, Vim, and Visual Studio Code — +syntax highlighting, delimiter matching, structural reindentation, table +alignment, diagnostics — is in [`doc/edit/`](doc/edit/), together with the +shared implementation and the Klammertext language server they build on. ## Provenance @@ -39,7 +41,7 @@ are regenerated on each release — patches cannot be merged directly. Report problems (or send patches) to the author; accepted changes are applied to the development tree and appear in a following snapshot. -This snapshot was assembled from development commit `4e8c3db2882d`. +This snapshot was assembled from development commit `eb5baf9cbe59`. ## License diff --git a/doc/edit/README.md b/doc/edit/README.md new file mode 100644 index 0000000..7a4aec2 --- /dev/null +++ b/doc/edit/README.md @@ -0,0 +1,85 @@ +# Klammertext editor support + +Editing support for Klammertext files (`.kt` documents and `.k` klammer +definitions) in four editors, built around one shared implementation. + +| Directory | Contents | +|---|---| +| `shared/` | **The shared editor core** (`klammertext_edit.py`) and the **Klammertext language server** (`klammertext_ls.py`). Everything structure-aware — indentation, table alignment, delimiter matching, delimiter diagnostics — implemented once, in dependency-free Python. | +| `emacs/` | Emacs major mode. An independent elisp implementation of the same algorithms (Emacs cannot call Python per keystroke), held byte-equal to the shared core by the test suite. | +| `sublime/` | Sublime Text package. Syntax file plus thin plugin wrappers that import the shared core directly (Sublime's plugin host is Python). | +| `vim/` | Vim plugin. Syntax/ftplugin files plus commands that run the shared core's CLI; with `+python3`, the core also runs in-process (`=` operator, live match highlighting). | +| `vscode/` | Visual Studio Code extension. TextMate grammar plus a dependency-free extension that spawns the language server and speaks LSP to it. | + +## The architecture + +Klammertext's *structural layer* — @-run tiers, bar-run dimension, +`^`-escapes, `#` removal, literal spans, nesting depth — is independent of +both the Klammermachine and any klammer set, and every editor needs the +same operations on it. Those operations live in +**`shared/klammertext_edit.py`**: + +- **Indentation** — 2 spaces per nesting level; bar runs and closing + delimiters sit at their opener's column; `@document` is transparent; + verbatim/`@eval`/removed content is never touched. Explicit-only in + every editor: whitespace is content in Klammertext. +- **Table alignment** — pad a `@table`'s rows so the depth-0 `|` + separators line up (rows end with `||`; cells over 30 characters or + spanning lines opt their row out; over 100 columns the command declines; + never a space inside a bar run). +- **Delimiter matching** — open ↔ close for klammer applications, literal + klammers matched by name with opaque content, everything else by depth. +- **Diagnostics** — whole-buffer balance check over all three `@`-tiers: + unclosed openings, extra closes, name and tier mismatches, unclosed + literal spans and `#[` blocks. +- **A CLI** (`indent | align | match | check` over stdin/stdout) for + editors that shell out (Vim), and for scripts. + +**`shared/klammertext_ls.py`** puts a language server in front of the same +core: JSON-RPC over stdio, no dependencies. It serves publishDiagnostics, +document/range formatting (reindentation), documentHighlight and +definition (the matcher), and a `klammertext.alignTable` command. The +VS Code extension is its first client; any LSP client works — Neovim's +built-in LSP, Emacs eglot, Sublime's LSP package — with a one-line +configuration pointing at `python3 klammertext_ls.py`. + +Syntax highlighting and cursor-latency features stay native in each editor +(a tokenizer or a per-keystroke matcher cannot round-trip to Python), so +each editor directory carries its own syntax artifact and, where relevant, +its own thin glue. + +## Locating the shared core + +The Sublime, Vim, and VS Code integrations find `klammertext_edit.py` (and +VS Code additionally `klammertext_ls.py`) in this order: a copy at the +integration's own root (the **editing zip** vendors one there, so each +unpacked folder is self-contained), `../shared/` relative to the +integration (this repository's layout — using an editor directory straight +from a checkout just works), then `$KLAMMERTEXT_HOME/doc/edit/shared/`. + +## Keeping things in sync + +`klammertext_edit.py` is the source of truth for the policy lists +(`LITERAL_KLAMMERS`, `TRANSPARENT_KLAMMERS`, `CODE_KLAMMERS`, +`ALIGN_KLAMMERS`) and limits (`INDENT_OFFSET`, `CELL_MAX`, `ROW_MAX`). +Two kinds of artifact cannot read it and restate parts of it by hand: + +1. **The Emacs mode** — a full independent implementation with its own + defcustoms, checked byte-for-byte against the shared core by + `tst/editor_test.sh`. +2. **The static syntax files** — the literal-klammer set (`@code`) appears + in the Sublime `.sublime-syntax`, the Vim `syntax/klammertext.vim`, and + the VS Code `tmLanguage.json` grammar (and the Emacs defcustom). When + you add a literal klammer, change them together; each file's header + carries the same SYNC note. + +## Testing + +`tst/editor_test.sh` (run by `dbg/rebuild.sh` and `make -C tst test`) +drives the fixture pairs in `tst/editor/` through the shared core's API +and CLI, checks idempotence, checks the Emacs implementation for +byte-equality, runs the language server through a scripted LSP client +(`ls_test.py`), exercises the VS Code extension against the real server +under a stubbed VS Code API (`vscode_ext_test.js`), and runs the Vim +plugin's commands headlessly. Emacs, Vim, and Node/VS Code halves skip +gracefully where not installed. diff --git a/doc/edit/emacs/klammertext-align.el b/doc/edit/emacs/klammertext-align.el index 9193b92..b75e13d 100644 --- a/doc/edit/emacs/klammertext-align.el +++ b/doc/edit/emacs/klammertext-align.el @@ -14,7 +14,7 @@ ;; (require 'klammertext-align) ;; ;; Comment that line out to disable alignment entirely. The Sublime Text -;; port doc/sublime/Klammertext_align.py implements the same algorithm — +;; port doc/edit/sublime/Klammertext_align.py implements the same algorithm — ;; keep the two in step. ;; ;; Alignment is for SMALL data items (2026-07-27): @@ -42,8 +42,11 @@ ;; aligned row; run TAB / `indent-region' first if the rows disagree. ;; ;; SYNC: `klammertext-align-klammers' / `-cell-max' / `-row-max' are -;; mirrored as ALIGN_KLAMMERS / CELL_MAX / ROW_MAX in Klammertext_align.py -;; (a Sublime plugin cannot read these defcustoms). +;; mirrored as ALIGN_KLAMMERS / CELL_MAX / ROW_MAX in the shared Python +;; core doc/edit/shared/klammertext_edit.py (the single implementation +;; behind the Sublime, Vim, and VS Code integrations and the language +;; server; this elisp unit stays independent, held equal by +;; tst/editor_test.sh's byte-equality checks). ;;; Code: diff --git a/doc/edit/emacs/klammertext-indent.el b/doc/edit/emacs/klammertext-indent.el index ce574fb..3debd17 100644 --- a/doc/edit/emacs/klammertext-indent.el +++ b/doc/edit/emacs/klammertext-indent.el @@ -48,11 +48,14 @@ ;; Known limitation: a raw @ inside a ^'...'^ literal region would confuse ;; the depth scan (the same limitation as the font-lock scanner). ;; -;; SYNC: the Sublime Text port doc/sublime/Klammertext_indent.py duplicates -;; this file's policy (a Sublime plugin cannot read these defcustoms). When -;; you change `klammertext-indent-offset', `klammertext-transparent-klammers' -;; or `klammertext-code-klammers', mirror the change in that file's -;; INDENT_OFFSET / TRANSPARENT_KLAMMERS / CODE_KLAMMERS. +;; SYNC: the shared Python core doc/edit/shared/klammertext_edit.py — the +;; single implementation behind the Sublime, Vim, and VS Code integrations +;; and the language server — carries this file's policy as INDENT_OFFSET / +;; TRANSPARENT_KLAMMERS / CODE_KLAMMERS (an elisp defcustom cannot be read +;; from Python, so this unit remains an independent implementation, held +;; equal by tst/editor_test.sh's byte-equality checks). When you change +;; `klammertext-indent-offset', `klammertext-transparent-klammers' or +;; `klammertext-code-klammers', mirror the change there. ;;; Code: diff --git a/doc/edit/emacs/klammertext-mode.el b/doc/edit/emacs/klammertext-mode.el index 74149b6..c393990 100644 --- a/doc/edit/emacs/klammertext-mode.el +++ b/doc/edit/emacs/klammertext-mode.el @@ -121,14 +121,17 @@ Register one with `klammertext-add-literal-klammer', e.g. in your init file: :type '(repeat string) :group 'klammertext) -;; SYNC: the Sublime Text port in doc/sublime/ duplicates this list statically -;; (a Sublime syntax/plugin cannot read this Emacs defcustom). When you add or -;; remove a literal klammer, mirror it in ALL of: -;; * LITERAL_KLAMMERS in doc/sublime/Klammertext.py -;; * LITERAL_KLAMMERS in doc/sublime/Klammertext_indent.py +;; SYNC: the shared Python core doc/edit/shared/klammertext_edit.py (used by +;; the Sublime, Vim, and VS Code integrations and the language server) holds +;; this list as LITERAL_KLAMMERS, and the static per-editor syntax files +;; restate it (a tokenizer cannot read a defcustom or a Python module). When +;; you add or remove a literal klammer, mirror it in ALL of: +;; * LITERAL_KLAMMERS in doc/edit/shared/klammertext_edit.py ;; * the @NAME literal rule + literal_NAME context in -;; doc/sublime/Klammertext.sublime-syntax -;; All four are currently seeded with just "code". +;; doc/edit/sublime/Klammertext.sublime-syntax +;; * the @NAME verbatim region in doc/edit/vim/syntax/klammertext.vim +;; * the @NAME rule in doc/edit/vscode/syntaxes/klammertext.tmLanguage.json +;; All are currently seeded with just "code". (defun klammertext-add-literal-klammer (name) "Register NAME as a klammer whose literal content must not be interpreted. diff --git a/doc/edit/shared/klammertext_edit.py b/doc/edit/shared/klammertext_edit.py new file mode 100644 index 0000000..6eae53c --- /dev/null +++ b/doc/edit/shared/klammertext_edit.py @@ -0,0 +1,1083 @@ +#!/usr/bin/env python3 +# klammertext_edit.py +# +# The SHARED implementation of Klammertext's editor-support algorithms: +# everything about the language's *structural* layer — @-run tiers, bar-run +# dimension, ^-escapes, # removal, literal spans, nesting depth — that more +# than one editor integration needs. One algorithm, one file: +# +# * scanner helpers name_char_p, escaped_p, block_end, at_run_end +# * delimiter matching app_delim_info, app_match, ... (jump / highlight) +# * indentation reindent_lines, target_column, ... +# * table alignment enclosing_span, compute_edits, ... +# * diagnostics diagnostics(): unclosed / mismatched delimiters +# * a CLI indent | align | match | check over stdin/stdout +# +# Consumers: +# * doc/edit/sublime/ the Sublime Text plugins import this module +# directly (Sublime's plugin host is Python) +# * doc/edit/vim/ the Vim plugin shells out to the CLI +# * doc/edit/shared/klammertext_ls.py +# the language server (used by VS Code, and by any +# LSP client: Neovim, Emacs eglot, Sublime LSP) +# * tst/editor_test.sh the regression suite drives the fixtures through +# this module and through the CLI +# * doc/edit/emacs/ NOT a consumer: the Emacs mode is an independent +# elisp implementation of the same algorithms, +# held equal to this file by the byte-equality +# checks in tst/editor_test.sh +# +# This file is the SOURCE OF TRUTH for the policy lists (LITERAL_KLAMMERS, +# TRANSPARENT_KLAMMERS, CODE_KLAMMERS, ALIGN_KLAMMERS) and the numeric limits +# (INDENT_OFFSET, CELL_MAX, ROW_MAX). It replaces the former per-editor +# copies in Klammertext.py / Klammertext_indent.py / Klammertext_align.py. +# SYNC: the following per-editor artifacts cannot import Python and must be +# kept in step by hand: +# * the Emacs defcustoms (klammertext-literal-klammers, -transparent-, +# -code-, -align-klammers, -indent-offset, -align-cell-max, -align-row-max) +# in doc/edit/emacs/klammertext-mode.el / -indent.el / -align.el +# * the '@code' rule + literal_code context in +# doc/edit/sublime/Klammertext.sublime-syntax +# * the '@code' verbatim region in doc/edit/vim/syntax/klammertext.vim +# * the '@code' rule in doc/edit/vscode/syntaxes/klammertext.tmLanguage.json +# +# Installation note: editors locate this file either next to their own plugin +# files (a vendored copy, placed there by doc/make_editing_zip.sh), as +# ../shared/klammertext_edit.py relative to the plugin directory (the layout +# of this repository), or under $KLAMMERTEXT_HOME/doc/edit/shared/. +# +# Python floor: 3.8 (the Sublime Text 4 plugin host) — no 3.9+ syntax here. +# +# Known limitation (inherited by every consumer): a raw @ or # inside a +# ^'...'^ literal region confuses the scanners; use ^@ / ^# there instead. + +import sys + +# --- policy ---------------------------------------------------------------- + +# Klammer names whose content is a literal argument (verbatim interior, +# closed by a named NAME@ delimiter). +LITERAL_KLAMMERS = set(["code"]) + +# Klammers that contribute no indentation level (a @document's paragraphs +# stay at the left margin). +TRANSPARENT_KLAMMERS = set(["document"]) + +# Klammers whose argument span holds code (inline Python is +# indentation-sensitive): lines inside are never reindented. +CODE_KLAMMERS = set(["eval"]) + +# Klammers whose rows the table-alignment command aligns. +ALIGN_KLAMMERS = set(["table"]) + +# Spaces per nesting level. +INDENT_OFFSET = 2 + +# Alignment limits: a row with a cell longer than CELL_MAX characters (or +# spanning lines) is left untouched; if the aligned rows would exceed ROW_MAX +# columns, nothing is changed. +CELL_MAX = 30 +ROW_MAX = 100 + + +# --- scanner helpers ------------------------------------------------------- + +def name_char_p(ch): + """True if CH can be part of a klammer name (letter, digit or _). + A hyphen is NOT a name char: @name-arg1 ends the name at the first hyphen.""" + if ch is None: + return False + return (('a' <= ch <= 'z') or ('A' <= ch <= 'Z') + or ('0' <= ch <= '9') or ch == '_') + + +def escaped_p(s, pos): + """True if the char at POS is escaped by an odd run of ^ before it. + In Klammertext ^# and ^@ are literal, so such a char is not a delimiter.""" + n = 0 + i = pos - 1 + while i >= 0 and s[i] == '^': + n += 1 + i -= 1 + return (n % 2) == 1 + + +def block_end(s, frm): + """Index just after the ]# that closes a #[ block opened at FROM (the index + just after the opening #[). Counts nested #[ ... ]#; len(s) if unclosed.""" + depth = 1 + i = frm + n = len(s) + while depth > 0: + a = s.find('#[', i) + b = s.find(']#', i) + if a == -1 and b == -1: + return n + if b == -1 or (a != -1 and a < b): + depth += 1 + i = a + 2 + else: + depth -= 1 + i = b + 2 + return i + + +def at_run_end(s, pos): + """Index just after the run of @ that begins at POS.""" + p = pos + n = len(s) + while p < n and s[p] == '@': + p += 1 + return p + + +def _name_forward(s, pos): + """Index just past the run of name chars starting at POS.""" + n = len(s) + k = pos + while k < n and name_char_p(s[k]): + k += 1 + return k + + +# --- delimiter matching (klammer APPLICATIONS, the single-@ tier) ---------- +# +# Matching is context-dependent (the same @ is both open and close, decided +# by its neighbors), so no editor's built-in bracket matching can express it; +# every editor integration routes its jump-to-match and live match +# highlighting through these functions (Emacs excepted; see the header). +# A literal klammer's own delimiters are matched BY NAME (@code <-> code@, +# content opaque); all others match by depth. + +def next_app_delim(s, i, limit): + """From index I, find the next single-@ application delimiter before LIMIT. + Step over @@/@@@ runs, removed text, literal spans, escaped ^@, and the + abbreviated @name-arg form. Return (pos, kind, next_i) with kind 'open' or + 'close' and next_i the index to resume from, or None when none is found.""" + n = len(s) + if limit is None: + limit = n + while i < limit: + j = i + while j < limit and s[j] != '@' and s[j] != '#': + j += 1 + if j >= limit: + return None + hit = j + i = hit + 1 # default: advance past the hit + if escaped_p(s, hit): # ^@ / ^# : keep going + continue + nxt = s[hit + 1] if hit + 1 < n else None + if s[hit] == '#': # removal: step over it + if nxt == '#': + i = n + elif nxt == '[': + i = block_end(s, hit + 2) + elif nxt in ('+', '/', '-'): + i = hit + 1 + else: # to end of line + eol = s.find('\n', hit) + i = n if eol == -1 else eol + continue + # s[hit] == '@' + if nxt == '@': # @@ / @@@ : step over the run + i = at_run_end(s, hit) + continue + if name_char_p(nxt): # @name : opening? + k = _name_forward(s, hit + 1) + name = s[hit + 1:k] + after = s[k] if k < n else None + if name in LITERAL_KLAMMERS: # literal span: skip to its close + close = name + '@' + idx = s.find(close, k) + i = n if idx == -1 else idx + len(close) + continue + elif after == '-': # @name-arg : opens no span + i = k + continue + else: + return (hit, 'open', k) + else: # name@ / bare @ : closing + return (hit, 'close', hit + 1) + return None + + +def match_forward(s, open_pos): + """OPEN_POS is the @ of an opening application. Return the matching close @ + index, or None if unbalanced.""" + i = _name_forward(s, open_pos + 1) # past the opening name + depth = 1 + while depth > 0: + d = next_app_delim(s, i, None) + if d is None: + return None + pos, kind, nxt = d + i = nxt + if kind == 'open': + depth += 1 + else: + depth -= 1 + if depth == 0: + return pos + return None + + +def match_backward(s, close_pos): + """CLOSE_POS is the @ of a closing application. Return the matching open @ + index, or None if unbalanced. Scans forward from 0 with a stack.""" + stack = [] + i = 0 + limit = close_pos + 1 + while True: + d = next_app_delim(s, i, limit) + if d is None: + return None + pos, kind, nxt = d + i = nxt + if kind == 'open': + stack.append(pos) + else: + open_pos = stack.pop() if stack else None + if pos == close_pos: + return open_pos + + +def app_delim_info(s, pos): + """If the char at POS is a single-@ application delimiter, return + (pos, kind) with kind 'open' or 'close'; else None. The abbreviated + @name-arg form (which opens no span) returns None.""" + n = len(s) + if not (0 <= pos < n): + return None + if s[pos] != '@': + return None + if pos > 0 and s[pos - 1] == '@': + return None + if pos + 1 < n and s[pos + 1] == '@': + return None + if escaped_p(s, pos): + return None + nxt = s[pos + 1] if pos + 1 < n else None + if name_char_p(nxt): + k = _name_forward(s, pos + 1) + after = s[k] if k < n else None + if after == '-': + return None + return (pos, 'open') + return (pos, 'close') + + +def open_name(s, open_pos): + """Name of the opening @name whose @ is at OPEN_POS.""" + return s[open_pos + 1:_name_forward(s, open_pos + 1)] + + +def close_name(s, close_pos): + """Name of a named close NAME@ whose @ is at CLOSE_POS, or None for a bare @ + (including the compact @name@ form, whose name belongs to the opening).""" + ns = close_pos + while ns > 0 and name_char_p(s[ns - 1]): + ns -= 1 + if ns < close_pos and (ns == 0 or s[ns - 1] != '@'): + return s[ns:close_pos] + return None + + +def paren_mismatch(s, open_pos, close_pos): + """True if the pair is unbalanced (either side None) or the named close + disagrees with the opening name.""" + if open_pos is None or close_pos is None: + return True + cname = close_name(s, close_pos) + return cname is not None and cname != open_name(s, open_pos) + + +def token_region(s, pos, kind): + """(start, end) of the whole delimiter token whose @ is at POS. + Opening: @ plus its name. Named close: the name plus @. Bare @: just @.""" + if kind == 'open': + return (pos, _name_forward(s, pos + 1)) + ns = pos + while ns > 0 and name_char_p(s[ns - 1]): + ns -= 1 + if ns < pos and (ns == 0 or s[ns - 1] != '@'): + return (ns, pos + 1) # named close NAME@ + return (pos, pos + 1) # bare @ (or @name@) + + +def literal_delim_name(s, pos, kind): + """If the application delimiter at POS (kind 'open'/'close') belongs to a + literal klammer (name in LITERAL_KLAMMERS), return its name; else None.""" + name = open_name(s, pos) if kind == 'open' else close_name(s, pos) + if name and name in LITERAL_KLAMMERS: + return name + return None + + +def literal_match_forward(s, open_pos, name): + """Index of the @ of the NAME@ that closes the literal @NAME at OPEN_POS, or + None. The content is opaque, so search for the literal close string.""" + start = open_pos + 1 + len(name) + idx = s.find(name + '@', start) + return idx + len(name) if idx != -1 else None + + +def literal_match_backward(s, close_pos, name): + """Index of the @ of the @NAME that opens the literal NAME@ whose @ is at + CLOSE_POS, or None. Literal spans do not nest, so the nearest preceding + real @NAME is the opener (not @@NAME, and not escaped).""" + open_str = '@' + name + end = close_pos + while True: + idx = s.rfind(open_str, 0, end) + if idx == -1: + return None + before = s[idx - 1] if idx > 0 else None + if before != '@' and not escaped_p(s, idx): + return idx + end = idx + + +def app_match(s, pos, kind): + """Matching application delimiter for the delimiter at POS of KIND + ('open'/'close'), or None. A literal klammer matches by name (@NAME <-> + NAME@) with content opaque; other klammers match by depth.""" + lit = literal_delim_name(s, pos, kind) + if lit is not None: + return (literal_match_forward(s, pos, lit) if kind == 'open' + else literal_match_backward(s, pos, lit)) + return match_forward(s, pos) if kind == 'open' else match_backward(s, pos) + + +def match_at(s, pos): + """The full matching story for a caret at POS (used by jump-to-match and + the live highlighters). The delimiter is taken at POS or, failing that, + just before POS (the on-or-just-after rule). Return None when POS is not + at an application delimiter; else a dict: + {'pos', 'kind', 'token': (start, end), + 'match': int or None, 'match_token': (start, end) or None, + 'mismatch': bool, 'message': str or None}""" + info = app_delim_info(s, pos) + if info is None and pos > 0: + info = app_delim_info(s, pos - 1) + if info is None: + return None + dpos, kind = info + match = app_match(s, dpos, kind) + open_pos = dpos if kind == 'open' else match + close_pos = match if kind == 'open' else dpos + mism = paren_mismatch(s, open_pos, close_pos) + message = None + if mism: + if match is None: + if kind == 'open': + message = ("opening @%s has no matching close" + % open_name(s, open_pos)) + else: + message = "closing delimiter has no matching open" + else: + message = ("closing %s@ does not match opening @%s" + % (close_name(s, close_pos) or '?', + open_name(s, open_pos))) + other_kind = 'close' if kind == 'open' else 'open' + return {'pos': dpos, 'kind': kind, + 'token': token_region(s, dpos, kind), + 'match': match, + 'match_token': (token_region(s, match, other_kind) + if match is not None else None), + 'mismatch': mism, 'message': message} + + +# --- indentation ----------------------------------------------------------- +# +# The convention (2026-07-27): a line indents to INDENT_OFFSET x (effective +# depth); a line that BEGINS with a bar run (|, ||, ...) or a closing +# delimiter sits at its opener's column. Effective depth counts every +# enclosing span uniformly — applications (@), definitions (@@), system +# commands (@@@) — except that TRANSPARENT_KLAMMERS contribute no level, and +# lines inside literal/verbatim content, CODE_KLAMMERS argument spans, or +# removed regions are never touched. Reindentation is explicit-only in every +# editor: whitespace is content in Klammertext, so nothing reformats as a +# side effect of typing. + +def state_at(s, pos): + """Scan s[0:POS] (POS a line beginning). Return (stack, opaque): STACK is + the list of names of the klammer applications, @@ definitions and @@@ + commands open at POS, outermost first; OPAQUE is True when POS lies inside + content that indentation must not touch (removed text, a literal klammer's + verbatim span, or a code klammer's argument span).""" + stack = [] + n = len(s) + i = 0 + while i < pos: + j = i + while j < pos and s[j] != '@' and s[j] != '#': + j += 1 + if j >= pos: + break + hit = j + i = hit + 1 + if escaped_p(s, hit): # ^@ / ^# : plain text + continue + nxt = s[hit + 1] if hit + 1 < n else None + if s[hit] == '#': + if nxt == '#': # ## removes to end of buffer + return (stack, True) + elif nxt == '[': # #[ ... ]# (nestable) + end = block_end(s, hit + 2) + if pos < end: + return (stack, True) + i = end + elif nxt in ('+', '/', '-'): # whitespace operators + pass + else: # # to end of line + eol = s.find('\n', hit) + i = n if eol == -1 else eol + continue + # an @-run + run_end = at_run_end(s, hit) + run_len = run_end - hit + after = s[run_end] if run_end < n else None + if name_char_p(after): + # @name / @@name / @@@name : an opener (or, for a literal + # klammer, a verbatim span to step over). + k = _name_forward(s, run_end) + name = s[run_end:k] + i = k + if run_len == 1 and name in LITERAL_KLAMMERS: + # Verbatim interior: find the closing NAME@ by name. + idx = s.find(name + '@', k) + if idx == -1: # never closed + return (stack, True) + close_end = idx + len(name) + 1 + if pos < close_end: + return (stack, True) + i = close_end + elif run_len == 1 and k < n and s[k] == '-': + pass # @name-arg : opens no span + else: + stack.append(name) + else: + # a bare @-run, or the run of a named close: a close. + if stack: + stack.pop() + i = run_end + opaque = any(name in CODE_KLAMMERS for name in stack) + return (stack, opaque) + + +def depth(stack): + """Number of indentation levels STACK contributes. + Transparent klammers contribute none.""" + return sum(1 for name in stack if name not in TRANSPARENT_KLAMMERS) + + +def dedent_line_p(s, bol): + """True when the line starting at BOL begins with a token that sits at its + owner's opening column: a bar run (|, ||, ...), a bare close run (@, @@, + @@@), or a named close (name@, name@@, name@@@). A line beginning with an + opener (@name, @@name, @@@name) is content-level.""" + n = len(s) + i = bol + while i < n and s[i] in ' \t': + i += 1 + if i >= n: + return False + c = s[i] + if c == '|': + return True + if c == '@': + run_end = at_run_end(s, i) + return not name_char_p(s[run_end] if run_end < n else None) + if name_char_p(c): + # A named close: name chars followed by an @-run (an unescaped @ can + # only be a delimiter). + k = _name_forward(s, i) + return k < n and s[k] == '@' + return False + + +def target_column(s, bol): + """Column for the line starting at BOL, or None for lines that must not + be touched (verbatim, code, or removed content).""" + stack, opaque = state_at(s, bol) + if opaque: + return None + if dedent_line_p(s, bol) and stack: + stack = stack[:-1] + return INDENT_OFFSET * depth(stack) + + +def line_beginnings(s): + """Offsets of every line beginning in S.""" + return [0] + [i + 1 for i, ch in enumerate(s) + if ch == '\n' and i + 1 < len(s)] + + +def reindent_lines(s, bols): + """Compute the edits that reindent the lines whose beginnings are BOLS. + Return a list of (start, end, replacement) triples over S, in ascending + order, replacing each line's leading whitespace; untouchable lines and + already-correct lines produce no edit. Pure function.""" + n = len(s) + edits = [] + for bol in bols: + tgt = target_column(s, bol) + if tgt is None: + continue + i = bol + while i < n and s[i] in ' \t': + i += 1 + if s[bol:i] != ' ' * tgt: + edits.append((bol, i, ' ' * tgt)) + return edits + + +# --- table alignment ------------------------------------------------------- +# +# Pads the cells of the enclosing @table's rows so the depth-0 | separators +# line up. Alignment is for SMALL data items: a row with a cell longer than +# CELL_MAX or spanning lines is untouched (and contributes no width); if the +# aligned rows would exceed ROW_MAX columns nothing changes. No whitespace +# is ever inserted inside a bar run (|| is a row separator; | | is an empty +# cell — the load-bearing-whitespace trap). Bars inside a nested klammer +# belong to that klammer: only depth-0 bars count, the same rule the +# Klammermachine applies to @cond. + +def enclosing_span(s, pos, names): + """Innermost span of a klammer named in NAMES that contains POS. + Return (name, content_start, content_end) with content_start just after + the opening @name token and content_end at the start of the closing + delimiter token, or None. Scans S from the start with a position stack, + stepping over removed text, literal spans, escaped characters, and the + abbreviated @name-arg form.""" + stack = [] # (name, open_token_start, content_start) + n = len(s) + i = 0 + while i < n: + j = i + while j < n and s[j] != '@' and s[j] != '#': + j += 1 + if j >= n: + break + hit = j + i = hit + 1 + if escaped_p(s, hit): + continue + nxt = s[hit + 1] if hit + 1 < n else None + if s[hit] == '#': + if nxt == '#': + break # ## removes the rest of the buffer + elif nxt == '[': + i = block_end(s, hit + 2) + elif nxt in ('+', '/', '-'): + pass + else: + eol = s.find('\n', hit) + i = n if eol == -1 else eol + continue + run_end = at_run_end(s, hit) + run_len = run_end - hit + after = s[run_end] if run_end < n else None + if name_char_p(after): + k = _name_forward(s, run_end) + name = s[run_end:k] + i = k + if run_len == 1 and name in LITERAL_KLAMMERS: + idx = s.find(name + '@', k) + if idx == -1: + break + i = idx + len(name) + 1 + elif run_len == 1 and k < n and s[k] == '-': + pass # @name-arg : opens no span + else: + stack.append((name, hit, k)) + else: + # a close: the token starts at the preceding name run, if any + ns = hit + while ns > 0 and name_char_p(s[ns - 1]): + ns -= 1 + tok_start = ns if (ns < hit and (ns == 0 or s[ns - 1] != '@')) else hit + if stack: + name, open_start, content_start = stack.pop() + if name in names and open_start <= pos <= run_end: + return (name, content_start, tok_start) + i = run_end + return None + + +def scan_lines(content): + """Scan CONTENT (the text of a table span). Return a list of line + records, one per line: dicts with start, end (offsets into CONTENT, end + excludes the newline), start_depth, end_depth (klammer nesting relative + to the span), bars (list of (pos, runlen) for unescaped depth-0 bar + runs), blocked (inside removed/verbatim content), comment (a # removes + the rest of the line).""" + import bisect + n = len(content) + line_starts = [0] + for idx, ch in enumerate(content): + if ch == '\n': + line_starts.append(idx + 1) + nlines = len(line_starts) + + def line_index(p): + return bisect.bisect_right(line_starts, p) - 1 + + lines = [{'start': line_starts[k], + 'end': (line_starts[k + 1] - 1 if k + 1 < nlines else n), + 'bars': [], 'blocked': False, 'comment': False, + 'start_depth': None, 'end_depth': None} + for k in range(nlines)] + lines[0]['start_depth'] = 0 + + def block_range(a, b): + """Mark every line touched by [a, b) as blocked.""" + last = max(a, b - 1) + for k in range(line_index(a), line_index(min(last, n - 1)) + 1): + lines[k]['blocked'] = True + + depth = 0 + i = 0 + while i < n: + j = i + while j < n and content[j] not in '@#|\n': + j += 1 + if j >= n: + break + hit = j + i = hit + 1 + c = content[hit] + if c == '\n': + k = line_index(hit) + lines[k]['end_depth'] = depth + if k + 1 < nlines: + lines[k + 1]['start_depth'] = depth + continue + if escaped_p(content, hit): + continue + nxt = content[hit + 1] if hit + 1 < n else None + if c == '#': + if nxt == '#': + block_range(hit, n) + break + elif nxt == '[': + e = block_end(content, hit + 2) + if line_index(max(hit, e - 1)) != line_index(hit): + block_range(hit, e) + i = e + elif nxt in ('+', '/', '-'): + pass + else: # # to end of line + lines[line_index(hit)]['comment'] = True + eol = content.find('\n', hit) + i = n if eol == -1 else eol + continue + if c == '|': + if hit > 0 and content[hit - 1] == '|': + continue # mid-run (after an escaped ^|) + k = hit + while k < n and content[k] == '|': + k += 1 + if depth == 0: + lines[line_index(hit)]['bars'].append((hit, k - hit)) + i = k + continue + # '@' + run_end = at_run_end(content, hit) + run_len = run_end - hit + after = content[run_end] if run_end < n else None + if name_char_p(after): + k = _name_forward(content, run_end) + name = content[run_end:k] + i = k + if run_len == 1 and name in LITERAL_KLAMMERS: + idx = content.find(name + '@', k) + e = n if idx == -1 else idx + len(name) + 1 + if line_index(max(hit, e - 1)) != line_index(hit): + block_range(hit, e) + i = e + elif run_len == 1 and k < n and content[k] == '-': + pass + else: + depth += 1 + else: + depth = max(0, depth - 1) + i = run_end + + for ln in lines: + if ln['start_depth'] is None: + ln['blocked'] = True + if ln['end_depth'] is None: + ln['end_depth'] = depth + return lines + + +def compute_edits(content): + """Compute the alignment edits for CONTENT (a table span's text). + Return (edits, message): edits is a list of (start, end, new_text) + triples relative to CONTENT, ascending; message is a status string (a + reason when edits is empty).""" + lines = scan_lines(content) + + # The last line holding actual content (a row there may omit its ||). + last_content = None + for k in range(len(lines) - 1, 0, -1): + if content[lines[k]['start']:lines[k]['end']].strip(): + last_content = k + break + + rows = [] # (line record, cells, trailing_p) + chain_ok = True # a row must START a row: the previous + for k in range(1, len(lines)): # content line ended with || (or was the + ln = lines[k] # opener / an option line / blank) + text = content[ln['start']:ln['end']] + stripped = text.strip() + if not stripped: + continue # blank line: chain unchanged + if stripped.startswith(':') and not ln['bars']: + continue # option line: chain unchanged + row = _parse_row(content, ln, text, chain_ok, k == last_content) + trailing = _trailing_rowsep(content, ln) + chain_ok = trailing + if row is not None: + rows.append(row) + + if not rows: + return ([], "no alignable rows found") + + widths = [] + for _ln, cells, _tr in rows: + for c_idx, cell in enumerate(cells): + if c_idx >= len(widths): + widths.append(0) + widths[c_idx] = max(widths[c_idx], len(cell)) + + indent = ' ' * _indent_width(content, rows[0][0]) + if ROW_MAX is not None: + longest = 0 + for _ln, cells, trailing in rows: + m = len(cells) + w = (len(indent) + sum(widths[:m]) + 3 * (m - 1) + + (3 if trailing else 0)) + longest = max(longest, w) + if longest > ROW_MAX: + return ([], "aligned rows would be %d characters (limit %d); " + "not aligning" % (longest, ROW_MAX)) + + edits = [] + for ln, cells, trailing in rows: + parts = [cells[c].ljust(widths[c]) for c in range(len(cells) - 1)] + last = cells[-1] + if trailing: + last = last.ljust(widths[len(cells) - 1]) + parts.append(last) + new = indent + ' | '.join(parts) + (' ||' if trailing else '') + if new != content[ln['start']:ln['end']]: + edits.append((ln['start'], ln['end'], new)) + msg = ("aligned %d rows" % len(rows)) if edits else "already aligned" + return (edits, msg) + + +def _indent_width(content, ln): + i = ln['start'] + while i < ln['end'] and content[i] in ' \t': + i += 1 + return i - ln['start'] + + +def _trailing_rowsep(content, ln): + """True when the line's LAST depth-0 bar run is a || sitting at the end of + the line (only whitespace after it).""" + if not ln['bars']: + return False + pos, runlen = ln['bars'][-1] + return (runlen == 2 + and content[pos + 2:ln['end']].strip() == '') + + +def _parse_row(content, ln, text, chain_ok, is_last_content): + """If the line is an alignable row, return (ln, cells, trailing_p); + else None.""" + if ln['blocked'] or ln['comment'] or not chain_ok: + return None + if ln['start_depth'] != 0 or ln['end_depth'] != 0: + return None + if not ln['bars']: + return None + trailing = _trailing_rowsep(content, ln) + singles = ln['bars'][:-1] if trailing else ln['bars'] + for _pos, runlen in singles: + if runlen != 1: + return None # a mid-line || (or |||): not one row + if not trailing and not is_last_content: + return None # row continues onto the next line + cell_start = ln['start'] + _indent_width(content, ln) + cell_end = ln['bars'][-1][0] if trailing else ln['end'] + bounds = [cell_start] + [p for p, _r in singles] + [cell_end] + cells = [] + for b_idx in range(len(bounds) - 1): + a = bounds[b_idx] + (1 if b_idx > 0 else 0) # skip the | itself + cell = content[a:bounds[b_idx + 1]].strip() + if len(cell) > CELL_MAX: + return None + cells.append(cell) + return (ln, cells, trailing) + + +# --- diagnostics ----------------------------------------------------------- +# +# A whole-buffer balance check over all three @-tiers, for editor problem +# panels (LSP publishDiagnostics, Vim quickfix). The scan is the uniform +# one the indentation uses — every @-run is an opener when a name follows it +# and a close otherwise — extended with positions and names so problems can +# be reported where they are: +# +# * a closing delimiter with no opening to match +# * a named close whose name disagrees with its opening +# * a close whose @-run length differs from its opening's (@name ... @@) +# * an opening never closed (reported at the opening, at end of scan) +# * a literal klammer never closed (@code without code@) +# * an unclosed #[ removal block +# +# Content removed by ## is not scanned (it is not part of the document). + +def diagnostics(s): + """Scan S and return a list of problems, each a dict: + {'start': int, 'end': int, 'message': str, 'severity': 'error'|'warning'}. + Positions are character offsets into S (token start/end).""" + probs = [] + stack = [] # (name, run_len, tok_start, tok_end) + n = len(s) + i = 0 + while i < n: + j = i + while j < n and s[j] != '@' and s[j] != '#': + j += 1 + if j >= n: + break + hit = j + i = hit + 1 + if escaped_p(s, hit): + continue + nxt = s[hit + 1] if hit + 1 < n else None + if s[hit] == '#': + if nxt == '#': # rest of file removed: stop scanning + break + elif nxt == '[': + end = block_end(s, hit + 2) + if end >= n and not s.endswith(']#'): + probs.append({'start': hit, 'end': hit + 2, + 'message': "#[ has no closing ]#", + 'severity': 'warning'}) + i = end + elif nxt in ('+', '/', '-'): + pass + else: + eol = s.find('\n', hit) + i = n if eol == -1 else eol + continue + # an @-run + run_end = at_run_end(s, hit) + run_len = run_end - hit + after = s[run_end] if run_end < n else None + if name_char_p(after): + k = _name_forward(s, run_end) + name = s[run_end:k] + i = k + if run_len == 1 and name in LITERAL_KLAMMERS: + idx = s.find(name + '@', k) + if idx == -1: + probs.append({'start': hit, 'end': k, + 'message': ("literal klammer @%s has no " + "closing %s@" % (name, name)), + 'severity': 'error'}) + break # everything after is verbatim + i = idx + len(name) + 1 + elif run_len == 1 and k < n and s[k] == '-': + pass # @name-arg : opens no span + else: + stack.append((name, run_len, hit, k)) + else: + # a close; find the token (a preceding name run makes it named) + ns = hit + while ns > 0 and name_char_p(s[ns - 1]): + ns -= 1 + named = ns < hit and (ns == 0 or s[ns - 1] != '@') + cname = s[ns:hit] if named else None + tok_start = ns if named else hit + if not stack: + probs.append({'start': tok_start, 'end': run_end, + 'message': ("closing delimiter %s has no " + "matching opening" + % s[tok_start:run_end]), + 'severity': 'error'}) + else: + oname, orun, ostart, oend = stack.pop() + if cname is not None and cname != oname: + probs.append({'start': tok_start, 'end': run_end, + 'message': ("closing %s does not match " + "opening %s" + % (s[tok_start:run_end], + s[ostart:oend])), + 'severity': 'error'}) + elif orun != run_len: + probs.append({'start': tok_start, 'end': run_end, + 'message': ("closing %s does not match " + "opening %s (%d-@ close for " + "a %d-@ opening)" + % (s[tok_start:run_end], + s[ostart:oend], + run_len, orun)), + 'severity': 'error'}) + i = run_end + for name, run_len, ostart, oend in stack: + probs.append({'start': ostart, 'end': oend, + 'message': "opening %s is never closed" % s[ostart:oend], + 'severity': 'error'}) + probs.sort(key=lambda p: p['start']) + return probs + + +# --- whole-buffer conveniences (used by the CLI, the LSP server, tests) ---- + +def apply_edits(s, edits): + """Apply (start, end, new) EDITS (ascending, non-overlapping) to S.""" + out = [] + last = 0 + for a, b, new in edits: + out.append(s[last:a]) + out.append(new) + last = b + out.append(s[last:]) + return ''.join(out) + + +def indent_text(s, first=None, last=None): + """Reindent S (whole buffer, or 1-based inclusive line range FIRST..LAST). + Return the new text.""" + bols = line_beginnings(s) + if first is not None: + lo = max(first, 1) + hi = len(bols) if last is None else min(last, len(bols)) + bols = bols[lo - 1:hi] + return apply_edits(s, reindent_lines(s, bols)) + + +def align_text(s, pos): + """Align the table klammer enclosing POS. Return (new_text, message).""" + span = enclosing_span(s, pos, ALIGN_KLAMMERS) + if span is None: + return (s, "not inside a table klammer (%s)" + % ", ".join("@" + name for name in sorted(ALIGN_KLAMMERS))) + _name, cs, ce = span + edits, msg = compute_edits(s[cs:ce]) + shifted = [(cs + a, cs + b, new) for a, b, new in edits] + return (apply_edits(s, shifted), msg) + + +def offset_of(s, line, col): + """Character offset of 1-based LINE, 1-based character column COL.""" + bols = line_beginnings(s) + line = max(1, min(line, len(bols))) + bol = bols[line - 1] + eol = s.find('\n', bol) + if eol == -1: + eol = len(s) + return min(bol + max(col - 1, 0), eol) + + +def line_col(s, offset): + """(1-based line, 1-based character column) of character OFFSET.""" + offset = max(0, min(offset, len(s))) + line = s.count('\n', 0, offset) + 1 + bol = s.rfind('\n', 0, offset) + 1 + return (line, offset - bol + 1) + + +# --- CLI ------------------------------------------------------------------- +# +# The shell-out interface for editors that are neither Python-hosted nor LSP +# clients (the Vim plugin). Reads the buffer on stdin (UTF-8), writes the +# transformed buffer on stdout; status messages go to stderr. +# +# klammertext_edit.py indent [FIRST[-LAST]] reindent all / a line range +# klammertext_edit.py align LINE COL align the enclosing table +# klammertext_edit.py match LINE COL print "match L C" | "none MSG" +# (with "mismatch" when the +# pair disagrees); no buffer +# klammertext_edit.py check print "L:C: message" lines; +# no buffer output +# +# LINE and COL are 1-based; COL counts characters. Exit code: 0 (including +# "nothing to do"), 2 on a usage error. + +def _cli(argv): + def usage(): + sys.stderr.write( + "usage: klammertext_edit.py indent [FIRST[-LAST]] |" + " align LINE COL | match LINE COL | check\n") + return 2 + + if len(argv) < 1: + return usage() + mode = argv[0] + s = sys.stdin.read() + + if mode == 'indent': + first = last = None + if len(argv) > 1: + rng = argv[1] + try: + if '-' in rng: + a, b = rng.split('-', 1) + first, last = int(a), int(b) + else: + first = last = int(rng) + except ValueError: + return usage() + sys.stdout.write(indent_text(s, first, last)) + return 0 + + if mode == 'align': + if len(argv) != 3: + return usage() + try: + pos = offset_of(s, int(argv[1]), int(argv[2])) + except ValueError: + return usage() + out, msg = align_text(s, pos) + sys.stdout.write(out) + sys.stderr.write(msg + "\n") + return 0 + + if mode == 'match': + if len(argv) != 3: + return usage() + try: + pos = offset_of(s, int(argv[1]), int(argv[2])) + except ValueError: + return usage() + m = match_at(s, pos) + if m is None: + print("none point is not on a klammer application delimiter (@)") + elif m['match'] is None: + print("none " + (m['message'] or "no matching delimiter")) + else: + line, col = line_col(s, m['match']) + kind = "mismatch" if m['mismatch'] else "match" + msg = (" " + m['message']) if m['mismatch'] and m['message'] else "" + print("%s %d %d%s" % (kind, line, col, msg)) + return 0 + + if mode == 'check': + for p in diagnostics(s): + line, col = line_col(s, p['start']) + print("%d:%d: %s" % (line, col, p['message'])) + return 0 + + return usage() + + +if __name__ == '__main__': + sys.exit(_cli(sys.argv[1:])) diff --git a/doc/edit/shared/klammertext_ls.py b/doc/edit/shared/klammertext_ls.py new file mode 100644 index 0000000..3645796 --- /dev/null +++ b/doc/edit/shared/klammertext_ls.py @@ -0,0 +1,356 @@ +#!/usr/bin/env python3 +# klammertext_ls.py +# +# The Klammertext language server: a small, dependency-free implementation of +# the Language Server Protocol (JSON-RPC over stdio) on top of the shared +# editor core, doc/edit/shared/klammertext_edit.py. One server, any LSP +# client — the VS Code extension (doc/edit/vscode/) spawns it, and Neovim's +# built-in LSP, Emacs eglot, or Sublime's LSP package can attach to it with a +# one-line configuration (see doc/edit/README.md). +# +# What it serves: +# +# textDocument/publishDiagnostics unclosed / mismatched delimiters, on +# every open and change +# textDocument/formatting reindent the whole document +# textDocument/rangeFormatting reindent the selected lines +# textDocument/documentHighlight the matching application delimiter for +# the cursor position (live match +# highlighting in clients that request it +# on cursor movement) +# textDocument/definition jump-to-match: on an @ delimiter, +# "go to definition" goes to its match +# workspace/executeCommand klammertext.alignTable — align the +# @table enclosing the given position +# (applied via workspace/applyEdit) +# +# Formatting is EXPLICIT-ONLY by design: the server does not implement +# on-type formatting, because whitespace is content in Klammertext; nothing +# reformats as a side effect of typing. +# +# Protocol notes: full-text document sync (documents are small); positions in +# UTF-16 code units per the LSP default. No external libraries — the framing +# and dispatch below are the whole protocol layer. +# +# Usage: python3 klammertext_ls.py (talks LSP on stdin/stdout) +# +# The shared core is located next to this file (the repository layout and the +# vendored layouts both put the two files side by side), or under +# $KLAMMERTEXT_HOME/doc/edit/shared. +# +# Python floor: 3.8. + +import json +import os +import sys + + +def _import_shared(): + here = os.path.dirname(os.path.abspath(__file__)) + candidates = [here] + kh = os.environ.get('KLAMMERTEXT_HOME') + if kh: + candidates.append(os.path.join(kh, 'doc', 'edit', 'shared')) + for d in candidates: + if os.path.isfile(os.path.join(d, 'klammertext_edit.py')): + if d not in sys.path: + sys.path.insert(0, d) + break + import klammertext_edit + return klammertext_edit + + +KE = _import_shared() + + +# --- positions: LSP {line, character} (UTF-16) <-> python string offsets --- + +def _utf16_len(s): + n = 0 + for ch in s: + n += 2 if ord(ch) > 0xFFFF else 1 + return n + + +def pos_to_offset(text, pos): + """Offset in TEXT of LSP position POS ({'line', 'character'}, UTF-16).""" + line = pos.get('line', 0) + character = pos.get('character', 0) + off = 0 + for _ in range(line): + nl = text.find('\n', off) + if nl == -1: + return len(text) + off = nl + 1 + units = 0 + n = len(text) + while off < n and text[off] != '\n' and units < character: + units += 2 if ord(text[off]) > 0xFFFF else 1 + off += 1 + return off + + +def offset_to_pos(text, offset): + """LSP position of character OFFSET in TEXT.""" + offset = max(0, min(offset, len(text))) + line = text.count('\n', 0, offset) + bol = text.rfind('\n', 0, offset) + 1 + return {'line': line, 'character': _utf16_len(text[bol:offset])} + + +def offsets_to_range(text, start, end): + return {'start': offset_to_pos(text, start), + 'end': offset_to_pos(text, end)} + + +# --- the server ------------------------------------------------------------ + +class Server: + + def __init__(self, instream, outstream): + self.instream = instream + self.outstream = outstream + self.docs = {} # uri -> text + self.shutdown_received = False + self.running = True + self.next_server_id = 1 # ids for server->client requests + + # -- transport -- + + def read_message(self): + length = None + while True: + line = self.instream.readline() + if not line: + return None # EOF + line = line.strip() + if not line: + break # end of headers + if line.lower().startswith(b'content-length:'): + length = int(line.split(b':', 1)[1]) + if length is None: + return None + body = self.instream.read(length) + if len(body) < length: + return None + return json.loads(body.decode('utf-8')) + + def send(self, message): + body = json.dumps(message, ensure_ascii=False).encode('utf-8') + self.outstream.write(b'Content-Length: ' + str(len(body)).encode() + + b'\r\n\r\n' + body) + self.outstream.flush() + + def reply(self, msg_id, result): + self.send({'jsonrpc': '2.0', 'id': msg_id, 'result': result}) + + def reply_error(self, msg_id, code, message): + self.send({'jsonrpc': '2.0', 'id': msg_id, + 'error': {'code': code, 'message': message}}) + + def notify(self, method, params): + self.send({'jsonrpc': '2.0', 'method': method, 'params': params}) + + def request(self, method, params): + """Server->client request (fire and forget: the client's response is + consumed, and ignored, by the main loop).""" + self.send({'jsonrpc': '2.0', 'id': 'server-%d' % self.next_server_id, + 'method': method, 'params': params}) + self.next_server_id += 1 + + # -- diagnostics -- + + def publish_diagnostics(self, uri): + text = self.docs.get(uri, '') + diags = [] + for p in KE.diagnostics(text): + diags.append({ + 'range': offsets_to_range(text, p['start'], p['end']), + 'severity': 1 if p['severity'] == 'error' else 2, + 'source': 'klammertext', + 'message': p['message'], + }) + self.notify('textDocument/publishDiagnostics', + {'uri': uri, 'diagnostics': diags}) + + # -- dispatch -- + + def handle(self, msg): + method = msg.get('method') + msg_id = msg.get('id') + params = msg.get('params') or {} + + if method is None: + return # a response to a server->client request + + handler = getattr(self, 'on_' + method.replace('/', '_') + .replace('$', 'dollar'), None) + if handler is not None: + handler(msg_id, params) + elif msg_id is not None: # unknown request: MethodNotFound + self.reply_error(msg_id, -32601, 'method not found: ' + method) + # unknown notification: ignored + + # -- lifecycle -- + + def on_initialize(self, msg_id, params): + self.reply(msg_id, { + 'capabilities': { + 'textDocumentSync': 1, # full + 'documentFormattingProvider': True, + 'documentRangeFormattingProvider': True, + 'documentHighlightProvider': True, + 'definitionProvider': True, + 'executeCommandProvider': { + 'commands': ['klammertext.alignTable'], + }, + }, + 'serverInfo': {'name': 'klammertext-ls'}, + }) + + def on_initialized(self, msg_id, params): + pass + + def on_shutdown(self, msg_id, params): + self.shutdown_received = True + self.reply(msg_id, None) + + def on_exit(self, msg_id, params): + self.running = False + + def on_dollar_cancelRequest(self, msg_id, params): + pass # requests here are all fast + + # -- document sync (full text) -- + + def on_textDocument_didOpen(self, msg_id, params): + doc = params['textDocument'] + self.docs[doc['uri']] = doc.get('text', '') + self.publish_diagnostics(doc['uri']) + + def on_textDocument_didChange(self, msg_id, params): + uri = params['textDocument']['uri'] + changes = params.get('contentChanges') or [] + if changes: + self.docs[uri] = changes[-1].get('text', '') + self.publish_diagnostics(uri) + + def on_textDocument_didClose(self, msg_id, params): + uri = params['textDocument']['uri'] + self.docs.pop(uri, None) + self.notify('textDocument/publishDiagnostics', + {'uri': uri, 'diagnostics': []}) + + def on_textDocument_didSave(self, msg_id, params): + pass + + # -- formatting (reindentation) -- + + def _format_edits(self, text, bols): + edits = [] + for a, b, new in KE.reindent_lines(text, bols): + edits.append({'range': offsets_to_range(text, a, b), + 'newText': new}) + return edits + + def on_textDocument_formatting(self, msg_id, params): + text = self.docs.get(params['textDocument']['uri'], '') + self.reply(msg_id, self._format_edits(text, KE.line_beginnings(text))) + + def on_textDocument_rangeFormatting(self, msg_id, params): + text = self.docs.get(params['textDocument']['uri'], '') + rng = params['range'] + start = pos_to_offset(text, rng['start']) + end = pos_to_offset(text, rng['end']) + bols = [b for b in KE.line_beginnings(text) + if b <= end and (text.find('\n', b) == -1 + or text.find('\n', b) >= start)] + self.reply(msg_id, self._format_edits(text, bols)) + + # -- matching (highlight + jump) -- + + def on_textDocument_documentHighlight(self, msg_id, params): + text = self.docs.get(params['textDocument']['uri'], '') + m = KE.match_at(text, pos_to_offset(text, params['position'])) + if m is None: + self.reply(msg_id, None) + return + highlights = [{'range': offsets_to_range(text, *m['token']), + 'kind': 1}] + if m['match_token'] is not None: + highlights.append({'range': offsets_to_range(text, + *m['match_token']), + 'kind': 1}) + self.reply(msg_id, highlights) + + def on_textDocument_definition(self, msg_id, params): + uri = params['textDocument']['uri'] + text = self.docs.get(uri, '') + m = KE.match_at(text, pos_to_offset(text, params['position'])) + if m is None or m['match_token'] is None: + self.reply(msg_id, None) + return + self.reply(msg_id, {'uri': uri, + 'range': offsets_to_range(text, + *m['match_token'])}) + + # -- commands -- + + def on_workspace_executeCommand(self, msg_id, params): + command = params.get('command') + args = params.get('arguments') or [] + if command != 'klammertext.alignTable' or not args: + self.reply_error(msg_id, -32602, + 'unknown command or missing arguments') + return + arg = args[0] + uri = arg['uri'] + text = self.docs.get(uri, '') + pos = pos_to_offset(text, arg['position']) + span = KE.enclosing_span(text, pos, KE.ALIGN_KLAMMERS) + if span is None: + self.reply(msg_id, None) + self.notify('window/showMessage', + {'type': 3, 'message': + 'Klammertext: the cursor is not inside a table ' + 'klammer (%s)' % ', '.join( + '@' + n for n in sorted(KE.ALIGN_KLAMMERS))}) + return + _name, cs, ce = span + edits, message = KE.compute_edits(text[cs:ce]) + self.reply(msg_id, None) + if edits: + lsp_edits = [{'range': offsets_to_range(text, cs + a, cs + b), + 'newText': new} + for a, b, new in edits] + self.request('workspace/applyEdit', + {'label': 'Klammertext: align table', + 'edit': {'changes': {uri: lsp_edits}}}) + self.notify('window/showMessage', + {'type': 3, 'message': 'Klammertext: ' + message}) + + # -- main loop -- + + def run(self): + while self.running: + msg = self.read_message() + if msg is None: + break # EOF or malformed stream + try: + self.handle(msg) + except Exception as e: # a bug must not kill the server + if msg.get('id') is not None and 'method' in msg: + self.reply_error(msg['id'], -32603, + 'internal error: %s' % e) + sys.stderr.write('klammertext_ls: %s\n' % e) + sys.stderr.flush() + return 0 if self.shutdown_received else 1 + + +def main(): + server = Server(sys.stdin.buffer, sys.stdout.buffer) + sys.exit(server.run()) + + +if __name__ == '__main__': + main() diff --git a/doc/edit/sublime/Klammertext.py b/doc/edit/sublime/Klammertext.py index 490bed4..b16bfb5 100644 --- a/doc/edit/sublime/Klammertext.py +++ b/doc/edit/sublime/Klammertext.py @@ -1,7 +1,8 @@ # Klammertext.py # # Sublime Text plugin for klammer APPLICATION (@) delimiters. Two features, -# both ports of doc/emacs/klammertext-mode.el, both reusing one matcher: +# both counterparts of doc/edit/emacs/klammertext-mode.el, both reusing the +# one context-sensitive matcher in the shared core: # # 1. Jump between an opening and its close — the Sublime equivalent of the # Emacs mode's `klammertext-jump-to-match' (bound C-c C-j). Command name @@ -9,465 +10,172 @@ # # 2. Live highlighting of the matching delimiter as the caret sits on one — # the equivalent of the Emacs mode's show-paren support. Implemented as a -# ViewEventListener (see KlammertextMatchHighlighter at the bottom); no -# language server is involved. A mismatched named close or an unbalanced -# delimiter is highlighted in red with a status-bar message, mirroring the -# Emacs mode's klammertext-mismatch-face + minibuffer report. +# ViewEventListener (see KlammertextMatchHighlighter at the bottom). A +# mismatched named close or an unbalanced delimiter is highlighted in red +# with a status-bar message, mirroring the Emacs mode's +# klammertext-mismatch-face + minibuffer report. # # This is the companion to Klammertext.sublime-syntax. The syntax file only # colors tokens; a tokenizer cannot match context-dependent delimiters, so the # jump is implemented here as a TextCommand. The keybinding lives in the # companion Default.sublime-keymap. # -# Command name (for keymaps / the command palette): klammertext_jump_to_match +# The matcher itself — on-or-just-after caret rule, literal klammers matched +# BY NAME with opaque content (@code <-> code@), everything else by depth, +# @@/@@@ runs and removed text stepped over — lives in the shared core, +# doc/edit/shared/klammertext_edit.py, together with the LITERAL_KLAMMERS +# policy list. This file is only the Sublime wrapper. # -# --------------------------------------------------------------------------- -# What it does (a direct port of the elisp matcher): -# * On an opening @name, move to its closing @ or name@. -# * On a close (bare @ or name@), move to the opening @name. -# * Triggers when the caret is ON the @ or immediately AFTER it (the same -# on-or-just-after rule the Emacs command uses). -# * Only single-@ APPLICATION delimiters match. @@/@@@ runs, removed text -# (#, ##, #[...]#), escaped ^@, and literal-klammer spans (@code ... code@) -# are stepped over, exactly as in the Emacs mode. The abbreviated -# @name-arg form opens no span. -# * Works at every caret when there are multiple selections. +# The shared core is located next to this file (a vendored copy — the +# installed-package layout produced by doc/make_editing_zip.sh), or in +# ../shared (the repository layout), or under $KLAMMERTEXT_HOME. # -# Literal klammers (identical to C-c C-j): a @code ... code@ span is opaque. -# The general depth scan still steps over such a span WHOLESALE when matching -# some OTHER klammer, so verbatim @ inside it never miscount. A literal -# klammer's OWN delimiters are matched BY NAME rather than by depth (see -# app_match): @code jumps to the next code@, and code@ to the nearest preceding -# @code — correct even when the content holds unbalanced @, e.g. @code x @ y -# code@. LITERAL_KLAMMERS lists these names; keep it in sync with the '@code' -# handling in Klammertext.sublime-syntax. -# -# LITERAL_KLAMMERS must stay in sync with the literal klammers recognized in -# Klammertext.sublime-syntax (seeded there as @code). The Emacs mode keeps this -# list in the `klammertext-literal-klammers' defcustom; a plugin has no access -# to it, so it is duplicated here. +# SYNC: the literal-klammer set in the shared core must agree with the @code +# rule + literal_code context in Klammertext.sublime-syntax (a static syntax +# file cannot read Python; both are seeded with just 'code'). -import sublime -import sublime_plugin +import os +import sys -# Klammer names whose content is a literal argument (verbatim interior). -# -# SYNC: this list is one of four copies that must agree. When you add or -# remove a literal klammer, mirror it in all four: -# * klammertext-literal-klammers in doc/emacs/klammertext-mode.el (the source -# of truth; a Sublime syntax/plugin cannot read that Emacs defcustom) -# * LITERAL_KLAMMERS here -# * LITERAL_KLAMMERS in Klammertext_indent.py (a deletable unit, so it does -# not import from this file) -# * the @NAME literal rule + literal_NAME context in Klammertext.sublime-syntax -# All four are currently seeded with just "code". -LITERAL_KLAMMERS = set(["code"]) +try: + import sublime + import sublime_plugin + _IN_SUBLIME = True +except ImportError: # standalone import outside Sublime Text + _IN_SUBLIME = False -# --- pure helpers (operate on the whole buffer as a string) ---------------- - -def name_char_p(ch): - """True if CH can be part of a klammer name (letter, digit or _). - A hyphen is NOT a name char: @name-arg1 ends the name at the first hyphen.""" - if ch is None: - return False - return (('a' <= ch <= 'z') or ('A' <= ch <= 'Z') - or ('0' <= ch <= '9') or ch == '_') +def _import_shared(): + here = os.path.dirname(os.path.abspath(__file__)) + candidates = [here, os.path.join(os.path.dirname(here), 'shared')] + kh = os.environ.get('KLAMMERTEXT_HOME') + if kh: + candidates.append(os.path.join(kh, 'doc', 'edit', 'shared')) + for d in candidates: + if os.path.isfile(os.path.join(d, 'klammertext_edit.py')): + if d not in sys.path: + sys.path.insert(0, d) + break + import klammertext_edit + return klammertext_edit -def escaped_p(s, pos): - """True if the char at POS is escaped by an odd run of ^ before it. - In Klammertext ^# and ^@ are literal, so such a char is not a delimiter.""" - n = 0 - i = pos - 1 - while i >= 0 and s[i] == '^': - n += 1 - i -= 1 - return (n % 2) == 1 +KE = _import_shared() -def block_end(s, frm): - """Index just after the ]# that closes a #[ block opened at FROM (the index - just after the opening #[). Counts nested #[ ... ]#; len(s) if unclosed.""" - depth = 1 - i = frm - n = len(s) - while depth > 0: - a = s.find('#[', i) - b = s.find(']#', i) - if a == -1 and b == -1: - return n - if b == -1 or (a != -1 and a < b): - depth += 1 - i = a + 2 - else: - depth -= 1 - i = b + 2 - return i +if _IN_SUBLIME: + class KlammertextJumpToMatchCommand(sublime_plugin.TextCommand): + """Jump between a klammer application's opening and closing delimiter. + Sublime equivalent of the Emacs mode's C-c C-j.""" -def at_run_end(s, pos): - """Index just after the run of @ that begins at POS.""" - p = pos - n = len(s) - while p < n and s[p] == '@': - p += 1 - return p + def run(self, edit): + view = self.view + s = view.substr(sublime.Region(0, view.size())) + new_regions = [] + moved = False + message = None + for region in view.sel(): + m = KE.match_at(s, region.b) + if m is None: + new_regions.append(region) + message = ("point is not on a klammer application " + "delimiter (@)") + continue + if m['match'] is None: + new_regions.append(region) + message = ("no matching delimiter for this %s klammer" + % ("opening" if m['kind'] == 'open' + else "closing")) + continue + new_regions.append(sublime.Region(m['match'], m['match'])) + moved = True -def next_app_delim(s, i, limit): - """From index I, find the next single-@ application delimiter before LIMIT. - Step over @@/@@@ runs, removed text, literal spans, escaped ^@, and the - abbreviated @name-arg form. Return (pos, kind, next_i) with kind 'open' or - 'close' and next_i the index to resume from, or None when none is found.""" - n = len(s) - if limit is None: - limit = n - while i < limit: - # find next @ or # at or after i (emacs re-search-forward "[@#]") - j = i - while j < limit and s[j] != '@' and s[j] != '#': - j += 1 - if j >= limit: - return None - hit = j - i = hit + 1 # default: advance past the hit - if escaped_p(s, hit): # ^@ / ^# : keep going - continue - nxt = s[hit + 1] if hit + 1 < n else None - if s[hit] == '#': # removal: step over it - if nxt == '#': - i = n - elif nxt == '[': - i = block_end(s, hit + 2) - elif nxt in ('+', '/', '-'): - i = hit + 1 - else: # to end of line - eol = s.find('\n', hit) - i = n if eol == -1 else eol - continue - # s[hit] == '@' - if nxt == '@': # @@ / @@@ : step over the run - i = at_run_end(s, hit) - continue - if name_char_p(nxt): # @name : opening? - k = hit + 1 - while k < n and name_char_p(s[k]): - k += 1 - name = s[hit + 1:k] - after = s[k] if k < n else None - if name in LITERAL_KLAMMERS: # literal span: skip to its close - close = name + '@' - idx = s.find(close, k) - i = n if idx == -1 else idx + len(close) - continue - elif after == '-': # @name-arg : opens no span - i = k - continue + view.sel().clear() + for r in new_regions: + view.sel().add(r) + + if moved: + view.show(view.sel()[0].b) + elif message: + sublime.status_message("Klammertext: " + message) + + def is_enabled(self): + # Only meaningful in Klammertext buffers. + return self.view.match_selector(0, "text.klammertext") + + # --- live matched-delimiter highlighting (show-paren equivalent) ------- + + class KlammertextMatchHighlighter(sublime_plugin.ViewEventListener): + """Highlight the matching klammer application delimiter as the caret + sits on one. The Sublime equivalent of the Emacs mode's show-paren + support — driven by cursor movement, reusing the shared matcher. + + A matched pair is boxed (region.bluish); a mismatch or unbalanced + delimiter is boxed in red (region.redish) with a status-bar message. + Both the token under the caret and its match are boxed; the Emacs mode + highlights only the single @ character, but boxing the whole + @name / name@ reads better here. To highlight only the far delimiter, + drop the first region in _update().""" + + MATCH_KEY = 'klammertext_paren_match' + MISMATCH_KEY = 'klammertext_paren_mismatch' + + @classmethod + def is_applicable(cls, settings): + return str(settings.get('syntax', '')).endswith( + 'Klammertext.sublime-syntax') + + def __init__(self, view): + super().__init__(view) + self._change_count = -1 + self._text = '' + + def _buffer(self): + # Re-read the buffer only when it has actually changed, so plain + # cursor movement over a large file does not re-copy the document. + cc = self.view.change_count() + if cc != self._change_count: + self._text = self.view.substr( + sublime.Region(0, self.view.size())) + self._change_count = cc + return self._text + + def on_selection_modified_async(self): + self._update() + + def on_activated_async(self): + self._update() + + def _clear(self): + self.view.erase_regions(self.MATCH_KEY) + self.view.erase_regions(self.MISMATCH_KEY) + + def _update(self): + view = self.view + sel = view.sel() + if len(sel) == 0: + self._clear() + return + s = self._buffer() + m = KE.match_at(s, sel[0].b) + if m is None: + self._clear() + return + + regions = [sublime.Region(*m['token'])] + if m['match_token'] is not None: + regions.append(sublime.Region(*m['match_token'])) + + flags = sublime.DRAW_NO_FILL + if m['mismatch']: + view.erase_regions(self.MATCH_KEY) + view.add_regions(self.MISMATCH_KEY, regions, + 'region.redish', '', flags) + if m['message']: + sublime.status_message("Klammertext: " + m['message']) else: - return (hit, 'open', k) - else: # name@ / bare @ : closing - return (hit, 'close', hit + 1) - return None - - -def match_forward(s, open_pos): - """OPEN_POS is the @ of an opening application. Return the matching close @ - index, or None if unbalanced.""" - n = len(s) - i = open_pos + 1 - while i < n and name_char_p(s[i]): # past the opening name - i += 1 - depth = 1 - while depth > 0: - d = next_app_delim(s, i, None) - if d is None: - return None - pos, kind, nxt = d - i = nxt - if kind == 'open': - depth += 1 - else: - depth -= 1 - if depth == 0: - return pos - return None - - -def match_backward(s, close_pos): - """CLOSE_POS is the @ of a closing application. Return the matching open @ - index, or None if unbalanced. Scans forward from 0 with a stack.""" - stack = [] - i = 0 - limit = close_pos + 1 - while True: - d = next_app_delim(s, i, limit) - if d is None: - return None - pos, kind, nxt = d - i = nxt - if kind == 'open': - stack.append(pos) - else: - open_pos = stack.pop() if stack else None - if pos == close_pos: - return open_pos - - -def app_delim_info(s, pos): - """If the char at POS is a single-@ application delimiter, return - (pos, kind) with kind 'open' or 'close'; else None. The abbreviated - @name-arg form (which opens no span) returns None.""" - n = len(s) - if not (0 <= pos < n): - return None - if s[pos] != '@': - return None - if pos > 0 and s[pos - 1] == '@': - return None - if pos + 1 < n and s[pos + 1] == '@': - return None - if escaped_p(s, pos): - return None - nxt = s[pos + 1] if pos + 1 < n else None - if name_char_p(nxt): - k = pos + 1 - while k < n and name_char_p(s[k]): - k += 1 - after = s[k] if k < n else None - if after == '-': - return None - return (pos, 'open') - return (pos, 'close') - - -# --- name / mismatch helpers (for the live highlighter) -------------------- - -def _name_forward(s, pos): - """Index just past the run of name chars starting at POS.""" - n = len(s) - k = pos - while k < n and name_char_p(s[k]): - k += 1 - return k - - -def open_name(s, open_pos): - """Name of the opening @name whose @ is at OPEN_POS.""" - return s[open_pos + 1:_name_forward(s, open_pos + 1)] - - -def close_name(s, close_pos): - """Name of a named close NAME@ whose @ is at CLOSE_POS, or None for a bare @ - (including the compact @name@ form, whose name belongs to the opening).""" - ns = close_pos - while ns > 0 and name_char_p(s[ns - 1]): - ns -= 1 - if ns < close_pos and (ns == 0 or s[ns - 1] != '@'): - return s[ns:close_pos] - return None - - -def paren_mismatch(s, open_pos, close_pos): - """True if the pair is unbalanced (either side None) or the named close - disagrees with the opening name.""" - if open_pos is None or close_pos is None: - return True - cname = close_name(s, close_pos) - return cname is not None and cname != open_name(s, open_pos) - - -def token_region(s, pos, kind): - """(start, end) of the whole delimiter token whose @ is at POS. - Opening: @ plus its name. Named close: the name plus @. Bare @: just @.""" - if kind == 'open': - return (pos, _name_forward(s, pos + 1)) - ns = pos - while ns > 0 and name_char_p(s[ns - 1]): - ns -= 1 - if ns < pos and (ns == 0 or s[ns - 1] != '@'): - return (ns, pos + 1) # named close NAME@ - return (pos, pos + 1) # bare @ (or @name@) - - -# --- matching dispatch: literal klammers by name, others by depth ---------- - -def literal_delim_name(s, pos, kind): - """If the application delimiter at POS (kind 'open'/'close') belongs to a - literal klammer (name in LITERAL_KLAMMERS), return its name; else None. - A literal klammer's @NAME open and NAME@ close are matched by name, not by - depth counting, because its content is verbatim.""" - name = open_name(s, pos) if kind == 'open' else close_name(s, pos) - if name and name in LITERAL_KLAMMERS: - return name - return None - - -def literal_match_forward(s, open_pos, name): - """Index of the @ of the NAME@ that closes the literal @NAME at OPEN_POS, or - None. The content is opaque, so search for the literal close string.""" - start = open_pos + 1 + len(name) - idx = s.find(name + '@', start) - return idx + len(name) if idx != -1 else None - - -def literal_match_backward(s, close_pos, name): - """Index of the @ of the @NAME that opens the literal NAME@ whose @ is at - CLOSE_POS, or None. Literal spans do not nest, so the nearest preceding - real @NAME is the opener (not @@NAME, and not escaped).""" - open_str = '@' + name - end = close_pos - while True: - idx = s.rfind(open_str, 0, end) - if idx == -1: - return None - before = s[idx - 1] if idx > 0 else None - if before != '@' and not escaped_p(s, idx): - return idx - end = idx - - -def app_match(s, pos, kind): - """Matching application delimiter for the delimiter at POS of KIND - ('open'/'close'), or None. A literal klammer matches by name (@NAME <-> - NAME@) with content opaque; other klammers match by depth.""" - lit = literal_delim_name(s, pos, kind) - if lit is not None: - return (literal_match_forward(s, pos, lit) if kind == 'open' - else literal_match_backward(s, pos, lit)) - return match_forward(s, pos) if kind == 'open' else match_backward(s, pos) - - -# --- the command ----------------------------------------------------------- - -class KlammertextJumpToMatchCommand(sublime_plugin.TextCommand): - """Jump between a klammer application's opening and closing delimiter. - Sublime equivalent of the Emacs mode's C-c C-j.""" - - def run(self, edit): - view = self.view - s = view.substr(sublime.Region(0, view.size())) - new_regions = [] - moved = False - message = None - - for region in view.sel(): - p = region.b - info = app_delim_info(s, p) - if info is None and p > 0: - info = app_delim_info(s, p - 1) - if info is None: - new_regions.append(region) - message = "point is not on a klammer application delimiter (@)" - continue - dpos, kind = info - match = app_match(s, dpos, kind) - if match is None: - new_regions.append(region) - message = ("no matching delimiter for this %s klammer" - % ("opening" if kind == 'open' else "closing")) - continue - new_regions.append(sublime.Region(match, match)) - moved = True - - view.sel().clear() - for r in new_regions: - view.sel().add(r) - - if moved: - view.show(view.sel()[0].b) - elif message: - sublime.status_message("Klammertext: " + message) - - def is_enabled(self): - # Only meaningful in Klammertext buffers. - return self.view.match_selector(0, "text.klammertext") - - -# --- live matched-delimiter highlighting (show-paren equivalent) ----------- - -class KlammertextMatchHighlighter(sublime_plugin.ViewEventListener): - """Highlight the matching klammer application delimiter as the caret sits - on one. The Sublime equivalent of the Emacs mode's show-paren support — - driven by cursor movement, reusing the same context-sensitive matcher. - - A matched pair is boxed (region.bluish); a mismatch or unbalanced delimiter - is boxed in red (region.redish) with a status-bar message. Both the token - under the caret and its match are boxed; the Emacs mode highlights only the - single @ character, but boxing the whole @name / name@ reads better here. - To highlight only the far delimiter, drop the first region in _update().""" - - MATCH_KEY = 'klammertext_paren_match' - MISMATCH_KEY = 'klammertext_paren_mismatch' - - @classmethod - def is_applicable(cls, settings): - return str(settings.get('syntax', '')).endswith('Klammertext.sublime-syntax') - - def __init__(self, view): - super().__init__(view) - self._change_count = -1 - self._text = '' - - def _buffer(self): - # Re-read the buffer only when it has actually changed, so plain cursor - # movement over a large file does not re-copy the whole document. - cc = self.view.change_count() - if cc != self._change_count: - self._text = self.view.substr(sublime.Region(0, self.view.size())) - self._change_count = cc - return self._text - - def on_selection_modified_async(self): - self._update() - - def on_activated_async(self): - self._update() - - def _clear(self): - self.view.erase_regions(self.MATCH_KEY) - self.view.erase_regions(self.MISMATCH_KEY) - - def _update(self): - view = self.view - sel = view.sel() - if len(sel) == 0: - self._clear() - return - p = sel[0].b - s = self._buffer() - - info = app_delim_info(s, p) - if info is None and p > 0: - info = app_delim_info(s, p - 1) - if info is None: - self._clear() - return - - dpos, kind = info - match = app_match(s, dpos, kind) - open_pos = dpos if kind == 'open' else match - close_pos = match if kind == 'open' else dpos - mism = paren_mismatch(s, open_pos, close_pos) - - regions = [sublime.Region(*token_region(s, dpos, kind))] - if match is not None: - other_kind = 'close' if kind == 'open' else 'open' - regions.append(sublime.Region(*token_region(s, match, other_kind))) - - flags = sublime.DRAW_NO_FILL - if mism: - view.erase_regions(self.MATCH_KEY) - view.add_regions(self.MISMATCH_KEY, regions, 'region.redish', '', flags) - if match is None: - if kind == 'open': - msg = "opening @%s has no matching close" % open_name(s, open_pos) - else: - msg = "closing delimiter has no matching open" - else: - msg = ("closing %s@ does not match opening @%s" - % (close_name(s, close_pos) or '?', open_name(s, open_pos))) - sublime.status_message("Klammertext: " + msg) - else: - view.erase_regions(self.MISMATCH_KEY) - view.add_regions(self.MATCH_KEY, regions, 'region.bluish', '', flags) + view.erase_regions(self.MISMATCH_KEY) + view.add_regions(self.MATCH_KEY, regions, + 'region.bluish', '', flags) diff --git a/doc/edit/sublime/Klammertext.sublime-syntax b/doc/edit/sublime/Klammertext.sublime-syntax index e3f66fd..90bb8b9 100644 --- a/doc/edit/sublime/Klammertext.sublime-syntax +++ b/doc/edit/sublime/Klammertext.sublime-syntax @@ -3,7 +3,7 @@ # Klammertext.sublime-syntax # # Sublime Text syntax highlighting for Klammertext (.kt and .k files). -# A port of the Emacs major mode doc/emacs/klammertext-mode.el. +# A port of the Emacs major mode doc/edit/emacs/klammertext-mode.el. # # --------------------------------------------------------------------------- # What it highlights (mirrors the Emacs mode's eight token classes): @@ -32,14 +32,18 @@ # '@code' rule and the 'literal_code' context below, replacing # code -> foo. # -# SYNC: the literal-klammer set is duplicated in three places that -# must agree (a .sublime-syntax file is static and cannot read the -# Emacs defcustom). When you add or remove one, mirror it in all: +# SYNC: the literal-klammer set's source of truth is +# LITERAL_KLAMMERS in doc/edit/shared/klammertext_edit.py (the +# shared core all the Python-side integrations import). A static +# syntax file cannot read it, so when you add or remove one, +# mirror it in the per-editor artifacts: # * klammertext-literal-klammers in -# doc/emacs/klammertext-mode.el (the source of truth) -# * LITERAL_KLAMMERS in Klammertext.py +# doc/edit/emacs/klammertext-mode.el # * the @NAME rule + literal_NAME context here -# All three are currently seeded with just 'code'. +# * the @NAME verbatim region in doc/edit/vim/syntax/klammertext.vim +# * the @NAME rule in +# doc/edit/vscode/syntaxes/klammertext.tmLanguage.json +# All are currently seeded with just 'code'. # # --------------------------------------------------------------------------- # How open vs. close is decided (the same rule the Emacs scanner uses): diff --git a/doc/edit/sublime/Klammertext_align.py b/doc/edit/sublime/Klammertext_align.py index f669ee7..01863a9 100644 --- a/doc/edit/sublime/Klammertext_align.py +++ b/doc/edit/sublime/Klammertext_align.py @@ -1,10 +1,9 @@ # Klammertext_align.py # # EXPERIMENTAL. Table alignment for Klammertext files — pads the cells of a -# klammer's rows so the | separators line up vertically. Companion to -# doc/emacs/klammertext-align.el (the same algorithm; keep the two in step). -# This file is a separate unit: delete it (or move it out of the package -# folder) to disable alignment entirely. +# klammer's rows so the | separators line up vertically. Counterpart of +# doc/edit/emacs/klammertext-align.el. This file is a separate unit: delete +# it (or move it out of the package folder) to disable alignment entirely. # # Command name (for keymaps / the command palette): klammertext_align_table # Keybinding: Ctrl+Alt+A (in Default.sublime-keymap), scoped to Klammertext @@ -16,389 +15,45 @@ # Row 2 | Text | Not as long || # @ # -# Alignment is for SMALL data items (2026-07-27): +# The algorithm, its rules (rows end with ||; a row with a cell over +# CELL_MAX or spanning lines is untouched; beyond ROW_MAX columns nothing +# changes; only depth-0 bars are separators; no whitespace ever inside a bar +# run), and the policy lists (ALIGN_KLAMMERS, CELL_MAX, ROW_MAX) live in the +# shared core, doc/edit/shared/klammertext_edit.py. This file is only the +# Sublime command wrapper. # -# * A row is one line ending with the row delimiter || (the customary -# trailing delimiter; the parser strips one trailing top-level delimiter, -# and it keeps every row uniform). The last row may omit the ||. -# * A row is LEFT UNTOUCHED when any of its cells is longer than CELL_MAX -# (30) characters, or when the row spans lines (a cell with a newline). -# Untouched rows do not contribute to the column widths. -# * If the aligned rows would exceed ROW_MAX (100) columns, nothing is -# changed and the status bar says so — the general case of long rows has -# no good answer, so the command declines rather than guessing. -# -# Cell padding is semantically free: the SKS strips cell content, and no -# whitespace is ever inserted inside a bar run (that would turn a || row -# separator into an empty | | cell — the load-bearing-whitespace trap). -# Bars inside a nested klammer (e.g. @frac 1 | 2 @ in a cell) belong to that -# klammer, not the table: only bars at nesting depth 0 within the table span -# count, the same depth rule the Klammermachine itself applies to @cond. -# Aligned rows adopt the leading whitespace of the first aligned row; run -# the reindent command (Ctrl+Alt+I) first if the rows disagree. -# -# SYNC: ALIGN_KLAMMERS / CELL_MAX / ROW_MAX mirror the Emacs defcustoms -# klammertext-align-klammers / -cell-max / -row-max in klammertext-align.el. -# LITERAL_KLAMMERS is the same four-way synced list as everywhere else. The -# scanning helpers are duplicated from Klammertext.py so this file stays a -# deletable unit with no import coupling. +# The shared core is located next to this file (a vendored copy — the +# installed-package layout produced by doc/make_editing_zip.sh), or in +# ../shared (the repository layout), or under $KLAMMERTEXT_HOME. + +import os +import sys try: import sublime import sublime_plugin _IN_SUBLIME = True -except ImportError: # standalone testing outside Sublime Text +except ImportError: # standalone import outside Sublime Text _IN_SUBLIME = False -import bisect -ALIGN_KLAMMERS = set(["table"]) -CELL_MAX = 30 -ROW_MAX = 100 -LITERAL_KLAMMERS = set(["code"]) - - -# --- pure helpers (duplicated from Klammertext.py; see SYNC note above) ----- - -def name_char_p(ch): - """True if CH can be part of a klammer name (letter, digit or _).""" - if ch is None: - return False - return (('a' <= ch <= 'z') or ('A' <= ch <= 'Z') - or ('0' <= ch <= '9') or ch == '_') - - -def escaped_p(s, pos): - """True if the char at POS is escaped by an odd run of ^ before it.""" - n = 0 - i = pos - 1 - while i >= 0 and s[i] == '^': - n += 1 - i -= 1 - return (n % 2) == 1 - - -def block_end(s, frm): - """Index just after the ]# that closes a #[ block opened at FROM (the index - just after the opening #[). Counts nested #[ ... ]#; len(s) if unclosed.""" - depth = 1 - i = frm - n = len(s) - while depth > 0: - a = s.find('#[', i) - b = s.find(']#', i) - if a == -1 and b == -1: - return n - if b == -1 or (a != -1 and a < b): - depth += 1 - i = a + 2 - else: - depth -= 1 - i = b + 2 - return i - - -def at_run_end(s, pos): - """Index just after the run of @ that begins at POS.""" - p = pos - n = len(s) - while p < n and s[p] == '@': - p += 1 - return p - - -# --- finding the enclosing table span --------------------------------------- - -def enclosing_span(s, pos, names): - """Innermost span of a klammer named in NAMES that contains POS. - Return (name, content_start, content_end) with content_start just after - the opening @name token and content_end at the start of the closing - delimiter token, or None. Scans S from the start with a position stack, - stepping over removed text, literal spans, escaped characters, and the - abbreviated @name-arg form.""" - stack = [] # (name, open_token_start, content_start) - n = len(s) - i = 0 - while i < n: - j = i - while j < n and s[j] != '@' and s[j] != '#': - j += 1 - if j >= n: +def _import_shared(): + here = os.path.dirname(os.path.abspath(__file__)) + candidates = [here, os.path.join(os.path.dirname(here), 'shared')] + kh = os.environ.get('KLAMMERTEXT_HOME') + if kh: + candidates.append(os.path.join(kh, 'doc', 'edit', 'shared')) + for d in candidates: + if os.path.isfile(os.path.join(d, 'klammertext_edit.py')): + if d not in sys.path: + sys.path.insert(0, d) break - hit = j - i = hit + 1 - if escaped_p(s, hit): - continue - nxt = s[hit + 1] if hit + 1 < n else None - if s[hit] == '#': - if nxt == '#': - break # ## removes the rest of the buffer - elif nxt == '[': - i = block_end(s, hit + 2) - elif nxt in ('+', '/', '-'): - pass - else: - eol = s.find('\n', hit) - i = n if eol == -1 else eol - continue - run_end = at_run_end(s, hit) - run_len = run_end - hit - after = s[run_end] if run_end < n else None - if name_char_p(after): - k = run_end - while k < n and name_char_p(s[k]): - k += 1 - name = s[run_end:k] - i = k - if run_len == 1 and name in LITERAL_KLAMMERS: - idx = s.find(name + '@', k) - if idx == -1: - break - i = idx + len(name) + 1 - elif run_len == 1 and k < n and s[k] == '-': - pass # @name-arg : opens no span - else: - stack.append((name, hit, k)) - else: - # a close: the token starts at the preceding name run, if any - ns = hit - while ns > 0 and name_char_p(s[ns - 1]): - ns -= 1 - tok_start = ns if (ns < hit and (ns == 0 or s[ns - 1] != '@')) else hit - if stack: - name, open_start, content_start = stack.pop() - if name in names and open_start <= pos <= run_end: - return (name, content_start, tok_start) - i = run_end - return None + import klammertext_edit + return klammertext_edit -# --- scanning the span content, line by line -------------------------------- +KE = _import_shared() -def scan_lines(content): - """Scan CONTENT (the text of a table span). Return a list of line - records, one per line: dicts with start, end (offsets into CONTENT, end - excludes the newline), start_depth, end_depth (klammer nesting relative - to the span), bars (list of (pos, runlen) for unescaped depth-0 bar - runs), blocked (inside removed/verbatim content), comment (a # removes - the rest of the line).""" - n = len(content) - line_starts = [0] - for idx, ch in enumerate(content): - if ch == '\n': - line_starts.append(idx + 1) - nlines = len(line_starts) - - def line_index(p): - return bisect.bisect_right(line_starts, p) - 1 - - lines = [{'start': line_starts[k], - 'end': (line_starts[k + 1] - 1 if k + 1 < nlines else n), - 'bars': [], 'blocked': False, 'comment': False, - 'start_depth': None, 'end_depth': None} - for k in range(nlines)] - lines[0]['start_depth'] = 0 - - def block_range(a, b): - """Mark every line touched by [a, b) as blocked.""" - last = max(a, b - 1) - for k in range(line_index(a), line_index(min(last, n - 1)) + 1): - lines[k]['blocked'] = True - - depth = 0 - i = 0 - while i < n: - j = i - while j < n and content[j] not in '@#|\n': - j += 1 - if j >= n: - break - hit = j - i = hit + 1 - c = content[hit] - if c == '\n': - k = line_index(hit) - lines[k]['end_depth'] = depth - if k + 1 < nlines: - lines[k + 1]['start_depth'] = depth - continue - if escaped_p(content, hit): - continue - nxt = content[hit + 1] if hit + 1 < n else None - if c == '#': - if nxt == '#': - block_range(hit, n) - break - elif nxt == '[': - e = block_end(content, hit + 2) - if line_index(max(hit, e - 1)) != line_index(hit): - block_range(hit, e) - i = e - elif nxt in ('+', '/', '-'): - pass - else: # # to end of line - lines[line_index(hit)]['comment'] = True - eol = content.find('\n', hit) - i = n if eol == -1 else eol - continue - if c == '|': - if hit > 0 and content[hit - 1] == '|': - continue # mid-run (after an escaped ^|) - k = hit - while k < n and content[k] == '|': - k += 1 - if depth == 0: - lines[line_index(hit)]['bars'].append((hit, k - hit)) - i = k - continue - # '@' - run_end = at_run_end(content, hit) - run_len = run_end - hit - after = content[run_end] if run_end < n else None - if name_char_p(after): - k = run_end - while k < n and name_char_p(content[k]): - k += 1 - name = content[run_end:k] - i = k - if run_len == 1 and name in LITERAL_KLAMMERS: - idx = content.find(name + '@', k) - e = n if idx == -1 else idx + len(name) + 1 - if line_index(max(hit, e - 1)) != line_index(hit): - block_range(hit, e) - i = e - elif run_len == 1 and k < n and content[k] == '-': - pass - else: - depth += 1 - else: - depth = max(0, depth - 1) - i = run_end - - for ln in lines: - if ln['start_depth'] is None: - ln['blocked'] = True - if ln['end_depth'] is None: - ln['end_depth'] = depth - return lines - - -# --- the alignment ---------------------------------------------------------- - -def compute_edits(content): - """Compute the alignment edits for CONTENT (a table span's text). - Return (edits, message): edits is a list of (start, end, new_text) - triples relative to CONTENT, ascending; message is a status string (a - reason when edits is empty).""" - lines = scan_lines(content) - n = len(content) - - # The last line holding actual content (a row there may omit its ||). - last_content = None - for k in range(len(lines) - 1, 0, -1): - if content[lines[k]['start']:lines[k]['end']].strip(): - last_content = k - break - - rows = [] # (line record, cells, trailing_p) - chain_ok = True # a row must START a row: the previous - for k in range(1, len(lines)): # content line ended with || (or was the - ln = lines[k] # opener / an option line / blank) - text = content[ln['start']:ln['end']] - stripped = text.strip() - if not stripped: - continue # blank line: chain unchanged - if stripped.startswith(':') and not ln['bars']: - continue # option line: chain unchanged - row = _parse_row(content, ln, text, chain_ok, k == last_content) - trailing = _trailing_rowsep(content, ln) - chain_ok = trailing - if row is not None: - rows.append(row) - - if not rows: - return ([], "no alignable rows found") - - widths = [] - for _ln, cells, _tr in rows: - for c_idx, cell in enumerate(cells): - if c_idx >= len(widths): - widths.append(0) - widths[c_idx] = max(widths[c_idx], len(cell)) - - indent = ' ' * _indent_width(content, rows[0][0]) - if ROW_MAX is not None: - longest = 0 - for _ln, cells, trailing in rows: - m = len(cells) - w = (len(indent) + sum(widths[:m]) + 3 * (m - 1) - + (3 if trailing else 0)) - longest = max(longest, w) - if longest > ROW_MAX: - return ([], "aligned rows would be %d characters (limit %d); " - "not aligning" % (longest, ROW_MAX)) - - edits = [] - for ln, cells, trailing in rows: - parts = [cells[c].ljust(widths[c]) for c in range(len(cells) - 1)] - last = cells[-1] - if trailing: - last = last.ljust(widths[len(cells) - 1]) - parts.append(last) - new = indent + ' | '.join(parts) + (' ||' if trailing else '') - if new != content[ln['start']:ln['end']]: - edits.append((ln['start'], ln['end'], new)) - msg = ("aligned %d rows" % len(rows)) if edits else "already aligned" - return (edits, msg) - - -def _indent_width(content, ln): - i = ln['start'] - while i < ln['end'] and content[i] in ' \t': - i += 1 - return i - ln['start'] - - -def _trailing_rowsep(content, ln): - """True when the line's LAST depth-0 bar run is a || sitting at the end of - the line (only whitespace after it).""" - if not ln['bars']: - return False - pos, runlen = ln['bars'][-1] - return (runlen == 2 - and content[pos + 2:ln['end']].strip() == '') - - -def _parse_row(content, ln, text, chain_ok, is_last_content): - """If the line is an alignable row, return (ln, cells, trailing_p); - else None. See the file header for the rules.""" - if ln['blocked'] or ln['comment'] or not chain_ok: - return None - if ln['start_depth'] != 0 or ln['end_depth'] != 0: - return None - if not ln['bars']: - return None - trailing = _trailing_rowsep(content, ln) - singles = ln['bars'][:-1] if trailing else ln['bars'] - for _pos, runlen in singles: - if runlen != 1: - return None # a mid-line || (or |||): not one row - if not trailing and not is_last_content: - return None # row continues onto the next line - cell_start = ln['start'] + _indent_width(content, ln) - cell_end = ln['bars'][-1][0] if trailing else ln['end'] - bounds = [cell_start] + [p for p, _r in singles] + [cell_end] - cells = [] - for b_idx in range(len(bounds) - 1): - a = bounds[b_idx] + (1 if b_idx > 0 else 0) # skip the | itself - cell = content[a:bounds[b_idx + 1]].strip() - if len(cell) > CELL_MAX: - return None - cells.append(cell) - return (ln, cells, trailing) - - -# --- the command ------------------------------------------------------------ if _IN_SUBLIME: @@ -410,14 +65,15 @@ if _IN_SUBLIME: view = self.view s = view.substr(sublime.Region(0, view.size())) pos = view.sel()[0].b if len(view.sel()) else 0 - span = enclosing_span(s, pos, ALIGN_KLAMMERS) + span = KE.enclosing_span(s, pos, KE.ALIGN_KLAMMERS) if span is None: sublime.status_message( "Klammertext: the caret is not inside a table klammer (%s)" - % ", ".join("@" + name for name in sorted(ALIGN_KLAMMERS))) + % ", ".join("@" + name + for name in sorted(KE.ALIGN_KLAMMERS))) return _name, cs, ce = span - edits, msg = compute_edits(s[cs:ce]) + edits, msg = KE.compute_edits(s[cs:ce]) for a, b, new in sorted(edits, reverse=True): view.replace(edit, sublime.Region(cs + a, cs + b), new) sublime.status_message("Klammertext: " + msg) diff --git a/doc/edit/sublime/Klammertext_indent.py b/doc/edit/sublime/Klammertext_indent.py index b122781..c30d2c6 100644 --- a/doc/edit/sublime/Klammertext_indent.py +++ b/doc/edit/sublime/Klammertext_indent.py @@ -1,9 +1,9 @@ # Klammertext_indent.py # -# EXPERIMENTAL. Reindentation for Klammertext files — the Sublime Text port -# of doc/emacs/klammertext-indent.el. This file is a separate unit: delete it -# (or move it out of the package folder) to disable indentation entirely; the -# rest of the Klammertext package is unaffected. +# EXPERIMENTAL. Reindentation for Klammertext files — the Sublime Text +# counterpart of doc/edit/emacs/klammertext-indent.el. This file is a +# separate unit: delete it (or move it out of the package folder) to disable +# indentation entirely; the rest of the Klammertext package is unaffected. # # Command name (for keymaps / the command palette): klammertext_reindent # Keybinding: Ctrl+Alt+I (in Default.sublime-keymap), scoped to Klammertext @@ -11,245 +11,45 @@ # line when there is just a caret. Nothing reformats automatically (no # on-Enter auto-indent), because whitespace is content in Klammertext. # -# The convention (2026-07-27): +# The algorithm, the indentation convention, and the policy lists +# (TRANSPARENT_KLAMMERS, CODE_KLAMMERS, INDENT_OFFSET, ...) live in the +# shared core, doc/edit/shared/klammertext_edit.py — the single Python +# implementation used by the Sublime, Vim, and VS Code integrations and by +# the language server. This file is only the Sublime command wrapper. # -# @ol <- opener at its context's content column -# Item one <- content: opener column + 2 -# | Item two <- bar run at the OPENER's column ("| " is two -# @ol characters, so item text aligns with "Item one") -# Embedded item one -# | Embedded item two -# @ <- close at its opener's column -# | Item four -# @ -# -# Formal rule: a line indents to offset x (effective depth); a line that -# BEGINS with a bar run (|, ||, ...) or a closing delimiter (a bare @-run or -# a named close) indents one level less, i.e. to its owner's opening column. -# The bar-run rule is dimension-independent: | (list items), || (table rows) -# and any longer run all drop to the opener's column. Effective depth counts -# every enclosing span uniformly -- applications (@), definitions (@@), and -# system commands (@@@) -- with these exceptions: -# -# * Klammers in TRANSPARENT_KLAMMERS (seeded with "document") contribute no -# level, so ordinary paragraphs of a document sit at the left margin. -# * Lines inside a literal klammer's verbatim content (@code ... code@) and -# inside the argument span of a klammer in CODE_KLAMMERS (seeded with -# "eval" -- inline Python is indentation-sensitive!) are NEVER touched. -# Removed regions (#[ ... ]#, everything after ##) are likewise left -# alone. -# -# Known limitation (shared with the Emacs scanner): a raw @ inside a ^'...'^ -# literal region would confuse the depth scan. -# -# SYNC: the policy lists below must agree with the Emacs side: -# * LITERAL_KLAMMERS with klammertext-literal-klammers (also duplicated in -# Klammertext.py and Klammertext.sublime-syntax; all seeded "code") -# * TRANSPARENT_KLAMMERS with klammertext-transparent-klammers ("document") -# * CODE_KLAMMERS with klammertext-code-klammers ("eval") -# * INDENT_OFFSET with klammertext-indent-offset (2) -# The scanning helpers (name_char_p, escaped_p, block_end, at_run_end) are -# duplicated from Klammertext.py so this file stays a deletable unit with no -# import coupling. +# The shared core is located next to this file (a vendored copy — the +# installed-package layout produced by doc/make_editing_zip.sh), or in +# ../shared (the repository layout), or under $KLAMMERTEXT_HOME. + +import os +import sys try: import sublime import sublime_plugin _IN_SUBLIME = True -except ImportError: # standalone testing outside Sublime Text +except ImportError: # standalone import outside Sublime Text _IN_SUBLIME = False -INDENT_OFFSET = 2 -LITERAL_KLAMMERS = set(["code"]) -TRANSPARENT_KLAMMERS = set(["document"]) -CODE_KLAMMERS = set(["eval"]) - -# --- pure helpers (duplicated from Klammertext.py; see SYNC note above) ----- - -def name_char_p(ch): - """True if CH can be part of a klammer name (letter, digit or _).""" - if ch is None: - return False - return (('a' <= ch <= 'z') or ('A' <= ch <= 'Z') - or ('0' <= ch <= '9') or ch == '_') - - -def escaped_p(s, pos): - """True if the char at POS is escaped by an odd run of ^ before it.""" - n = 0 - i = pos - 1 - while i >= 0 and s[i] == '^': - n += 1 - i -= 1 - return (n % 2) == 1 - - -def block_end(s, frm): - """Index just after the ]# that closes a #[ block opened at FROM (the index - just after the opening #[). Counts nested #[ ... ]#; len(s) if unclosed.""" - depth = 1 - i = frm - n = len(s) - while depth > 0: - a = s.find('#[', i) - b = s.find(']#', i) - if a == -1 and b == -1: - return n - if b == -1 or (a != -1 and a < b): - depth += 1 - i = a + 2 - else: - depth -= 1 - i = b + 2 - return i - - -def at_run_end(s, pos): - """Index just after the run of @ that begins at POS.""" - p = pos - n = len(s) - while p < n and s[p] == '@': - p += 1 - return p - - -# --- the depth scanner (port of klammertext-indent--state-at) --------------- - -def state_at(s, pos): - """Scan s[0:POS] (POS a line beginning). Return (stack, opaque): STACK is - the list of names of the klammer applications, @@ definitions and @@@ - commands open at POS, outermost first; OPAQUE is True when POS lies inside - content that indentation must not touch (removed text, a literal klammer's - verbatim span, or a code klammer's argument span).""" - stack = [] - n = len(s) - i = 0 - while i < pos: - j = i - while j < pos and s[j] != '@' and s[j] != '#': - j += 1 - if j >= pos: +def _import_shared(): + here = os.path.dirname(os.path.abspath(__file__)) + candidates = [here, os.path.join(os.path.dirname(here), 'shared')] + kh = os.environ.get('KLAMMERTEXT_HOME') + if kh: + candidates.append(os.path.join(kh, 'doc', 'edit', 'shared')) + for d in candidates: + if os.path.isfile(os.path.join(d, 'klammertext_edit.py')): + if d not in sys.path: + sys.path.insert(0, d) break - hit = j - i = hit + 1 - if escaped_p(s, hit): # ^@ / ^# : plain text - continue - nxt = s[hit + 1] if hit + 1 < n else None - if s[hit] == '#': - if nxt == '#': # ## removes to end of buffer - return (stack, True) - elif nxt == '[': # #[ ... ]# (nestable) - end = block_end(s, hit + 2) - if pos < end: - return (stack, True) - i = end - elif nxt in ('+', '/', '-'): # whitespace operators - pass - else: # # to end of line - eol = s.find('\n', hit) - i = n if eol == -1 else eol - continue - # an @-run - run_end = at_run_end(s, hit) - run_len = run_end - hit - after = s[run_end] if run_end < n else None - if name_char_p(after): - # @name / @@name / @@@name : an opener (or, for a literal - # klammer, a verbatim span to step over). - k = run_end - while k < n and name_char_p(s[k]): - k += 1 - name = s[run_end:k] - i = k - if run_len == 1 and name in LITERAL_KLAMMERS: - # Verbatim interior: find the closing NAME@ by name. - idx = s.find(name + '@', k) - if idx == -1: # never closed - return (stack, True) - close_end = idx + len(name) + 1 - if pos < close_end: - return (stack, True) - i = close_end - elif run_len == 1 and k < n and s[k] == '-': - pass # @name-arg : opens no span - else: - stack.append(name) - else: - # a bare @-run, or the run of a named close: a close. - if stack: - stack.pop() - i = run_end - opaque = any(name in CODE_KLAMMERS for name in stack) - return (stack, opaque) + import klammertext_edit + return klammertext_edit -def depth(stack): - """Number of indentation levels STACK contributes. - Transparent klammers contribute none.""" - return sum(1 for name in stack if name not in TRANSPARENT_KLAMMERS) +KE = _import_shared() -def dedent_line_p(s, bol): - """True when the line starting at BOL begins with a token that sits at its - owner's opening column: a bar run (|, ||, ...), a bare close run (@, @@, - @@@), or a named close (name@, name@@, name@@@). A line beginning with an - opener (@name, @@name, @@@name) is content-level.""" - n = len(s) - i = bol - while i < n and s[i] in ' \t': - i += 1 - if i >= n: - return False - c = s[i] - if c == '|': - return True - if c == '@': - run_end = at_run_end(s, i) - return not name_char_p(s[run_end] if run_end < n else None) - if name_char_p(c): - # A named close: name chars followed by an @-run (an unescaped @ can - # only be a delimiter). - k = i - while k < n and name_char_p(s[k]): - k += 1 - return k < n and s[k] == '@' - return False - - -def target_column(s, bol): - """Column for the line starting at BOL, or None for lines that must not - be touched (verbatim, code, or removed content).""" - stack, opaque = state_at(s, bol) - if opaque: - return None - if dedent_line_p(s, bol) and stack: - stack = stack[:-1] - return INDENT_OFFSET * depth(stack) - - -def reindent_lines(s, bols): - """Compute the edits that reindent the lines whose beginnings are BOLS. - Return a list of (start, end, replacement) triples over S, in ascending - order, replacing each line's leading whitespace; untouchable lines and - already-correct lines produce no edit. Pure function -- also used by the - standalone tests.""" - n = len(s) - edits = [] - for bol in bols: - tgt = target_column(s, bol) - if tgt is None: - continue - i = bol - while i < n and s[i] in ' \t': - i += 1 - if s[bol:i] != ' ' * tgt: - edits.append((bol, i, ' ' * tgt)) - return edits - - -# --- the command ------------------------------------------------------------ - if _IN_SUBLIME: class KlammertextReindentCommand(sublime_plugin.TextCommand): @@ -269,7 +69,7 @@ if _IN_SUBLIME: bols.append(line.a) # Compute all edits from the original text, then apply from the # bottom up so earlier offsets stay valid. - for a, b, new in sorted(reindent_lines(s, bols), reverse=True): + for a, b, new in sorted(KE.reindent_lines(s, bols), reverse=True): view.replace(edit, sublime.Region(a, b), new) def is_enabled(self): diff --git a/doc/edit/sublime/README.md b/doc/edit/sublime/README.md index 0467704..ef6b26c 100644 --- a/doc/edit/sublime/README.md +++ b/doc/edit/sublime/README.md @@ -1,18 +1,24 @@ # Klammertext for Sublime Text -A Sublime Text port of the Emacs major mode for Klammertext -(`doc/emacs/klammertext-mode.el`). It brings syntax highlighting, delimiter -matching, and comment toggling to `.kt` and `.k` files. Behavior mirrors the -Emacs mode closely; where the two intentionally differ, the file headers say so. +A Sublime Text package for Klammertext: syntax highlighting, delimiter +matching, comment toggling, reindentation, and table alignment for `.kt` and +`.k` files. The structural features run on the **shared editor core** +(`klammertext_edit.py`) — the single Python implementation used by the Vim +and VS Code integrations and the Klammertext language server — so all the +editors behave identically; the plugin files here are Sublime command +wrappers. Behavior mirrors the Emacs mode closely (an independent elisp +implementation, held equal by the Klammertext test suite); where the two +intentionally differ, the file headers say so. ## Files | File | Purpose | |------|---------| | `Klammertext.sublime-syntax` | Syntax highlighting. Colors the text-removal constructs (`#`, `##`, `#[...]#`) and the three `@`-tiers — application `@`, definition `@@`, system `@@@` — each as an opening vs. a close, plus `^`-escapes and verbatim `@code ... code@` spans. | -| `Klammertext.py` | Plugin with two features that share one context-sensitive matcher: jump between an opening and its close, and live highlighting of the matching delimiter as the caret moves (mismatched or unbalanced delimiters flag in red). | +| `Klammertext.py` | Jump between an opening and its close, and live highlighting of the matching delimiter as the caret moves (mismatched or unbalanced delimiters flag in red). Both reuse the shared core's context-sensitive matcher. | | `Klammertext_indent.py` | **Experimental.** Reindentation per the Klammertext convention (see below). A separate unit: delete this one file to disable indentation; nothing else is affected. | | `Klammertext_align.py` | **Experimental.** Table alignment (see below). Also a separate, deletable unit. | +| `klammertext_edit.py` | The shared editor core the three plugins import. Ships in the editing zip; when you install from the Klammertext repository instead, copy it in from `doc/edit/shared/` (or leave the package inside the repository tree, where the plugins find `../shared/` themselves). | | `Default.sublime-keymap` | Binds jump-to-match to **Ctrl+M**, reindent to **Ctrl+Alt+I**, and table alignment to **Ctrl+Alt+A**, scoped to Klammertext files. | | `Comments.tmPreferences` | Comment toggling: **Ctrl+/** inserts `# ` (line removal), **Ctrl+Shift+/** wraps in `#[ ... ]#` (block removal). | | `Breakers` / `Celeste` / `Mariana` / `Monokai` / `Sixteen` `.sublime-color-scheme` | Color overrides for Sublime's five built-in schemes — one hue system, full intensity on the dark schemes, scaled down on the light ones. Additive: they recolor only the Klammertext delimiters and leave the rest of each scheme unchanged. | @@ -32,7 +38,10 @@ directory: The quickest way to find it: **Preferences ￫ Browse Packages…** opens the `Packages` directory. Create the `Klammertext` folder there and copy the files in. Sublime loads them live — no restart — and applies the syntax to `.kt` and -`.k` files automatically. +`.k` files automatically. The package must include `klammertext_edit.py` +(see the file table above): the editing zip ships it in place; from a +repository checkout, copy `doc/edit/shared/klammertext_edit.py` into the +folder alongside the plugin files. Use a dedicated folder (not `Packages/User/`) so the bundled keymap does not merge into your personal one. If you want highlighting only, the @@ -60,7 +69,7 @@ prefer `super+m` can change it in `Default.sublime-keymap`. ## Indentation (experimental) `Klammertext_indent.py` ports the Emacs mode's indentation -(`doc/emacs/klammertext-indent.el`): **Ctrl+Alt+I** reindents the line(s) +(`doc/edit/emacs/klammertext-indent.el`): **Ctrl+Alt+I** reindents the line(s) touched by the selection to reflect the klammer nesting, two spaces per level: ``` @@ -82,8 +91,9 @@ belong to. All three `@`-tiers indent uniformly. Exceptions: `@document` contributes no level (a document's paragraphs stay at the left margin); lines inside verbatim `@code` content, inside `@eval` argument spans (inline Python is indentation-sensitive), and inside removed regions are never touched. The -policy lists (`TRANSPARENT_KLAMMERS`, `CODE_KLAMMERS`, `INDENT_OFFSET`) are at -the top of `Klammertext_indent.py`, kept in sync with the Emacs defcustoms. +policy lists (`TRANSPARENT_KLAMMERS`, `CODE_KLAMMERS`, `INDENT_OFFSET`) are +in the shared core (`klammertext_edit.py`), kept in sync with the Emacs +defcustoms. Sublime's own Reindent (Edit ￫ Line ￫ Reindent) is driven by single-line regex patterns that cannot express Klammertext nesting, so this is a plugin @@ -96,7 +106,7 @@ keep the command available from plugins). ## Table alignment (experimental) `Klammertext_align.py` ports the Emacs mode's table alignment -(`doc/emacs/klammertext-align.el`): **Ctrl+Alt+A** with the caret anywhere +(`doc/edit/emacs/klammertext-align.el`): **Ctrl+Alt+A** with the caret anywhere inside a `@table` span pads the cells of its rows so the `|` separators line up: @@ -118,8 +128,8 @@ bar run (which would turn a `||` row separator into an empty `| |` cell), and bars inside a nested klammer in a cell (`@frac 1 | 2 @`) belong to that klammer, not the table. Aligned rows adopt the leading whitespace of the first aligned row — run Ctrl+Alt+I first if the rows disagree. The limits -sit at the top of `Klammertext_align.py`, mirrored from the Emacs -defcustoms. +(`ALIGN_KLAMMERS`, `CELL_MAX`, `ROW_MAX`) are in the shared core +(`klammertext_edit.py`), mirrored from the Emacs defcustoms. ## Colors @@ -143,25 +153,30 @@ exact values are in each file's header comment. ## Keeping literal klammers in sync -Klammers whose content is verbatim (`@code ... code@`) are listed in four -places that must agree — a Sublime syntax/plugin cannot read the Emacs -defcustom, so the list is duplicated: +Klammers whose content is verbatim (`@code ... code@`) are listed in the +shared core — `LITERAL_KLAMMERS` in `klammertext_edit.py`, the source of +truth — and restated in the static per-editor artifacts, which cannot read +Python: -- `klammertext-literal-klammers` in `doc/emacs/klammertext-mode.el` (the source of truth) -- `LITERAL_KLAMMERS` in `Klammertext.py` -- `LITERAL_KLAMMERS` in `Klammertext_indent.py` - the `@code` rule and `literal_code` context in `Klammertext.sublime-syntax` +- `klammertext-literal-klammers` in `doc/edit/emacs/klammertext-mode.el` + (the independent elisp implementation) +- the `@code` region in `doc/edit/vim/syntax/klammertext.vim` and the rule + in `doc/edit/vscode/syntaxes/klammertext.tmLanguage.json` -All four are seeded with just `code`. When you add or remove a literal -klammer, change all four. +All are seeded with just `code`. When you add or remove a literal klammer, +change them together. ## Not included -Whole-file semantic validation — persistent error underlines when the cursor is -elsewhere, klammer-name completion, go-to-definition — is not part of this -package. That would need a language server (used through the Sublime LSP -package), a separate program, and is unrelated to the highlighting and matching -provided here. +Whole-file diagnostics (persistent error underlines when the cursor is +elsewhere) are not part of this package, but they exist: the **Klammertext +language server** (`doc/edit/shared/klammertext_ls.py`, the same program the +VS Code extension uses) serves them to Sublime through the community LSP +package. Install "LSP" from Package Control and add a client with +`command: ["python3", "/path/to/klammertext_ls.py"]` for the +`text.klammertext` selector. Everything in this package works the same +with or without it. ## Troubleshooting diff --git a/doc/edit/vim/README.md b/doc/edit/vim/README.md new file mode 100644 index 0000000..060cdef --- /dev/null +++ b/doc/edit/vim/README.md @@ -0,0 +1,117 @@ +# Klammertext support for Vim + +Vim support for editing Klammertext files (`.kt` documents and `.k` klammer +definitions): syntax highlighting, delimiter matching and jumping, structural +reindentation, table alignment, and a delimiter checker. + +The structure-aware features are thin wrappers around the **shared editor +core** (`klammertext_edit.py`) — the single Python implementation of +Klammertext's structural layer used by the Sublime Text and VS Code +integrations and by the Klammertext language server. The Vim plugin runs it +through `python3`; there is no Vim-specific reimplementation to drift out of +sync. + +## Requirements + +- Vim 8+ (or Neovim) with `+eval` — any normal Vim; only `vim-tiny` lacks it. +- `python3` on `PATH` (Klammertext itself already requires Python). +- The shared core, found automatically in this order: + 1. `g:klammertext_edit_py`, if you set it (a full path to + `klammertext_edit.py`); + 2. a `klammertext_edit.py` vendored at this plugin's root (the layout the + Klammertext editing zip ships); + 3. `../shared/klammertext_edit.py` relative to the plugin directory (the + layout of the Klammertext repository — using the plugin straight from a + checkout just works); + 4. `$KLAMMERTEXT_HOME/doc/edit/shared/klammertext_edit.py`. + +## Install + +Copy (or symlink) this `vim/` directory into a Vim native package path, +renaming it as you like: + + mkdir -p ~/.vim/pack/klammertext/start + cp -R vim ~/.vim/pack/klammertext/start/klammertext + +(Neovim: `~/.local/share/nvim/site/pack/klammertext/start/klammertext`.) +If you copy the directory out of the Klammertext tree, also copy +`shared/klammertext_edit.py` into the copied folder's root — or set +`g:klammertext_edit_py`. The editing zip from the Klammertext website ships +the vendored copy already in place. + +`.kt` and `.k` files then get the `klammertext` filetype. **Note:** `.kt` +is also Kotlin's extension, and stock Vim maps it to Kotlin; this plugin's +`ftdetect` overrides that unconditionally. If you edit both languages, see +the comment in `ftdetect/klammertext.vim`. + +## What you get + +**Syntax highlighting** — the same token classes and palette as the Emacs +and Sublime Text support: text removal (`#`, `##`, nestable `#[ ... ]#`), +the three `@`-tiers — application (`@`, blue), definition (`@@`, green), +system (`@@@`, orange) — each as an opening (`@name`) or a close (`name@`, +bare `@`), opens bright and closes the same hue darker; `^`-escapes; +verbatim `@code ... code@` interiors. All groups are `hi def` — override +them with `:highlight` in your vimrc. + +**Commands** (buffer-local; default mappings below): + +| Command | Does | +|---|---| +| `:[range]KlammertextReindent` | reindent the range (default: whole buffer) | +| `:KlammertextAlign` | align the `@table` enclosing the cursor | +| `:KlammertextJumpToMatch` | jump between a klammer application's opening and closing delimiter | +| `:KlammertextCheck` | list unclosed/mismatched delimiters in the location list | + +Default mappings (buffer-local; suppress them all with +`let g:klammertext_no_mappings = 1`): `i` reindents the current +line (in visual mode, the selection), `a` aligns the enclosing +table, `j` jumps to the matching delimiter. `LocalLeader` +defaults to backslash; set `maplocalleader` to taste. + +Reindentation is **explicit-only**: whitespace is content in Klammertext, +so nothing reformats as a side effect of typing (`indentkeys` is emptied). +Alignment follows the shared rules: rows end with `||`; a row with a cell +over 30 characters or spanning lines is left untouched; beyond 100 columns +the command declines; bars inside a nested klammer are not separators; no +whitespace is ever inserted inside a bar run. + +**With `+python3`** (check `:echo has('python3')`) the shared core also runs +in-process: the `=` operator reindents through `'indentexpr'` (`==`, `gg=G`), +and the matching delimiter is highlighted live as the cursor sits on one — +the show-paren equivalent, with a mismatched or unbalanced delimiter shown +in red plus a message. Without `+python3` the commands above still work; +they shell out to `python3`. + +**Comment toggling** — `'commentstring'` is set to `# %s`, so Vim 9.1's +built-in commenting and Neovim's `gcc`/`gc` (or the commentary plugin) +toggle `#` line removal. + +## Configuration + +| Variable | Meaning (default) | +|---|---| +| `g:klammertext_edit_py` | full path to `klammertext_edit.py` (auto-detected) | +| `g:klammertext_python` | Python interpreter (`python3`) | +| `g:klammertext_no_mappings` | set to define no default mappings | + +## Neovim and the language server + +Neovim users can additionally attach Neovim's built-in LSP client to the +Klammertext language server (`klammertext_ls.py`, next to the shared core) +for diagnostics as you type, `gq`/format-document reindentation, and +cursor-hold match highlighting: + +```lua +vim.api.nvim_create_autocmd('FileType', { + pattern = 'klammertext', + callback = function() + vim.lsp.start { + name = 'klammertext-ls', + cmd = { 'python3', '/path/to/doc/edit/shared/klammertext_ls.py' }, + } + end, +}) +``` + +The plugin's own commands work the same with or without it. diff --git a/doc/edit/vim/autoload/klammertext.vim b/doc/edit/vim/autoload/klammertext.vim new file mode 100644 index 0000000..29dd317 --- /dev/null +++ b/doc/edit/vim/autoload/klammertext.vim @@ -0,0 +1,299 @@ +" autoload/klammertext.vim — implementation of the Klammertext commands. +" +" All the structure-aware work (indentation, table alignment, delimiter +" matching, diagnostics) is done by the shared Python core, +" klammertext_edit.py — the single implementation used by the Sublime, +" Vim, and VS Code integrations and by the language server. This file is +" glue: it locates the core, runs it (as a stdin/stdout filter, or +" in-process via +python3 where that is faster), and applies the results +" to the buffer. +" +" Configuration: +" g:klammertext_edit_py full path to klammertext_edit.py (overrides the +" search described below) +" g:klammertext_python the Python interpreter (default "python3") + +" --- locating the shared core --------------------------------------------- +" Search order: an explicit g:klammertext_edit_py; a vendored copy at this +" plugin's root (the layout the editing zip installs); ../shared relative +" to the plugin root (the repository layout); $KLAMMERTEXT_HOME. + +let s:plugin_root = expand(':p:h:h') + +function! s:EditScript() abort + if exists('g:klammertext_edit_py') + return g:klammertext_edit_py + endif + for candidate in [ + \ s:plugin_root . '/klammertext_edit.py', + \ fnamemodify(s:plugin_root, ':h') . '/shared/klammertext_edit.py', + \ (empty($KLAMMERTEXT_HOME) ? '' : + \ $KLAMMERTEXT_HOME . '/doc/edit/shared/klammertext_edit.py')] + if !empty(candidate) && filereadable(candidate) + return candidate + endif + endfor + return '' +endfunction + +function! s:Python() abort + return exists('g:klammertext_python') ? g:klammertext_python : 'python3' +endfunction + +" Run the core CLI over the whole buffer. ARGS is the argument string +" (e.g. "indent 3-7"). Returns a dict {ok, lines, msg}: LINES is the +" transformed buffer (for the filter modes), MSG the stderr message. +function! s:RunCore(args) abort + let script = s:EditScript() + if empty(script) + return {'ok': 0, 'lines': [], 'msg': + \ 'Klammertext: cannot locate klammertext_edit.py' + \ . ' (set g:klammertext_edit_py)'} + endif + let errfile = tempname() + let cmd = s:Python() . ' ' . shellescape(script) . ' ' . a:args + \ . ' 2>' . shellescape(errfile) + let out = systemlist(cmd, getline(1, '$')) + let msg = filereadable(errfile) ? join(readfile(errfile), ' ') : '' + call delete(errfile) + if v:shell_error + return {'ok': 0, 'lines': [], 'msg': + \ empty(msg) ? 'Klammertext: the shared core failed' : msg} + endif + return {'ok': 1, 'lines': out, 'msg': msg} +endfunction + +" Replace the buffer with LINES, touching only the lines that changed (so +" undo stays small and the cursor does not move). The transformations +" never add or remove lines; refuse if the count disagrees. +function! s:ApplyLines(lines) abort + if len(a:lines) != line('$') + echohl ErrorMsg + echo 'Klammertext: unexpected line count from the shared core' + echohl None + return 0 + endif + let changed = 0 + for i in range(1, line('$')) + if getline(i) !=# a:lines[i - 1] + call setline(i, a:lines[i - 1]) + let changed += 1 + endif + endfor + return changed +endfunction + +function! s:CursorArgs() abort + let col = exists('*charcol') ? charcol('.') : col('.') + return line('.') . ' ' . col +endfunction + +" --- the commands ---------------------------------------------------------- + +function! klammertext#Reindent(first, last) abort + let r = s:RunCore('indent ' . a:first . '-' . a:last) + if !r.ok + echohl ErrorMsg | echo r.msg | echohl None + return + endif + call s:ApplyLines(r.lines) +endfunction + +function! klammertext#AlignTable() abort + let r = s:RunCore('align ' . s:CursorArgs()) + if !r.ok + echohl ErrorMsg | echo r.msg | echohl None + return + endif + call s:ApplyLines(r.lines) + if !empty(r.msg) + echo 'Klammertext: ' . r.msg + endif +endfunction + +function! klammertext#JumpToMatch() abort + let script = s:EditScript() + if empty(script) + echohl ErrorMsg + echo 'Klammertext: cannot locate klammertext_edit.py' + echohl None + return + endif + let out = systemlist(s:Python() . ' ' . shellescape(script) + \ . ' match ' . s:CursorArgs(), getline(1, '$')) + if v:shell_error || empty(out) + echohl ErrorMsg | echo 'Klammertext: the shared core failed' | echohl None + return + endif + let parts = split(out[0], ' ') + if parts[0] ==# 'none' + echo 'Klammertext: ' . join(parts[1:], ' ') + return + endif + " "match L C" or "mismatch L C message..." + let lnum = str2nr(parts[1]) + let ccol = str2nr(parts[2]) + if exists('*setcursorcharpos') + call setcursorcharpos(lnum, ccol) + else + call cursor(lnum, ccol) + endif + if parts[0] ==# 'mismatch' + echohl WarningMsg + echo 'Klammertext: ' . join(parts[3:], ' ') + echohl None + endif +endfunction + +function! klammertext#Check() abort + let script = s:EditScript() + if empty(script) + echohl ErrorMsg + echo 'Klammertext: cannot locate klammertext_edit.py' + echohl None + return + endif + let out = systemlist(s:Python() . ' ' . shellescape(script) . ' check', + \ getline(1, '$')) + if v:shell_error + echohl ErrorMsg | echo 'Klammertext: the shared core failed' | echohl None + return + endif + let items = [] + for line in out + let m = matchlist(line, '^\(\d\+\):\(\d\+\): \(.*\)$') + if !empty(m) + call add(items, {'bufnr': bufnr('%'), 'lnum': str2nr(m[1]), + \ 'col': str2nr(m[2]), 'text': m[3], 'type': 'E'}) + endif + endfor + call setloclist(0, items, ' ') + call setloclist(0, [], 'a', {'title': 'Klammertext delimiter check'}) + if empty(items) + lclose + echo 'Klammertext: no delimiter problems found' + else + lopen + endif +endfunction + +" --- +python3: in-process indentexpr and live match highlighting ---------- +" These load the shared core into Vim's embedded Python once, so the = +" operator and the per-cursor-move matcher run without spawning processes. + +let s:py_ready = 0 + +function! s:PySetup() abort + if s:py_ready + return 1 + endif + let script = s:EditScript() + if empty(script) || !has('python3') + return 0 + endif + let g:klammertext_py_dir = fnamemodify(script, ':h') + py3 << EOF +import sys +import vim +_kt_dir = vim.eval('g:klammertext_py_dir') +if _kt_dir not in sys.path: + sys.path.insert(0, _kt_dir) +import klammertext_edit as _kt + +def _kt_buffer_and_offset(): + buf = vim.current.buffer + lines = buf[:] + s = '\n'.join(lines) + row, bytecol = vim.current.window.cursor # bytecol is 0-based bytes + line = lines[row - 1] if row <= len(lines) else '' + charcol = len(line.encode('utf-8')[:bytecol].decode('utf-8', 'replace')) + offset = sum(len(l) + 1 for l in lines[:row - 1]) + charcol + return s, lines, offset + +def _kt_indent(lnum): + buf = vim.current.buffer + lines = buf[:] + s = '\n'.join(lines) + bol = sum(len(l) + 1 for l in lines[:lnum - 1]) + col = _kt.target_column(s, bol) + return -1 if col is None else col + +def _kt_pos(lines, offset): + """(1-based line, 0-based char col, the line's text) for char OFFSET.""" + line = 0 + while line < len(lines) and offset > len(lines[line]): + offset -= len(lines[line]) + 1 + line += 1 + text = lines[line] if line < len(lines) else '' + return (line + 1, offset, text) + +def _kt_match(): + """[] when not on a delimiter; else [mismatch, message, + l1, c1, len1, (l2, c2, len2)?] with byte columns for matchaddpos().""" + s, lines, offset = _kt_buffer_and_offset() + m = _kt.match_at(s, offset) + if m is None: + return [] + out = [1 if m['mismatch'] else 0, m['message'] or ''] + for tok in [m['token'], m['match_token']]: + if tok is None: + continue + line, ccol, text = _kt_pos(lines, tok[0]) + bcol = len(text[:ccol].encode('utf-8')) + 1 + blen = len(text[ccol:ccol + (tok[1] - tok[0])].encode('utf-8')) + out.extend([line, bcol, blen]) + return out +EOF + let s:py_ready = 1 + return 1 +endfunction + +function! klammertext#IndentExpr() abort + if !s:PySetup() + return -1 + endif + return py3eval('_kt_indent(' . v:lnum . ')') +endfunction + +function! s:ClearMatchHighlight() abort + if exists('w:klammertext_match_ids') + for id in w:klammertext_match_ids + silent! call matchdelete(id) + endfor + endif + let w:klammertext_match_ids = [] +endfunction + +function! s:UpdateMatchHighlight() abort + call s:ClearMatchHighlight() + let r = py3eval('_kt_match()') + if empty(r) + return + endif + let group = r[0] ? 'KlammertextMismatch' : 'KlammertextMatch' + let pos = [] + let i = 2 + while i + 3 <= len(r) + call add(pos, [r[i], r[i + 1], r[i + 2]]) + let i += 3 + endwhile + if !empty(pos) + call add(w:klammertext_match_ids, matchaddpos(group, pos)) + endif + if r[0] && !empty(r[1]) + echo 'Klammertext: ' . r[1] + endif +endfunction + +function! klammertext#SetupMatchHighlight() abort + if !s:PySetup() + return + endif + hi def link KlammertextMatch MatchParen + hi def KlammertextMismatch guifg=#ff5555 gui=bold ctermfg=203 cterm=bold + augroup klammertextMatch + autocmd! * + autocmd CursorMoved,CursorMovedI call s:UpdateMatchHighlight() + autocmd BufLeave,WinLeave call s:ClearMatchHighlight() + augroup END +endfunction diff --git a/doc/edit/vim/ftdetect/klammertext.vim b/doc/edit/vim/ftdetect/klammertext.vim new file mode 100644 index 0000000..561ba87 --- /dev/null +++ b/doc/edit/vim/ftdetect/klammertext.vim @@ -0,0 +1,10 @@ +" Klammertext filetype detection (.kt source files, .k klammer definitions). +" +" NOTE: .kt is also Kotlin's extension, and recent Vim/Neovim runtimes map +" *.kt to the kotlin filetype. This file overrides that unconditionally +" (`set filetype=`, not `setfiletype`, so it wins over the runtime's +" earlier detection). If you edit both Kotlin and Klammertext, replace the +" *.kt line with a content heuristic of your choice, or drop it and set the +" filetype per file with a modeline (# vim: ft=klammertext) or :set. +au BufRead,BufNewFile *.kt set filetype=klammertext +au BufRead,BufNewFile *.k set filetype=klammertext diff --git a/doc/edit/vim/ftplugin/klammertext.vim b/doc/edit/vim/ftplugin/klammertext.vim new file mode 100644 index 0000000..514a05b --- /dev/null +++ b/doc/edit/vim/ftplugin/klammertext.vim @@ -0,0 +1,63 @@ +" Klammertext filetype plugin: comment format, the structural commands, and +" (when Vim has +python3) fast in-process indentation and live delimiter +" match highlighting. The implementations are in autoload/klammertext.vim; +" the algorithms themselves are the shared Python core +" (klammertext_edit.py — see doc/edit/README.md for how it is located). +" +" Commands (buffer-local): +" :[range]KlammertextReindent reindent the range (default: whole buffer) +" :KlammertextAlign align the @table enclosing the cursor +" :KlammertextJumpToMatch jump between a klammer application's +" opening and closing delimiter +" :KlammertextCheck unbalanced/mismatched delimiters -> the +" location list +" +" Default mappings (set g:klammertext_no_mappings to define none): +" i reindent the current line (visual: the selection) +" a align the enclosing table +" j jump to the matching delimiter +" +" Reindentation is EXPLICIT-ONLY: whitespace is content in Klammertext, so +" indentkeys is emptied and nothing reformats as a side effect of typing. +" With +python3 the ftplugin also sets 'indentexpr', so the = operator +" (e.g. ==, gg=G) reindents through the same shared implementation. + +if exists("b:did_ftplugin") + finish +endif +let b:did_ftplugin = 1 + +setlocal commentstring=#\ %s +setlocal comments=b:# +setlocal indentkeys= + +command! -buffer -range=% KlammertextReindent + \ call klammertext#Reindent(, ) +command! -buffer KlammertextAlign call klammertext#AlignTable() +command! -buffer KlammertextJumpToMatch call klammertext#JumpToMatch() +command! -buffer KlammertextCheck call klammertext#Check() + +if !exists("g:klammertext_no_mappings") + nnoremap i :.KlammertextReindent + xnoremap i :KlammertextReindent + nnoremap a :KlammertextAlign + nnoremap j :KlammertextJumpToMatch +endif + +let b:undo_ftplugin = "setlocal commentstring< comments< indentkeys<" + \ . " | delcommand KlammertextReindent" + \ . " | delcommand KlammertextAlign" + \ . " | delcommand KlammertextJumpToMatch" + \ . " | delcommand KlammertextCheck" + +" With +python3 the shared core runs in-process: 'indentexpr' makes the = +" operator work, and the matching delimiter is highlighted live as the +" cursor sits on one (the show-paren equivalent; a mismatch shows in red +" with a message). Without +python3 the commands above still work — they +" shell out to python3 — and Neovim users can get live matching from the +" language server instead (see doc/edit/README.md). +if has('python3') + setlocal indentexpr=klammertext#IndentExpr() + let b:undo_ftplugin .= " | setlocal indentexpr<" + call klammertext#SetupMatchHighlight() +endif diff --git a/doc/edit/vim/syntax/klammertext.vim b/doc/edit/vim/syntax/klammertext.vim new file mode 100644 index 0000000..5ea6126 --- /dev/null +++ b/doc/edit/vim/syntax/klammertext.vim @@ -0,0 +1,113 @@ +" Vim syntax highlighting for Klammertext (.kt and .k files). +" The Vim counterpart of doc/edit/emacs/klammertext-mode.el's highlighting +" and doc/edit/sublime/Klammertext.sublime-syntax. +" +" What it highlights (the same token classes as the other editors): +" +" Text removal (#): +" # ... remove to end of line (marker + removed text) +" ## ... remove to end of file (marker + removed text) +" #[ ... ]# remove enclosed text, nestable (markers + removed) +" #- #+ #/ whitespace operators: NOT removals, left unhighlighted +" +" Klammer applications (@), definitions (@@), system commands (@@@): +" @name @@name @@@name opening (@ and name are one unit) +" name@ name@@ name@@@ named closing +" @ @@ @@@ bare closing +" +" Escapes: ^@ ^# ^| ^^ — the caret makes the next character literal; the +" two characters are consumed as one (unhighlighted) unit, so the escaped +" character is never read as a delimiter. A run of carets pairs +" left-to-right, reproducing the language's parity rule. +" +" Literal klammers: @code ... code@ — the interior is verbatim (no # or @ +" interpreted). SYNC: the literal-klammer set's source of truth is +" LITERAL_KLAMMERS in doc/edit/shared/klammertext_edit.py; a static +" syntax file cannot read it, so when you add a literal klammer 'foo', +" copy the klammertextVerbatim region below with code -> foo (and mirror +" it in the Emacs, Sublime, and VS Code artifacts; all are seeded with +" just 'code'). +" +" How open vs. close is decided (the same rule as every other integration): +" a delimiter whose NAME follows the @-run (@name) is an OPENING; a bare +" @-run, or one whose NAME precedes it (name@), is a CLOSING. The +" look-ahead \%(\w\|@\)\@! on every closing keeps 'foo@bar' correct: that @ +" is followed by a name, so it opens @bar and 'foo' stays plain text. The +" abbreviated @name-arg form colors only @name (the name ends at the first +" hyphen), exactly like the other editors. +" +" Colors come from the shared Klammertext palette +" (notes/klammertext_palette.md in the development tree): application blue, +" definition green, system orange; each opening bright and its close the +" same hue darker; full intensity on dark backgrounds, deepened (0.60x) on +" light. All groups are `hi def`, so :highlight in your vimrc overrides. +" Delimiter matching (jump + live highlight) is not a tokenizer concern — +" it lives in the ftplugin/autoload files. + +if exists("b:current_syntax") + finish +endif + +" --- escapes: ^X makes X literal; consumed so # / @ are not delimiters ---- +" (Defined first; it wins by the earlier-start rule, since the ^ precedes.) +syn match klammertextEscape /\^./ + +" --- text removal (#) ----------------------------------------------------- +" Order matters: at the same start position, the LAST defined item wins. +syn match klammertextRemovedLine /#.*$/ contains=klammertextMarkerLine +syn match klammertextMarkerLine /#/ contained +" whitespace operators #- #+N #/N : not removals, left unhighlighted +syn match klammertextWhitespaceOp "#[-+/]\d*" +syn region klammertextRemovedBlock matchgroup=klammertextMarker start=/#\[/ end=/\]#/ contains=klammertextRemovedBlock +syn region klammertextRemovedFile matchgroup=klammertextMarker start=/##/ end=/\%$/ + +" --- system / target commands @@@ ---------------------------------------- +syn match klammertextSysOpen /@\@1 the Problems panel +// * Format Document / Format Selection -> textDocument/(range)formatting +// (structural reindentation; explicit-only — no format-on-type) +// * occurrences highlighting -> textDocument/documentHighlight (the +// matching delimiter lights up as the cursor sits on one) +// * Go to Definition on a delimiter -> its matching delimiter +// * klammertext.jumpToMatch (Ctrl+Alt+J) -> move the cursor to the match +// * klammertext.alignTable (Ctrl+Alt+A) -> workspace/executeCommand; the +// server answers with workspace/applyEdit +// +// The server is located via the klammertext.serverPath setting, a vendored +// copy next to this file (the editing-zip layout), ../shared/ relative to +// it (the Klammertext repository layout), or $KLAMMERTEXT_HOME. + +'use strict'; + +const vscode = require('vscode'); +const cp = require('child_process'); +const fs = require('fs'); +const path = require('path'); + +// --- a minimal LSP client over a child process ----------------------------- + +class LspClient { + constructor(command, args, log) { + this.log = log; + this.nextId = 1; + this.pending = new Map(); // id -> {resolve, reject} + this.handlers = new Map(); // method -> fn(params, id) + this.buffer = Buffer.alloc(0); + this.dead = false; + this.proc = cp.spawn(command, args, { stdio: ['pipe', 'pipe', 'pipe'] }); + this.proc.stdout.on('data', (chunk) => this._onData(chunk)); + this.proc.stderr.on('data', (chunk) => log(chunk.toString())); + this.proc.on('error', (err) => { this.dead = true; log('spawn error: ' + err.message); }); + this.proc.on('exit', (code) => { this.dead = true; log('server exited: ' + code); }); + } + + _onData(chunk) { + this.buffer = Buffer.concat([this.buffer, chunk]); + for (;;) { + const sep = this.buffer.indexOf('\r\n\r\n'); + if (sep === -1) return; + const header = this.buffer.slice(0, sep).toString(); + const m = /content-length:\s*(\d+)/i.exec(header); + if (!m) { this.buffer = this.buffer.slice(sep + 4); continue; } + const length = parseInt(m[1], 10); + if (this.buffer.length < sep + 4 + length) return; + const body = this.buffer.slice(sep + 4, sep + 4 + length).toString(); + this.buffer = this.buffer.slice(sep + 4 + length); + let msg; + try { msg = JSON.parse(body); } catch (e) { continue; } + this._dispatch(msg); + } + } + + _dispatch(msg) { + if (msg.method !== undefined) { + const handler = this.handlers.get(msg.method); + if (handler) { + Promise.resolve(handler(msg.params, msg.id)).then((result) => { + if (msg.id !== undefined && msg.id !== null) { + this._send({ jsonrpc: '2.0', id: msg.id, result: result === undefined ? null : result }); + } + }); + } else if (msg.id !== undefined && msg.id !== null) { + this._send({ jsonrpc: '2.0', id: msg.id, error: { code: -32601, message: 'method not found' } }); + } + } else if (this.pending.has(msg.id)) { + const p = this.pending.get(msg.id); + this.pending.delete(msg.id); + if (msg.error) p.reject(new Error(msg.error.message)); + else p.resolve(msg.result); + } + } + + _send(msg) { + if (this.dead) return; + const body = Buffer.from(JSON.stringify(msg), 'utf8'); + this.proc.stdin.write('Content-Length: ' + body.length + '\r\n\r\n'); + this.proc.stdin.write(body); + } + + request(method, params) { + if (this.dead) return Promise.reject(new Error('server not running')); + const id = this.nextId++; + return new Promise((resolve, reject) => { + this.pending.set(id, { resolve, reject }); + this._send({ jsonrpc: '2.0', id, method, params }); + }); + } + + notify(method, params) { + this._send({ jsonrpc: '2.0', method, params }); + } + + onRequest(method, handler) { this.handlers.set(method, handler); } + + stop() { + if (this.dead) return; + this.request('shutdown', null).then( + () => { this.notify('exit', null); }, + () => { try { this.proc.kill(); } catch (e) { /* gone */ } }); + } +} + +// --- LSP <-> VS Code conversions ------------------------------------------- + +function toVsRange(r) { + return new vscode.Range(r.start.line, r.start.character, r.end.line, r.end.character); +} + +function toVsEdits(edits) { + return (edits || []).map((e) => new vscode.TextEdit(toVsRange(e.range), e.newText)); +} + +function fromVsPosition(p) { + return { line: p.line, character: p.character }; +} + +function docParams(document) { + return { textDocument: { uri: document.uri.toString() } }; +} + +// --- locating the server --------------------------------------------------- + +function findServer(context) { + const configured = vscode.workspace.getConfiguration('klammertext').get('serverPath'); + const candidates = []; + if (configured) candidates.push(configured); + candidates.push(path.join(context.extensionPath, 'klammertext_ls.py')); + candidates.push(path.join(context.extensionPath, '..', 'shared', 'klammertext_ls.py')); + if (process.env.KLAMMERTEXT_HOME) { + candidates.push(path.join(process.env.KLAMMERTEXT_HOME, 'doc', 'edit', 'shared', 'klammertext_ls.py')); + } + return candidates.find((c) => { try { return fs.statSync(c).isFile(); } catch (e) { return false; } }); +} + +// --- activation ------------------------------------------------------------ + +let client = null; + +function activate(context) { + const output = vscode.window.createOutputChannel('Klammertext'); + const log = (s) => output.append(s.endsWith('\n') ? s : s + '\n'); + + const serverPath = findServer(context); + if (!serverPath) { + vscode.window.showWarningMessage( + 'Klammertext: cannot locate klammertext_ls.py — set the ' + + 'klammertext.serverPath setting. Highlighting works; ' + + 'diagnostics, formatting, matching and alignment need the server.'); + return; + } + const python = vscode.workspace.getConfiguration('klammertext').get('pythonPath') || 'python3'; + client = new LspClient(python, [serverPath], log); + + const diagnostics = vscode.languages.createDiagnosticCollection('klammertext'); + context.subscriptions.push(diagnostics, output); + + client.onRequest('textDocument/publishDiagnostics', (params) => { + diagnostics.set(vscode.Uri.parse(params.uri), (params.diagnostics || []).map((d) => { + const diag = new vscode.Diagnostic( + toVsRange(d.range), d.message, + d.severity === 1 ? vscode.DiagnosticSeverity.Error + : vscode.DiagnosticSeverity.Warning); + diag.source = d.source; + return diag; + })); + }); + + client.onRequest('workspace/applyEdit', (params) => { + const we = new vscode.WorkspaceEdit(); + const changes = (params.edit && params.edit.changes) || {}; + for (const uri of Object.keys(changes)) { + for (const e of changes[uri]) { + we.replace(vscode.Uri.parse(uri), toVsRange(e.range), e.newText); + } + } + return vscode.workspace.applyEdit(we).then((applied) => ({ applied })); + }); + + client.onRequest('window/showMessage', (params) => { + vscode.window.setStatusBarMessage(params.message, 5000); + }); + + // -- document sync (full text) -- + const isKt = (doc) => doc.languageId === 'klammertext'; + const open = (doc) => { + if (!isKt(doc)) return; + client.notify('textDocument/didOpen', { + textDocument: { uri: doc.uri.toString(), languageId: 'klammertext', + version: doc.version, text: doc.getText() }, + }); + }; + + client.request('initialize', { + processId: process.pid, + rootUri: null, + capabilities: {}, + }).then(() => { + client.notify('initialized', {}); + vscode.workspace.textDocuments.forEach(open); + }, (err) => log('initialize failed: ' + err.message)); + + context.subscriptions.push( + vscode.workspace.onDidOpenTextDocument(open), + vscode.workspace.onDidChangeTextDocument((event) => { + if (!isKt(event.document)) return; + client.notify('textDocument/didChange', { + textDocument: { uri: event.document.uri.toString(), + version: event.document.version }, + contentChanges: [{ text: event.document.getText() }], + }); + }), + vscode.workspace.onDidCloseTextDocument((doc) => { + if (!isKt(doc)) return; + client.notify('textDocument/didClose', docParams(doc)); + })); + + // -- providers -- + context.subscriptions.push( + vscode.languages.registerDocumentFormattingEditProvider('klammertext', { + provideDocumentFormattingEdits(document) { + return client.request('textDocument/formatting', + Object.assign(docParams(document), { options: {} })) + .then(toVsEdits); + }, + }), + vscode.languages.registerDocumentRangeFormattingEditProvider('klammertext', { + provideDocumentRangeFormattingEdits(document, range) { + return client.request('textDocument/rangeFormatting', + Object.assign(docParams(document), { + range: { start: fromVsPosition(range.start), + end: fromVsPosition(range.end) }, + options: {}, + })).then(toVsEdits); + }, + }), + vscode.languages.registerDocumentHighlightProvider('klammertext', { + provideDocumentHighlights(document, position) { + return client.request('textDocument/documentHighlight', + Object.assign(docParams(document), + { position: fromVsPosition(position) })) + .then((result) => (result || []).map((h) => + new vscode.DocumentHighlight(toVsRange(h.range)))); + }, + }), + vscode.languages.registerDefinitionProvider('klammertext', { + provideDefinition(document, position) { + return client.request('textDocument/definition', + Object.assign(docParams(document), + { position: fromVsPosition(position) })) + .then((result) => result + ? new vscode.Location(vscode.Uri.parse(result.uri), + toVsRange(result.range)) + : null); + }, + })); + + // -- commands -- + context.subscriptions.push( + vscode.commands.registerCommand('klammertext.jumpToMatch', () => { + const editor = vscode.window.activeTextEditor; + if (!editor || !isKt(editor.document)) return; + return client.request('textDocument/definition', + Object.assign(docParams(editor.document), + { position: fromVsPosition(editor.selection.active) })) + .then((result) => { + if (!result) { + vscode.window.setStatusBarMessage( + 'Klammertext: no matching delimiter here', 5000); + return; + } + const pos = toVsRange(result.range).start; + editor.selection = new vscode.Selection(pos, pos); + editor.revealRange(new vscode.Range(pos, pos)); + }); + }), + vscode.commands.registerCommand('klammertext.alignTable', () => { + const editor = vscode.window.activeTextEditor; + if (!editor || !isKt(editor.document)) return; + return client.request('workspace/executeCommand', { + command: 'klammertext.alignTable', + arguments: [{ uri: editor.document.uri.toString(), + position: fromVsPosition(editor.selection.active) }], + }); + })); +} + +function deactivate() { + if (client) client.stop(); + client = null; +} + +module.exports = { activate, deactivate }; diff --git a/doc/edit/vscode/language-configuration.json b/doc/edit/vscode/language-configuration.json new file mode 100644 index 0000000..e8a3bbb --- /dev/null +++ b/doc/edit/vscode/language-configuration.json @@ -0,0 +1,11 @@ +{ + "comments": { + "lineComment": "#", + "blockComment": ["#[", "]#"] + }, + "brackets": [ + ["#[", "]#"] + ], + "autoClosingPairs": [], + "surroundingPairs": [] +} diff --git a/doc/edit/vscode/package.json b/doc/edit/vscode/package.json new file mode 100644 index 0000000..b9f4a2e --- /dev/null +++ b/doc/edit/vscode/package.json @@ -0,0 +1,85 @@ +{ + "name": "klammertext", + "displayName": "Klammertext", + "description": "Klammertext language support: syntax highlighting, delimiter matching, structural reindentation, table alignment, and delimiter diagnostics.", + "version": "0.1.0", + "publisher": "klammertext", + "license": "SEE LICENSE IN THE KLAMMERTEXT DISTRIBUTION", + "engines": { + "vscode": "^1.75.0" + }, + "categories": [ + "Programming Languages" + ], + "main": "./extension.js", + "activationEvents": [ + "onLanguage:klammertext" + ], + "capabilities": { + "untrustedWorkspaces": { + "supported": false, + "description": "The extension runs the Klammertext language server (a local Python process)." + } + }, + "contributes": { + "languages": [ + { + "id": "klammertext", + "aliases": [ + "Klammertext" + ], + "extensions": [ + ".kt", + ".k" + ], + "configuration": "./language-configuration.json" + } + ], + "grammars": [ + { + "language": "klammertext", + "scopeName": "text.klammertext", + "path": "./syntaxes/klammertext.tmLanguage.json" + } + ], + "commands": [ + { + "command": "klammertext.jumpToMatch", + "title": "Klammertext: Jump to Matching Delimiter" + }, + { + "command": "klammertext.alignTable", + "title": "Klammertext: Align Table" + } + ], + "keybindings": [ + { + "command": "klammertext.jumpToMatch", + "key": "ctrl+alt+j", + "mac": "cmd+alt+j", + "when": "editorTextFocus && editorLangId == klammertext" + }, + { + "command": "klammertext.alignTable", + "key": "ctrl+alt+a", + "mac": "cmd+alt+a", + "when": "editorTextFocus && editorLangId == klammertext" + } + ], + "configuration": { + "title": "Klammertext", + "properties": { + "klammertext.pythonPath": { + "type": "string", + "default": "python3", + "description": "Python interpreter used to run the Klammertext language server." + }, + "klammertext.serverPath": { + "type": "string", + "default": "", + "description": "Full path to klammertext_ls.py. Leave blank to auto-locate: a copy next to the extension, ../shared/ relative to it (the Klammertext repository layout), or $KLAMMERTEXT_HOME/doc/edit/shared/." + } + } + } + } +} diff --git a/doc/edit/vscode/syntaxes/klammertext.tmLanguage.json b/doc/edit/vscode/syntaxes/klammertext.tmLanguage.json new file mode 100644 index 0000000..c31f1df --- /dev/null +++ b/doc/edit/vscode/syntaxes/klammertext.tmLanguage.json @@ -0,0 +1,132 @@ +{ + "//": [ + "TextMate grammar for Klammertext — the VS Code port of", + "doc/edit/sublime/Klammertext.sublime-syntax (same token classes,", + "same scope names; see that file's header for the full rationale).", + "", + "What it highlights: text removal (#, ##, nestable #[ ... ]#,", + "whitespace operators left unscoped), the three @-tiers — application", + "(@), definition (@@), system (@@@) — each as an opening (@name, one", + "unit) or a close (name@, bare @), ^-escapes (consumed, unscoped),", + "and verbatim @code ... code@ interiors.", + "", + "How open vs. close is decided (the same rule as every integration):", + "a delimiter whose NAME follows the @-run is an OPENING; a bare", + "@-run, or one whose NAME precedes it, is a CLOSING. The", + "(?![A-Za-z0-9_@]) look-ahead on every closing keeps foo@bar correct.", + "", + "SYNC: the literal-klammer set's source of truth is LITERAL_KLAMMERS", + "in doc/edit/shared/klammertext_edit.py. A static grammar cannot", + "read it: to add a literal klammer 'foo', copy the @code begin/end", + "rule below with code -> foo (and mirror it in the Emacs, Sublime,", + "and Vim artifacts; all are seeded with just 'code').", + "", + "Delimiter matching, indentation, alignment, and diagnostics are not", + "tokenizer concerns — they come from the Klammertext language server", + "via extension.js.", + "", + "The ## rule's end pattern never matches, so the region runs to the", + "end of the file (## removes the rest of the file by definition)." + ], + "name": "Klammertext", + "scopeName": "text.klammertext", + "patterns": [ + { + "match": "\\^." + }, + { + "begin": "#\\[", + "beginCaptures": { + "0": { "name": "punctuation.definition.comment.klammertext" } + }, + "end": "\\]#", + "endCaptures": { + "0": { "name": "punctuation.definition.comment.klammertext" } + }, + "name": "comment.block.klammertext", + "patterns": [ + { "include": "#removal-block" } + ] + }, + { + "match": "#[-+/]\\d*" + }, + { + "begin": "##", + "beginCaptures": { + "0": { "name": "punctuation.definition.comment.klammertext" } + }, + "end": "$never^", + "name": "comment.block.klammertext" + }, + { + "begin": "#", + "beginCaptures": { + "0": { "name": "punctuation.definition.comment.klammertext" } + }, + "end": "$", + "name": "comment.line.klammertext" + }, + { + "begin": "@code(?![A-Za-z0-9_])", + "beginCaptures": { + "0": { "name": "entity.name.function.begin.klammertext" } + }, + "end": "code@", + "endCaptures": { + "0": { "name": "entity.name.function.end.klammertext" } + } + }, + { + "match": "@@@[A-Za-z0-9_]+", + "name": "keyword.control.begin.klammertext" + }, + { + "match": "@@@(?![A-Za-z0-9_@])", + "name": "keyword.control.end.klammertext" + }, + { + "match": "[A-Za-z0-9_]+@@@(?![A-Za-z0-9_@])", + "name": "keyword.control.end.klammertext" + }, + { + "match": "@@[A-Za-z0-9_]+", + "name": "storage.type.begin.klammertext" + }, + { + "match": "@@(?![A-Za-z0-9_@])", + "name": "storage.type.end.klammertext" + }, + { + "match": "[A-Za-z0-9_]+@@(?![A-Za-z0-9_@])", + "name": "storage.type.end.klammertext" + }, + { + "match": "@[A-Za-z0-9_]+", + "name": "entity.name.function.begin.klammertext" + }, + { + "match": "@(?![A-Za-z0-9_@])", + "name": "entity.name.function.end.klammertext" + }, + { + "match": "[A-Za-z0-9_]+@(?![A-Za-z0-9_@])", + "name": "entity.name.function.end.klammertext" + } + ], + "repository": { + "removal-block": { + "begin": "#\\[", + "beginCaptures": { + "0": { "name": "punctuation.definition.comment.klammertext" } + }, + "end": "\\]#", + "endCaptures": { + "0": { "name": "punctuation.definition.comment.klammertext" } + }, + "patterns": [ + { "include": "#removal-block" } + ] + } + } +} diff --git a/tst/editor/editor_driver.py b/tst/editor/editor_driver.py index 76b26e6..f97c69d 100644 --- a/tst/editor/editor_driver.py +++ b/tst/editor/editor_driver.py @@ -1,45 +1,50 @@ #!/usr/bin/env python3 -"""Drive the Sublime Text editor cores over fixture files, outside Sublime. +"""Drive the shared editor core over fixture files. -Usage: editor_driver.py SUBLIME_DIR (MODE INFILE OUTFILE)... +Usage: editor_driver.py SHARED_DIR (MODE INFILE OUTFILE)... -MODE is `indent` or `align`; each triple applies that tool to INFILE and -writes the result to OUTFILE. The Sublime plugin files import without the -`sublime` module (their try/except guard), so the pure cores run under plain -python3. Also asserts the built-in error path (no enclosing table). +MODE is `indent`, `align` (call the shared core's API), or `indent-cli`, +`align-cli` (run the same operation through the core's command-line +interface, as the Vim plugin does). Each triple applies that tool to INFILE +and writes the result to OUTFILE. Also asserts the built-in error paths +(no enclosing table; empty `check` output on balanced fixtures). Called by editor_test.sh; exits nonzero on an internal error. """ +import subprocess import sys -def apply_indent(KI, s): - bols = [0] + [i + 1 for i, ch in enumerate(s) - if ch == '\n' and i + 1 < len(s)] - out = s - for a, b, new in sorted(KI.reindent_lines(s, bols), reverse=True): - out = out[:a] + new + out[b:] - return out +def apply_indent(KE, s): + return KE.indent_text(s) -def apply_align(KA, s): +def apply_align(KE, s): caret = s.index('|') if '|' in s else 0 - span = KA.enclosing_span(s, caret, KA.ALIGN_KLAMMERS) - if span is None: - return s - _name, cs, ce = span - edits, _msg = KA.compute_edits(s[cs:ce]) - out = s - for a, b, new in sorted(edits, reverse=True): - out = out[:cs + a] + new + out[cs + b:] - return out + return KE.align_text(s, caret)[0] + + +def line_col_of_first_bar(s): + pos = s.index('|') if '|' in s else 0 + line = s.count('\n', 0, pos) + 1 + col = pos - (s.rfind('\n', 0, pos) + 1) + 1 + return line, col + + +def run_cli(script, args, s): + r = subprocess.run([sys.executable, script] + args, + input=s.encode('utf-8'), stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + if r.returncode != 0: + sys.exit("editor_driver.py: CLI failed: %s" % r.stderr.decode()) + return r.stdout.decode('utf-8') def main(): - sublime_dir = sys.argv[1] - sys.path.insert(0, sublime_dir) - import Klammertext_indent as KI - import Klammertext_align as KA + shared_dir = sys.argv[1] + sys.path.insert(0, shared_dir) + import klammertext_edit as KE + script = shared_dir + '/klammertext_edit.py' args = sys.argv[2:] for k in range(0, len(args), 3): @@ -47,16 +52,27 @@ def main(): with open(infile) as f: s = f.read() if mode == 'indent': - out = apply_indent(KI, s) + out = apply_indent(KE, s) elif mode == 'align': - out = apply_align(KA, s) + out = apply_align(KE, s) + elif mode == 'indent-cli': + out = run_cli(script, ['indent'], s) + elif mode == 'align-cli': + line, col = line_col_of_first_bar(s) + out = run_cli(script, ['align', str(line), str(col)], s) else: sys.exit("editor_driver.py: unknown mode: " + mode) with open(outfile, 'w') as f: f.write(out) - # Error path: no enclosing table klammer. - assert KA.enclosing_span("no table here\n", 3, KA.ALIGN_KLAMMERS) is None + # Error and diagnostics paths. + assert KE.enclosing_span("no table here\n", 3, KE.ALIGN_KLAMMERS) is None + assert KE.diagnostics("@i abc @ #[ x ]# ^@\n") == [] + probs = KE.diagnostics("@i abc\n@ol x ul@\n") + assert any('never closed' in p['message'] for p in probs), probs + assert any('ul@' in p['message'] for p in probs), probs + assert run_cli(script, ['check'], "@i abc @\n") == "" + assert '1:1:' in run_cli(script, ['check'], "@i abc\n") if __name__ == '__main__': diff --git a/tst/editor/ls_test.py b/tst/editor/ls_test.py new file mode 100644 index 0000000..e782425 --- /dev/null +++ b/tst/editor/ls_test.py @@ -0,0 +1,262 @@ +#!/usr/bin/env python3 +"""Protocol test for the Klammertext language server (klammertext_ls.py). + +Spawns the server as an LSP client would (JSON-RPC over stdio) and exercises +every capability against the editor fixtures: + + * initialize / initialized handshake and the advertised capabilities + * didOpen + publishDiagnostics (clean fixture -> no diagnostics; + unbalanced text -> the expected errors) + * textDocument/formatting on every indent fixture == its expected file + * textDocument/rangeFormatting (a sub-range only) + * textDocument/documentHighlight + definition (the delimiter matcher) + * workspace/executeCommand klammertext.alignTable -> workspace/applyEdit, + applied result == every align fixture's expected file + * shutdown / exit (exit code 0) + +Usage: ls_test.py SHARED_DIR FIXTURE_DIR +Prints one PASS/FAIL line per check; exits nonzero on any failure. +""" + +import json +import subprocess +import sys + + +class Client: + """A minimal scripted LSP client over a server subprocess.""" + + def __init__(self, server_path): + self.proc = subprocess.Popen( + [sys.executable, server_path], + stdin=subprocess.PIPE, stdout=subprocess.PIPE) + self.next_id = 1 + self.notifications = [] # collected server notifications + self.server_requests = [] # collected server->client requests + + def send(self, message): + body = json.dumps(message).encode('utf-8') + self.proc.stdin.write(b'Content-Length: %d\r\n\r\n' % len(body) + body) + self.proc.stdin.flush() + + def read_message(self): + length = None + while True: + line = self.proc.stdout.readline() + if not line: + return None + line = line.strip() + if not line: + break + if line.lower().startswith(b'content-length:'): + length = int(line.split(b':', 1)[1]) + return json.loads(self.proc.stdout.read(length).decode('utf-8')) + + def request(self, method, params): + """Send a request and pump messages until its response arrives.""" + msg_id = self.next_id + self.next_id += 1 + self.send({'jsonrpc': '2.0', 'id': msg_id, + 'method': method, 'params': params}) + while True: + msg = self.read_message() + assert msg is not None, 'server closed during ' + method + if msg.get('id') == msg_id and 'method' not in msg: + assert 'error' not in msg, 'error response: %r' % msg + return msg.get('result') + self._collect(msg) + + def notify(self, method, params): + self.send({'jsonrpc': '2.0', 'method': method, 'params': params}) + + def _collect(self, msg): + if 'method' in msg and msg.get('id') is not None: + self.server_requests.append(msg) + # acknowledge server->client requests (e.g. applyEdit) + self.send({'jsonrpc': '2.0', 'id': msg['id'], + 'result': {'applied': True}}) + elif 'method' in msg: + self.notifications.append(msg) + + def wait_notification(self, method): + """Pump messages until a notification with METHOD arrives.""" + while True: + for i, msg in enumerate(self.notifications): + if msg['method'] == method: + return self.notifications.pop(i)['params'] + msg = self.read_message() + assert msg is not None, 'server closed waiting for ' + method + self._collect(msg) + + def wait_server_request(self, method): + while True: + for i, msg in enumerate(self.server_requests): + if msg['method'] == method: + return self.server_requests.pop(i)['params'] + msg = self.read_message() + assert msg is not None, 'server closed waiting for ' + method + self._collect(msg) + + +PASS = 0 +FAIL = 0 + + +def check(name, ok, detail=''): + global PASS, FAIL + if ok: + PASS += 1 + print('PASS %s' % name) + else: + FAIL += 1 + print('FAIL %s %s' % (name, detail)) + + +def apply_edits(ls, text, edits): + """Apply LSP TextEdits to TEXT (offsets via the server's own helpers).""" + resolved = [(ls.pos_to_offset(text, e['range']['start']), + ls.pos_to_offset(text, e['range']['end']), + e['newText']) for e in edits] + for a, b, new in sorted(resolved, reverse=True): + text = text[:a] + new + text[b:] + return text + + +def main(): + shared_dir, fixture_dir = sys.argv[1], sys.argv[2] + sys.path.insert(0, shared_dir) + import klammertext_ls as ls + + client = Client(shared_dir + '/klammertext_ls.py') + uri = 'file:///ls_test.kt' + + # -- handshake -- + result = client.request('initialize', {'capabilities': {}}) + caps = result.get('capabilities', {}) + check('initialize capabilities', + caps.get('documentFormattingProvider') is True + and caps.get('documentHighlightProvider') is True + and 'klammertext.alignTable' + in caps.get('executeCommandProvider', {}).get('commands', [])) + client.notify('initialized', {}) + + # -- diagnostics: clean fixture, then unbalanced text -- + text = open(fixture_dir + '/indent_list_expected.kt').read() + client.notify('textDocument/didOpen', + {'textDocument': {'uri': uri, 'languageId': 'klammertext', + 'version': 1, 'text': text}}) + diags = client.wait_notification('textDocument/publishDiagnostics') + check('clean fixture: no diagnostics', diags['diagnostics'] == [], + repr(diags['diagnostics'])) + + bad = '@i abc\n@ol x ul@\n' + client.notify('textDocument/didChange', + {'textDocument': {'uri': uri, 'version': 2}, + 'contentChanges': [{'text': bad}]}) + diags = client.wait_notification('textDocument/publishDiagnostics') + msgs = [d['message'] for d in diags['diagnostics']] + check('unbalanced text: diagnostics', + any('@i' in m and 'never closed' in m for m in msgs) + and any('ul@' in m for m in msgs), repr(msgs)) + + # -- formatting == every indent fixture's expected file -- + indent_fixtures = ['indent_list', 'indent_document', 'indent_table', + 'indent_untouched', 'indent_defs', 'indent_escapes', + 'indent_named_close'] + for f in indent_fixtures: + src = open(fixture_dir + '/%s.kt' % f).read() + exp = open(fixture_dir + '/%s_expected.kt' % f).read() + client.notify('textDocument/didChange', + {'textDocument': {'uri': uri, 'version': 3}, + 'contentChanges': [{'text': src}]}) + client.wait_notification('textDocument/publishDiagnostics') + edits = client.request('textDocument/formatting', + {'textDocument': {'uri': uri}, + 'options': {'tabSize': 2, + 'insertSpaces': True}}) + check('formatting %s' % f, apply_edits(ls, src, edits or []) == exp) + + # -- rangeFormatting: only the requested lines change -- + src = '@ol\nzero\n one\n@\n' + client.notify('textDocument/didChange', + {'textDocument': {'uri': uri, 'version': 4}, + 'contentChanges': [{'text': src}]}) + client.wait_notification('textDocument/publishDiagnostics') + edits = client.request('textDocument/rangeFormatting', + {'textDocument': {'uri': uri}, + 'range': {'start': {'line': 2, 'character': 0}, + 'end': {'line': 2, 'character': 0}}, + 'options': {}}) + check('rangeFormatting touches only its lines', + apply_edits(ls, src, edits or []) == '@ol\nzero\n one\n@\n') + + # -- matcher: documentHighlight + definition -- + src = '@i abc @\n' + client.notify('textDocument/didChange', + {'textDocument': {'uri': uri, 'version': 5}, + 'contentChanges': [{'text': src}]}) + client.wait_notification('textDocument/publishDiagnostics') + hl = client.request('textDocument/documentHighlight', + {'textDocument': {'uri': uri}, + 'position': {'line': 0, 'character': 0}}) + check('documentHighlight pair', hl is not None and len(hl) == 2, repr(hl)) + defn = client.request('textDocument/definition', + {'textDocument': {'uri': uri}, + 'position': {'line': 0, 'character': 0}}) + check('definition = the matching close', + defn is not None and defn['range']['start']['character'] == 7, + repr(defn)) + off = client.request('textDocument/documentHighlight', + {'textDocument': {'uri': uri}, + 'position': {'line': 0, 'character': 4}}) + check('documentHighlight off-delimiter is null', off is None, repr(off)) + + # -- alignTable via executeCommand -> applyEdit, on every align fixture -- + align_fixtures = ['align_mixed', 'align_empty_cells', 'align_boundary', + 'align_colspan', 'align_escapes'] + for f in align_fixtures: + src = open(fixture_dir + '/%s.kt' % f).read() + exp = open(fixture_dir + '/%s_expected.kt' % f).read() + client.notify('textDocument/didChange', + {'textDocument': {'uri': uri, 'version': 6}, + 'contentChanges': [{'text': src}]}) + client.wait_notification('textDocument/publishDiagnostics') + pos = ls.offset_to_pos(src, src.index('|')) + client.request('workspace/executeCommand', + {'command': 'klammertext.alignTable', + 'arguments': [{'uri': uri, 'position': pos}]}) + if src == exp: # already-aligned fixture: no applyEdit + continue + params = client.wait_server_request('workspace/applyEdit') + edits = params['edit']['changes'][uri] + check('alignTable %s' % f, apply_edits(ls, src, edits) == exp) + + # align_too_wide must produce NO applyEdit (limit refusal) — verify via + # a message arriving with no pending applyEdit request. + src = open(fixture_dir + '/align_too_wide.kt').read() + client.notify('textDocument/didChange', + {'textDocument': {'uri': uri, 'version': 7}, + 'contentChanges': [{'text': src}]}) + client.wait_notification('textDocument/publishDiagnostics') + client.notifications.clear() # drop showMessages from the loop above + pos = ls.offset_to_pos(src, src.index('|')) + client.request('workspace/executeCommand', + {'command': 'klammertext.alignTable', + 'arguments': [{'uri': uri, 'position': pos}]}) + msg = client.wait_notification('window/showMessage') + check('alignTable respects ROW_MAX (refusal message, no edit)', + 'not aligning' in msg['message'] and not client.server_requests, + repr(msg)) + + # -- shutdown -- + client.request('shutdown', None) + client.notify('exit', None) + code = client.proc.wait(timeout=10) + check('clean exit', code == 0, 'exit code %d' % code) + + print('ls_test: %d passed, %d failed' % (PASS, FAIL)) + sys.exit(1 if FAIL else 0) + + +if __name__ == '__main__': + main() diff --git a/tst/editor/vim_feature_test.vim b/tst/editor/vim_feature_test.vim new file mode 100644 index 0000000..52377e3 --- /dev/null +++ b/tst/editor/vim_feature_test.vim @@ -0,0 +1,179 @@ +" vim_feature_test.vim — comprehensive checks of the Klammertext Vim plugin, +" beyond the fixture runs in editor_test.sh (which cover the Reindent and +" Align commands' output bytes). Run headlessly: +" +" KT_FIX= KT_OUT= \ +" vim -N -n -u NONE -i NONE -es --not-a-term \ +" --cmd 'set rtp^=' -c 'source vim_feature_test.vim' +" +" What is checked: +" * ftdetect: .kt maps to the klammertext filetype, overriding the stock +" runtime's .kt -> kotlin mapping +" * syntax: token classes at known positions (three @-tiers open/close, +" removal line/block/file with markers, verbatim @code interior, +" ^-escapes, abbreviated @name-arg) +" * commands: KlammertextJumpToMatch (open->close, close->open, +" multibyte columns), KlammertextCheck (location list) +" * with +python3: 'indentexpr' drives the = operator (gg=G equals the +" expected fixture), and the CursorMoved match highlighter sets +" KlammertextMatch / KlammertextMismatch matches +" +" Writes PASS/FAIL lines to $KT_OUT; exits nonzero if anything failed. + +let s:results = [] +function! s:Check(name, ok, ...) abort + let detail = (a:ok || !a:0) ? '' : ' -- ' . string(a:1) + call add(s:results, (a:ok ? 'PASS ' : 'FAIL ') . a:name . detail) +endfunction + +function! s:Syn(l, c) abort + return synIDattr(synID(a:l, a:c, 1), 'name') +endfunction + +filetype plugin on +if has('syntax') + syntax enable +endif + +" --- ftdetect: .kt is klammertext (the Kotlin override) -------------------- +execute 'edit! ' . fnameescape($KT_FIX . '/indent_list.kt') +call s:Check('ftdetect .kt -> klammertext (overrides kotlin)', + \ &filetype ==# 'klammertext', &filetype) + +" --- syntax token classes --------------------------------------------------- +if has('syntax') +enew! +call setline(1, [ + \ '@i abc @ x', + \ '@@def : x @@', + \ '@@@target x @@@', + \ '# removed line', + \ '#[ rem ]# after', + \ '^@ escaped', + \ '@code x @ y code@', + \ 'name@ x', + \ '@name-arg x', + \ '## tail', + \ 'still removed']) +set filetype=klammertext +call s:Check('syntax loaded', b:current_syntax ==# 'klammertext') +call s:Check('app open @i', s:Syn(1, 1) ==# 'klammertextAppOpen', s:Syn(1, 1)) +call s:Check('app bare close', s:Syn(1, 8) ==# 'klammertextAppClose', s:Syn(1, 8)) +call s:Check('plain text', s:Syn(1, 10) ==# '', s:Syn(1, 10)) +call s:Check('def open @@def', s:Syn(2, 1) ==# 'klammertextDefOpen', s:Syn(2, 1)) +call s:Check('def bare close', s:Syn(2, 11) ==# 'klammertextDefClose', s:Syn(2, 11)) +call s:Check('sys open @@@target', s:Syn(3, 1) ==# 'klammertextSysOpen', s:Syn(3, 1)) +call s:Check('sys bare close', s:Syn(3, 13) ==# 'klammertextSysClose', s:Syn(3, 13)) +call s:Check('line marker #', s:Syn(4, 1) ==# 'klammertextMarkerLine', s:Syn(4, 1)) +call s:Check('line removed text', s:Syn(4, 5) ==# 'klammertextRemovedLine', s:Syn(4, 5)) +call s:Check('block marker #[', s:Syn(5, 1) ==# 'klammertextMarker', s:Syn(5, 1)) +call s:Check('block removed text', s:Syn(5, 5) ==# 'klammertextRemovedBlock', s:Syn(5, 5)) +call s:Check('block marker ]#', s:Syn(5, 8) ==# 'klammertextMarker', s:Syn(5, 8)) +call s:Check('text after block', s:Syn(5, 12) ==# '', s:Syn(5, 12)) +call s:Check('escape ^@ consumed', s:Syn(6, 1) ==# 'klammertextEscape' + \ && s:Syn(6, 2) ==# 'klammertextEscape', s:Syn(6, 2)) +call s:Check('verbatim open', s:Syn(7, 1) ==# 'klammertextAppOpen', s:Syn(7, 1)) +call s:Check('verbatim interior @', s:Syn(7, 9) ==# 'klammertextVerbatim', s:Syn(7, 9)) +call s:Check('verbatim close', s:Syn(7, 17) ==# 'klammertextAppClose', s:Syn(7, 17)) +call s:Check('named close name@', s:Syn(8, 1) ==# 'klammertextAppClose' + \ && s:Syn(8, 5) ==# 'klammertextAppClose', s:Syn(8, 1)) +call s:Check('abbreviated @name-', s:Syn(9, 1) ==# 'klammertextAppOpen' + \ && s:Syn(9, 6) ==# '' && s:Syn(9, 7) ==# '', s:Syn(9, 6)) +call s:Check('file marker ##', s:Syn(10, 1) ==# 'klammertextMarker', s:Syn(10, 1)) +call s:Check('## removes to EOF', s:Syn(11, 3) ==# 'klammertextRemovedFile', s:Syn(11, 3)) +else + call add(s:results, 'SKIP syntax checks (this Vim lacks +syntax)') +endif + +" --- jump-to-match (the CLI shell-out path) --------------------------------- +enew! +call setline(1, ['@i abc @ x']) +set filetype=klammertext +call cursor(1, 1) +KlammertextJumpToMatch +call s:Check('jump open -> close', col('.') == 8, col('.')) +KlammertextJumpToMatch +call s:Check('jump close -> open', col('.') == 1, col('.')) + +" multibyte: 2-byte umlauts before the delimiters +enew! +call setline(1, ['üü @i x @']) +set filetype=klammertext +call cursor(1, 6) " byte col 6 = the @ of @i (2 x 2-byte ü) +KlammertextJumpToMatch +call s:Check('jump with multibyte line', + \ (exists('*charcol') ? charcol('.') : col('.')) + \ == (exists('*charcol') ? 9 : 11), + \ [col('.'), getline('.')]) + +" --- KlammertextCheck: diagnostics into the location list ------------------- +enew! +call setline(1, ['@i abc', '@ol x ul@']) +set filetype=klammertext +KlammertextCheck +let s:ll = getloclist(0) +call s:Check('check fills the location list', len(s:ll) == 2, s:ll) +call s:Check('check positions', len(s:ll) == 2 + \ && s:ll[0].lnum == 1 && s:ll[0].col == 1 + \ && s:ll[1].lnum == 2 && s:ll[1].col == 7, s:ll) +call s:Check('check messages', len(s:ll) == 2 + \ && s:ll[0].text =~# 'never closed' && s:ll[1].text =~# 'ul@', s:ll) +lclose + +enew! +call setline(1, ['@i balanced @']) +set filetype=klammertext +KlammertextCheck +call s:Check('check clean buffer -> empty list', len(getloclist(0)) == 0) + +" --- +python3: indentexpr (= operator) and live match highlighting --------- +if has('python3') + for s:fix in ['indent_list', 'indent_document', 'indent_named_close'] + execute 'edit! ' . fnameescape($KT_FIX . '/' . s:fix . '.kt') + call s:Check(s:fix . ': indentexpr is set', + \ &indentexpr ==# 'klammertext#IndentExpr()', &indentexpr) + normal! gg=G + call s:Check(s:fix . ': gg=G equals expected fixture', + \ getline(1, '$') ==# readfile($KT_FIX . '/' . s:fix . '_expected.kt')) + edit! + endfor + + " live match highlighting via the CursorMoved autocmd + enew! + call setline(1, ['@i abc @ x']) + set filetype=klammertext + call cursor(1, 1) + doautocmd CursorMoved + let s:m = filter(getmatches(), 'v:val.group =~# "^Klammertext"') + call s:Check('highlight: matched pair boxed', + \ len(s:m) == 1 && s:m[0].group ==# 'KlammertextMatch' + \ && len(filter(keys(s:m[0]), 'v:val =~# "^pos"')) == 2, s:m) + call cursor(1, 5) " not on a delimiter + doautocmd CursorMoved + call s:Check('highlight: cleared off-delimiter', + \ empty(filter(getmatches(), 'v:val.group =~# "^Klammertext"'))) + + enew! + call setline(1, ['@ol x ul@']) + set filetype=klammertext + call cursor(1, 1) + doautocmd CursorMoved + let s:m = filter(getmatches(), 'v:val.group =~# "^Klammertext"') + call s:Check('highlight: mismatch in red group', + \ len(s:m) == 1 && s:m[0].group ==# 'KlammertextMismatch', s:m) + + " default mappings exist (buffer-local, LocalLeader) + call s:Check('mappings defined', + \ !empty(maparg('j', 'n')) + \ && !empty(maparg('i', 'n')) + \ && !empty(maparg('a', 'n'))) +else + call add(s:results, 'SKIP +python3 checks (this Vim lacks python3)') +endif + +" --- write results and exit ------------------------------------------------- +call writefile(s:results, $KT_OUT) +if len(filter(copy(s:results), 'v:val =~# "^FAIL"')) > 0 + cquit! +endif +qa! diff --git a/tst/editor/vscode_ext_test.js b/tst/editor/vscode_ext_test.js new file mode 100644 index 0000000..2815d09 --- /dev/null +++ b/tst/editor/vscode_ext_test.js @@ -0,0 +1,225 @@ +// vscode_ext_test.js — behavioral test for the Klammertext VS Code +// extension (doc/edit/vscode/extension.js) OUTSIDE VS Code. +// +// The `vscode` module is stubbed with just enough API for activation, and +// the extension then talks to the REAL language server it spawned — so this +// exercises the whole chain: extension glue -> hand-rolled LSP client -> +// klammertext_ls.py -> shared core. Checks: activation + handshake, +// publishDiagnostics reaching the diagnostic collection, Format Document +// equalling the indent fixtures, documentHighlight pairs, and the +// alignTable command round-tripping through workspace/applyEdit. +// +// Runs under any Node >= 16 — including VS Code's own Electron binary +// (ELECTRON_RUN_AS_NODE=1 code vscode_ext_test.js EXT_DIR FIXTURE_DIR). +// Prints PASS/FAIL lines; exits nonzero on any failure. + +'use strict'; + +const fs = require('fs'); +const path = require('path'); +const Module = require('module'); + +const extDir = path.resolve(process.argv[2]); +const fixDir = path.resolve(process.argv[3]); + +let passed = 0, failed = 0; +function check(name, ok, detail) { + if (ok) { passed++; console.log('PASS ' + name); } + else { failed++; console.log('FAIL ' + name + ' ' + (detail || '')); } +} + +// --- the vscode stub ------------------------------------------------------- + +class Position { + constructor(line, character) { this.line = line; this.character = character; } +} +class Range { + constructor(a, b, c, d) { + if (typeof a === 'number') { this.start = new Position(a, b); this.end = new Position(c, d); } + else { this.start = a; this.end = b; } + } +} +class Selection extends Range { + constructor(a, b) { super(a, b); this.active = b; } +} +class TextEdit { + constructor(range, newText) { this.range = range; this.newText = newText; } +} +class Diagnostic { + constructor(range, message, severity) { this.range = range; this.message = message; this.severity = severity; } +} +class Location { + constructor(uri, range) { this.uri = uri; this.range = range; } +} +class DocumentHighlight { + constructor(range) { this.range = range; } +} +class WorkspaceEdit { + constructor() { this.edits = []; } + replace(uri, range, newText) { this.edits.push({ uri, range, newText }); } +} + +const listeners = { open: [], change: [], close: [] }; +const providers = {}; +const commands = {}; +const collections = {}; +let appliedEdits = []; +let statusMessages = []; + +const fakeDoc = { + uri: { toString: () => 'file:///ext_test.kt' }, + languageId: 'klammertext', + version: 1, + text: '@i abc\n', + getText() { return this.text; }, +}; + +const vscodeStub = { + Position, Range, Selection, TextEdit, Diagnostic, Location, + DocumentHighlight, WorkspaceEdit, + Uri: { parse: (s) => ({ toString: () => s }) }, + DiagnosticSeverity: { Error: 0, Warning: 1 }, + workspace: { + getConfiguration: () => ({ get: (k) => (k === 'pythonPath' ? 'python3' : '') }), + textDocuments: [fakeDoc], + onDidOpenTextDocument: (fn) => { listeners.open.push(fn); return { dispose() {} }; }, + onDidChangeTextDocument: (fn) => { listeners.change.push(fn); return { dispose() {} }; }, + onDidCloseTextDocument: (fn) => { listeners.close.push(fn); return { dispose() {} }; }, + applyEdit: (we) => { appliedEdits.push(we); return Promise.resolve(true); }, + }, + window: { + createOutputChannel: () => ({ append() {}, dispose() {} }), + showWarningMessage: (m) => { statusMessages.push(m); }, + setStatusBarMessage: (m) => { statusMessages.push(m); }, + activeTextEditor: null, + }, + languages: { + createDiagnosticCollection: (name) => { + const c = { + store: new Map(), + set(uri, diags) { this.store.set(uri.toString(), diags); }, + dispose() {}, + }; + collections[name] = c; + return c; + }, + registerDocumentFormattingEditProvider: (lang, p) => { providers.format = p; return { dispose() {} }; }, + registerDocumentRangeFormattingEditProvider: (lang, p) => { providers.rangeFormat = p; return { dispose() {} }; }, + registerDocumentHighlightProvider: (lang, p) => { providers.highlight = p; return { dispose() {} }; }, + registerDefinitionProvider: (lang, p) => { providers.definition = p; return { dispose() {} }; }, + }, + commands: { + registerCommand: (name, fn) => { commands[name] = fn; return { dispose() {} }; }, + }, +}; + +const realResolve = Module._resolveFilename; +Module._resolveFilename = function (request, ...rest) { + if (request === 'vscode') return 'vscode'; + return realResolve.call(this, request, ...rest); +}; +require.cache.vscode = { id: 'vscode', filename: 'vscode', loaded: true, exports: vscodeStub }; + +// --- helpers --------------------------------------------------------------- + +function sleep(ms) { return new Promise((r) => setTimeout(r, ms)); } + +async function waitFor(pred, what, ms = 8000) { + const t0 = Date.now(); + while (Date.now() - t0 < ms) { + if (pred()) return true; + await sleep(25); + } + throw new Error('timeout waiting for ' + what); +} + +function applyEditsToText(text, edits) { + // Convert {line, character} ranges to offsets (fixtures are ASCII). + const lineStart = [0]; + for (let i = 0; i < text.length; i++) if (text[i] === '\n') lineStart.push(i + 1); + const off = (p) => lineStart[p.line] + p.character; + const resolved = edits.map((e) => ({ a: off(e.range.start), b: off(e.range.end), t: e.newText })); + resolved.sort((x, y) => y.a - x.a); + for (const e of resolved) text = text.slice(0, e.a) + e.t + text.slice(e.b); + return text; +} + +function setDocText(text) { + fakeDoc.text = text; + fakeDoc.version++; + listeners.change.forEach((fn) => fn({ document: fakeDoc })); +} + +// --- the test -------------------------------------------------------------- + +async function main() { + const ext = require(path.join(extDir, 'extension.js')); + const context = { extensionPath: extDir, subscriptions: [] }; + ext.activate(context); + + // Activation opens the (unbalanced) preloaded document; diagnostics + // arrive from the real server. + await waitFor(() => (collections.klammertext + && (collections.klammertext.store.get('file:///ext_test.kt') || []).length > 0), + 'diagnostics'); + const diags = collections.klammertext.store.get('file:///ext_test.kt'); + check('activation + publishDiagnostics', + diags.length === 1 && /never closed/.test(diags[0].message), + JSON.stringify(diags.map((d) => d.message))); + + // Format Document == every indent fixture's expected file. + for (const f of ['indent_list', 'indent_document', 'indent_table', + 'indent_untouched', 'indent_defs', 'indent_escapes', + 'indent_named_close']) { + const src = fs.readFileSync(path.join(fixDir, f + '.kt'), 'utf8'); + const exp = fs.readFileSync(path.join(fixDir, f + '_expected.kt'), 'utf8'); + setDocText(src); + const edits = await providers.format.provideDocumentFormattingEdits(fakeDoc); + check('format ' + f, applyEditsToText(src, edits) === exp); + } + + // Balanced text clears the diagnostics. + setDocText('@i abc @\n'); + await waitFor(() => (collections.klammertext.store.get('file:///ext_test.kt') || []).length === 0, + 'diagnostics cleared'); + check('diagnostics cleared on balanced text', true); + + // documentHighlight: the delimiter pair. + const hl = await providers.highlight.provideDocumentHighlights(fakeDoc, new Position(0, 0)); + check('documentHighlight pair', hl.length === 2, JSON.stringify(hl)); + + // jumpToMatch moves the cursor to the close. + vscodeStub.window.activeTextEditor = { + document: fakeDoc, + selection: new Selection(new Position(0, 0), new Position(0, 0)), + revealRange() {}, + }; + await commands['klammertext.jumpToMatch'](); + const sel = vscodeStub.window.activeTextEditor.selection; + check('jumpToMatch cursor at close', sel.active.character === 7, + JSON.stringify(sel)); + + // alignTable round-trips through workspace/applyEdit. + const src = fs.readFileSync(path.join(fixDir, 'align_mixed.kt'), 'utf8'); + const exp = fs.readFileSync(path.join(fixDir, 'align_mixed_expected.kt'), 'utf8'); + setDocText(src); + const barOffset = src.indexOf('|'); + const barLine = src.slice(0, barOffset).split('\n').length - 1; + const barCol = barOffset - (src.lastIndexOf('\n', barOffset - 1) + 1); + vscodeStub.window.activeTextEditor.selection = + new Selection(new Position(barLine, barCol), new Position(barLine, barCol)); + appliedEdits = []; + await commands['klammertext.alignTable'](); + await waitFor(() => appliedEdits.length > 0, 'applyEdit'); + const edits = appliedEdits[0].edits.map((e) => ({ range: e.range, newText: e.newText })); + check('alignTable via applyEdit', applyEditsToText(src, edits) === exp); + + ext.deactivate(); + console.log('vscode_ext_test: %d passed, %d failed', passed, failed); + process.exit(failed ? 1 : 0); +} + +main().catch((err) => { + console.error('vscode_ext_test: ' + err.stack); + process.exit(1); +}); diff --git a/tst/editor_test.sh b/tst/editor_test.sh index f99a9ed..479bf55 100755 --- a/tst/editor_test.sh +++ b/tst/editor_test.sh @@ -1,30 +1,39 @@ #!/bin/bash # # editor_test.sh — Regression tests for the Klammertext editor support -# (the Emacs mode's indentation/alignment units and their Sublime Text ports). +# (doc/edit/: the shared Python core, the language server, and the Emacs, +# Sublime Text, Vim, and VS Code integrations built on them). # # Why this lives in tst/: the editor code implements the LANGUAGE's structural # layer — @-run length, bar-run dimension, ^-escapes, # removal, literal # spans, nesting depth — independently of both the SKS and the Klammermachine. # tst/ is the SKS-independent tier, and these suites check that the -# independent implementations of Klammertext's structure agree with the -# canonical formatting conventions AND with each other. The klammer names -# that appear in the fixtures (@ol, @table, @document, @code) are seeded -# configuration of the editor tools, not SKS dependencies; nothing here runs -# ktext or loads a klammer set. +# implementations of Klammertext's structure agree with the canonical +# formatting conventions AND with each other. The klammer names that appear +# in the fixtures (@ol, @table, @document, @code) are seeded configuration of +# the editor tools, not SKS dependencies; nothing here runs ktext or loads a +# klammer set. # # What is checked, for every fixture pair .kt / _expected.kt in # tst/editor/: -# * the Sublime Python core's output equals the expected file +# * the shared core's output equals the expected file — through the Python +# API (what Sublime uses) AND through the core's CLI (what Vim uses) # * idempotence: the tool applied to the expected file leaves it unchanged -# * when Emacs is installed: the Emacs unit's output equals the expected -# file, and equals the Python output byte for byte (the sync check across -# the SYNC-noted policy lists) — skipped gracefully otherwise +# * the three Sublime adapter plugins import outside Sublime and resolve +# to the shared core +# * the language server passes its protocol test (tst/editor/ls_test.py: +# handshake, diagnostics, formatting, matching, alignTable — the VS Code +# extension's whole surface) +# * when Emacs is installed: the Emacs units' output equals the expected +# file, and equals the shared core's output byte for byte (the sync +# check across the independent elisp implementation) — skipped otherwise +# * when a Vim with +eval is installed: the Vim plugin's commands produce +# the same bytes (exercising the CLI shell-out glue) — skipped otherwise # -# The editor code is located automatically: doc/emacs + doc/sublime in the -# development tree, doc/edit/emacs + doc/edit/sublime in the distribution. +# The editor code lives in doc/edit/ (same layout in the development tree and +# the distribution). # -# Usage: ./editor_test.sh Requires python3; Emacs is optional. +# Usage: ./editor_test.sh Requires python3; Emacs and Vim optional. # Exit code: 0 if all tests pass, 1 otherwise. PASS=0 @@ -37,16 +46,15 @@ reset=$'\033[0m' HERE="$(cd "$(dirname "$0")" && pwd)" ROOT="$(cd "$HERE/.." && pwd)" -if [ -d "$ROOT/doc/emacs" ]; then - EMACS_DIR="$ROOT/doc/emacs" - SUBLIME_DIR="$ROOT/doc/sublime" -elif [ -d "$ROOT/doc/edit/emacs" ]; then - EMACS_DIR="$ROOT/doc/edit/emacs" - SUBLIME_DIR="$ROOT/doc/edit/sublime" -else - echo "editor_test.sh: cannot locate the editor support (doc/emacs or doc/edit/emacs)" >&2 +EDIT_DIR="$ROOT/doc/edit" +if [ ! -d "$EDIT_DIR/emacs" ]; then + echo "editor_test.sh: cannot locate the editor support (doc/edit)" >&2 exit 1 fi +EMACS_DIR="$EDIT_DIR/emacs" +SUBLIME_DIR="$EDIT_DIR/sublime" +SHARED_DIR="$EDIT_DIR/shared" +VIM_DIR="$EDIT_DIR/vim" FIX="$HERE/editor" OUT="$(mktemp -d)" trap 'rm -rf "$OUT"' EXIT @@ -62,25 +70,62 @@ py_args=() el_args=() add_fixture() { # add_fixture MODE NAME local mode="$1" name="$2" - py_args+=("$mode" "$FIX/$name.kt" "$OUT/$name.py.out") - py_args+=("$mode" "$FIX/${name}_expected.kt" "$OUT/$name.py.idem") - el_args+=("$mode" "$FIX/$name.kt" "$OUT/$name.el.out") - el_args+=("$mode" "$FIX/${name}_expected.kt" "$OUT/$name.el.idem") + py_args+=("$mode" "$FIX/$name.kt" "$OUT/$name.py.out") + py_args+=("$mode" "$FIX/${name}_expected.kt" "$OUT/$name.py.idem") + py_args+=("$mode-cli" "$FIX/$name.kt" "$OUT/$name.cli.out") + el_args+=("$mode" "$FIX/$name.kt" "$OUT/$name.el.out") + el_args+=("$mode" "$FIX/${name}_expected.kt" "$OUT/$name.el.idem") } for f in $INDENT_FIXTURES; do add_fixture indent "$f"; done for f in $ALIGN_FIXTURES; do add_fixture align "$f"; done -# --- run the Sublime Python cores (required) ------------------------------- +check() { # check NAME FILE_A FILE_B + if diff -q "$2" "$3" >/dev/null 2>&1; then + echo "${green}PASS${reset} $1" + PASS=$((PASS + 1)) + else + echo "${red}FAIL${reset} $1" + diff "$2" "$3" | head -10 + FAIL=$((FAIL + 1)) + fi +} + +# --- the shared core: API and CLI (required) ------------------------------- if ! command -v python3 >/dev/null 2>&1; then echo "editor_test.sh: python3 not found" >&2 exit 1 fi -if ! PYTHONDONTWRITEBYTECODE=1 python3 "$FIX/editor_driver.py" "$SUBLIME_DIR" "${py_args[@]}"; then +if ! PYTHONDONTWRITEBYTECODE=1 python3 "$FIX/editor_driver.py" "$SHARED_DIR" "${py_args[@]}"; then echo "editor_test.sh: the Python driver failed" >&2 exit 1 fi -# --- run the Emacs units (optional) ---------------------------------------- +# --- the Sublime adapters resolve to the shared core ----------------------- +if PYTHONDONTWRITEBYTECODE=1 python3 -c " +import sys +sys.path.insert(0, '$SUBLIME_DIR') +import Klammertext, Klammertext_indent, Klammertext_align +assert Klammertext.KE.__file__.startswith('$SHARED_DIR') +" 2>"$OUT/sublime.log"; then + echo "${green}PASS${reset} sublime adapters import the shared core" + PASS=$((PASS + 1)) +else + echo "${red}FAIL${reset} sublime adapters import the shared core" + cat "$OUT/sublime.log" + FAIL=$((FAIL + 1)) +fi + +# --- the language server protocol test ------------------------------------- +if PYTHONDONTWRITEBYTECODE=1 python3 "$FIX/ls_test.py" "$SHARED_DIR" "$FIX" >"$OUT/ls.log" 2>&1; then + echo "${green}PASS${reset} language server protocol ($(grep -c '^PASS' "$OUT/ls.log") checks)" + PASS=$((PASS + 1)) +else + echo "${red}FAIL${reset} language server protocol:" + cat "$OUT/ls.log" + FAIL=$((FAIL + 1)) +fi + +# --- the Emacs units (optional) -------------------------------------------- HAVE_EMACS=0 if command -v emacs >/dev/null 2>&1; then if emacs --batch -L "$EMACS_DIR" \ @@ -93,29 +138,93 @@ if command -v emacs >/dev/null 2>&1; then FAIL=$((FAIL + 1)) fi else - echo "(Emacs not installed — the Emacs half is skipped; the Python cores still run)" + echo "(Emacs not installed — the Emacs half is skipped; the shared core still runs)" +fi + +# --- the Vim plugin (optional) --------------------------------------------- +HAVE_VIM=0 +VIM_BIN="$(command -v vim || true)" +if [ -n "$VIM_BIN" ] && "$VIM_BIN" --version 2>/dev/null | grep -q '+eval'; then + HAVE_VIM=1 + run_vim() { # run_vim MODE INFILE OUTFILE + local cmd + if [ "$1" = indent ]; then + cmd='KlammertextReindent' + else + cmd='call search("|") | KlammertextAlign' + fi + "$VIM_BIN" -N -n -u NONE -i NONE -es --not-a-term \ + --cmd "set rtp^=$VIM_DIR" \ + -c 'filetype plugin on' \ + -c "edit! $2" -c 'set ft=klammertext' \ + -c "$cmd" -c "saveas! $3" -c 'qa!' /dev/null 2>&1 + } + for f in $INDENT_FIXTURES; do + run_vim indent "$FIX/$f.kt" "$OUT/$f.vim.out" + run_vim indent "$FIX/${f}_expected.kt" "$OUT/$f.vim.idem" + done + for f in $ALIGN_FIXTURES; do + run_vim align "$FIX/$f.kt" "$OUT/$f.vim.out" + run_vim align "$FIX/${f}_expected.kt" "$OUT/$f.vim.idem" + done + # The comprehensive plugin checks: ftdetect (Kotlin override), syntax + # token classes, jump-to-match (incl. multibyte columns), the location- + # list check, and — with +python3 — indentexpr (gg=G) and the live + # match highlighter. Sections skip inside the driver per Vim feature. + if KT_FIX="$FIX" KT_OUT="$OUT/vim_feature.txt" \ + "$VIM_BIN" -N -n -u NONE -i NONE -es --not-a-term \ + --cmd "set rtp^=$VIM_DIR" \ + -c "source $FIX/vim_feature_test.vim" /dev/null 2>&1; then + echo "${green}PASS${reset} vim plugin features ($(grep -c '^PASS' "$OUT/vim_feature.txt") checks$( + grep -q '^SKIP' "$OUT/vim_feature.txt" && printf '; %s' "$(grep -c '^SKIP' "$OUT/vim_feature.txt") section(s) skipped"))" + PASS=$((PASS + 1)) + else + echo "${red}FAIL${reset} vim plugin features:" + grep -v '^PASS' "$OUT/vim_feature.txt" 2>/dev/null || echo " (the Vim driver itself failed)" + FAIL=$((FAIL + 1)) + fi +else + echo "(no Vim with +eval installed — the Vim half is skipped; the shared core still runs)" +fi + +# --- the VS Code extension (optional: needs a Node runtime) ---------------- +# node itself, or VS Code's Electron binary run as Node. +run_node() { + if command -v node >/dev/null 2>&1; then + node "$@" + elif [ -x /usr/share/code/code ]; then + ELECTRON_RUN_AS_NODE=1 /usr/share/code/code "$@" + else + return 127 + fi +} +if command -v node >/dev/null 2>&1 || [ -x /usr/share/code/code ]; then + if run_node "$FIX/vscode_ext_test.js" "$EDIT_DIR/vscode" "$FIX" >"$OUT/vscode.log" 2>&1; then + echo "${green}PASS${reset} vscode extension ($(grep -c '^PASS' "$OUT/vscode.log") checks)" + PASS=$((PASS + 1)) + else + echo "${red}FAIL${reset} vscode extension:" + cat "$OUT/vscode.log" + FAIL=$((FAIL + 1)) + fi +else + echo "(no Node runtime — the VS Code extension test is skipped; the language server is still tested)" fi # --- compare --------------------------------------------------------------- -check() { # check NAME FILE_A FILE_B - if diff -q "$2" "$3" >/dev/null 2>&1; then - echo "${green}PASS${reset} $1" - PASS=$((PASS + 1)) - else - echo "${red}FAIL${reset} $1" - diff "$2" "$3" | head -10 - FAIL=$((FAIL + 1)) - fi -} - for f in $INDENT_FIXTURES $ALIGN_FIXTURES; do check "$f (python)" "$OUT/$f.py.out" "$FIX/${f}_expected.kt" check "$f (python idempotent)" "$OUT/$f.py.idem" "$FIX/${f}_expected.kt" + check "$f (cli)" "$OUT/$f.cli.out" "$FIX/${f}_expected.kt" if [ "$HAVE_EMACS" = 1 ]; then check "$f (emacs)" "$OUT/$f.el.out" "$FIX/${f}_expected.kt" check "$f (emacs idempotent)" "$OUT/$f.el.idem" "$FIX/${f}_expected.kt" check "$f (emacs == python)" "$OUT/$f.el.out" "$OUT/$f.py.out" fi + if [ "$HAVE_VIM" = 1 ]; then + check "$f (vim)" "$OUT/$f.vim.out" "$FIX/${f}_expected.kt" + check "$f (vim idempotent)" "$OUT/$f.vim.idem" "$FIX/${f}_expected.kt" + fi done echo