ack/klammertext
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Target coverage: a klammer states the targets it serves

Browse Source 
kdesc gains --coverage, which reports for every klammer the set of targets it
can render to, and — the point of it — which klammers' coverage cannot be
derived and must therefore be declared.  Three rules: coverage is DERIVED
where the definitions determine it (a general body of klammer calls covers
the intersection of what those klammers cover, by a greatest fixpoint after
loading), DECLARED where the engine cannot interpret what decides it (an
@eval body, whose targets are undecidable), and UNKNOWN where nothing is
written — which never means "deliberately unavailable".

Two new spellings in a definition's name.  A comma-separated target list,
"@@table.html,tex :: ...", gives one body several targets; it is surface
syntax, expanded at registration, and each member goes through the
redefinition rules on its own.  And "@@date.* :: ..." writes the general
target out, asserting that the klammer works for EVERY target including ones
not yet defined — a stronger claim than a list of the targets defined today,
and the one target declaration that could be mechanically falsified.

The Standard Klammer Set was swept accordingly: it now has no general
definitions at all, every klammer names the targets it serves, six use ".*",
and tex and pdf are at zero undecided.

kdesc's flags are reorganised on two rules: a flag reached for often gets a
single letter (-k klammers, -t targets, -c characters, -i input), a more
specialised topic a multi-letter name (--argtypes, --katoms, --rewrite,
--optionsets, --coverage, --klammerset, --font); and -v says how much to show
about PROCESSING, never what the RESULT contains — so the katom regex column
is "--katoms full" and the coverage detail "--coverage all".  NOTE: "-k" now
lists klammers (optionally filtered by a name/description search); the katom
table moved to "--katoms".

Fixes carried along: an option written with no value crashed the command with
SIGSEGV instead of reporting the mistake; two required positional arguments
never parsed; kdesc and kdiag printed an error and exited 0; and definition
diagnostics counted registrations rather than what was written, so one line
could be reported as two definitions and then printed twice.

Four new test suites: target_list, coverage, command_option, kdesc.

(from dev 46f54080bd9a)
This commit is contained in:
Andy Kopra
2026-08-12 17:20:23 +02:00
parent 61987c8b1f
commit 59c1599bc9
49 changed files with 1960 additions and 431 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
mac/Makefile
View File

@@ -11,7 +11,7 @@ BASENAMES := util error locator file argv character ktype katom katom_list \
log show command argument argument_set argtype argtype_registry \
state eval eval_python eval_cpp klammer klammer_registry klammerset klammerset_registry \
option_set option_set_registry deftype \
target target_registry machine font_store check
target target_registry machine font_store check coverage
SOURCES := $(addsuffix .cpp,$(BASENAMES))
OBJECTS := $(addsuffix .o,$(BASENAMES))

60
mac/argv.cpp
View File

@@ -145,7 +145,8 @@ void Argv::opt(const std::string& name, const std::string& desc, const std::stri
update_width(arg);
}
void Argv::var(const std::string& name, const std::string& desc)
void Argv::var(const std::string& name, const std::string& desc,
const std::string& parameter)
{
(void)K::log(2, name, desc);
Arg arg {};
@@ -153,6 +154,9 @@ void Argv::var(const std::string& name, const std::string& desc)
arg.m_name = name;
arg.m_desc = get_regex_desc(desc);
arg.m_syntax = arg.symbol();
if (!parameter.empty()) {
arg.m_syntax += " [<" + parameter + ">]";
}
m_args[name] = arg;
m_names.push_back(name);
m_var_names.push_back(name);
@@ -291,18 +295,23 @@ void Argv::parse_optional(strings_t& words, string_map& named_args)
// std::cout << " Found: " << words[index] << "\n";
//std::vector<std::string> opt_args = {};
index++;
/*
// An option declared with opt() takes a value, so the word after
// the flag must exist. Written last with nothing after it -- a
// bare "kdesc -v", or "ktext doc.kt -t" -- this read past the end
// of the vector and the command died with SIGSEGV, naming
// nothing. (Two bounds checks used to sit here, commented out;
// they would have returned a HALF-PARSED option rather than
// reporting the mistake, so this reports it instead.) A valueless
// option is declared with flag(), not opt().
if (index >= words.size()) {
break;
throw Argument_error(
"The option " + flag_name(opt) + " needs a value: " +
m_args[opt].m_syntax + ". Enter \"" +
file_basename(command_name) + "\" for the list of arguments.",
Locator(), false);
}
*/
std::string opt_arg = words[index] + " ";
index++;
/*
if (index >= words.size()) {
break;
}
*/
std::regex opt_regex = m_args[opt].m_rgx;
// std::cout << "Regex match? " << std::regex_match(trim(opt_arg), opt_regex) << "\n";
@@ -346,7 +355,19 @@ void Argv::parse_positional(const std::string& command, //strings_t words,
throw Argument_error(ss.str(), Locator(), false);
}
named_args[req] = substring;
pos_args = rest;
// Drop the whitespace that separated this positional from the next.
// regex_split_prefix() requires its match at position 0 and returns
// the remainder verbatim, so without this the SECOND required
// argument is always "not found": its pattern is offered " second"
// and cannot match a leading space.
//
// Trimming here rather than at the top of the loop is deliberate: the
// first positional still receives pos_args exactly as before, so a
// command with ONE required argument -- which is every command that
// ships (ktext's `filenames`, kdiag's `input`) -- parses
// bit-identically. Only the case that never worked changes.
size_t next = rest.find_first_not_of(" \t");
pos_args = (next == std::string::npos) ? "" : rest.substr(next);
}
// std::cout << "Remaining words: " << words << "\n" << pos_args << "\n";
}
@@ -436,7 +457,13 @@ void Argv::parse(int argc, char* argv[], bool full_parse)
{
(void)K::log(2);
command_name = argv[0];
describe();
// No describe() here. A call sat at this point, before any value has
// been assigned, so it could only ever print a table of "<none>" -- and
// because set_verbose_level() parses a throwaway Argv before the real
// one, EVERY command would print that table on every run. Silencing
// Argv::describe() itself was the wrong half of the fix: it also silenced
// the callers that legitimately want it (kdesc -v, argv_test). The
// caller decides; parse() does not print.
auto input_args = classify_arguments(argc, argv, full_parse);
// std::cout << "parse() classify:\n" << input_args << "\n";
@@ -610,10 +637,11 @@ void Argv::describe()
std::stringstream ss {};
ss << " "<< std::setfill(' ') << std::setw(width)
<< m_args[name].symbol() << " : " << value;
// The line was built and then dropped, so this printed nothing at
// all -- which is why "kdesc -v 1" showed no arguments and argv_test,
// whose whole job is to display what Argv parsed, was silent. The
// callers already decide whether to call it (the commands gate it on
// verbose_level > 0), so it prints unconditionally here.
std::cout << ss.str() << "\n";
}
/*
if (verbose_level == 1) {
std::cout << "\n";
}
*/
}

8
mac/argv.h
View File

@@ -58,7 +58,13 @@ public:
// space-joined; given(name) distinguishes "--name with no words"
// from an absent --name. Used for subcommand-style interfaces
// (kdesc --font install <dir>).
void var(const std::string& name, const std::string& desc);
// A variadic option: the words after the flag, up to the next one.
// `parameter` is shown in the usage line as "[<parameter>]" -- an option
// whose words are a free argument (a search phrase) should name it, so
// the usage says what may follow; a subcommand-style option (--font,
// --coverage) leaves it empty and documents its words in its own help.
void var(const std::string& name, const std::string& desc,
const std::string& parameter = "");
void update_width(Arg arg);

481
mac/coverage.cpp Normal file
View File

@@ -0,0 +1,481 @@
#include <algorithm>
#include <functional>
#include <iomanip>
#include <ostream>
#include <set>
#include "coverage.h"
#include "machine.h"
#include "katom.h"
#include "util.h"
namespace {
using target_set = std::set<std::string>;
// The written definitions of one klammer, split by what they tell us. A
// definition filed under the general name is the only one whose coverage has
// to be worked out; a definition written for a target IS its own statement.
struct Written
{
const Klammer::components* m_general { nullptr };
target_set m_targets {}; // targets with a definition of their own
bool m_declared { false }; // has a ".k" declaration
};
Written written_definitions(const Klammer& klammer)
{
Written w {};
for (const auto& def : klammer.m_defs) {
if (def.target == Target_registry::general_name) {
w.m_general = &def;
} else if (def.target == Target_registry::declare_name) {
w.m_declared = true;
} else if (def.target != Target_registry::optionset_name) {
w.m_targets.insert(def.target);
}
}
return w;
}
// What a general body is made of. The three findings are ordered by how
// firmly they settle the question: anything the engine cannot interpret ends
// the analysis, and only a body of plain klammer calls is derivable.
struct Body_scan
{
bool m_undecidable { false };
std::string m_reason {};
std::vector<std::string> m_calls {};
};
Body_scan scan_body(const katom_list& body)
{
Body_scan scan {};
target_set seen {};
for (const auto& k : body) {
switch (k.m_type) {
case katom_t::eval_begin:
// Which targets a Python function answers for is undecidable, so
// the analysis stops here and the targets must be declared.
scan.m_undecidable = true;
scan.m_reason = "@eval body";
return scan;
case katom_t::read_begin:
scan.m_undecidable = true;
scan.m_reason = "@read body";
return scan;
case katom_t::literal_begin:
// A ^'...'^ span exists to carry raw target markup past the
// escaping pass. A general body holding one is target-specific
// with nothing for the intersection rule to see -- the blind spot
// a body of plain text would otherwise hide.
scan.m_undecidable = true;
scan.m_reason = "^'...'^ literal span";
return scan;
case katom_t::apply_begin: {
// The body read here is the STORED body, which is the body as
// written with one exception: a general klammer that takes no
// parameters is a constant, and a constant's body is spliced into
// later definitions at definition time. So a call to a constant
// does not appear here -- what appears is whatever the constant
// expanded to. That is the right thing for coverage (the calls
// that remain are the ones that will be applied), but it makes
// the "from" list a statement about the stored body, not about
// the source text.
std::string name = trim_char(k.m_text, '@');
if (seen.insert(name).second) {
scan.m_calls.push_back("@" + name);
}
break;
}
default:
break;
}
}
return scan;
}
target_set intersect(const target_set& a, const target_set& b)
{
target_set result {};
std::set_intersection(a.begin(), a.end(), b.begin(), b.end(),
std::inserter(result, result.begin()));
return result;
}
std::vector<std::string> as_vector(const target_set& s)
{
return { s.begin(), s.end() };
}
// Shorten a definition's pathname for the report. What identifies a
// definition to a reader is its tail -- "sks/block/block.k" -- not the
// absolute path the file happened to be read from, which is the same long
// prefix on every row. A klammer set outside $KLAMMERTEXT_HOME keeps its
// path in full rather than being shortened to something ambiguous.
std::string short_path(const std::string& path)
{
auto pos = path.rfind("/sks/");
if (pos != std::string::npos) {
return path.substr(pos + 1);
}
const char* home = std::getenv("KLAMMERTEXT_HOME");
if (home != nullptr) {
std::string prefix = std::string(home) + "/";
if (path.size() > prefix.size() && path.compare(0, prefix.size(), prefix) == 0) {
return path.substr(prefix.size());
}
}
return path;
}
} // namespace
std::vector<Klammer_coverage> klammer_coverage(const Machine& machine)
{
target_set all_targets {};
for (const auto& t : machine.m_targets.user_defined()) {
all_targets.insert(t);
}
// Pass 1: what each klammer's definitions say, without resolving anything.
std::map<std::string, Written> written {};
std::map<std::string, Body_scan> scans {};
for (const auto& [name, klammer] : machine.m_klammers.m_klammers) {
written[name] = written_definitions(klammer);
if (written[name].m_general != nullptr) {
scans[name] = scan_body(written[name].m_general->body);
}
}
// Pass 2: the greatest fixpoint. Every general klammer starts optimistic
// -- all targets -- and the intersection rule is applied until nothing
// shrinks. Starting optimistic is what makes a cycle terminate: two
// klammers calling each other simply keep each other's sets, and a set
// that only loses members cannot iterate forever.
std::map<std::string, target_set> general {};
for (const auto& [name, scan] : scans) {
general[name] = all_targets;
}
// A klammer's coverage, for use as an operand of the intersection: what
// its own definitions cover, plus whatever its general body covers.
auto coverage_of = [&](const std::string& name) -> target_set {
auto w = written.find(name);
if (w == written.end()) return {}; // not defined; contributes nothing
target_set result = w->second.m_targets;
auto g = general.find(name);
if (g != general.end()) {
result.insert(g->second.begin(), g->second.end());
}
return result;
};
bool changed = true;
while (changed) {
changed = false;
for (auto& [name, targets] : general) {
const Body_scan& scan = scans[name];
if (scan.m_undecidable || scan.m_calls.empty()) continue;
target_set next = all_targets;
for (const auto& call : scan.m_calls) {
std::string called = trim_char(call, '@');
if (called == name) continue; // self-reference constrains nothing
next = intersect(next, coverage_of(called));
}
if (next != targets) {
targets = next;
changed = true;
}
}
}
// Pass 3: classify and collect.
std::vector<Klammer_coverage> result {};
for (const auto& [name, klammer] : machine.m_klammers.m_klammers) {
const Written& w = written.at(name);
Klammer_coverage kc {};
kc.m_name = name;
kc.m_declared = w.m_declared;
kc.m_written = as_vector(w.m_targets);
// Written definitions only (m_defs), so this is where the klammer is
// WRITTEN. m_defloc would also carry the targets a general body was
// copied to, which name the same file again.
for (const auto& def : klammer.m_defs) {
std::string file = short_path(def.loc.m_filename);
if (!is_in(file, kc.m_files)) {
kc.m_files.push_back(file);
}
}
for (const auto& [target, loc] : klammer.m_defloc) {
if (target != Target_registry::declare_name &&
target != Target_registry::general_name &&
target != Target_registry::optionset_name) {
kc.m_effective.push_back(target);
}
}
if (klammer.m_general_declared) {
// The author has said "every target". That is a statement, not
// something to be re-derived from the body: the whole point of
// writing ".*" is to assert what an @eval body cannot be read to
// mean.
kc.m_kind = coverage_t::all_declared;
target_set covered = w.m_targets;
covered.insert(all_targets.begin(), all_targets.end());
kc.m_targets = as_vector(covered);
} else if (w.m_general == nullptr) {
kc.m_kind = w.m_targets.empty() ? coverage_t::none : coverage_t::declared;
kc.m_targets = kc.m_written;
} else {
const Body_scan& scan = scans.at(name);
target_set covered = w.m_targets;
if (scan.m_undecidable) {
kc.m_kind = coverage_t::undecidable;
kc.m_reason = scan.m_reason;
// What it covers is not knowable here; report what is written.
kc.m_targets = kc.m_written;
} else if (scan.m_calls.empty()) {
kc.m_kind = coverage_t::all;
covered.insert(all_targets.begin(), all_targets.end());
kc.m_targets = as_vector(covered);
} else {
kc.m_kind = coverage_t::derived;
kc.m_from = scan.m_calls;
const target_set& g = general.at(name);
covered.insert(g.begin(), g.end());
kc.m_targets = as_vector(covered);
}
}
result.push_back(kc);
}
return result;
}
namespace {
std::string list_of(const std::vector<std::string>& v)
{
return v.empty() ? "--" : join(v, " ");
}
void section(std::ostream& os, const std::string& title, size_t count,
const std::string& explanation)
{
os << "\n" << title << " (" << count << ")\n";
if (!explanation.empty()) {
os << explanation;
}
}
} // namespace
void report_coverage(const Machine& machine,
const std::vector<Klammer_coverage>& coverage,
bool full, std::ostream& os)
{
strings_t targets = machine.m_targets.user_defined();
os << "Klammer coverage\n"
<< "================\n\n"
<< coverage.size() << " klammers, " << targets.size()
<< " targets: " << join(targets, " ") << "\n";
auto of_kind = [&coverage](coverage_t kind) {
std::vector<const Klammer_coverage*> result {};
for (const auto& kc : coverage) {
if (kc.m_kind == kind) result.push_back(&kc);
}
return result;
};
size_t width = 0;
for (const auto& kc : coverage) {
width = std::max(width, kc.m_name.size());
}
// The indent of a continuation line, under the name column.
std::string continuation(2 + 1 + width, ' ');
auto name_of = [&](const Klammer_coverage& kc) {
std::stringstream ss {};
ss << " @" << std::left << std::setw(width) << kc.m_name;
return ss.str();
};
// A section's rows, held until the whole section is built so the
// source-file column can be aligned. The file goes LAST because it is
// reference information: what the row says comes first, and the reader
// looks right only when they want to go and edit it. Only a row naming
// ONE klammer carries a file -- the "All targets" section lists many
// names on a line and has nothing to attach one to.
using Row = std::pair<std::string, std::string>; // text, file
auto emit = [&os, full](const std::vector<Row>& rows) {
size_t text_width = 0;
if (full) {
for (const auto& [text, file] : rows) {
if (!file.empty()) text_width = std::max(text_width, text.size());
}
}
for (const auto& [text, file] : rows) {
if (full && !file.empty()) {
os << std::left << std::setw(text_width) << text << " " << file << "\n";
} else {
os << trim_right(text) << "\n";
}
}
};
// ---- what works, first --------------------------------------------
//
// The reporting categories come before the problems because a terminal is
// read from the BOTTOM: an eighty-klammer listing scrolls a three-line
// warning off the screen entirely, so the actionable part has to be last,
// where "| tail" finds it.
//
// The two halves of the report hide an empty category for different
// reasons, so they are two functions rather than one with a flag.
using Row_of = std::function<std::string(const Klammer_coverage&)>;
// The explanation under a heading is for a reader learning the categories,
// so it appears only under "all" -- the same argument that shows the empty
// ones. A default report is headings and rows.
auto write = [&](const std::string& title,
const std::vector<const Klammer_coverage*>& group,
const std::string& explanation, const Row_of& row) {
section(os, title, group.size(), full ? explanation : "");
std::vector<Row> rows {};
for (const auto* kc : group) {
rows.push_back({row(*kc), join(kc->m_files, ", ")});
}
emit(rows);
};
// A REPORTING category describes the shape of the klammer set, so an
// empty one still says something ("nothing here uses .*") and "all" shows
// it as a designer's checklist.
auto reporting = [&](const std::string& title,
const std::vector<const Klammer_coverage*>& group,
const std::string& explanation, const Row_of& row) {
if (group.empty() && !full) return;
write(title, group, explanation, row);
};
// A PROBLEM category is different: it sits under a banner that says
// "Needs attention", and an empty one does not. Printing "Covers no
// target (0)" there states the opposite of the heading above it, so it is
// hidden whether or not "all" was given -- "all" is for information that
// is missing, and a category with nothing in it is not missing anything.
auto problem = [&](const std::string& title,
const std::vector<const Klammer_coverage*>& group,
const std::string& explanation, const Row_of& row) {
if (group.empty()) return;
write(title, group, explanation, row);
};
auto with_targets = [&](const Klammer_coverage& kc) {
return name_of(kc) + " " + list_of(kc.m_targets);
};
reporting("Defined per target", of_kind(coverage_t::declared), "", with_targets);
reporting("All targets, declared", of_kind(coverage_t::all_declared),
" Written \".*\": the author states that these work for every target,\n"
" including targets that do not exist yet.\n", with_targets);
// Many names on one line, so no file column: there is nothing for a file
// to attach to.
auto all = of_kind(coverage_t::all);
if (!all.empty() || full) {
section(os, "All targets, derived", all.size(),
full ? " No target suffix and a body of plain text, so nothing in them is\n"
" target-specific. This is the writer's macro form -- a repeated\n"
" phrase, not a klammer set -- and needs no declaration.\n" : "");
strings_t names {};
for (const auto* kc : all) {
names.push_back("@" + kc->m_name);
}
if (!names.empty()) {
os << " " << join(names, " ") << "\n";
}
}
// A derived klammer covering NOTHING is unusable, so it is reported with
// the problems rather than here.
std::vector<const Klammer_coverage*> derived {};
std::vector<const Klammer_coverage*> uncoverable {};
for (const auto* kc : of_kind(coverage_t::derived)) {
(kc->m_targets.empty() ? uncoverable : derived).push_back(kc);
}
reporting("Derived from the klammers the body calls", derived,
" No target suffix and a body of klammer calls, so the coverage is the\n"
" intersection of what those klammers cover.\n",
[&](const Klammer_coverage& kc) {
std::stringstream ss {};
ss << name_of(kc) << " " << std::left << std::setw(22) << list_of(kc.m_targets)
<< " from " << join(kc.m_from, " ");
return ss.str();
});
// ---- then what needs doing ----------------------------------------
//
// Four categories, ordered by severity: the first two mean the klammer
// cannot be used at all, the third that its coverage is a guess, the
// fourth that it is undocumented. Counted by DISTINCT klammer -- "no .k"
// is orthogonal to the others, so a klammer can be in two categories and
// summing the counts would overstate the work.
auto undecidable = of_kind(coverage_t::undecidable);
auto none = of_kind(coverage_t::none);
std::vector<const Klammer_coverage*> undescribed {};
for (const auto& kc : coverage) {
if (!kc.m_declared) undescribed.push_back(&kc);
}
target_set needing {};
for (const auto* group : { &none, &uncoverable, &undecidable, &undescribed }) {
for (const auto* kc : *group) needing.insert(kc->m_name);
}
// Nothing to attend to, nothing said -- the same rule as the categories
// below it. "Needs attention: 0" under "all" was the banner contradicting
// itself, exactly as an empty category under it would.
if (!needing.empty()) {
os << "\nNeeds attention: " << needing.size() << " "
<< plural("klammer", static_cast<int>(needing.size())) << "\n";
}
problem("Declared but never defined", none,
" A \".k\" declaration with no definition for any target, so the klammer\n"
" can never be applied.\n",
[&](const Klammer_coverage& kc) { return name_of(kc); });
problem("Covers no target", uncoverable,
" The klammers this one calls have no target in common, so the\n"
" intersection is empty and it can never be applied. The klammers\n"
" named are the ones to look at.\n",
[&](const Klammer_coverage& kc) {
return name_of(kc) + " from " + join(kc.m_from, " ");
});
problem("Must be declared", undecidable,
" A general definition whose body the engine cannot interpret, so it is\n"
" offered to EVERY target whether or not its code answers for that\n"
" target. Name the targets it does answer for -- @@name.html,tex :: --\n"
" or, if it works for any target at all, @@name.* ::\n",
[&](const Klammer_coverage& kc) { return name_of(kc) + " " + kc.m_reason; });
problem("No \".k\" declaration", undescribed,
" These render, but nothing describes them: a klammer without a \".k\"\n"
" has no description, so kdesc can say nothing about what it does and\n"
" \"kdesc -k <text>\" can only find it by name.\n", with_targets);
// The progress number. "Undecided" is the accurate label, and the one
// that asserts no more than was measured: a klammer that does not cover a
// target has not been declared unavailable there -- no notation for that
// exists yet -- it simply has no definition. "Unsupported" or "excluded"
// would each claim a decision nobody made.
os << "\nBy target\n";
for (const auto& target : targets) {
size_t covered = 0;
size_t offered = 0;
for (const auto& kc : coverage) {
if (is_in(target, kc.m_targets)) covered++;
if (is_in(target, kc.m_effective)) offered++;
}
os << " " << std::left << std::setw(10) << target
<< std::right << std::setw(4) << covered << " covered"
<< std::setw(6) << (coverage.size() - covered) << " undecided";
if (offered > covered) {
os << " (" << offered - covered
<< " more currently offered by an underivable general body)";
}
os << "\n";
}
}

110
mac/coverage.h Normal file
View File

@@ -0,0 +1,110 @@
#pragma once
#include <iosfwd>
#include <string>
#include <vector>
class Machine;
// Target coverage: which targets a klammer can actually render to.
//
// Coverage is a FACT about a klammer, distinct from a klammer set's claim
// about what it supports and from what happens when a document meets a
// target. This module computes the fact, and computes only what can be
// computed -- it changes nothing about how the Machine behaves. See
// notes/target_coverage.md for why the fact has to come first.
//
// Three rules, and the whole of the analysis is deciding which one applies:
//
// DERIVED where coverage is structurally determined. A general
// definition (no target suffix) whose body is text and nothing
// else covers every target. One whose body calls other klammers
// covers the INTERSECTION of what those klammers cover -- a
// klammer can only render where everything it is made of renders.
//
// DECLARED where coverage depends on something the engine cannot
// interpret. A general body holding an @eval is the main case:
// deciding which targets a Python function answers for is
// undecidable, so the targets have to be written down (which is
// what the comma-separated target list is for). A general body
// holding a ^'...'^ literal span is the same problem wearing
// different clothes -- the span exists precisely to carry raw
// target markup past the escaping pass, so a body containing one
// is target-specific with nothing for the intersection rule to
// see. @read is included: its content is not known statically.
//
// DECLARED-ALL a definition written ".*" asserts that the klammer works for
// EVERY target, including targets that do not exist yet. A list
// of the targets defined today cannot say that. It is the one
// target declaration a machine could later falsify: a ".*"
// klammer whose implementation branches per target is
// contradicting itself, which is a structural property.
//
// UNKNOWN where neither applies -- no definition at all. Absence means
// "not decided yet", never "deliberately unavailable": the SKS is
// incomplete on schedule rather than by design, so nothing may
// read a missing definition as a statement of intent.
//
// The intersection is computed as a GREATEST FIXPOINT rather than by
// recursion: general klammers may call each other, and a cycle would not
// terminate. Every general klammer starts at "all targets" and the rule is
// applied until nothing shrinks, which terminates because the sets only ever
// lose members. The pass runs over the whole registry AFTER loading, not at
// definition time -- definitions load in file order, so a body may call a
// klammer defined later.
enum class coverage_t {
all_declared, // written ".*": every target, including ones not yet defined
all, // general body, no klammer calls: every target
derived, // general body of klammer calls: their intersection
declared, // written per target, no general body to derive from
undecidable, // general body holding @eval, @read, or a literal span
none, // declared (.k) but never defined
};
struct Klammer_coverage
{
std::string m_name {};
coverage_t m_kind { coverage_t::none };
// The targets this klammer can render to, as computed.
std::vector<std::string> m_targets {};
// Targets the Machine currently offers it for. These differ exactly
// where a general body is copied to targets it cannot really serve, which
// is the hazard the report exists to surface.
std::vector<std::string> m_effective {};
// Targets named in a written definition (the comma-list, or one per
// definition), empty for a purely general klammer.
std::vector<std::string> m_written {};
// For `derived`: the klammers the body calls. For `undecidable`: why.
std::vector<std::string> m_from {};
std::string m_reason {};
// Whether a ".k" declaration exists. A klammer without one still works
// -- its parameters can be declared on the definition itself -- but it
// has no DESCRIPTION, so kdesc can say nothing about what it does and
// "kdesc -k <text>" can only find it by name.
bool m_declared { false };
// The file(s) the klammer is written in, in definition order. Usually
// one -- a klammer's targets are declared together -- but a klammer whose
// definitions are spread over several files lists them all. Shown by
// "--coverage -v", and only on a row that names ONE klammer.
std::vector<std::string> m_files {};
};
// Compute the coverage of every klammer the machine has loaded. Analysis
// only: nothing in the Machine is modified.
std::vector<Klammer_coverage> klammer_coverage(const Machine& machine);
// The report behind "kdesc --coverage". `full` ("--coverage all") adds the
// source-file column and shows every "Needs attention" category, including
// the empty ones; without it those categories appear only when they have
// entries, so a klammer set with nothing wrong produces a short report.
//
// It is an argument rather than a verbosity level because the two are
// independent: -v says how much to show about the command's PROCESSING, and
// this says what the command's RESULT contains.
//
// Audiences, in the project's terms: an AUTHOR runs it to see what is
// available for a target; a DESIGNER runs "--coverage all" to be reminded of
// the full set of categories while building a klammer set.
void report_coverage(const Machine& machine,
const std::vector<Klammer_coverage>& coverage,
bool full, std::ostream& os);

128
mac/klammer.cpp
View File

@@ -10,34 +10,82 @@
using namespace std::literals;
std::regex Klammer::name_re = std::regex(R"((\w+)(?:\.(\w+))?)");
std::regex Klammer::name_re = std::regex(R"((\w+)(?:\.((?:\w+|\*)(?:,(?:\w+|\*))*))?)");
std::tuple<std::string, std::string>
std::tuple<std::string, strings_t, bool>
parse_name(const Target_registry& targets, const Katom& name_katom)
{
std::string name_with_target = trim_char(name_katom.m_text, '@');
std::smatch match {};
if (!std::regex_match(name_with_target, match, Klammer::name_re)) {
throw Parsing_error(
"The klammer name \"" + name_with_target + "\" is not correctly defined. "
"The form is \"<klammer-name>\" for general klammers or \"<klammer-name>.<target-name>\" "
"for a specialized target. The klammer defined as \"<klammer-name>.k\" specifies the "
"for a specialized target. Several targets that share one body are written as a "
"comma-separated list: \"<klammer-name>.<target-name>,<target-name>\". The klammer "
"defined as \"<klammer-name>.k\" specifies the "
"arguments and contains a description of the klammer in the definition body.",
name_katom.m_loc);
}
std::string klammer_name = match[1];
std::string target_name = match[2];
if (target_name.empty()) {
target_name = Target_registry::general_name;
std::string target_part = match[2];
if (target_part.empty()) {
// No suffix at all: the general target, but not a STATEMENT about
// coverage. This is the writer's macro form -- someone defining a
// repeated phrase is not building a klammer set -- so the third
// result is false and the coverage analysis treats the body on its
// merits rather than as an assertion.
return { klammer_name, { Target_registry::general_name }, false };
}
if (!targets.has(target_name)) {
strings_t target_names = regex_split(target_part, std::regex(","));
strings_t seen {};
for (const auto& target_name : target_names) {
if (!targets.has(target_name)) {
throw Target_error(
"The target \"" + target_name + "\" in klammer definition \"" + name_with_target + "\" "
"is not defined. Enter \"kdesc -t\" to see the targets defined by the Standard Klammer Set.",
name_katom.m_loc);
}
if (is_in(target_name, seen)) {
throw Target_error(
"The target \"" + target_name + "\" is named more than once in klammer definition \"" +
name_with_target + "\".",
name_katom.m_loc);
}
seen.push_back(target_name);
}
// A ".k" declaration states ONE interface for every target, and a ".o"
// declares an option set; neither produces output, so neither has any
// meaning as a member of a list of output targets.
if (target_names.size() > 1) {
for (const auto& reserved :
{ Target_registry::declare_name, Target_registry::optionset_name }) {
if (is_in(reserved, target_names)) {
throw Target_error(
"The klammer definition \"" + name_with_target + "\" names \"" + reserved +
"\" in a list of targets. A \"." + reserved + "\" definition declares an "
"interface for all targets rather than producing output for one, so it must "
"be written on its own.",
name_katom.m_loc);
}
}
}
// "*" written out is an ASSERTION: this klammer works for every target,
// including targets that do not exist yet. A list of "all the targets
// defined today" cannot say that, and the difference matters the moment a
// new target is added. It may not appear IN a list -- "all targets and
// also html" is either redundant or a misunderstanding.
bool general_declared = is_in(Target_registry::general_name, target_names);
if (general_declared && target_names.size() > 1) {
throw Target_error(
"The target \"" + target_name + "\" in klammer definition \"" + name_with_target + "\" "
"is not defined. Enter \"kdesc -t\" to see the targets defined by the Standard Klammer Set.",
"The klammer definition \"" + name_with_target + "\" names \"" +
Target_registry::general_name + "\" in a list of targets. \"" +
Target_registry::general_name + "\" already means every target, so it "
"must be written on its own.",
name_katom.m_loc);
}
return { klammer_name, target_name };
return { klammer_name, target_names, general_declared };
}
std::tuple<Katom, Parameter_set, katom_list, Locator>
@@ -146,12 +194,50 @@ void Klammer::remove_target_definition(const std::string& target_name)
// Rationalize multiple definitions
// ONE definition in the source can register more than once. A target that
// "provides" another registers both (the SKS's pdf includes tex, so
// "@@fraktur.tex : ..." becomes a tex definition and a pdf one), and so does
// every member of a comma-separated target list. Counting or listing those
// registrations reports work the author did not do: "2 definitions" for a
// single line, followed by that same line printed twice -- which sends the
// reader hunting for a second definition that does not exist.
//
// These two report what was WRITTEN. Registrations are grouped by source
// location, and a location that produced several targets names them, so the
// count and the listing agree with the file.
using location_group = std::pair<std::string, strings_t>;
std::vector<location_group> group_by_location(const auto& components)
{
std::vector<location_group> groups {};
for (const auto& c : components) {
std::string loc = c.loc.desc();
auto it = std::find_if(groups.begin(), groups.end(),
[&loc](const location_group& g) { return g.first == loc; });
if (it == groups.end()) {
groups.push_back({loc, {c.target}});
} else {
it->second.push_back(c.target);
}
}
return groups;
}
int written_count(const auto& components)
{
return static_cast<int>(group_by_location(components).size());
}
std::string error_list(const std::string& label, const auto& components, const std::string& after="")
{
std::stringstream ss {};
ss << label << ":\n";
for (const auto& c : components) {
ss << " " << c.loc.desc() << "\n";
for (const auto& [loc, targets] : group_by_location(components)) {
ss << " " << loc;
if (targets.size() > 1) {
ss << " (targets " << join(targets, ", ") << ")";
}
ss << "\n";
}
ss << after;
return ss.str();
@@ -182,7 +268,7 @@ void Klammer::disallow_instances() //Klammer::components declaration)
{
auto instances = instance_defs();
if (!instances.empty()) {
int icount = instances.size();
int icount = written_count(instances);
std::stringstream ss {};
ss << "There " << to_be(icount) << " " << icount << " "
<< plural("instance", icount) << " (defined by \"::\"), but "
@@ -201,9 +287,9 @@ bool Klammer::copy_to_instances(const Target_registry& targets)
[] (const auto& def) {
return def.deftype != katom_t::klammer_instance
&& def.deftype != katom_t::klammer_override; });
int dcount = definitions.size();
int dcount = written_count(definitions);
if (dcount != 1) {
int icount = instances.size();
int icount = written_count(instances);
std::stringstream ss {};
ss << "There " << to_be(icount) << " " << icount << " "
<< plural("instance", icount) << " (defined by \"::\"), but "
@@ -268,11 +354,13 @@ void Klammer::check_for_declaration_and_definitions()
}
}
if (!definitions.empty()) {
auto defsize = definitions.size();
std::string desc = defsize == 1 ? "a definition" :
std::to_string(defsize) + " definitions";
int defsize = written_count(definitions);
std::string desc = defsize == 1 ? "a definition that declares its own parameters"
: std::to_string(defsize) + " definitions that declare their own parameters";
throw Definition_error(
error_list("A klammer has both a declaration (.k) as well as " + desc + "\n(instances are defined by \"::\")",
error_list("A klammer has both a \".k\" declaration and " + desc +
".\nWrite \"::\" instead of \":\" so the definition takes its parameters "
"from the declaration",
definitions),
declares[0].loc, false);
}

21
mac/klammer.h
View File

@@ -18,7 +18,9 @@ public:
using variable_map_t = std::map<std::string, std::vector<int>>;
using target_variable_map_t = std::map<std::string, variable_map_t>;
static std::regex name_re; // = std::regex(R"((\w+)(?:\.(\w+))?)");
// <name>[.<target>[,<target>...]] -- the target part is a comma-separated
// list so that one body can serve several targets (see parse_name).
static std::regex name_re; // = std::regex(R"((\w+)(?:\.(\w+(?:,\w+)*))?)");
struct components {
std::string target;
@@ -93,11 +95,26 @@ public:
// target -> true if this target's body came from a general ("*") definition
// (writer content, subject to target escaping) vs a target-specific one.
std::map<std::string, bool> m_body_generic {};
// True when a definition wrote the general target out as ".*" instead of
// omitting the suffix. Identical to the engine; to a reader it is the
// difference between "I did not say" and "I say: every target, whatever
// they turn out to be". Only the second is a claim the coverage report
// can repeat.
bool m_general_declared { false };
};
std::string klammer_name_from_katom(const std::string& s, const Locator& loc);
std::tuple<std::string,std::string>
// Split "@@<name>[.<target>[,<target>...]]" into the klammer name and the
// targets the definition is for. Always at least one name: an absent suffix
// is the general target. A comma list is surface syntax only -- the caller
// registers one definition per target, so nothing downstream of registration
// knows a list was written.
// The third result is true when the general target was written out as "*"
// rather than left off. Both mean the same to the engine; they mean
// different things to a reader and to the coverage analysis -- see
// Klammer::m_general_declared.
std::tuple<std::string,strings_t,bool>
parse_name(const Target_registry& targets, const Katom& name_katom);
// Split a definition's katoms into its definition separator, parameters,

86
mac/klammer_registry.cpp
View File

@@ -18,13 +18,11 @@ void Klammer_registry::add(
{
(void)K::log(3, *begin, *(end - 1));
restore_initial_type(begin, end);
auto [klammer_name, target_name] = parse_name(targets, *begin);
if (!targets.has(target_name)) {
throw Argument_error("The target \"" + target_name + "\" is not defined", begin->m_loc);
}
if (target_name == Target_registry::optionset_name) {
auto [klammer_name, target_names, general_declared] = parse_name(targets, *begin);
if (is_in(Target_registry::optionset_name, target_names)) {
// The Machine routes an ".o" definition to the option set registry;
// reaching here means it did not.
// reaching here means it did not. parse_name has already rejected
// ".o" as a member of a list, so this is the bare ".o" form.
throw Internal_error(
"The option set declaration \"" + klammer_name + ".o\" reached the klammer registry",
begin->m_loc);
@@ -41,37 +39,47 @@ void Klammer_registry::add(
if (m_klammers.count(klammer_name) == 0) {
m_klammers[klammer_name] = Klammer(klammer_name);
} else if (m_klammers[klammer_name].m_defloc.count(target_name) > 0) {
defmode_t existing_mode = m_klammers[klammer_name].m_defmode[target_name];
const auto& result = defmode_transition(existing_mode, incoming_mode);
std::string name_target = klammer_name + "." + target_name;
std::string at_desc = m_klammers[klammer_name].m_defloc[target_name].desc();
if (!result.replace) {
if (result.message.empty()) {
// Silent ignore (e.g., create + default)
modify_type(katom_t::replaced, begin, end);
ignore_whitespace(end, katoms);
return;
}
std::string msg = result.message;
msg = string_replace(msg, "NAME", q_(name_target));
msg = string_replace(msg, "AT", at_desc);
throw Definition_error(msg, begin->m_loc);
}
if (result.warn) {
std::string msg = result.message;
msg = string_replace(msg, "NAME", q_(name_target));
msg = string_replace(msg, "AT", at_desc);
warning(msg, begin->m_loc);
}
m_klammers[klammer_name].remove_target_definition(target_name);
}
m_klammers[klammer_name].add_target_definition(
target_name, argtypes, option_sets, begin + 1, end - 1);
// Sticky: once a definition has written ".*", the klammer carries the
// claim. A klammer with both a ".*" body and a target-specific one still
// asserts that the general body serves everything else.
if (general_declared) {
m_klammers[klammer_name].m_general_declared = true;
}
// One definition per target named. A comma-separated list is surface
// syntax: each target goes through the same registration, including the
// redefinition transition table, so a list that collides with an existing
// definition is decided per target -- one member may be silently ignored
// or rejected while the others are created.
for (const auto& target_name : target_names) {
if (m_klammers[klammer_name].m_defloc.count(target_name) > 0) {
defmode_t existing_mode = m_klammers[klammer_name].m_defmode[target_name];
const auto& result = defmode_transition(existing_mode, incoming_mode);
std::string name_target = klammer_name + "." + target_name;
std::string at_desc = m_klammers[klammer_name].m_defloc[target_name].desc();
if (!result.replace) {
if (result.message.empty()) {
// Silent ignore (e.g., create + default)
continue;
}
std::string msg = result.message;
msg = string_replace(msg, "NAME", q_(name_target));
msg = string_replace(msg, "AT", at_desc);
throw Definition_error(msg, begin->m_loc);
}
if (result.warn) {
std::string msg = result.message;
msg = string_replace(msg, "NAME", q_(name_target));
msg = string_replace(msg, "AT", at_desc);
warning(msg, begin->m_loc);
}
m_klammers[klammer_name].remove_target_definition(target_name);
}
m_klammers[klammer_name].add_target_definition(
target_name, argtypes, option_sets, begin + 1, end - 1);
// This add's target:
Target target = targets.get(target_name, begin->m_loc);
if (!target.m_provides.empty()) {
// This add's target:
Target target = targets.get(target_name, begin->m_loc);
for (const auto& provide_name : target.m_provides) {
if (m_klammers[klammer_name].m_defloc.count(provide_name) > 0) {
m_klammers[klammer_name].remove_target_definition(provide_name);
@@ -164,7 +172,7 @@ std::string Klammer_registry::instance_list(int margin) const
return ss.str();
}
std::string Klammer_registry::describe(int margin) const
std::string Klammer_registry::describe(int margin, const std::string& search) const
{
/*
strings_t names {};
@@ -172,7 +180,13 @@ std::string Klammer_registry::describe(int margin) const
strings_t locations {};
*/
std::string result;
std::string query = collapse_whitespace(search);
for (const auto& [name, k] : m_klammers) {
if (!query.empty() &&
!contains_fold(name, query) &&
!contains_fold(collapse_whitespace(k.description_text()), query)) {
continue;
}
result += k.describe(margin) + "\n";
/*
names.push_back(name);

7
mac/klammer_registry.h
View File

@@ -14,7 +14,12 @@ public:
void check_klammer(const std::string& name, const std::string& target, const Locator& loc) const;
const std::vector<Katom>* constant_body(const std::string& name) const;
std::string instance_list(int margin) const;
std::string describe(int margin=0) const;
// With a search string, only the klammers whose NAME or DESCRIPTION
// contains it, case-insensitively and with whitespace collapsed on both
// sides (a description written across several lines in a ".k" file must
// still match a phrase typed on one). Empty when nothing matched, which
// is how the caller knows to say so.
std::string describe(int margin=0, const std::string& search="") const;
std::map<std::string, Klammer> m_klammers {};
};

12
mac/ktype.h
View File

@@ -35,6 +35,16 @@ inline const std::string ws_newline_s { R"(#/\d*)" };
// const std::string k_name = R"([^^@#|\:]+)";
inline const std::string definition_name = R"([a-zA-Z][a-zA-Z0-9_.]*)";
// The name at the START of a definition may carry a comma-separated list of
// targets ("@@table.html,tex"), so its katom runs through commas. This is
// deliberately looser than the grammar: the katomizer's job is to delimit the
// token, and parse_name() in klammer.cpp validates the list, where a
// malformed one can be reported with its location. The comma stays out of
// definition_name itself -- that pattern also delimits klammer APPLICATIONS,
// option names and "*arg*" variables, where a comma is ordinary writer text.
// "*" is here for the same reason: "@@date.*" declares a klammer for every
// target explicitly, and "*" is a Klammertext special everywhere else.
inline const std::string definition_begin_name = R"([a-zA-Z][a-zA-Z0-9_.,*]*)";
enum class katom_t {
@@ -163,7 +173,7 @@ std::vector<Ktype> katom_types {
Ktype(katom_t::apply_begin, "apply-begin", at_s + definition_name, "Beginning of a klammer call"),
Ktype(katom_t::apply_end, "apply-end", "(" + definition_name + ")?" + at_s, "End of a klammer call"),
Ktype(katom_t::option_name, "option-name", ":" + definition_name, "Optional argument name"),
Ktype(katom_t::define_begin, "define-begin", at2_s + definition_name, "Beginning of a klammer definition"),
Ktype(katom_t::define_begin, "define-begin", at2_s + definition_begin_name, "Beginning of a klammer definition"),
Ktype(katom_t::define_end, "define-end", "(" + definition_name + ")?" + at2_s, "End of a klammer definition"),
Ktype(katom_t::klammer_default, "klammer-default", "::::", "Define klammer default value for possible override"),
Ktype(katom_t::klammer_override, "klammer-override", ":::", "Override existing klammer definition"),

7
mac/machine.cpp
View File

@@ -528,8 +528,11 @@ void Machine::add_definition(katom_list& katoms, const Katom& op, const Katom& c
{
expand_constant_klammers(katoms, op, cl);
auto [begin, end] = find_span_katoms(katoms, op, cl);
auto [name, target] = parse_name(m_targets, *begin);
if (target == Target_registry::optionset_name) {
auto [name, target_names, general_declared] = parse_name(m_targets, *begin);
(void)general_declared; // routing only cares whether this is a ".o"
// parse_name rejects ".o" as a member of a target list, so an option set
// declaration is always the single-target form.
if (target_names.size() == 1 && target_names[0] == Target_registry::optionset_name) {
m_option_sets.add(m_argtypes, name, begin, end, katoms);
} else {
m_klammers.add(m_argtypes, m_targets, m_option_sets, begin, end, katoms);

2
mac/target_registry.cpp
View File

@@ -17,7 +17,7 @@ Target_registry::Target_registry()
: m_parameters(Parameter_set("name | desc :after_apply :after_write :includes :escape | transforms.rest"))
{
Target k(declare_name, "Description of parameters and klammer result", Locator());
Target general(general_name, "General target, used when a target is not specified", Locator());
Target general(general_name, "Every target: written \".*\" to declare a klammer works for all of them, or implied when a definition names no target", Locator());
Target option_set(optionset_name, "Declaration of an option set: parameters shared by klammers", Locator());
add(k);
add(general);

30
mac/util.cpp
View File

@@ -134,6 +134,36 @@ strings_t word_split(const std::string& s)
return regex_split(s, std::regex("\\s+"));
}
std::string collapse_whitespace(const std::string& s)
{
std::string result {};
bool in_space = true; // leading whitespace is dropped
for (char c : s) {
if (std::isspace(static_cast<unsigned char>(c)) != 0) {
in_space = true;
} else {
if (in_space && !result.empty()) {
result += ' ';
}
in_space = false;
result += c;
}
}
return result;
}
bool contains_fold(const std::string& haystack, const std::string& needle)
{
auto fold = [](const std::string& s) {
std::string result {};
for (char c : s) {
result += static_cast<char>(std::tolower(static_cast<unsigned char>(c)));
}
return result;
};
return fold(haystack).find(fold(needle)) != std::string::npos;
}
bool is_in(const std::string& s, const strings_t& v)
{
return find(v.begin(), v.end(), s) != v.end();

13
mac/util.h
View File

@@ -38,6 +38,19 @@ bool contains(const std::string& str, const std::string& substr);
bool contains(const std::vector<std::string>& strings, const std::string& element);
std::vector<std::string> regex_split(const std::string& s, const std::regex& re, bool trim_parts=true);
std::vector<std::string> word_split(const std::string& s);
// Every run of whitespace becomes one space, and the ends are trimmed. Needed
// wherever a phrase typed on one line is matched against text that was written
// across several: a klammer's description sits on its own lines in a ".k" file
// and may wrap, so "displayed verbatim" would otherwise match the klammer
// whose description happens to fit one line and miss the identically worded
// one that does not.
std::string collapse_whitespace(const std::string& s);
// Case-insensitive substring test, ASCII folding only. The project takes no
// ICU or locale dependency (the same rule as the language tables and
// ":decimal"), so a non-ASCII query matches only exactly.
bool contains_fold(const std::string& haystack, const std::string& needle);
bool is_in(const std::string& s, const std::vector<std::string>& v);
bool is_not_in(const std::string& s, const std::vector<std::string>& v);
std::vector<std::string> find_all(const std::string& str, const std::regex& pattern, int match_group=0);