fix(novelty): keep a skill whose only spec error is an over-long description in the routing field

dokimi/changelog.d/20261104_120000_napetrov_desc-over-limit-kept.md

@@ -0,0 +1,16 @@
+### Changed
+
+- **A skill whose only spec error is a description over 1024 characters stays in the routing
+  field** — `dokimi novelty routing` (single-artifact and `--all-vs-all`) left such a skill out
+  unless `--include-invalid` was passed, but harnesses differ: Claude Code lists it and truncates
+  the description past about 1,536 characters, pi.dev rejects it. The skill now stays in the
+  field, from `--catalog` or `--competitors skill:`, with its full description. The run warns
+  with the skill name and its length, and the saved result lists it under `field.spec_warnings`
+  (an empty list when none); an over-limit skill under test (`--artifact`) is warned about and
+  listed there too.
+  Any other spec error (name not kebab-case or not its directory name, broken frontmatter) still
+  leaves a skill out unless `--include-invalid`. `dokimi catalog check` no longer treats such a
+  copy as unlisted, so it reports a `name-conflict` with a different copy of the same name, as
+  routing does.
+  The routing field of a catalog with such a skill changes, so routing rates measured before
+  this release on it differ: re-run a saved baseline before comparing with it.

dokimi/docs/external-content.md

@@ -186,7 +186,8 @@ with the root directory's name), and its `description` at most 1024 characters:
 `error`, printed under the skill. It exits 1 only with `--strict`; without it, the scan reports the
 error and still exits 0. A `name` missing from the frontmatter, or a `SKILL.md` byte-identical to
 another skill's, is a `warn`. So is a skill whose `name` another skill shares with different content
-(`name-conflict`), when both are listed: neither is `disable-model-invocation` or has a spec error.
+(`name-conflict`), when both are listed: neither is `disable-model-invocation` or has a spec error
+(a description over 1024 characters, as the only error, does not unlist a skill for routing).
 The finding names the other path, and `dokimi novelty routing --catalog` skips that name rather than
 the whole catalog. Of identical copies, the one nearest the root (fewest path parts, ties to path
 order) is the original, the copy the `skill-catalog:` arm offers; each other copy is the `warn` and

dokimi/docs/skill-novelty.md

@@ -635,10 +635,17 @@ names every path of that name: the rest of the field stays. Only the copies the
 a disabled copy (unless `--include-disabled`) or a spec-error copy (unless `--include-invalid`) is
 left out first, so it does not make its name a conflict. Rename one, or delete the copy you do
 not mean to route against, to route the name. `dokimi catalog check` lists the same names as
-`name-conflict` warnings, for the copies it lists. A skill with a spec error (for example a description over 1024 characters, or a
-name that is not kebab-case or not its directory name) is also left out, with a warning naming the error; pass
+`name-conflict` warnings, for the copies it lists. A skill with a spec error (a name that is not
+kebab-case or not its directory name) is also left out, with a warning naming the error; pass
 `--include-invalid` to keep it, as a harness that refuses to list such a skill would not. A `skill:`
-competitor with a spec error is skipped with the same warning.
+competitor with a spec error is skipped with the same warning. The one exception is a description
+over 1024 characters as the only error: harnesses differ on it, so the skill stays in the field with
+its full description, and the run warns with its name and length (`WARNING: skill 'x': description is
+1100 characters ...`) and records it under `spec_warnings` in the saved field rule. Claude Code
+(2.1.318, probed 2026-11-04) lists such a skill with the description cut to about 1,536 characters and
+an ellipsis, so a tail past that is never seen by its model (anthropics/skills#881); pi.dev rejects the
+skill (#1622). Where a harness truncates, the rate for such a skill may differ from the one measured on the
+full text; shorten the description to 1024 characters or fewer to make every harness agree.
 The directory-name check applies to a `skill:` competitor under a `skills/` directory
 (`skills/<dir>/SKILL.md`, `plugins/x/skills/<dir>/SKILL.md`), as it does in a catalog. A `skill:`
 competitor anywhere else is read as a root skill, and its name is not checked against its
@@ -719,8 +726,9 @@ is no attractor, and a tied skill's own rate is lower by the picks inside its gr
 `n_scored` and `n_unscored` per skill say how many, a query with no content word or a failed
 model call is unscored, and a skill with none scored has an own rate of `null`, not 0.0. A run
 with any failed model call writes the result and exits non-zero. The saved file records the
-field rule under `field` (`include_disabled`, `include_invalid`, the `competitors` and what was `left_out`, which
-also names each skill left out for a conflicting name or a spec error); a
+field rule under `field` (`include_disabled`, `include_invalid`, the `competitors`, what was `left_out`, which
+also names each skill left out for a conflicting name or a spec error, and `spec_warnings`, one
+`{skill, length, limit, message}` per kept skill whose description is over 1024 characters); a
 single-artifact run records it too. It takes no `--artifact`, `--queries` or `--queries-from-headings`. `--repeats` needs `--model`
 and asks each query that many times, as a single-artifact run does: the model arm then carries a
 `noise` block (agreement per query, the rate of each repeat, `half_range`), and its counts are

dokimi/dokimi/commands/novelty.py

@@ -20,7 +20,11 @@
 from typing import TYPE_CHECKING, NoReturn
 
 from dokimi.commands.inputs import placebo_text_for_arms_or_exit
-from dokimi.commands.novelty_routing_inputs import load_all_vs_all_field, load_routing_field
+from dokimi.commands.novelty_routing_inputs import (
+    field_record,
+    load_all_vs_all_field,
+    load_routing_field,
+)
 from dokimi.commands.novelty_routing_report import (
     exit_partial_all_vs_all,
     print_all_vs_all_model,
@@ -344,7 +348,7 @@ def _cmd_routing_all_vs_all(args: argparse.Namespace) -> None:
     out.parent.mkdir(parents=True, exist_ok=True)
     payload = {"mode": "all_vs_all", "queries_by_skill": args.queries_by_skill,
                "n_candidates": len(field), "result": result}
-    payload["field"] = _field_record(args, field, left_out)
+    payload["field"] = field_record(args, field, left_out)
     if selector is not None:
         payload["cost"] = roll_up_cost([selector])
     out.write_text(json.dumps(payload, indent=2), encoding="utf-8")
@@ -357,12 +361,6 @@ def _cmd_routing_all_vs_all(args: argparse.Namespace) -> None:
     print_all_vs_all_model(model_arm, payload.get("cost"))
 
 
-def _field_record(args: argparse.Namespace, competitors: list, left_out: list[str]) -> dict:
-    """The field rule a routing result was measured under, saved next to the result."""
-    return {**{name: bool(getattr(args, name)) for name in ("include_disabled", "include_invalid")},
-            "competitors": [c.name for c in competitors], "left_out": left_out}
-
-
 def _check_all_vs_all_args(args: argparse.Namespace) -> None:
     """Refuse the flag combinations ``--all-vs-all`` cannot run with."""
     if args.artifact:
@@ -427,7 +425,7 @@ def cmd_novelty_routing(args: argparse.Namespace) -> None:
         "n_candidates": len(candidates),
         "queries_self_derived": self_derived,
         "queries_source": queries_source,
-        "field": _field_record(args, competitors, left_out),
+        "field": field_record(args, competitors, left_out, gold),
         "lexical": lexical,
     }
     if args.model:
@@ -609,7 +607,9 @@ def register(sub, positive_int) -> None:
                                 "select them and a card for one measures a choice that does "
                                 "not exist")
     routing_p.add_argument("--include-invalid", action="store_true", dest="include_invalid",
-                           help="Keep catalog and skill: competitors with a spec error in the field")
+                           help="Keep catalog and skill: competitors with a spec error in the field. "
+                                "A description over 1024 characters alone never leaves a skill "
+                                "out: it is kept and warned about")
     routing_p.add_argument("--queries", default=None,
                            help="JSON list (or one per line) of user-shaped requests that "
                                 "should select this artifact, written independently of it")

dokimi/dokimi/commands/novelty_routing_inputs.py

@@ -59,12 +59,13 @@ def routing_field(args: argparse.Namespace, exclude: set[str],
     """The field ``--competitors`` and ``--catalog`` name, without the names in ``exclude``.
 
     What was left out of the field, and why, is appended to ``left_out`` when given. Each
-    name a rule leaves out maps to its rules in ``dropped`` (when given).
+    name a rule leaves out maps to its rules in ``dropped`` (when given). A skill kept with a
+    description over 1024 characters is warned about here and listed in the saved field rule.
 
     Raises:
         OSError, ValueError, RecursionError: A reference or the catalog cannot be loaded.
     """
-    from dokimi.novelty.routing import catalog_competitors
+    from dokimi.novelty.routing import catalog_competitors, over_limit_warnings
 
     competitors = named_competitors(args, left_out, dropped)
     if args.catalog:
@@ -83,6 +84,8 @@ def routing_field(args: argparse.Namespace, exclude: set[str],
                   file=sys.stderr)
         print(f"--catalog {args.catalog}: {len(catalog)} competitors", file=sys.stderr)
         competitors += catalog
+    for warning in over_limit_warnings(competitors):
+        print(f"WARNING: skill {warning['skill']!r}: {warning['message']}", file=sys.stderr)
     return competitors
 
 
@@ -101,7 +104,7 @@ def load_routing_field(
     args: argparse.Namespace,
 ) -> tuple[Candidate, str, list[Candidate], list[str]]:
     """The gold card, its body, the competitors and the ones left out, or InputError naming the cause."""
-    from dokimi.novelty.routing import disables_model_invocation, skill_file
+    from dokimi.novelty.routing import disables_model_invocation, over_limit_warnings, skill_file
 
     try:
         gold, body = routing_card(args.artifact)
@@ -112,8 +115,24 @@ def load_routing_field(
                   file=sys.stderr)
         left_out: list[str] = []
         competitors = routing_field(args, {gold.name}, left_out)
+        for warning in over_limit_warnings([gold], under_test=True):
+            print(f"WARNING: skill {warning['skill']!r}: {warning['message']}", file=sys.stderr)
     except InputError:
         raise
     except (OSError, ValueError, RecursionError) as exc:
         raise InputError(f"cannot load an artifact: {exc}") from exc
     return gold, body, competitors, left_out
+
+
+def field_record(args: argparse.Namespace, competitors: list[Candidate], left_out: list[str],
+                 gold: Candidate | None = None) -> dict:
+    """The field rule a routing result was measured under, saved next to the result.
+
+    ``spec_warnings`` lists ``gold`` (the skill under test, when given) as well as the field.
+    """
+    from dokimi.novelty.routing import over_limit_warnings
+
+    return {**{name: bool(getattr(args, name)) for name in ("include_disabled", "include_invalid")},
+            "competitors": [c.name for c in competitors], "left_out": left_out,
+            "spec_warnings": (over_limit_warnings([gold], under_test=True) if gold else [])
+            + over_limit_warnings(competitors)}

dokimi/dokimi/external/listing.py

@@ -0,0 +1,29 @@
+# Copyright (C) 2026 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+"""Which reviewed skills a harness lists, and so the routing field and ``catalog check`` agree on."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from .validate import SkillReview
+
+
+def only_description_too_long(review: SkillReview) -> bool:
+    """Whether every spec error of ``review`` is a description over 1024 characters.
+
+    Harnesses differ on such a skill: pi.dev rejects it, Claude Code lists it and truncates the
+    description past about 1,536 characters. The routing field keeps it, so a result is not
+    silently measured without a skill that some harness would list. Any other spec error still
+    leaves a skill out.
+    """
+    errors = [f for f in review.findings if f.level == "error"]
+    return bool(errors) and all(f.code == "description-too-long" for f in errors)
+
+
+def in_field(review: SkillReview) -> bool:
+    """Whether the routing field lists the skill by default: not disabled, no spec error but length."""
+    return not review.disable_model_invocation and (
+        review.state != "error" or only_description_too_long(review))

dokimi/dokimi/external/validate.py

@@ -27,9 +27,9 @@
 Mirrors are common (one skill checked in under two plugin directories), and
 counting them twice inflates routing and coverage numbers.
 
-Same name, different content: a skill whose ``name`` another listed skill shares, with other bytes, is a
-``warn`` (``name-conflict``) naming the other path. A listed skill is one a harness offers (no
-``disable-model-invocation``, no spec error). Routing leaves that name out of the field rather than aborting.
+Same name, different content: a skill whose ``name`` another listed skill shares, with other bytes, is a ``warn`` (``name-conflict``) naming the other path. A listed skill is one a harness offers (no
+``disable-model-invocation``, no spec error other than a description over 1024 characters; see ``dokimi.external.listing.in_field``).
+Routing leaves that name out of the field rather than aborting.
 
 ``disable-model-invocation: true`` is not a finding: the model is never offered
 that skill, so it is reported as a flag (the same reading as the skill arm's
@@ -72,6 +72,7 @@
 from dokimi.utils import parse_frontmatter_strict
 
 from .catalog import SkillCatalogEntry, _within
+from .listing import in_field
 
 NAME_PATTERN = re.compile(r"[a-z0-9]+(?:-[a-z0-9]+)*")
 NAME_MAX = 64
@@ -778,13 +779,12 @@ def _name_conflicts(by_name: dict[str, list[tuple[SkillReview, str, str]]]) -> N
     """A warning on each listed skill that shares its name with a listed skill of different content.
 
     Listed means a harness offers it: a skill with ``disable-model-invocation`` or a spec error is
-    left out of the routing field by default, so its copy is no conflict. Identical copies are not
+    left out of the routing field by default (see ``in_field``), so its copy is no conflict. Identical copies are not
     a conflict (they are the duplicate above). The routing field skips such a name, so the scan
     names both paths to say which skills are affected.
     """
     for name, group in by_name.items():
-        group = [entry for entry in group
-                 if not entry[0].disable_model_invocation and entry[0].state != "error"]
+        group = [entry for entry in group if in_field(entry[0])]
         for review, _, digest in group:
             others = sorted({other for _, other, other_digest in group if other_digest != digest})
             if not others:

dokimi/dokimi/novelty/routing.py

@@ -41,6 +41,7 @@
 from pathlib import Path
 from typing import TYPE_CHECKING
 
+from dokimi.external.listing import only_description_too_long
 from dokimi.skills import Skill
 from dokimi.utils import InputError, parse_frontmatter
 
@@ -482,11 +483,30 @@ def non_skill_reason(ref: str) -> str | None:
     return None
 
 
+def over_limit_warnings(competitors: list[Candidate], *, under_test: bool = False) -> list[dict]:
+    """One record per skill whose description is over the 1024-character limit.
+
+    ``competitors`` are skills of the field, or, with ``under_test``, the skill under test,
+    which the message names as such.
+    """
+    from dokimi.external.validate import DESCRIPTION_MAX
+
+    kept = ("the skill under test, measured with its full description" if under_test
+            else "kept in the field with its full description")
+    return [{"skill": c.name, "length": len(c.description), "limit": DESCRIPTION_MAX,
+             "message": f"description is {len(c.description)} characters, over the {DESCRIPTION_MAX} "
+                        f"limit; {kept} (harnesses differ: "
+                        "pi.dev rejects the skill, Claude Code truncates past about 1,536 "
+                        "characters)"}
+            for c in competitors if len(c.description) > DESCRIPTION_MAX]
+
+
 def competitor_left_out(ref: str, *, include_disabled: bool, include_invalid: bool) -> str | None:
     """Why an explicit ``skill:`` competitor is left out of the field, as the note says, or None.
 
     The same two rules as a catalog skill: a ``disable-model-invocation`` skill (unless
-    ``include_disabled``) and a skill with a spec error (unless ``include_invalid``).
+    ``include_disabled``) and a skill with a spec error (unless ``include_invalid``). A
+    description over 1024 characters, as the only spec error, does not leave a skill out.
     """
     from dokimi.external.catalog import SkillCatalogEntry
     from dokimi.external.validate import review_skills
@@ -509,7 +529,7 @@ def competitor_left_out(ref: str, *, include_disabled: bool, include_invalid: bo
     entry = SkillCatalogEntry(name=skill.name, status=None, path=path.parent,
                               source_root=root, skill=skill)
     review = review_skills([entry])[0]
-    if review is None or review.state != "error":
+    if review is None or review.state != "error" or only_description_too_long(review):
         return None
     problems = "; ".join(f.message for f in review.findings if f.level == "error")
     return f"spec error ({problems}; --include-invalid keeps it)"
@@ -578,8 +598,9 @@ def catalog_competitors(root: str, exclude: set[str],
     the catalog has. A declared skill that cannot be loaded is left out and described in
     ``notes`` (when given) rather than aborting the run. A skill with
     ``disable-model-invocation`` is left out too, unless ``include_disabled``. A skill with a
-    spec error (``catalog check`` reports it as an error, e.g. a description over 1024
-    characters) is left out too, unless ``include_invalid``. A mirror, a byte-identical
+    spec error (``catalog check`` reports it as an error, e.g. a name that is not its directory
+    name) is left out too, unless ``include_invalid``. A description over 1024 characters, as the
+    only error, keeps the skill (harnesses differ on it; see ``only_description_too_long``). A mirror, a byte-identical
     ``SKILL.md`` under another path (a plugin copy of the ``skills/`` tree), is left out too:
     the copy nearest the catalog root stands for the card. Disabled and spec-error copies are
     filtered according to the flags first. A name whose remaining copies differ in content leaves
@@ -650,7 +671,8 @@ def _left_out_by_flag(skill: Skill, review: SkillReview, include_disabled: bool,
     The note names the first rule; ``dropped`` (when given) gets every rule that applies.
     """
     disabled = not include_disabled and review.disable_model_invocation
-    invalid = not include_invalid and review.state == "error"
+    invalid = (not include_invalid and review.state == "error"
+               and not only_description_too_long(review))
     if dropped is not None and (disabled or invalid):
         dropped.setdefault(skill.name, set()).update(
             why for why, hit in (("disabled", disabled), ("invalid", invalid)) if hit)