Skip to content
Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
fcd8390
Sync living-doc glossary with canonical definitions, update project p…
miroslavpojer Sep 15, 2026
3351f4f
fix: align living-doc-update description with schema (Features no sta…
miroslavpojer Sep 15, 2026
d3d4d43
fix: clarify stale AC update location in gherkin-living-doc-sync
miroslavpojer Sep 15, 2026
fc9711a
feat: add standard artifact locations for BDD automation in copilot i…
miroslavpojer Sep 15, 2026
c609578
fix: update living-doc BDD schemas and related documentation for clar…
miroslavpojer Sep 16, 2026
c095dff
fix: update expected output formatting in evals and enhance deprecati…
miroslavpojer Sep 16, 2026
429db05
fix: update surface type validation to only allow UI and API, and adj…
miroslavpojer Sep 16, 2026
f525f3f
fix: refine expected output for Notifications Service feature and upd…
miroslavpojer Sep 16, 2026
c6c9d48
fix: update glossary sync script to use environment variable for refe…
miroslavpojer Sep 16, 2026
96c58bd
fix: enhance living-doc BDD schemas and documentation for clarity, ad…
miroslavpojer Sep 16, 2026
7607373
fix: update expected output and validation details in evals for livin…
miroslavpojer Sep 16, 2026
377c154
fix: update descoped scenario handling in documentation and evals, cl…
miroslavpojer Sep 16, 2026
5a02872
fix: enhance validation for ac_states in profile, ensuring schema com…
miroslavpojer Sep 16, 2026
22f3404
fix: add validation to prevent duplicate ac_states in profile and enh…
miroslavpojer Sep 16, 2026
822aad2
fix: enhance validation for profile ac_states to reject non-string it…
miroslavpojer Sep 16, 2026
e7a8e00
fix: add tests to reject duplicate values in profile ac_states and up…
miroslavpojer Sep 16, 2026
602a738
fix: update living doc guidelines to clarify API surface types and en…
miroslavpojer Sep 16, 2026
d5187ca
fix: enhance living doc guidelines for AC versioning and improve vali…
miroslavpojer Sep 16, 2026
2378d51
fix: update living doc guidelines and improve validation for user sto…
miroslavpojer Sep 16, 2026
748b767
fix: enhance error handling in profile loading to catch malformed YAML
miroslavpojer Sep 16, 2026
1ae73b5
fix: update documentation and tests for validation and coverage impro…
miroslavpojer Sep 17, 2026
d1ec392
fix: enforce error handling for unrecognized AC states in validation …
miroslavpojer Sep 17, 2026
0318a13
fix: update living doc guidelines and JSON structure for User Stories…
miroslavpojer Sep 17, 2026
ff2e547
fix: enhance validation for deprecated features to include deprecated…
miroslavpojer Sep 17, 2026
4b46118
fix: update living doc guidelines and JSON structure for Feature and …
miroslavpojer Sep 17, 2026
fad91af
Updated from consistency review.
miroslavpojer Sep 26, 2026
412a098
feat: generate both canon copies from one SHA-pinned sync mechanism
miroslavpojer Sep 28, 2026
89f0eb6
Updated to introduce news from canon.
miroslavpojer Sep 28, 2026
2ef9ff0
feat: add dependency update path and feature_dependencies checks; ali…
miroslavpojer Sep 29, 2026
5d356b3
feat: add dependency update path and feature_dependencies checks; ali…
miroslavpojer Sep 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/agents/evals/living-doc-bdd-copilot/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"id": 3,
"category": "happy-path",
"prompt": "Generate Gherkin scenarios for US-007 — Place an Online Order. ACs: (1) active — happy path: customer places order with saved payment. (2) active — error: order rejected when card declined.",
"expected_output": "Agent generates a .feature file named us-007-place-an-online-order.feature. Feature header uses the As-a/I-can/so-that narrative from US-007. Two scenarios generated — one per active AC. Each Scenario: is immediately preceded by a '# AC: US-007-0n (v1.0.0 – active) — ...' traceability tag. Step text uses domain language (no HTTP calls, selectors, or DB). For steps without a matching step definition, generates stubs: Case A (PageObject method exists) = full stub; Case B (no PageObject method) = stub with NotImplementedError and flag to extend the PageObject.",
"expected_output": "Agent generates a .feature file named us-007-place-an-online-order.feature. Feature header uses the As-a/I-can/so-that narrative from US-007. Two scenarios generated - one per active AC. Each Scenario: is immediately preceded by a '# AC:US-007-0n (v1.0.0 - active) - ...' traceability tag. Step text uses domain language (no HTTP calls, selectors, or DB). For steps without a matching step definition, generates stubs: Case A (PageObject method exists) = full stub; Case B (no PageObject method) = stub with NotImplementedError and flag to extend the PageObject.",
"files": [],
"expectations": [
"Feature file named us-007-place-an-online-order.feature",
Expand Down Expand Up @@ -215,7 +215,7 @@
"id": 16,
"category": "skill-dispatch",
"prompt": "The @AC: traceability tags in checkout.feature are out of sync with the living doc. Sync them.",
"expected_output": "Agent loads the `gherkin-living-doc-sync` skill. Runs scan_ac_links.py to audit the @AC: tags and # AC: comments across checkout.feature. For each scenario: (1) verifies the @AC: tag matches a live AC in the catalog; (2) checks the # AC: comment format is canonical; (3) flags stale or missing links. Produces a sync diff showing what changed. Does not generate new scenarios — routes new-scenario requests to living-doc-scenario-creator.",
"expected_output": "Agent loads the `gherkin-living-doc-sync` skill. Runs scan_ac_links.py to audit the @AC: tags and # AC: comments across checkout.feature. For each scenario: (1) verifies the @AC: tag matches a live AC in the catalog; (2) checks the # AC: comment format is canonical; (3) flags stale or missing links. Produces a sync diff showing what changed. Does not generate new scenarios - routes new-scenario requests to living-doc-scenario-creator.",
"files": [],
"expectations": [
"Loads gherkin-living-doc-sync skill",
Expand Down Expand Up @@ -261,7 +261,7 @@
"files": [],
"expectations": [
"Loads bdd-maintain skill",
"REMOVE mode — identifies scenarios via # AC: tags linked to deprecated Feature",
"REMOVE mode - identifies scenarios via # AC: tags linked to deprecated Feature",
"Runs all three audit scripts to identify exclusively-used artifacts",
"Checks fixtures.ts for PageObject imports",
"Presents deletion list for confirmation before touching any file"
Expand All @@ -271,7 +271,7 @@
"id": 20,
"category": "skill-dispatch",
"prompt": "Document the Notifications Service as a Feature entity in the living doc — it exposes a REST API at /api/notifications.",
"expected_output": "Agent loads the `living-doc-create-feature` skill. Assigns the next sequential FEAT-nnn ID using next_id.py. Creates a Feature entity JSON with: id, name ('Notifications Service'), type ('api'), route ('/api/notifications'), status ('active'), owners (asks user), and empty functionalities and user_stories arrays. Adds the Feature to feature_registry.json. Prompts: 'Do you want to create Functionality entities for specific behaviors of this service?'",
"expected_output": "Agent loads the `living-doc-create-feature` skill. Assigns the next sequential FEAT-nnn ID using next_id.py. Creates a Feature entity JSON with: id, name ('Notifications Service'), surface_type ('API'), purpose, owners (asks user), external_dependencies, and provisional functionalities/user_stories arrays seeded with placeholder IDs rather than left empty — the prompt does not say the links are unknown — no `status` field, since a Feature's state is derived from its Functionalities. Adds the Feature to feature_registry.json. Prompts: 'Do you want to create Functionality entities for specific behaviors of this service?'",
"files": [],
"expectations": [
"Loads living-doc-create-feature skill",
Expand Down Expand Up @@ -314,7 +314,7 @@
"id": 23,
"category": "skill-dispatch",
"prompt": "Generate BDD scenarios for US-007 — Place an Online Order. It has 3 active ACs and 1 planned AC.",
"expected_output": "Agent loads the `living-doc-scenario-creator` skill. Generates a .feature file for US-007 with 3 scenarios — one per active AC. The planned AC is skipped (not generated until active). Each scenario is preceded by a '# AC: US-007-0n (v1.0.0 – active)' traceability comment. Merge policy: if a scenario already exists for an AC, applies the 4-row decision table (skip if intent matches, update if GWT is stale, propose replacement if deprecated, flag if multiple scenarios exist per AC).",
"expected_output": "Agent loads the `living-doc-scenario-creator` skill. Generates a .feature file for US-007 with 3 scenarios - one per active AC. The planned AC is skipped (not generated until active). Each scenario is preceded by a '# AC:US-007-0n (v1.0.0 - active)' traceability comment. Merge policy: if a scenario already exists for an AC, applies the 4-row decision table (skip if intent matches, update if GWT is stale, propose replacement if deprecated, flag if multiple scenarios exist per AC).",
"files": [],
"expectations": [
"Loads living-doc-scenario-creator skill",
Expand Down
8 changes: 4 additions & 4 deletions .github/agents/living-doc-bdd-copilot.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,15 +178,15 @@ When a User Story or Feature is deprecated, three skills fire in sequence. Compl

| Step | Skill | Action |
|---|---|---|
| 1 | `living-doc-update` | Set entity `status: deprecated`; add `deprecated_at`, `deprecation_reason`, and optionally `superseded_by` |
| 1 | `living-doc-update` | For a User Story, set `status: deprecated`. A Feature has no `status` field — its state is derived from its Functionalities; deprecate the Feature by deprecating every Functionality it owns instead. Either way, add `deprecated_at`, `deprecation_reason`, and optionally `superseded_by` |
Comment thread
miroslavpojer marked this conversation as resolved.
Resolved
| 2 | `gherkin-living-doc-sync` | Find all scenarios tagged `@AC:<id>` for the deprecated entity's ACs; add `@deprecated` and `@review-needed` |
| 3 | `bdd-maintain` (REMOVE) | Confirm file deletion list with user; remove confirmed `.feature` files, PageObjects, and step definitions; update `manifest.json` |

Do not skip steps or run them out of order. Complete catalog changes (step 1) before touching any Gherkin or automation files.

**Manifest loading rule:** Use targeted line ranges for the current route(s). Load full manifest only for RE-SCAN. `seed.yaml`: always load in full. When PageObject generation discovers a route with no linked Feature entity, set `feature_id: FEAT-UNKNOWN`, flag the route as needing a Feature entity, and cross-load `living-doc-create-feature` to create it before continuing.

**living-doc-bdd-schemas:** Load [skills/shared/references/living-doc-bdd-schemas.md](skills/shared/references/living-doc-bdd-schemas.md) only when generating or validating feature file headers, PageObject headers, ExplorationFixture entries, seed.yaml form_fixtures, or manifest.json route entries.
**living-doc-bdd-schemas:** Load [skills/shared/references/living-doc-bdd-schemas.md](skills/shared/references/living-doc-bdd-schemas.md) only when generating or validating feature file headers, PageObject headers, seed.yaml form_fixtures, or manifest.json route entries.

---

Expand Down Expand Up @@ -275,12 +275,12 @@ Full model: [living-doc-glossary](skills/shared/references/living-doc-glossary.m

**Entity IDs:** `US-<nnn>` · `FEAT-<nnn>` · `FUNC-<nnn>`

**AC reference format:** `AC:<parent-id>-<nn> (v<version> – <state>) — <description>`
**AC reference format:** `AC:<parent-id>-<nn> (v<version> - <state>) - <description>`
State: `planned | in_review | active | deprecated`

**Gherkin traceability:** every scenario in the living-doc feature directories (`feature_dirs.user_story` and `feature_dirs.functionality` from the Project Profile, defaults `features/liv_doc_us/` and `features/liv_doc_func/`) requires:
```gherkin
# AC:US-1-01 (v1.0.0 - active) — <description>
# AC:US-1-01 (v1.0.0 - active) - <description>
@AC:US-1-01
Scenario: ...
```
Expand Down
16 changes: 16 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,22 @@ See [Token-Saving Skill](../docs/guides/token-saving.md) for details, including

---

## BDD Artifacts

When working with living documentation and BDD automation, use these standard artifact locations:
- **Business Seed:** `.copilot/bdd/seed.yaml` — App configuration, routes, test users, form fixtures
- **Exploration Manifest:** `.copilot/bdd/manifest.json` — Discovered surfaces, PageObject paths, UI elements
- **Project Profile:** `.copilot/bdd/.project-profile.yaml` — BDD conventions for this project
- **Feature Files (User Story):** `features/liv_doc_us/` — E2E scenarios linked to User Stories
- **Feature Files (Functionality):** `features/liv_doc_func/` — System-test scenarios linked to Functionalities
- **PageObjects:** `playwright/pages/` — Locators and page interaction methods
- **Step Definitions:** `playwright/steps/` — Gherkin step implementations
- **Living Doc Catalog:** `docs/living-doc/` — User Stories, Features, Functionalities, and Acceptance Criteria

These paths are configurable in `.copilot/bdd/.project-profile.yaml`.

---

## PR Review

When reviewing a pull request, load and apply:
Expand Down
35 changes: 34 additions & 1 deletion .github/workflows/test-scripts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,45 @@ on:
pull_request:
paths:
- "skills/**/*.py"
- "skills/shared/references/schemas/*.schema.json"
- "scripts/**"
- "skills/shared/references/living-doc-glossary.md"
Comment thread
miroslavpojer marked this conversation as resolved.
Resolved
- ".github/workflows/test-scripts.yml"

permissions:
contents: read

jobs:
glossary-sync-check:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Extract synced ref from the glossary header
id: ref
run: |
set -euo pipefail
header="$(head -n1 skills/shared/references/living-doc-glossary.md)"
ref="$(echo "$header" | sed -nE 's/.*synced from AbsaOSS\/living-doc@([^ ]+).*/\1/p')"
if [ -z "$ref" ]; then
echo "Error: could not find a 'synced from AbsaOSS/living-doc@<ref>' header comment on line 1 of the glossary." >&2
exit 1
fi
echo "ref=$ref" >> "$GITHUB_OUTPUT"

- name: Re-run the sync script and diff against the committed copy
run: |
set -euo pipefail
cp skills/shared/references/living-doc-glossary.md /tmp/committed-glossary.md
bash scripts/sync-living-doc-glossary.sh "${{ steps.ref.outputs.ref }}"
if ! diff -u /tmp/committed-glossary.md skills/shared/references/living-doc-glossary.md; then
echo
echo "skills/shared/references/living-doc-glossary.md is out of sync with AbsaOSS/living-doc@${{ steps.ref.outputs.ref }}." >&2
echo "Run: scripts/sync-living-doc-glossary.sh ${{ steps.ref.outputs.ref }}" >&2
exit 1
fi
Comment thread
miroslavpojer marked this conversation as resolved.
Resolved

run-tests:
runs-on: ubuntu-latest
steps:
Expand All @@ -22,7 +55,7 @@ jobs:
python-version: "3.11"

- name: Install dependencies
run: pip install pyyaml
run: pip install pyyaml jsonschema

- name: Discover and run test_*.py scripts
env:
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/agent-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,12 +234,12 @@ IDs are stable — never change an ID after creation. Bump the `version` field f
### Gherkin traceability tag format

```gherkin
# AC:US-007-01 (v1.0.0 - active) — <description>
# AC:US-007-01 (v1.0.0 - active) - <description>
@AC:US-007-01
Scenario: ...
```

One `# AC:` + `@AC:` pair per AC. The `@AC:` tag is the machine-readable traceability anchor — never delete or rename it without syncing the catalog entity.
One `# AC:` + `@AC:` pair per AC. The `@AC:` tag is the machine-readable traceability anchor - never delete or rename it without syncing the catalog entity.

### Cooperating agent boundary

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/gherkin-living-doc-sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The skill audits traceability and produces sync reports:
|--------|-------------|
| **AC link audit** | Finds missing `@AC:` tags, stale links, orphaned scenarios |
| **Step text drift** | Flags scenarios where step text diverged from AC descriptions |
| **Deprecation tagging** | Tags scenarios `@deprecated` when ACs are descoped |
| **Deprecation / descope tagging** | Tags scenarios `@deprecated` + `@review-needed` when their AC is deprecated, or `@wip` + `@review-needed` when their AC is descoped back to `planned` |
| **AC change propagation** | Maps AC edits back to affected scenarios |

---
Expand Down
8 changes: 4 additions & 4 deletions docs/guides/living-doc-update.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The skill updates and validates entity changes:
| Operation | Effect |
|-----------|--------|
| **Add/modify/remove AC** | Version tracked; may trigger scenario sync |
| **Change status** | draft → ready → in_review → deprecated |
| **Change status** | planned → in_review → active → deprecated (User Story / Functionality only — a Feature has no status field) |
| **Change ownership** | Update team responsible for entity |
| **Link entities** | Connect User Stories to Features, Functionalities to Features |
| **Deprecate entity** | Tag dependent scenarios `@deprecated`; cleanup triggered |
Expand Down Expand Up @@ -40,9 +40,9 @@ change entity status
|------|------------------|
| **Add AC** | None — create scenarios via [Living Doc Scenario Creator](./living-doc-scenario-creator.md) |
| **Modify AC** | [Gherkin ↔ Living Doc Sync](./gherkin-living-doc-sync.md) to sync scenarios |
| **Descope AC** | Gherkin sync tags scenarios `@deprecated` |
| **Descope AC** | AC drops back to `planned` (no `descoped` state); Gherkin sync tags scenarios `@wip` + `@review-needed` |
| **Deprecate entity** | [BDD Maintain](./bdd-maintain.md) removes test files |
| **Change status to ready** | Scenario creation enabled |
| **Change status to active** | Scenario creation enabled |

---

Expand All @@ -66,7 +66,7 @@ One Python utility available in `skills/living-doc-update/scripts/`:

This skill has been validated with **16 test cases** covering:
- Adding/modifying/removing ACs
- Status transitions (draft → ready → deprecated)
- Status transitions (planned → in_review → active → deprecated)
- Ownership changes and notifications
- Entity linking workflows
- Propagation to downstream skills
75 changes: 75 additions & 0 deletions scripts/sync-living-doc-glossary.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
# sync-living-doc-glossary.sh — pull the canon glossary from AbsaOSS/living-doc and write the
# local copy that agentic-toolkit's skills load.
#
# The canon lives at living-doc's docs/guides/living-doc-glossary.md today. A later living-doc
# change relocates guides under docs/reference/ — when that lands, update SOURCE_PATH below (and
# the link-rewriting rules, which assume the source and its cross-references live under
# docs/guides/ and docs/examples/).
#
# Usage:
# scripts/sync-living-doc-glossary.sh <living-doc-ref>
#
# <living-doc-ref> is any git ref in AbsaOSS/living-doc: a commit SHA (preferred, for
# reproducibility), a tag, or a branch name.
#
# Writes: skills/shared/references/living-doc-glossary.md
# Exits 1 if the ref is missing, the fetch fails, or the fetched file is empty.

set -euo pipefail

REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
DEST="${REPO_ROOT}/skills/shared/references/living-doc-glossary.md"
SOURCE_PATH="docs/guides/living-doc-glossary.md"
LIVING_DOC_REPO="AbsaOSS/living-doc"

if [ "${1:-}" = "" ]; then
echo "Usage: $0 <living-doc-ref>" >&2
echo " <living-doc-ref>: a commit SHA, tag, or branch in ${LIVING_DOC_REPO}" >&2
exit 1
fi
REF="$1"

RAW_URL="https://raw.githubusercontent.com/${LIVING_DOC_REPO}/${REF}/${SOURCE_PATH}"

TMP_FILE="$(mktemp)"
trap 'rm -f "${TMP_FILE}"' EXIT

echo "Fetching ${SOURCE_PATH} from ${LIVING_DOC_REPO}@${REF} ..." >&2
if ! curl -fsSL --connect-timeout 10 --max-time 30 --retry 2 "${RAW_URL}" -o "${TMP_FILE}"; then
echo "Error: failed to fetch ${RAW_URL}" >&2
echo "Check that the ref exists and ${SOURCE_PATH} is still the canon glossary path." >&2
exit 1
fi

if [ ! -s "${TMP_FILE}" ]; then
echo "Error: fetched file is empty — refusing to overwrite ${DEST}" >&2
exit 1
fi

# Rewrite the canon's relative links to absolute living-doc URLs so the synced copy reads
# correctly from inside agentic-toolkit. Same-page anchors (#section) are left untouched.
# Known relative link shapes in the source file:
# - "living-doc-header-types.md" / "living-doc-document-types.md" — sibling under docs/guides/
# - "../examples/..." — under docs/examples/
#
# REF is attacker-controllable input (any branch/tag name) and lands in the replacement side
# of the sed commands below. Escape backslash, ampersand (sed's "insert the match" token), and
# the "|" delimiter those commands use, so a ref like "release&docs" can't corrupt the rewritten
# links or break the command.
REF_SED_SAFE="$(printf '%s' "${REF}" | sed -e 's/[\&|]/\\&/g')"
BLOB_BASE="https://github.com/${LIVING_DOC_REPO}/blob/${REF_SED_SAFE}"
sed -E \
-e "s|\(living-doc-header-types\.md|(${BLOB_BASE}/docs/guides/living-doc-header-types.md|g" \
-e "s|\(living-doc-document-types\.md|(${BLOB_BASE}/docs/guides/living-doc-document-types.md|g" \
-e "s|\(\.\./examples/|(${BLOB_BASE}/docs/examples/|g" \
"${TMP_FILE}" > "${TMP_FILE}.rewritten"
mv "${TMP_FILE}.rewritten" "${TMP_FILE}"

{
echo "<!-- synced from ${LIVING_DOC_REPO}@${REF} — run scripts/sync-living-doc-glossary.sh ${REF} to refresh -->"
echo
cat "${TMP_FILE}"
} > "${DEST}"

echo "Wrote $(wc -l < "${DEST}") lines to ${DEST#${REPO_ROOT}/}" >&2
2 changes: 1 addition & 1 deletion skills/bdd-maintain/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ All three scripts exit `0` on clean, `1` on findings, `2` on bad arguments — s
**Recommended script order for a full audit:** run in the sequence steps → PO methods → PO components. Deleting unused steps can expose unused PO methods; deleting unused PO methods can then expose unused PO classes. Running in this order ensures each pass builds on the previous one rather than missing transitively dead code.

**Unused step def — distinguish before deleting:**
- If the step belongs to a **deprecated entity** (the US/Feature has `status: deprecated` in the catalog), delete it — the coverage it provided is no longer needed.
- If the step belongs to a **deprecated entity** — a User Story with `status: deprecated`, or a Feature whose Functionalities are all deprecated (a Feature carries no `status` field) — delete it, the coverage it provided is no longer needed.
- If the step belongs to an **active entity** but has no exercising scenario, it is a stale draft or an orphan; flag it for team review before deleting. Someone may be about to add a scenario for it.
- Never delete without first verifying the step is not imported or re-exported by another step file — grep for the step file name as an import target as well.

Expand Down
Loading
Loading