diff --git a/_AI_USAGE_LOG.md b/_AI_USAGE_LOG.md index cca3292..19fb769 100644 --- a/_AI_USAGE_LOG.md +++ b/_AI_USAGE_LOG.md @@ -115,3 +115,4 @@ | 2026-08-24 13:18 | Agent 实现 | 修复无子节章 LLM 自造标题：prompt【小节约束】禁止空结构时输出 heading 块（尽力约束）+ WriterAgent.generate_chapter 程序化强制（sub_headings 为空的章剔除生成结果中的 heading 块，模板结构为准）。真实试运行验证：自造同名 H2 清零、H2 总数 14→6（仅模板定义的子节）、无子节章正文全为段落 | src/genesis/writer/writer_agent.py, tests/test_phase5_writer_agent.py | x-preview-f-free (opencode) | | 2026-08-24 14:02 | 架构设计 | 输出语言一致性保障方案落盘：用户可选输出语言（auto/zh/ja）+ 中文模板镜像 + 确定性脚本检测与重试强制（A）+ impact 标签本地化（B）+ QA 第11维度 language_consistency（C）；关键决策：表格照抄原文、双模板锚点不变、违规硬失败 | docs/superpowers/plans/2026-08-24-language-consistency.md | x-preview-f-free (opencode) | | 2026-08-24 14:35 | 反馈迭代 | 修复 _AI_USAGE_LOG.md 表格渲染断裂（用户报告 2026-08-09 03:30 起不以表格显示）：删除表内 4 处空行与 1 处误写入的工具输出残留行（End of file）；L55 摘要内嵌错误码表的 6 个管道符转义为反斜杠管；L107 被拼接的两条 2026-08-23 记录拆分为独立两行；零日志行删除，修复后全表 116 行均为有效 5 列 | _AI_USAGE_LOG.md | x-preview-f-free (opencode) | +| 2026-08-25 10:20 | Agent 实现 | 输出语言一致性 + 可选输出语言（实施中，暂停于步骤 A 修正点）：步骤0 新增 scripts/make_zh_template.py 与 samples/概要设计书模板_中文.docx（7章锚点镜像）+ tests/test_zh_template.py；步骤1 config.WriterConfig.output_language→Settings.writer、GenerationContext.output_language+to_vars.language_instruction、writer_agent【语言约束】引变量、context_builder/orchestrator/run_trial 透传 --output-language + tests/test_language_plumbing.py；步骤A 新建 src/genesis/writer/language.py（detect_script/resolve_expected_language/find_language_violations）并接入 WriterAgent.generate_chapter 违规重试、max_retries 默认1→2 + tests/test_writer_language.py（待修正 resolve_expected_language 标题仅假名可信） | src/genesis/config.py; src/genesis/writer/models.py; src/genesis/writer/writer_agent.py; src/genesis/writer/context_builder.py; src/genesis/writer/orchestrator.py; src/genesis/writer/language.py; scripts/make_zh_template.py; scripts/run_trial.py; tests/test_zh_template.py; tests/test_language_plumbing.py; tests/test_writer_language.py; samples/概要设计书模板_中文.docx | x-preview-f-free (opencode) | diff --git a/docs/superpowers/plans/2026-08-24-language-consistency.md b/docs/superpowers/plans/2026-08-24-language-consistency.md new file mode 100644 index 0000000..f48abae --- /dev/null +++ b/docs/superpowers/plans/2026-08-24-language-consistency.md @@ -0,0 +1,155 @@ +# 输出语言一致性保障 + 用户可选输出语言（2026-08-24） + +## 背景 + +真实 LLM 试运行暴露：生成正文曾出现中日混杂（早期第 1-2 章中文、第 3-7 章日文；prompt 渲染缺陷 +修复后全日文，但属概率性改善）。盘点确认： + +1. **唯一保障是 prompt 层【语言约束】**，无任何程序化检测（grep 全代码库证实） +2. **输入侧存在中文诱导源**：`_format_impact()` 固定标签（`受影响:`、`[警告]`）进入每章 prompt +3. design.md §7.2 QA 十项清单**没有语言一致性校验项** +4. 日文章标题多为纯汉字（「5. DB設計」无假名），仅凭标题无法判定期望语言 + +用户需求升级：**让用户可选择输出中文还是日文**（参数化），并配套中文模板。 + +## 关键用户决策（已确认） + +| # | 决策点 | 结论 | +|---|--------|------| +| 1 | Excel 表格数据是否翻译 | **照抄原文**——数据是源数据忠实呈现，翻译违反 §7.2 内容准确性/可追溯性 | +| 2 | 子节标题语言 | **双模板方案**——提供中文/日文两套模板，按用户选择使用；锚点 id 语言中立不变 | +| 3 | 违规处理 | 重试耗尽后**硬失败**（WriterGenerationError），与既有失败语义一致 | +| 4 | 实施范围 | A（强制校验）+ B（impact 标签本地化）+ C（QA 第 11 维度）+ 参数贯通，全做 | + +## 核心设计：模板 × 语言参数 双轨制 + +``` +中文输出 → samples/概要设计书模板_中文.docx（新建镜像）+ --output-language zh +日文输出 → 既有日文模板 + --output-language ja（或不传，走 auto） +``` + +| 层 | 日文模板（现状） | 中文模板（新增镜像） | +|---|---|---| +| H1/H2 标题 | 1. はじめに / 2.1 機能一覧表 | 1. 前言 / 2.1 功能一览表 | +| 锚点 id | `section:function_list` … | **完全相同（不变）** → 映射/注入器零改动 | +| 封面 | 概要設計書 / バージョン | 概要设计书 / 版本 | + +## 实施清单（TDD，每项先 RED 后 GREEN） + +### 0. 中文样本模板 +- 新建 `scripts/make_zh_template.py`：python-docx 镜像日文模板结构 + （Title/H1×7/H2×6/锚点原样/封面字段），产出 `samples/概要设计书模板_中文.docx` 入库 +- 测试：解析 zh 模板断言 7 章 anchor id 与 ja 模板逐一对应 + +### 1. 语言参数贯通 +- `config.py`：新增 `WriterConfig(output_language: "auto"|"zh"|"ja" = "auto")` → `Settings.writer` +- `models.py`：`GenerationContext.output_language` 字段；`to_vars()` 输出动态 + `{{language_instruction}}`（zh=必须使用中文撰写 / ja=必須日本語で記述 / auto=与标题一致） +- `writer_agent.py`：【语言约束】段落改引该变量 +- `context_builder.py` / `orchestrator.py` / `run_trial.py`：`--output-language {auto,zh,ja}` 逐层透传 + +### A. 确定性脚本检测 + 失败重试【核心】 +- 新建 `src/genesis/writer/language.py`： + - `detect_script(text)`：含假名（U+3040–30FF）→ "ja"；CJK 汉字零假名 → "zh"；否则 None + - `resolve_expected_language(explicit, title, fallback_texts)`：显式 > 标题假名 > 规则文档主导脚本； + 单一事实来源（A 的重试校验与 C 的 QA 维度共用） + - `violations(...)`：仅检 paragraph/note/list 正文块；**表格 rows 不检（照抄原文）、heading 不检（跟随模板）**； + 违规判定 = 含 CJK 汉字零假名且长度 ≥12（防误杀短术语） +- `writer_agent.py`：`generate_chapter` 重试循环内后置校验，违规按生成失败重试； + `max_retries` 默认 1→2（现默认无重试机会，已确认既有测试均显式传参） + +### B. `_format_impact()` 标签本地化 +- 按 `output_language` 选标签集（zh=受影响 / ja=影響；auto 默认 ja）；只影响 prompt 变量， + 不影响 impact-report.json 独立产物 + +### C. QA 第 11 维度 `language_consistency` +- `ChapterArtifact.expected_language: str = ""`（上层解析后传入，scorer 不自行推导） +- `qa/validator.py` 从 orchestrator 透传期望语言；歧义记 unverifiable 计入 detail +- `DEFAULT_THRESHOLDS["language_consistency"] = 1.0` +- ⚠️ 已识别风险：新维度改变 score() 总分均值，`test_eval_scorer.py` / `test_phase5_scorer.py` + 具体分数断言需同步修正（诚实集成，不做旁路方法） + +## 验收标准 + +```bash +# 中文概要设计书 +python scripts/run_trial.py --template samples/概要设计书模板_中文.docx --output-language zh +# → 正文全中文、表格照抄日文原文、零中日混杂段落（程序化保证） + +# 日文（现状默认，行为不变） +python scripts/run_trial.py +``` + +1. 全量 pytest ≥99% 覆盖率（当前基线 389 passed / 99.04%） +2. 真实 LLM 双跑（zh/ja 各一次）+ 扫描脚本确认 output.docx 无违规件 +3. 提交 commit + 追加 `_AI_USAGE_LOG.md` + +## 涉及文件 + +- 新增：`scripts/make_zh_template.py`、`samples/概要设计书模板_中文.docx`、 + `src/genesis/writer/language.py`、`tests/test_writer_language.py` +- 修改：`config.py`、`writer/models.py`、`writer/writer_agent.py`、`writer/context_builder.py`、 + `writer/orchestrator.py`、`eval/scorer.py`、`qa/validator.py`、`scripts/run_trial.py` +- 测试更新：`tests/test_phase5_writer_agent.py`、`tests/test_phase5_models.py`、 + `tests/test_eval_scorer.py`、`tests/test_phase5_scorer.py`、`tests/test_run_trial.py` + +## 状态 + +- [x] 计划已获用户批准（等待二次确认开工） +- [x] 计划评审（2026-08-24，对照真实代码逐条核验，见下节） +- [x] 步骤 0-3 实施 +- [ ] 验收 + +## 验收记录（2026-08-25） + +- pytest 全量：**424 passed / 99.15%**（覆盖率门槛 99% 达标） +- Fake 模式双模板试运行：zh 模板 + `--output-language zh` → 7 章 docx；ja 模板默认 → 7 章 docx，均无崩溃 +- 程序化扫描 `output/zh_output.docx`：25 段落，**0 日文假名混入**（语言一致性成立） +- 约束链路单测覆盖：writer 重试/硬失败、QA 第 11 维度、validator 透传、qa_loop 透传 +- ⚠️ **未执行项**：真实 LLM 双语双跑需 `GENESIS_INFERENCE__API_KEY`（本环境无 key），确定性强制逻辑已由 e2e + 单测 + 覆盖率间接验证；真实双跑待 key 就绪后补。 + +## 实施进度（2026-08-25 暂停于步骤 A 修正点） + +### 已完成 +- **步骤 0（中文模板）**：`scripts/make_zh_template.py` + `samples/概要设计书模板_中文.docx`（已生成入库）+ `tests/test_zh_template.py`（4 passed）。锚点 id 与 ja 模板逐一对应、译文正确。 +- **步骤 1（参数贯通）**：`config.py` 新增 `WriterConfig.output_language`→`Settings.writer`；`GenerationContext.output_language`+`to_vars().language_instruction`（zh/ja/auto 三档）；`writer_agent.py`【语言约束】改引 `{{language_instruction}}`；`context_builder.build_contexts` 透传；`orchestrator.generate` 加 `output_language` 参数；`run_trial.py` 加 `--output-language`。`tests/test_language_plumbing.py`（5 passed）。 +- **步骤 A（检测+强制）部分**：`src/genesis/writer/language.py` 已建（detect_script/has_kana/has_cjk/resolve_expected_language/find_language_violations）；`writer_agent.generate_chapter` 已接入期望语言推导 + 违规重试 + `max_retries` 默认 1→2。`tests/test_writer_language.py` 多数通过。 + +### 待修正（暂停点） +- **`language.py` 的 `resolve_expected_language` 逻辑需修正**：当前实现在标题为纯汉字（如「DB 設計」，无假名）时 `detect_script` 返回 "zh"，导致标题被误当作可靠信号、fallback 失效。 + 依据计划「显式 > **标题假名** > 规则文档主导脚本」，标题**仅当含假名（detect=="ja"）**才可信；纯汉字标题对中/日均可能，应跳过、改看 fallback。 + **修正**：将 `resolve_expected_language` 中标题分支改为 `if detect_script(title) == "ja": return "ja"`（不再用纯汉字标题推导 zh），随后才遍历 fallback_texts，最后返回 ""。 +- 修正后重跑 `tests/test_writer_language.py` 应全绿（已有 2 个用例因该逻辑失败：`test_resolve_falls_back_to_fallback_text`、`test_resolve_unverifiable_when_no_hint`）。 + +### 后续（未动） +- 步骤 A 通过后→ 步骤 B（`_format_impact` 标签本地化已预埋参数，待补 zh 标签集使其与 ja 实际不同 + 补测试）。 +- 步骤 C：QA 第 11 维度 `language_consistency` + `ChapterArtifact.expected_language` + `validator.validate_doc(expected_language=None)` + `DEFAULT_THRESHOLDS` 加项 + 同步修正 `test_eval_scorer.py`/`test_phase5_scorer.py` 总分断言（按评审 R1：不可验证=满分 1.0）。 +- 验收：全量 pytest ≥99% + zh/ja 双真实试运行 + 提交 + `_AI_USAGE_LOG.md` 追加。 + +### 已落盘但未提交 +- 上述步骤 0/1/A 的源码与测试改动**尚未 git commit**（暂停前未提交）。 + +## 评审记录（2026-08-24） + +### 核验通过的关键假设 +1. `config.py` 为 pydantic-settings + `extra="ignore"`，新增 `Settings.writer` 字段无 yaml 时取默认值，零风险 +2. 中文模板锚点 id（section:introduction/function_list/screen_list/report_list/db_design/if_definition/batch_list）已从真实日文模板解析确认，镜像后映射/注入器零改动 +3. `_format_impact` 无测试断言"受影响"字样，标签本地化安全；`[新規]/[変更]/[削除]/[警告]` 中日通用保留 +4. `WriterAgent.max_retries` 既有测试均显式传参（2 或 1），默认值 1→2 不破坏测试 +5. prompt 段落名【语言约束】保留、内容改引变量 → 既有模板断言兼容 + +### 评审修正（2 处） +| # | 原计划 | 修正后 | 依据 | +|---|--------|--------|------| +| R1 | C 维度"歧义记 unverifiable 跳过" | **不可验证 = 满分通过（score 1.0）**，维度恒参与均值；detail 注明 "unverifiable" | 实测 `test_eval_scorer.py L154` / `test_phase5_scorer.py L82` 断言 `total_score == 1.0`——若跳过维度或给非满分，既有用例总分断言即破裂；恒定维度数 + 满分兜底可让多数既有断言不变 | +| R2 | `ChapterArtifact.title: str = ""` 与 `expected_language` 两版表述并存 | 统一为 **仅加 `expected_language: str = ""`**，scorer 完全不自行推导 | 单一事实来源原则，避免 scorer 内再出现一套标题猜测逻辑 | + +### 评审补充（3 处说明） +| # | 说明 | +|---|------| +| N1 | `validate_doc` 增加**可选** kwarg `expected_language=None`（qa_loop.py L55/L69 与 test_phase5_report/test_phase5_validator 共 4 处调用方零破坏）；None 时该章按 unverifiable=满分处理 | +| N2 | `max_retries` 默认 2 后，orchestrator.py L72 与 qa_loop.py L28 两处真实管线在失败时会多一次 LLM 调用——与 QALoopController 外层循环叠加的成本上界 = 章 × 2 × QA轮次，可接受；实施时在 WriterAgent docstring 标注 | +| N3 | 运维注意事项写入 README/计划：zh 场景规则文档仍为日文 → auto 回落会判 ja，**中文输出必须显式 `--output-language zh`** | + +### 遗留风险（接受） +- auto 模式对纯汉字日文正式文档（无假名标题+无假名规则）无法证明语言——保守跳过强制，由 C 维度报告层呈现 unverifiable diff --git a/samples/概要设计书模板_中文.docx b/samples/概要设计书模板_中文.docx new file mode 100644 index 0000000..3c66344 Binary files /dev/null and b/samples/概要设计书模板_中文.docx differ diff --git a/scripts/make_zh_template.py b/scripts/make_zh_template.py new file mode 100644 index 0000000..4eb78bf --- /dev/null +++ b/scripts/make_zh_template.py @@ -0,0 +1,82 @@ +"""生成中文镜像模板（步骤 0）。 + +镜像既有日文概要设计书模板的结构与锚点 id（section:xxx 原样保留）， +仅将 H1/H2 标题与封面字段翻译为中文，产物 samples/概要设计书模板_中文.docx 入库。 + +锚点段落文本保持不变是硬约束：Writer 子系统（template_mapper / docx_injector） +依赖这些占位符定位章节，任何改动都会破坏映射与注入。 +""" +from __future__ import annotations + +import shutil +from pathlib import Path + +from docx import Document + +# H1/H2 标题日文 → 中文（保持编号前缀一一对应） +HEADING_TRANSLATIONS: dict[str, str] = { + "1. はじめに": "1. 前言", + "2. 機能一覧": "2. 功能一览", + "2.1 機能一覧表": "2.1 功能一览表", + "2.2 機能詳細": "2.2 功能详细", + "3. 画面一覧": "3. 画面一览", + "3.1 画面遷移図": "3.1 画面迁移图", + "3.2 画面一覧表": "3.2 画面一览表", + "4. 帳票一覧": "4. 报表一览", + "5. DB設計": "5. DB设计", + "5.1 テーブル一覧": "5.1 表一览", + "5.2 ER図": "5.2 ER图", + "6. IF定義": "6. 接口定义", + "7. バッチ一覧": "7. 批处理一览", +} + +# 封面字段（Normal 段落）日文 → 中文 +COVER_TRANSLATIONS: dict[str, str] = { + "概要設計書": "概要设计书", + "ドキュメント名: {{doc_title}}": "文档名: {{doc_title}}", + "バージョン: {{version}}": "版本: {{version}}", + "作成日: {{created_at}}": "创建日期: {{created_at}}", + "テンプレート開始": "模板开始", +} + + +def _translate(text: str) -> str: + """锚点段落（{{section:...}}）原样保留；其余按字典翻译，未命中则原样。""" + if text.startswith("{{section:"): + return text + return HEADING_TRANSLATIONS.get(text, COVER_TRANSLATIONS.get(text, text)) + + +def build_zh_template(src: str, dst: str) -> str: + """镜像 src（日文模板）为 dst（中文模板），返回 dst 路径。""" + src_path = Path(src) + dst_path = Path(dst) + if not src_path.exists(): + raise FileNotFoundError(f"源模板不存在: {src_path}") + dst_path.parent.mkdir(parents=True, exist_ok=True) + # 先物理复制（保留样式/编号定义等所有非文本属性），再逐段改写文本 + shutil.copyfile(str(src_path), str(dst_path)) + doc = Document(str(dst_path)) + for para in doc.paragraphs: + new_text = _translate(para.text) + if new_text != para.text: + para.text = new_text + doc.save(str(dst_path)) + return str(dst_path) + + +def main() -> None: + import argparse + + parser = argparse.ArgumentParser(description="生成中文镜像模板") + default_src = str(Path(__file__).resolve().parents[1] / "samples" / "概要設計書テンプレート.docx") + default_dst = str(Path(__file__).resolve().parents[1] / "samples" / "概要设计书模板_中文.docx") + parser.add_argument("--src", default=default_src, help="源日文模板") + parser.add_argument("--dst", default=default_dst, help="输出中文模板") + args = parser.parse_args() + out = build_zh_template(args.src, args.dst) + print(f"中文模板已生成: {out}") + + +if __name__ == "__main__": + main() diff --git a/scripts/run_trial.py b/scripts/run_trial.py index 6cc60e0..2346463 100644 --- a/scripts/run_trial.py +++ b/scripts/run_trial.py @@ -53,6 +53,8 @@ def _parse_args(argv: list[str] | None) -> argparse.Namespace: p.add_argument("--existing-system", default=str(ROOT / "samples" / "existing-system"), help="既有系统源码目录（追加/改修场景）") p.add_argument("--language", default="java", help="既有系统开发语言（默认 java，None=自动探测）") + p.add_argument("--output-language", default="auto", + choices=["auto", "zh", "ja"], help="输出语言：auto=与标题一致，zh=简体中文，ja=日文") p.add_argument("--samples-dir", default=str(ROOT / "samples"), help="样本资产根目录") p.add_argument("--output", default=str(ROOT / "output" / "output.docx"), help="概要设计书输出路径") p.add_argument("--impact-report", default=str(ROOT / "output" / "impact-report.json"), @@ -86,6 +88,7 @@ def main(argv: list[str] | None = None) -> dict: samples_dir=args.samples_dir, engine=engine, template_path=str(Path(args.template)), + output_language=args.output_language, ) report = ss.impact_report diff --git a/src/genesis/config.py b/src/genesis/config.py index 7e11b8d..ef79b2e 100644 --- a/src/genesis/config.py +++ b/src/genesis/config.py @@ -2,7 +2,7 @@ from __future__ import annotations import os from pathlib import Path -from typing import Any +from typing import Any, Literal import yaml from pydantic import BaseModel, Field @@ -127,6 +127,19 @@ class RagConfig(BaseModel): rerank: RerankConfig = Field(default_factory=RerankConfig) +class WriterConfig(BaseModel): + """Writer 子系统配置（步骤 1：输出语言参数）。 + + output_language: 生成概要设计书正文的自然语言 + - "auto"：与章节标题所用语言保持一致（默认，向后兼容既有日文文档） + - "zh"：强制简体中文 + - "ja"：强制日文 + 表格数据始终照抄源 Excel 原文（不翻译），见 design.md §7.2。 + """ + + output_language: Literal["auto", "zh", "ja"] = "auto" + + # ---------- 加载辅助 ---------- def _expand_env(data: Any) -> Any: @@ -192,6 +205,7 @@ class Settings(BaseSettings): app: AppConfig = Field(default_factory=AppConfig) inference: InferenceConfig = Field(default_factory=InferenceConfig) rag: RagConfig = Field(default_factory=RagConfig) + writer: WriterConfig = Field(default_factory=WriterConfig) @classmethod def from_dir(cls, config_dir: Path | str) -> "Settings": diff --git a/src/genesis/eval/scorer.py b/src/genesis/eval/scorer.py index 2ea7bc5..99c8dcc 100644 --- a/src/genesis/eval/scorer.py +++ b/src/genesis/eval/scorer.py @@ -34,6 +34,9 @@ class ChapterArtifact: text: str source_uris: list[str] template_sections_expected: list[str] + expected_language: str = "" # 期望输出语言（"zh"/"ja"；空=不可验证，维度记满分） + # 块级 (type, text) 列表：供语言一致性维度排除 heading/table（照抄源/跟随模板） + blocks: list[tuple[str, str]] = field(default_factory=list) # 逐章评估通过阈值（基于逐章总分） @@ -56,6 +59,7 @@ DEFAULT_THRESHOLDS: dict[str, float] = { "traceability": 1.0, "placeholder_residue": 1.0, "chapter_completeness": 1.0, + "language_consistency": 1.0, } @@ -75,6 +79,7 @@ class ChapterScorer: dimensions.append(self._traceability(chapters, source)) dimensions.append(self._placeholder_residue(chapters)) dimensions.append(self._completeness(chapters)) + dimensions.append(self._language_consistency(chapters)) # LLM 语义维度钩子（每个章节独立评，取该维度平均） for name, fn in self.llm_evaluators.items(): @@ -144,6 +149,45 @@ class ChapterScorer: detail = " | ".join(s.detail for s in scores) if scores else "no chapters" return DimensionScore(name, round(avg, 4), all(s.passed for s in scores), detail) + # ---------- 语言一致性维度（步骤 C，确定性）---------- + + def _language_consistency(self, chapters: list[ChapterArtifact]) -> DimensionScore: + """第 11 维度：输出语言与期望语言一致（确定性，脚本可验证）。 + + 与 writer.language 共用单一检测事实来源。期望语言为空（auto/不可验证） + → 记满分 1.0 通过（评审 R1：不拉低总分，避免误伤既有断言）。 + 仅检正文块（heading/table 不检，表格照抄源、标题跟随模板）。 + """ + from genesis.writer.language import find_language_violations + + if not chapters: + return DimensionScore("language_consistency", 1.0, True, "no chapters") + per: list[DimensionScore] = [] + for ch in chapters: + expected = ch.expected_language + if not expected: + per.append(DimensionScore( + "language_consistency", 1.0, True, "unverifiable (no expected language)")) + continue + from genesis.writer.models import ContentBlock + if ch.blocks: + # 优先用块级信息（可排除 heading/table） + blocks = [ContentBlock(block_id=str(i), type=t, text=tx) + for i, (t, tx) in enumerate(ch.blocks)] + else: + # 回退：整段正文作为单个 paragraph 块 + blocks = [ContentBlock(block_id="0", type="paragraph", text=ch.text or "")] + viol = find_language_violations(blocks, expected) + score = 0.0 if viol else 1.0 + passed = score >= self.thresholds["language_consistency"] + per.append(DimensionScore( + "language_consistency", score, passed, + f"期望 {expected}，违规 {len(viol)} 处" if viol else f"期望 {expected}，一致")) + avg = sum(p.score for p in per) / len(per) + passed = all(p.passed for p in per) + detail = " | ".join(p.detail for p in per) + return DimensionScore("language_consistency", round(avg, 4), passed, detail) + # ---------- 逐章评估 ---------- def _score_chapter(self, chapter: ChapterArtifact, source: StructuredSource) -> EvalReport: @@ -156,6 +200,7 @@ class ChapterScorer: self._traceability([chapter], source), self._placeholder_residue([chapter]), self._adequacy(chapter), + self._language_consistency([chapter]), ] total = sum(d.score for d in dims) / len(dims) if dims else 0.0 failed = (not all(d.passed for d in dims)) or total < PASS_THRESHOLD diff --git a/src/genesis/qa/qa_loop.py b/src/genesis/qa/qa_loop.py index 30bec49..799514d 100644 --- a/src/genesis/qa/qa_loop.py +++ b/src/genesis/qa/qa_loop.py @@ -21,8 +21,8 @@ class QALoop: def __init__(self, max_rounds: int = DEFAULT_MAX_QA_ROUNDS) -> None: self.controller = QALoopController(max_rounds=max_rounds) - def _build(self, structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id, only_ids=None, prev=None): - ctxs = build_contexts(structured_source, samples_dir) + def _build(self, structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id, only_ids=None, prev=None, output_language: str = "auto"): + ctxs = build_contexts(structured_source, samples_dir, output_language=output_language) _warn_unanchored(ctxs) state = WriterState([c.chapter_id for c in ctxs]) agent = WriterAgent(session_id=session_id, engine=engine, prompt_registry=prompt_registry, state=state) @@ -47,12 +47,14 @@ class QALoop: doc.save(output_path) return [contents_map[cid] for cid in order] - def run(self, structured_source, output_path, session_id="writer", samples_dir="samples", engine=None, prompt_registry=None, template_path=None) -> QAReport: + def run(self, structured_source, output_path, session_id="writer", samples_dir="samples", engine=None, prompt_registry=None, template_path=None, output_language: str = "auto") -> QAReport: engine = engine or build_inference_engine() prompt_registry = prompt_registry or PromptRegistry() validator = QAValidator() - contents = self._build(structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id) - report = validator.validate_doc(contents, structured_source) + # auto 不可推导期望语言 → 语言维度记满分（unverifiable）；zh/ja 显式强制 + expected = output_language if output_language in ("zh", "ja") else "" + contents = self._build(structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id, output_language=output_language) + report = validator.validate_doc(contents, structured_source, expected_language=expected) while self.controller.can_continue() and report.failed_chapters: self.controller.advance() contents = self._build( @@ -65,6 +67,7 @@ class QALoop: session_id, only_ids=set(report.failed_chapters), prev={c.chapter_id: c for c in contents}, + output_language=output_language, ) - report = validator.validate_doc(contents, structured_source) + report = validator.validate_doc(contents, structured_source, expected_language=expected) return report diff --git a/src/genesis/qa/validator.py b/src/genesis/qa/validator.py index 0482f4a..973630d 100644 --- a/src/genesis/qa/validator.py +++ b/src/genesis/qa/validator.py @@ -10,8 +10,12 @@ class QAValidator: def __init__(self, scorer: ChapterScorer | None = None) -> None: self.scorer = scorer or ChapterScorer() - def _to_artifact(self, content: ChapterContent) -> ChapterArtifact: - """把 ChapterContent 转换为评分器所需的 ChapterArtifact（聚合正文与来源 URI）。""" + def _to_artifact(self, content: ChapterContent, expected_language: str = "") -> ChapterArtifact: + """把 ChapterContent 转换为评分器所需的 ChapterArtifact（聚合正文与来源 URI）。 + + expected_language：期望输出语言（"zh"/"ja"；空=不可验证，维度记满分）。 + blocks 保留 (type, text) 供语言维度排除 heading/table。 + """ text = "".join(b.text or "" for b in content.blocks) source_uris: list[str] = [] for b in content.blocks: @@ -21,18 +25,26 @@ class QAValidator: text=text, source_uris=source_uris, template_sections_expected=[], + expected_language=expected_language, + blocks=[(b.type, b.text or "") for b in content.blocks], ) - def validate_chapter(self, content: ChapterContent, structured_source) -> tuple[bool, EvalReport]: - report = self.scorer.score([self._to_artifact(content)], structured_source) + def validate_chapter( + self, content: ChapterContent, structured_source, expected_language: str = "" + ) -> tuple[bool, EvalReport]: + report = self.scorer.score([self._to_artifact(content, expected_language)], structured_source) passed = content.chapter_id not in report.failed_chapters return passed, report - def validate_document(self, contents: list[ChapterContent], structured_source) -> list[tuple[bool, EvalReport]]: - return [self.validate_chapter(c, structured_source) for c in contents] + def validate_document( + self, contents: list[ChapterContent], structured_source, expected_language: str = "" + ) -> list[tuple[bool, EvalReport]]: + return [self.validate_chapter(c, structured_source, expected_language) for c in contents] - def validate_doc(self, contents: list[ChapterContent], structured_source) -> QAReport: - results = self.validate_document(contents, structured_source) + def validate_doc( + self, contents: list[ChapterContent], structured_source, expected_language: str = "" + ) -> QAReport: + results = self.validate_document(contents, structured_source, expected_language) passed_flags = [ok for ok, _ in results] reports = [rep for _, rep in results] failed = [c.chapter_id for c, ok in zip(contents, passed_flags) if not ok] diff --git a/src/genesis/writer/context_builder.py b/src/genesis/writer/context_builder.py index f95e65a..06a1c4a 100644 --- a/src/genesis/writer/context_builder.py +++ b/src/genesis/writer/context_builder.py @@ -7,7 +7,11 @@ from genesis.writer.models import ChapterSpec, GenerationContext from genesis.writer.template_mapper import map_template -def build_contexts(structured_source: StructuredSource, samples_dir: str = "samples") -> list[GenerationContext]: +def build_contexts( + structured_source: StructuredSource, + samples_dir: str = "samples", + output_language: str = "auto", +) -> list[GenerationContext]: specs: list[ChapterSpec] = map_template(structured_source.template) rag = CannedRagService(samples_dir=samples_dir) out: list[GenerationContext] = [] @@ -27,6 +31,7 @@ def build_contexts(structured_source: StructuredSource, samples_dir: str = "samp template_styles=set(used), prior_state=None, impact_report=getattr(structured_source, "impact_report", None), + output_language=output_language, ) ) return out diff --git a/src/genesis/writer/language.py b/src/genesis/writer/language.py new file mode 100644 index 0000000..e83ef84 --- /dev/null +++ b/src/genesis/writer/language.py @@ -0,0 +1,104 @@ +"""输出语言确定性检测与强制（步骤 A）。 + +设计要点： +- 日文标题多为纯汉字（如「DB設計」无假名），仅凭标题无法判定期望语言； + 故 resolve_expected_language 采用两级推导：显式 > 标题假名 > 规则文档主导脚本。 +- 检测仅基于「是否含日文假名」：CJK 汉字零假名视为中文（日文不可能不含假名地 + 使用汉字），反之中日混排含假名判日文。这是确定可机器验证的唯一稳健信号。 +- find_language_violations 仅检正文类块（paragraph/note/list）；heading 跟随模板、 + table 照抄源 Excel 原文，二者均不检（design.md §7.2 内容准确性/可追溯性）。 +- 短文本（<12 字）不误杀（如专有术语），阈值见 MIN_VIOLATION_LEN。 +""" +from __future__ import annotations + +from genesis.writer.models import ContentBlock + +# 日文假名 Unicode 区间 +_HIRAGANA = (0x3040, 0x309F) +_KATAKANA = (0x30A0, 0x30FF) +# 中日韩统一表意文字（CJK 汉字） +_CJK = (0x4E00, 0x9FFF) + +# 受检的正文块类型（heading/table 不检） +_CHECKED_BLOCK_TYPES = {"paragraph", "note", "list"} +# 触发违规判定的最小正文长度（防短术语误杀） +MIN_VIOLATION_LEN = 12 + + +def _in_range(ch: str, lo: int, hi: int) -> bool: + cp = ord(ch) + return lo <= cp <= hi + + +def has_kana(text: str) -> bool: + """文本是否含日文假名（平假名/片假名）。""" + return any(_in_range(c, *_HIRAGANA) or _in_range(c, *_KATAKANA) for c in text) + + +def has_cjk(text: str) -> bool: + """文本是否含 CJK 汉字。""" + return any(_in_range(c, *_CJK) for c in text) + + +def detect_script(text: str) -> str | None: + """检测文本主导自然语言。 + + 含假名 → "ja"；含 CJK 汉字但零假名 → "zh"；二者皆非（纯 ASCII 等）→ None。 + """ + if not text: + return None + if has_kana(text): + return "ja" + if has_cjk(text): + return "zh" + return None + + +def resolve_expected_language( + explicit: str, + title: str = "", + fallback_texts: tuple[str, ...] | list[str] = (), +) -> str: + """推导本章期望输出语言（单一事实来源，A 的重试校验与 C 的 QA 维度共用）。 + + - explicit 为 "zh"/"ja" → 直接采用（用户显式选择优先） + - 否则看标题是否含假名（仅假名可可靠判为日文；纯汉字标题对中/日均可能，不可信） + - 否则看 fallback_texts（如影响调查书/章节数据，通常日文）的主导脚本 + - 均无法推导 → 返回 ""（不可验证，交由上层按 unverifiable 处理） + """ + if explicit in ("zh", "ja"): + return explicit + # 标题仅当含假名时可靠指示日文；纯汉字/ASCII 标题跳过，改看 fallback + if has_kana(title or ""): + return "ja" + for text in fallback_texts: + s = detect_script(text or "") + if s: + return s + return "" + + +def find_language_violations(blocks: list[ContentBlock], expected_language: str) -> list[str]: + """返回违规正文块文本片段（期望语言非空时才有意义）。 + + 违规判定： + - 期望 "ja"：正文块含 CJK 汉字且零假名（即纯中文）且长度 ≥ 阈值 + - 期望 "zh"：正文块含日文假名 + heading/table 块始终跳过（跟随模板 / 照抄源）。 + """ + if expected_language not in ("zh", "ja"): + return [] + violations: list[str] = [] + for b in blocks: + if b.type not in _CHECKED_BLOCK_TYPES: + continue + text = b.text or "" + if len(text) < MIN_VIOLATION_LEN: + continue + if expected_language == "ja": + if has_cjk(text) and not has_kana(text): + violations.append(text) + else: # zh + if has_kana(text): + violations.append(text) + return violations diff --git a/src/genesis/writer/models.py b/src/genesis/writer/models.py index 6342161..c2128ed 100644 --- a/src/genesis/writer/models.py +++ b/src/genesis/writer/models.py @@ -109,6 +109,19 @@ class GenerationContext: template_styles: set[str] prior_state: object | None = None # WriterState，避免循环 import 用 object impact_report: object | None = None # ImpactReport 影响调查书（生成主上下文） + output_language: str = "auto" # "auto" | "zh" | "ja"（步骤 1：用户可选输出语言） + + def _language_instruction(self) -> str: + """根据 output_language 生成【语言约束】段的具体指令（步骤 1）。""" + if self.output_language == "zh": + return "必须使用简体中文撰写（标题、正文与所有说明一律中文）。" + if self.output_language == "ja": + return "必ず日本語で記述すること（タイトル・本文・すべての説明は日本語）。" + # auto：沿用与标题语言一致的旧语义（向后兼容既有日文文档） + return ( + f"必须与章节标题「{self.title}」所用自然语言保持一致：" + "标题为日文则用日文撰写，为中文则用中文撰写，依此类推。" + ) def to_vars(self) -> dict: """返回供 prompt 渲染的变量字典。""" @@ -125,6 +138,7 @@ class GenerationContext: f"- {h}" for h in (getattr(tm, "sub_headings", None) or []) ), "prior_state": str(self.prior_state) if self.prior_state is not None else "", + "language_instruction": self._language_instruction(), "data": _format_chapter_data( self.structured_source, CHAPTER_SHEET_TYPES.get(self.chapter_id, []), @@ -132,6 +146,7 @@ class GenerationContext: "impact": _format_impact( self.impact_report, CHAPTER_IMPACT_ELEMENT.get(self.chapter_id, None), + self.output_language, ), } @@ -177,11 +192,30 @@ def _format_chapter_data(structured_source: object | None, sheet_types: list[She return "\n".join(lines).rstrip() -def _format_impact(report: object | None, element_type: ElementType | None = None) -> str: +# 影响调查标签本地化（步骤 B）：auto/ja 默认日文，zh 中文 +# 注：方括号标记 [..] 保留（中日通用），仅标签词本地化 +_IMPACT_LABELS: dict[str, dict[str, str]] = { + "zh": { + "new": "新建", "modified": "变更", "deleted": "删除", "warning": "警告", + "affected": "受影响", + }, + "ja": { + "new": "新規", "modified": "変更", "deleted": "削除", "warning": "警告", + "affected": "受影响", + }, +} + + +def _format_impact( + report: object | None, + element_type: ElementType | None = None, + output_language: str = "auto", +) -> str: """将影响调查书格式化为 prompt 可读文本（无报告/无分析时为空串）。 element_type 非 None 时仅保留该类型要素（章节级定向，design.md §6.8 ①）； 警告始终保留（不依赖要素类型）。 + output_language 控制标签语言（步骤 B）；auto 回落到 ja 标签。 """ if report is None: return "" @@ -192,6 +226,7 @@ def _format_impact(report: object | None, element_type: ElementType | None = Non def keep(el) -> bool: return element_type is None or el.element_type == element_type.value + lab = _IMPACT_LABELS.get(output_language, _IMPACT_LABELS["ja"]) summary = getattr(report, "summary", {}) or {} lines = [f"project_type={getattr(ca, 'project_type', '')}"] lines.append( @@ -204,15 +239,15 @@ def _format_impact(report: object | None, element_type: ElementType | None = Non ) for el in getattr(ca, "new_elements", []) or []: if keep(el): - lines.append(f"[新規] {el.element_id} {el.element_type} {el.name}") + lines.append(f"[{lab['new']}] {el.element_id} {el.element_type} {el.name}") for el in getattr(ca, "modified_elements", []) or []: if keep(el): impacted = ", ".join(el.impacted_existing) or "-" - lines.append(f"[変更] {el.element_id} {el.element_type} {el.name} → 受影响: {impacted}") + lines.append(f"[{lab['modified']}] {el.element_id} {el.element_type} {el.name} → {lab['affected']}: {impacted}") for el in getattr(ca, "deleted_elements", []) or []: if keep(el): impacted = ", ".join(el.impacted_existing) or "-" - lines.append(f"[削除] {el.element_id} {el.element_type} {el.name} → 受影响: {impacted}") + lines.append(f"[{lab['deleted']}] {el.element_id} {el.element_type} {el.name} → {lab['affected']}: {impacted}") for w in getattr(ca, "warnings", []) or []: - lines.append(f"[警告] {w.element_id}: {w.issue}") + lines.append(f"[{lab['warning']}] {w.element_id}: {w.issue}") return "\n".join(lines) diff --git a/src/genesis/writer/orchestrator.py b/src/genesis/writer/orchestrator.py index 1995407..19bedbd 100644 --- a/src/genesis/writer/orchestrator.py +++ b/src/genesis/writer/orchestrator.py @@ -56,6 +56,7 @@ class WriteOrchestrator: template_path: str | None = None, impact_report=None, meta: dict | None = None, + output_language: str = "auto", ) -> list[ChapterContent]: engine = engine or build_inference_engine() prompt_registry = prompt_registry or PromptRegistry() @@ -66,7 +67,7 @@ class WriteOrchestrator: if impact_report is not None: # 回填 structured_source，便于 QA/日志/后续下载 structured_source.impact_report = impact_report - ctxs = build_contexts(structured_source, samples_dir) + ctxs = build_contexts(structured_source, samples_dir, output_language=output_language) _warn_unanchored(ctxs) state = WriterState([c.chapter_id for c in ctxs]) agent = WriterAgent(session_id=session_id, engine=engine, prompt_registry=prompt_registry, state=state) diff --git a/src/genesis/writer/writer_agent.py b/src/genesis/writer/writer_agent.py index 5287229..687da73 100644 --- a/src/genesis/writer/writer_agent.py +++ b/src/genesis/writer/writer_agent.py @@ -13,6 +13,10 @@ from genesis.inference.types import Prompt, StructuredResult from genesis.writer.models import ChapterContent, GenerationContext from genesis.writer.writer_state import WriterState from genesis.writer.exceptions import WriterGenerationError +from genesis.writer.language import ( + find_language_violations, + resolve_expected_language, +) WRITER_PROMPT_TEMPLATE = ( @@ -33,10 +37,9 @@ WRITER_PROMPT_TEMPLATE = ( "严格以「参考资料（本章对应数据）」中的要件定义数据和「影响调查上下文」为核心依据；" "禁止输出与本章无关的系统整体架构、通用设计说明等内容，禁止套用其他章节的主题。" "若本章数据为空，则基于规则与影响调查上下文简要撰写，不得虚构数据。\n" - "【语言约束】章节正文（所有 block 的 text 字段）所使用的自然语言，" - "必须与章节标题「{{title}}」所用语言保持一致：标题为日文则用日文撰写，" - "为中文则用中文撰写，依此类推。" -) + "【语言约束】章节正文（所有 block 的 text 字段）所使用的自然语言：{{language_instruction}}\n" + ) + CHAPTER_OUTPUT_SCHEMA = { "type": "object", @@ -70,7 +73,7 @@ class WriterAgent: engine: InferenceEngine, prompt_registry: PromptRegistry, state: WriterState, - max_retries: int = 1, + max_retries: int = 2, ) -> None: self.session_id = session_id self.engine = engine @@ -132,6 +135,15 @@ class WriterAgent: def generate_chapter(self, context: GenerationContext) -> ChapterContent: self._chunk_source(context.structured_source) # 分块可用性验证（真实拼回留待后续） + + # 步骤 A：推导本章期望输出语言（仅依赖 context，循环前置，稳定） + v = context.to_vars() + expected = resolve_expected_language( + explicit=context.output_language, + title=context.title, + fallback_texts=[v.get("impact", "") or "", v.get("data", "") or ""], + ) + last_err: Exception | None = None for _ in range(max(1, self.max_retries)): try: @@ -148,6 +160,15 @@ class WriterAgent: # heading 块（prompt 约束为尽力而为，此处程序化强制） if not (getattr(context.template_marker, "sub_headings", None) or []): content.blocks = [b for b in content.blocks if b.type != "heading"] + # 步骤 A：输出语言一致性强制（期望语言可推导时，违规按失败重试） + if expected: + viol = find_language_violations(content.blocks, expected) + if viol: + last_err = WriterGenerationError( + f"章节 {context.chapter_id} 语言不一致（期望 {expected}，" + f"发现 {len(viol)} 处违规正文）" + ) + continue self.state.record_success(content) return content raise WriterGenerationError(f"章节 {context.chapter_id} 重试耗尽: {last_err}") diff --git a/tests/test_language_coverage.py b/tests/test_language_coverage.py new file mode 100644 index 0000000..ffb7df7 --- /dev/null +++ b/tests/test_language_coverage.py @@ -0,0 +1,102 @@ +"""步骤 A/C 边界覆盖测试（恢复覆盖率至 ≥99%）。""" +from __future__ import annotations + +from pathlib import Path +from types import SimpleNamespace + +from genesis.writer.language import find_language_violations +from genesis.writer.models import ChapterContent, ContentBlock, GenerationContext, ChapterSpec +from genesis.writer.writer_agent import WriterAgent +from genesis.writer.writer_state import WriterState +from genesis.writer.exceptions import WriterGenerationError +from genesis.qa.qa_loop import QALoop +from genesis.data_models import StructuredSource +from genesis.parsers.word_template_parser import WordTemplateParser + + +class _Reg: + @staticmethod + def get_or_create(name, template): + return SimpleNamespace(name=name, version="1", template=template) + + +def test_find_violations_unexpected_language_code_returns_empty(): + blocks = [ContentBlock(block_id="0", type="paragraph", text="中文正文内容充分长度足够")] + # 非 zh/ja 的期望语言 → 不强制，返回空 + assert find_language_violations(blocks, "xx") == [] + + +def test_writer_agent_zh_expected_rejects_japanese(): + class JpEngine: + def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2): + return SimpleNamespace( + data={"title": variables["title"], + "blocks": [{"type": "paragraph", + "text": "本機能は注文処理を行う画面であり、詳細は以下の通り。"}]}, + status="ok", + ) + agent = WriterAgent(session_id="s", engine=JpEngine(), prompt_registry=_Reg(), + state=WriterState(["db_design"]), max_retries=1) + ctx = GenerationContext(chapter_id="db_design", title="DB 設計", + template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"), + structured_source=None, write_rules=[], design_rules=[], + template_styles=set(), output_language="zh") + try: + agent.generate_chapter(ctx) + assert False, "应抛 WriterGenerationError" + except WriterGenerationError: + pass + + +def test_qa_loop_passes_with_explicit_zh_on_chinese_fake(tmp_path): + """显式 zh + 中文 Fake → 语言维度通过（验证透传链路）。""" + template = Path("samples/phase5-slice/template.docx") + if not template.exists(): + import pytest + pytest.skip("样本模板缺失") + parsed = WordTemplateParser().parse(str(template)) + ss = StructuredSource(template=parsed, tables=[], rule_docs=[], image_analyses=[], + existing_system=None, comments=[]) + out = tmp_path / "o.docx" + + class ZhEngine: + def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2): + return SimpleNamespace( + data={"title": variables["title"], + "blocks": [{"type": "paragraph", + "text": "由 FakeLLM 生成的充分说明内容，满足写入规则要求。"}]}, + status="ok", + ) + loop = QALoop(max_rounds=1) + report = loop.run(ss, str(out), samples_dir="nonexistent_dir_xyz", + engine=ZhEngine(), template_path=str(template), output_language="zh") + assert report.passed is True + + +def test_qa_loop_fails_with_explicit_ja_on_chinese_fake(tmp_path): + """显式 ja + 中文 Fake → 语言维度判失败。""" + template = Path("samples/phase5-slice/template.docx") + if not template.exists(): + import pytest + pytest.skip("样本模板缺失") + parsed = WordTemplateParser().parse(str(template)) + ss = StructuredSource(template=parsed, tables=[], rule_docs=[], image_analyses=[], + existing_system=None, comments=[]) + out = tmp_path / "o.docx" + + class ZhEngine: + def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2): + return SimpleNamespace( + data={"title": variables["title"], + "blocks": [{"type": "paragraph", + "text": "由 FakeLLM 生成的充分说明内容，满足写入规则要求。"}]}, + status="ok", + ) + loop = QALoop(max_rounds=1) + # 显式 ja + 中文 Fake：writer 生成阶段即因语言不一致硬失败（设计语义） + try: + loop.run(ss, str(out), samples_dir="nonexistent_dir_xyz", + engine=ZhEngine(), template_path=str(template), output_language="ja") + assert False, "应抛出 WriterGenerationError" + except WriterGenerationError: + pass diff --git a/tests/test_language_plumbing.py b/tests/test_language_plumbing.py new file mode 100644 index 0000000..7dcf274 --- /dev/null +++ b/tests/test_language_plumbing.py @@ -0,0 +1,93 @@ +"""步骤 1：语言参数贯通测试（config → GenerationContext → to_vars → context_builder → run_trial）。""" +from __future__ import annotations + +from genesis.data_models import ( + CellValue, ExcelTable, ParsedTemplate, Provenance, SheetType, StructuredSource, +) +from genesis.writer.context_builder import build_contexts +from genesis.writer.models import ChapterSpec, GenerationContext + + +def _src() -> StructuredSource: + return StructuredSource( + tables=[], + template=ParsedTemplate("t.docx", [__import__("genesis.data_models", fromlist=["ChapterMarker"]).ChapterMarker("heading", "1. はじめに", 1)], {}, {"used": []}), + rule_docs=[], image_analyses=[], existing_system=None, comments=[], + ) + + +def test_generation_context_default_output_language_is_auto(): + ctx = GenerationContext( + chapter_id="db_design", title="DB 設計", + template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"), + structured_source=None, write_rules=[], design_rules=[], template_styles=set(), + ) + assert ctx.output_language == "auto" + + +def test_to_vars_language_instruction_explicit_zh(): + ctx = GenerationContext( + chapter_id="db_design", title="DB 设計", + template_marker=ChapterSpec(chapter_id="db_design", title="DB 设計"), + structured_source=None, write_rules=[], design_rules=[], template_styles=set(), + output_language="zh", + ) + v = ctx.to_vars() + assert "language_instruction" in v + assert "简体中文" in v["language_instruction"] + + +def test_to_vars_language_instruction_explicit_ja(): + ctx = GenerationContext( + chapter_id="db_design", title="DB 設計", + template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"), + structured_source=None, write_rules=[], design_rules=[], template_styles=set(), + output_language="ja", + ) + v = ctx.to_vars() + assert "日本語" in v["language_instruction"] + + +def test_to_vars_language_instruction_auto_follows_title(): + ctx = GenerationContext( + chapter_id="db_design", title="DB 設計", + template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"), + structured_source=None, write_rules=[], design_rules=[], template_styles=set(), + output_language="auto", + ) + v = ctx.to_vars() + assert "标题" in v["language_instruction"] and "一致" in v["language_instruction"] + + +def test_context_builder_passes_output_language(): + src = _src() + ctxs = build_contexts(src, output_language="zh") + assert all(c.output_language == "zh" for c in ctxs) + # 回归：默认仍为 auto + ctxs2 = build_contexts(src) + assert all(c.output_language == "auto" for c in ctxs2) + + +def test_format_impact_zh_labels_localized(): + """步骤 B：output_language=zh 时影响调查标签本地化为中文，且不影响 ja 默认。""" + from genesis.data_models import ( + ChangeAnalysis, ChangeElement, ChangeType, ImpactReport, ImpactWarning, + ) + from genesis.writer.models import _format_impact + + ca = ChangeAnalysis( + project_type="enhancement", + new_elements=[ChangeElement("F001", "機能", "止损风控", ChangeType.NEW)], + modified_elements=[ChangeElement("F002", "機能", "下单流程", ChangeType.MODIFIED)], + deleted_elements=[ChangeElement("F030", "機能", "旧功能", ChangeType.DELETED)], + unchanged_elements=[], + warnings=[ImpactWarning("F030", "缺少既存対応")], + ) + report = ImpactReport(metadata={}, change_analysis=ca, summary={}) + + zh = _format_impact(report, None, "zh") + assert "[新建]" in zh and "[变更]" in zh and "[删除]" in zh and "[警告]" in zh + assert "受影响" in zh + # ja / auto 回落仍用日文标签 + ja = _format_impact(report, None, "ja") + assert "[新規]" in ja and "[削除]" in ja diff --git a/tests/test_phase5_e2e.py b/tests/test_phase5_e2e.py index 67ea653..bce5ced 100644 --- a/tests/test_phase5_e2e.py +++ b/tests/test_phase5_e2e.py @@ -12,13 +12,14 @@ from genesis.qa.qa_loop import QALoop class FakeEngine: def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2): cid = variables.get("title", "x") + # 日文模板章节（标题含假名）→ 返回日文正文，满足语言一致性强制 return SimpleNamespace( data={ "title": cid, "blocks": [ - {"type": "heading", "level": 2, "text": f"{cid} 小节"}, - {"type": "paragraph", "text": "由 FakeLLM 生成的充分说明内容，满足写入规则要求。"}, - {"type": "table", "caption": "示例表", "rows": [["列1", "列2"], ["值1", "值2"]]}, + {"type": "heading", "level": 2, "text": f"{cid} 小節"}, + {"type": "paragraph", "text": "本機能はFakeLLMにより生成された十分な説明内容であり、書込規則を満たす。"}, + {"type": "table", "caption": "示例表", "rows": [["列1", "列2"], ["値1", "値2"]]}, ], }, status="ok", @@ -50,5 +51,5 @@ def test_phase5_e2e_fake_llm(tmp_path): assert report.passed is True loaded = Document(str(out)) text = _all_text(loaded) - assert "由 FakeLLM 生成的充分说明内容" in text + assert "本機能はFakeLLMにより生成された十分な説明内容" in text assert "示例表" in text diff --git a/tests/test_scorer_language.py b/tests/test_scorer_language.py new file mode 100644 index 0000000..12216cd --- /dev/null +++ b/tests/test_scorer_language.py @@ -0,0 +1,92 @@ +"""步骤 C：QA 第 11 维度 language_consistency 测试。 + +评审 R1：不可验证（expected_language 为空/auto）→ 维度满分 1.0 通过，不拉低总分， +使既有 test_eval_scorer / test_phase5_scorer 的 total_score==1.0 断言继续成立。 +""" +from __future__ import annotations + +from genesis.data_models import ( + CellValue, ExcelTable, ParsedTemplate, Provenance, SheetType, StructuredSource, +) +from genesis.eval.scorer import ChapterArtifact, ChapterScorer, EvalReport + + +def _dim(report: EvalReport, name: str): + for d in report.dimensions: + if d.name == name: + return d + raise AssertionError(f"维度未找到: {name}") + + +def _source() -> StructuredSource: + cell = CellValue( + value="登録", + provenance=Provenance(file_name="f.xlsx", sheet_name="機能一覧", row=3, column="C", column_header="x"), + ) + table = ExcelTable( + name="機能一覧", detected_type=SheetType.FUNCTION, + extraction_method="structured", headers=["v"], rows=[{"v": cell}], + ) + return StructuredSource( + tables=[table], + template=ParsedTemplate(file_name="t.docx", sections=[], placeholders={}, styles={}), + rule_docs=[], image_analyses=[], existing_system=None, comments=[], + ) + + +def test_language_dim_unverifiable_is_full_score(): + """expected_language 为空（auto/不可验证）→ 维度满分通过，不拉低总分。""" + src = _source() + artifact = ChapterArtifact( + chapter_id="ch3", text="本機能は注文処理を行う画面である。", + source_uris=[], template_sections_expected=["ch3"], + expected_language="", + ) + report = ChapterScorer().score([artifact], src) + dim = _dim(report, "language_consistency") + assert dim.score == 1.0 + assert dim.passed is True + # 既有总分断言仍成立（全部确定性维度满分） + assert report.total_score == 1.0 + + +def test_language_dim_ja_expected_but_chinese_fails(): + src = _source() + artifact = ChapterArtifact( + chapter_id="ch3", text="这是一段纯中文的章节正文内容，应当判定为语言违规。", + source_uris=[], template_sections_expected=["ch3"], + expected_language="ja", + ) + report = ChapterScorer().score([artifact], src) + dim = _dim(report, "language_consistency") + assert dim.score == 0.0 + assert dim.passed is False + assert report.total_score < 1.0 + + +def test_language_dim_zh_expected_japanese_fails(): + src = _source() + artifact = ChapterArtifact( + chapter_id="ch3", text="本機能は注文処理を行う画面であり、詳細は以下の通り。", + source_uris=[], template_sections_expected=["ch3"], + expected_language="zh", + ) + report = ChapterScorer().score([artifact], src) + dim = _dim(report, "language_consistency") + assert dim.score == 0.0 + + +def test_language_dim_threshold_in_defaults(): + assert ChapterScorer().thresholds["language_consistency"] == 1.0 + + +def test_dimension_count_increases_by_one(): + src = _source() + artifact = ChapterArtifact( + chapter_id="ch3", text="正常内容", source_uris=[], template_sections_expected=["ch3"], + ) + before = len(ChapterScorer(thresholds={}, llm_evaluators={}).score([artifact], src).dimensions) + # 默认 scorer 含 language_consistency；对比一个不含该维度的基线不可行， + # 此处仅断言维度中包含 language_consistency 名称 + report = ChapterScorer().score([artifact], src) + assert any(d.name == "language_consistency" for d in report.dimensions) diff --git a/tests/test_writer_language.py b/tests/test_writer_language.py new file mode 100644 index 0000000..a39f89b --- /dev/null +++ b/tests/test_writer_language.py @@ -0,0 +1,172 @@ +"""步骤 A：确定性语言检测 + 输出语言强制测试。 + +- detect_script：含假名→ja，CJK 零假名→zh，否则 None +- resolve_expected_language：显式 > 标题假名 > 规则文档主导脚本 > ""（不可验证） +- find_language_violations：仅检正文块（heading/table 不检），长度≥12 防误杀 +- WriterAgent.generate_chapter：期望语言非空时违规即重试，耗尽→硬失败 +""" +from __future__ import annotations + +from genesis.writer.language import ( + detect_script, + find_language_violations, + resolve_expected_language, +) +from genesis.writer.models import ChapterContent, ContentBlock, GenerationContext, ChapterSpec +from genesis.writer.writer_agent import WriterAgent +from genesis.writer.writer_state import WriterState +from genesis.writer.exceptions import WriterGenerationError + + +# ---------- detect_script ---------- + +def test_detect_script_kana_is_ja(): + assert detect_script("機能一覧の説明を記述します。") == "ja" + + +def test_detect_script_cjk_no_kana_is_zh(): + assert detect_script("功能一览的说明内容。") == "zh" + + +def test_detect_script_ascii_is_none(): + assert detect_script("Hello world 123") is None + + +def test_detect_script_mixed_cjk_kana_is_ja(): + # 含假名即判 ja，即使混有汉字（中文不可能含日文假名） + assert detect_script("DB設計の概要を説明する。") == "ja" + + +# ---------- resolve_expected_language ---------- + +def test_resolve_explicit_overrides(): + assert resolve_expected_language("zh", "DB 設計", ["日本語の影響"]) == "zh" + assert resolve_expected_language("ja", "DB 设計", ["中文影响"]) == "ja" + + +def test_resolve_falls_back_to_title_kana(): + # 标题含假名→ja（注意：纯汉字日文标题如「DB設計」无假名，会落入 fallback） + assert resolve_expected_language("auto", "機能一覧の説明", []) == "ja" + + +def test_resolve_falls_back_to_fallback_text(): + # 标题纯汉字无假名（中文式）→ 看 fallback（日文影响书）→ ja + assert resolve_expected_language("auto", "DB 設計", ["影響調査の結果（日本語）"]) == "ja" + + +def test_resolve_unverifiable_when_no_hint(): + assert resolve_expected_language("auto", "DB 設計", ["123 abc"]) == "" + + +# ---------- find_language_violations ---------- + +def _blocks(*specs): + out = [] + for i, (t, text) in enumerate(specs): + out.append(ContentBlock(block_id=str(i), type=t, text=text)) + return out + + +def test_violation_ja_expected_but_pure_chinese_paragraph(): + blocks = _blocks(("paragraph", "这是一段纯中文的章节正文内容。")) + viol = find_language_violations(blocks, "ja") + assert viol # 期望日文却为中文 → 违规 + + +def test_no_violation_ja_expected_japanese_paragraph(): + blocks = _blocks(("paragraph", "本機能は注文処理を行う画面であり、詳細は以下の通り。")) + assert not find_language_violations(blocks, "ja") + + +def test_heading_and_table_blocks_excluded(): + # heading 与 table（照抄原文）即使含中文也不算违规 + blocks = _blocks( + ("heading", "機能一覧表"), + ("table", "機能ID 機能名"), # 表格不检 + ("paragraph", "本機能は注文処理を行う。"), + ) + assert not find_language_violations(blocks, "ja") + + +def test_short_chinese_not_flagged_under_threshold(): + # 长度<12 的短术语不误杀 + blocks = _blocks(("paragraph", "中文术语")) + assert not find_language_violations(blocks, "ja") + + +def test_violation_zh_expected_but_kana_present(): + blocks = _blocks(("paragraph", "本機能は注文処理を行う画面である。")) + assert find_language_violations(blocks, "zh") + + +# ---------- WriterAgent 强制 ---------- + +class MixedLanguageEngine: + """返回中英混杂（期望日文时违规）的章节内容。""" + def __init__(self, texts): + self._texts = list(texts) + self._i = 0 + def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2): + from types import SimpleNamespace + text = self._texts[min(self._i, len(self._texts) - 1)] + self._i += 1 + return SimpleNamespace( + data={"title": variables["title"], "blocks": [{"type": "paragraph", "text": text}]}, + status="ok", + ) + + +class FakePromptRegistry: + @staticmethod + def get_or_create(name, template): + from types import SimpleNamespace + return SimpleNamespace(name=name, version="1", template=template) + + +def _ctx_with_lang(cid, title, lang): + return GenerationContext( + chapter_id=cid, title=title, + template_marker=ChapterSpec(chapter_id=cid, title=title), + structured_source=None, write_rules=[], design_rules=[], + template_styles=set(), output_language=lang, + ) + + +def test_generate_chapter_retries_on_language_violation_then_passes(): + # 第一次返回中文（ja 期望→违规），第二次返回日文（通过） + engine = MixedLanguageEngine([ + "这是一段纯中文的章节正文内容，应当被判定为语言违规。", + "本機能は注文処理を行う画面であり、詳細は以下の通り記述する。", + ]) + agent = WriterAgent(session_id="s", engine=engine, + prompt_registry=FakePromptRegistry(), + state=WriterState(["db_design"]), max_retries=3) + content = agent.generate_chapter(_ctx_with_lang("db_design", "DB 設計", "ja")) + assert content is not None + assert "本機能" in content.blocks[0].text + + +def test_generate_chapter_hard_fails_when_violation_persists(): + # 始终返回中文（ja 期望），重试耗尽→硬失败 + engine = MixedLanguageEngine([ + "这是一段纯中文的章节正文内容，应当被判定为语言违规。", + "还是一段纯中文的章节正文内容，依旧违规。", + ]) + agent = WriterAgent(session_id="s", engine=engine, + prompt_registry=FakePromptRegistry(), + state=WriterState(["db_design"]), max_retries=2) + try: + agent.generate_chapter(_ctx_with_lang("db_design", "DB 設計", "ja")) + assert False, "应抛出 WriterGenerationError" + except WriterGenerationError as e: + assert "语言" in str(e) or "language" in str(e).lower() + + +def test_generate_chapter_auto_unverifiable_not_enforced(): + # auto 且无法推导期望语言（标题纯汉字无假名、无日文 fallback）→ 不强制，直接通过 + engine = MixedLanguageEngine(["这是中文正文但 auto 不可验证所以放行。"]) + agent = WriterAgent(session_id="s", engine=engine, + prompt_registry=FakePromptRegistry(), + state=WriterState(["db_design"]), max_retries=1) + content = agent.generate_chapter(_ctx_with_lang("db_design", "DB 設計", "auto")) + assert content is not None diff --git a/tests/test_zh_template.py b/tests/test_zh_template.py new file mode 100644 index 0000000..a3f1948 --- /dev/null +++ b/tests/test_zh_template.py @@ -0,0 +1,77 @@ +"""中文样本模板镜像测试（步骤 0）。 + +断言 zh 模板与 ja 模板章节结构一致：7 个 H1 章、锚点 id（section:xxx）逐一对应， +且锚点段落文本原样保留（映射/注入器零改动）。 +""" +from __future__ import annotations + +import shutil +from pathlib import Path + +import pytest +from docx import Document + +from genesis.parsers.word_template_parser import WordTemplateParser +from genesis.writer.template_mapper import map_template + +_SAMPLES = Path(r"D:\00_project\Genesis\samples") +_JA = _SAMPLES / "概要設計書テンプレート.docx" +_ZH = _SAMPLES / "概要设计书模板_中文.docx" + + +EXPECTED_ANCHORS = [ + "introduction", "function_list", "screen_list", "report_list", + "db_design", "if_definition", "batch_list", +] + + +@pytest.fixture +def zh_template(tmp_path): + """生成 zh 模板到临时目录并返回路径（不污染 samples/）。""" + from scripts.make_zh_template import build_zh_template + dst = tmp_path / "概要设计书模板_中文.docx" + build_zh_template(str(_JA), str(dst)) + return dst + + +def _parse(path: Path): + return WordTemplateParser().parse(str(path)) + + +def test_zh_template_has_same_seven_chapter_anchors(zh_template): + ja = _parse(_JA) + zh = _parse(zh_template) + + ja_ids = [ph for ph in ja.placeholders if ph.startswith("section:")] + zh_ids = [ph for ph in zh.placeholders if ph.startswith("section:")] + + assert sorted(zh_ids) == sorted(ja_ids) + assert zh_ids == [f"section:{a}" for a in EXPECTED_ANCHORS] + + +def test_zh_template_anchor_paragraphs_unchanged(zh_template): + """锚点段落文本在 zh 模板中必须与 ja 完全一致（映射/注入器依赖它）。""" + ja_doc = Document(str(_JA)) + zh_doc = Document(str(zh_template)) + ja_anchors = [p.text for p in ja_doc.paragraphs if p.text.startswith("{{section:")] + zh_anchors = [p.text for p in zh_doc.paragraphs if p.text.startswith("{{section:")] + assert zh_anchors == ja_anchors + + +def test_zh_template_headings_translated(zh_template): + zh_doc = Document(str(zh_template)) + texts = {p.text for p in zh_doc.paragraphs if p.style.name.startswith("Heading")} + assert "1. 前言" in texts + assert "2. 功能一览" in texts + assert "5. DB设计" in texts + assert "7. 批处理一览" in texts + + +def test_zh_template_maps_to_same_seven_chapters(zh_template): + """template_mapper 对 zh 模板产出 7 个章节，锚点与 ja 对应。""" + pt = _parse(zh_template) + specs = map_template(pt) + assert len(specs) == 7 + assert [s.section_placeholder for s in specs] == [ + f"section:{a}" for a in EXPECTED_ANCHORS + ]