feat(writer): 输出语言一致性保障 + 用户可选输出语言

- 步骤0: 新增中文镜像模板 scripts/make_zh_template.py 与 samples/概要设计书模板_中文.docx(7章锚点原样保留)
- 步骤1: config.WriterConfig.output_language→Settings.writer;GenerationContext.output_language + to_vars.language_instruction(zh/ja/auto);writer_agent【语言约束】改引变量;context_builder/orchestrator/run_trial 透传 --output-language
- 步骤A: 新建 src/genesis/writer/language.py(detect_script/resolve_expected_language/find_language_violations);WriterAgent.generate_chapter 按期望语言强制、违规重试、耗尽硬失败;max_retries 默认 1→2
- 步骤B: _format_impact 影响调查标签按 output_language 本地化(zh 新建/变更/删除/警告)
- 步骤C: eval scorer 第 11 维度 language_consistency(不可验证=满分,不拉低总分);ChapterArtifact.expected_language;QAValidator.validate_doc 透传;QALoop.run 透传 output_language
- 测试: test_zh_template/test_language_plumbing/test_writer_language/test_scorer_language/test_language_coverage,并更新 test_phase5_e2e
- 全量 pytest 424 passed / 99.15%(覆盖率门槛 99% 达标)
This commit is contained in:
lhl
2026-08-25 23:30:24 +08:00
parent 9d0d3409c2
commit d01e1b720f
20 changed files with 1049 additions and 31 deletions
+1
View File
@@ -115,3 +115,4 @@
| 2026-08-24 13:18 | Agent 实现 | 修复无子节章 LLM 自造标题:prompt【小节约束】禁止空结构时输出 heading 块(尽力约束)+ WriterAgent.generate_chapter 程序化强制(sub_headings 为空的章剔除生成结果中的 heading 块,模板结构为准)。真实试运行验证:自造同名 H2 清零、H2 总数 14→6(仅模板定义的子节)、无子节章正文全为段落 | src/genesis/writer/writer_agent.py, tests/test_phase5_writer_agent.py | x-preview-f-free (opencode) |
| 2026-08-24 14:02 | 架构设计 | 输出语言一致性保障方案落盘:用户可选输出语言(auto/zh/ja)+ 中文模板镜像 + 确定性脚本检测与重试强制(A)+ impact 标签本地化(B+ QA 第11维度 language_consistencyC);关键决策:表格照抄原文、双模板锚点不变、违规硬失败 | docs/superpowers/plans/2026-08-24-language-consistency.md | x-preview-f-free (opencode) |
| 2026-08-24 14:35 | 反馈迭代 | 修复 _AI_USAGE_LOG.md 表格渲染断裂(用户报告 2026-08-09 03:30 起不以表格显示):删除表内 4 处空行与 1 处误写入的工具输出残留行(End of file);L55 摘要内嵌错误码表的 6 个管道符转义为反斜杠管;L107 被拼接的两条 2026-08-23 记录拆分为独立两行;零日志行删除,修复后全表 116 行均为有效 5 列 | _AI_USAGE_LOG.md | x-preview-f-free (opencode) |
| 2026-08-25 10:20 | Agent 实现 | 输出语言一致性 + 可选输出语言(实施中,暂停于步骤 A 修正点):步骤0 新增 scripts/make_zh_template.py 与 samples/概要设计书模板_中文.docx7章锚点镜像)+ tests/test_zh_template.py;步骤1 config.WriterConfig.output_language→Settings.writer、GenerationContext.output_language+to_vars.language_instruction、writer_agent【语言约束】引变量、context_builder/orchestrator/run_trial 透传 --output-language + tests/test_language_plumbing.py;步骤A 新建 src/genesis/writer/language.pydetect_script/resolve_expected_language/find_language_violations)并接入 WriterAgent.generate_chapter 违规重试、max_retries 默认1→2 + tests/test_writer_language.py(待修正 resolve_expected_language 标题仅假名可信) | src/genesis/config.py; src/genesis/writer/models.py; src/genesis/writer/writer_agent.py; src/genesis/writer/context_builder.py; src/genesis/writer/orchestrator.py; src/genesis/writer/language.py; scripts/make_zh_template.py; scripts/run_trial.py; tests/test_zh_template.py; tests/test_language_plumbing.py; tests/test_writer_language.py; samples/概要设计书模板_中文.docx | x-preview-f-free (opencode) |
@@ -0,0 +1,155 @@
# 输出语言一致性保障 + 用户可选输出语言(2026-08-24
## 背景
真实 LLM 试运行暴露:生成正文曾出现中日混杂(早期第 1-2 章中文、第 3-7 章日文;prompt 渲染缺陷
修复后全日文,但属概率性改善)。盘点确认:
1. **唯一保障是 prompt 层【语言约束】**,无任何程序化检测(grep 全代码库证实)
2. **输入侧存在中文诱导源**`_format_impact()` 固定标签(`受影响:``[警告]`)进入每章 prompt
3. design.md §7.2 QA 十项清单**没有语言一致性校验项**
4. 日文章标题多为纯汉字(「5. DB設計」无假名),仅凭标题无法判定期望语言
用户需求升级:**让用户可选择输出中文还是日文**(参数化),并配套中文模板。
## 关键用户决策(已确认)
| # | 决策点 | 结论 |
|---|--------|------|
| 1 | Excel 表格数据是否翻译 | **照抄原文**——数据是源数据忠实呈现,翻译违反 §7.2 内容准确性/可追溯性 |
| 2 | 子节标题语言 | **双模板方案**——提供中文/日文两套模板,按用户选择使用;锚点 id 语言中立不变 |
| 3 | 违规处理 | 重试耗尽后**硬失败**WriterGenerationError),与既有失败语义一致 |
| 4 | 实施范围 | A(强制校验)+ B(impact 标签本地化)+ C(QA 第 11 维度)+ 参数贯通,全做 |
## 核心设计:模板 × 语言参数 双轨制
```
中文输出 → samples/概要设计书模板_中文.docx(新建镜像)+ --output-language zh
日文输出 → 既有日文模板 + --output-language ja(或不传,走 auto
```
| 层 | 日文模板(现状) | 中文模板(新增镜像) |
|---|---|---|
| H1/H2 标题 | 1. はじめに / 2.1 機能一覧表 | 1. 前言 / 2.1 功能一览表 |
| 锚点 id | `section:function_list` … | **完全相同(不变)** → 映射/注入器零改动 |
| 封面 | 概要設計書 / バージョン | 概要设计书 / 版本 |
## 实施清单(TDD,每项先 RED 后 GREEN
### 0. 中文样本模板
- 新建 `scripts/make_zh_template.py`python-docx 镜像日文模板结构
Title/H1×7/H2×6/锚点原样/封面字段),产出 `samples/概要设计书模板_中文.docx` 入库
- 测试:解析 zh 模板断言 7 章 anchor id 与 ja 模板逐一对应
### 1. 语言参数贯通
- `config.py`:新增 `WriterConfig(output_language: "auto"|"zh"|"ja" = "auto")``Settings.writer`
- `models.py``GenerationContext.output_language` 字段;`to_vars()` 输出动态
`{{language_instruction}}`(zh=必须使用中文撰写 / ja=必須日本語で記述 / auto=与标题一致)
- `writer_agent.py`:【语言约束】段落改引该变量
- `context_builder.py` / `orchestrator.py` / `run_trial.py``--output-language {auto,zh,ja}` 逐层透传
### A. 确定性脚本检测 + 失败重试【核心】
- 新建 `src/genesis/writer/language.py`
- `detect_script(text)`:含假名(U+304030FF)→ "ja"CJK 汉字零假名 → "zh";否则 None
- `resolve_expected_language(explicit, title, fallback_texts)`:显式 > 标题假名 > 规则文档主导脚本;
单一事实来源(A 的重试校验与 C 的 QA 维度共用)
- `violations(...)`:仅检 paragraph/note/list 正文块;**表格 rows 不检(照抄原文)、heading 不检(跟随模板)**;
违规判定 = 含 CJK 汉字零假名且长度 ≥12(防误杀短术语)
- `writer_agent.py``generate_chapter` 重试循环内后置校验,违规按生成失败重试;
`max_retries` 默认 1→2(现默认无重试机会,已确认既有测试均显式传参)
### B. `_format_impact()` 标签本地化
-`output_language` 选标签集(zh=受影响 / ja=影響;auto 默认 ja);只影响 prompt 变量,
不影响 impact-report.json 独立产物
### C. QA 第 11 维度 `language_consistency`
- `ChapterArtifact.expected_language: str = ""`(上层解析后传入,scorer 不自行推导)
- `qa/validator.py` 从 orchestrator 透传期望语言;歧义记 unverifiable 计入 detail
- `DEFAULT_THRESHOLDS["language_consistency"] = 1.0`
- ⚠️ 已识别风险:新维度改变 score() 总分均值,`test_eval_scorer.py` / `test_phase5_scorer.py`
具体分数断言需同步修正(诚实集成,不做旁路方法)
## 验收标准
```bash
# 中文概要设计书
python scripts/run_trial.py --template samples/概要设计书模板_中文.docx --output-language zh
# → 正文全中文、表格照抄日文原文、零中日混杂段落(程序化保证)
# 日文(现状默认,行为不变)
python scripts/run_trial.py
```
1. 全量 pytest ≥99% 覆盖率(当前基线 389 passed / 99.04%
2. 真实 LLM 双跑(zh/ja 各一次)+ 扫描脚本确认 output.docx 无违规件
3. 提交 commit + 追加 `_AI_USAGE_LOG.md`
## 涉及文件
- 新增:`scripts/make_zh_template.py``samples/概要设计书模板_中文.docx`
`src/genesis/writer/language.py``tests/test_writer_language.py`
- 修改:`config.py``writer/models.py``writer/writer_agent.py``writer/context_builder.py`
`writer/orchestrator.py``eval/scorer.py``qa/validator.py``scripts/run_trial.py`
- 测试更新:`tests/test_phase5_writer_agent.py``tests/test_phase5_models.py`
`tests/test_eval_scorer.py``tests/test_phase5_scorer.py``tests/test_run_trial.py`
## 状态
- [x] 计划已获用户批准(等待二次确认开工)
- [x] 计划评审(2026-08-24,对照真实代码逐条核验,见下节)
- [x] 步骤 0-3 实施
- [ ] 验收
## 验收记录(2026-08-25
- pytest 全量:**424 passed / 99.15%**(覆盖率门槛 99% 达标)
- Fake 模式双模板试运行:zh 模板 + `--output-language zh` → 7 章 docxja 模板默认 → 7 章 docx,均无崩溃
- 程序化扫描 `output/zh_output.docx`:25 段落,**0 日文假名混入**(语言一致性成立)
- 约束链路单测覆盖:writer 重试/硬失败、QA 第 11 维度、validator 透传、qa_loop 透传
- ⚠️ **未执行项**:真实 LLM 双语双跑需 `GENESIS_INFERENCE__API_KEY`(本环境无 key),确定性强制逻辑已由 e2e + 单测 + 覆盖率间接验证;真实双跑待 key 就绪后补。
## 实施进度(2026-08-25 暂停于步骤 A 修正点)
### 已完成
- **步骤 0(中文模板)**`scripts/make_zh_template.py` + `samples/概要设计书模板_中文.docx`(已生成入库)+ `tests/test_zh_template.py`4 passed)。锚点 id 与 ja 模板逐一对应、译文正确。
- **步骤 1(参数贯通)**`config.py` 新增 `WriterConfig.output_language``Settings.writer``GenerationContext.output_language`+`to_vars().language_instruction`zh/ja/auto 三档);`writer_agent.py`【语言约束】改引 `{{language_instruction}}``context_builder.build_contexts` 透传;`orchestrator.generate``output_language` 参数;`run_trial.py``--output-language``tests/test_language_plumbing.py`5 passed)。
- **步骤 A(检测+强制)部分**`src/genesis/writer/language.py` 已建(detect_script/has_kana/has_cjk/resolve_expected_language/find_language_violations);`writer_agent.generate_chapter` 已接入期望语言推导 + 违规重试 + `max_retries` 默认 1→2。`tests/test_writer_language.py` 多数通过。
### 待修正(暂停点)
- **`language.py``resolve_expected_language` 逻辑需修正**:当前实现在标题为纯汉字(如「DB 設計」,无假名)时 `detect_script` 返回 "zh",导致标题被误当作可靠信号、fallback 失效。
依据计划「显式 > **标题假名** > 规则文档主导脚本」,标题**仅当含假名(detect=="ja"**才可信;纯汉字标题对中/日均可能,应跳过、改看 fallback。
**修正**:将 `resolve_expected_language` 中标题分支改为 `if detect_script(title) == "ja": return "ja"`(不再用纯汉字标题推导 zh),随后才遍历 fallback_texts,最后返回 ""。
- 修正后重跑 `tests/test_writer_language.py` 应全绿(已有 2 个用例因该逻辑失败:`test_resolve_falls_back_to_fallback_text``test_resolve_unverifiable_when_no_hint`)。
### 后续(未动)
- 步骤 A 通过后→ 步骤 B`_format_impact` 标签本地化已预埋参数,待补 zh 标签集使其与 ja 实际不同 + 补测试)。
- 步骤 CQA 第 11 维度 `language_consistency` + `ChapterArtifact.expected_language` + `validator.validate_doc(expected_language=None)` + `DEFAULT_THRESHOLDS` 加项 + 同步修正 `test_eval_scorer.py`/`test_phase5_scorer.py` 总分断言(按评审 R1:不可验证=满分 1.0)。
- 验收:全量 pytest ≥99% + zh/ja 双真实试运行 + 提交 + `_AI_USAGE_LOG.md` 追加。
### 已落盘但未提交
- 上述步骤 0/1/A 的源码与测试改动**尚未 git commit**(暂停前未提交)。
## 评审记录(2026-08-24
### 核验通过的关键假设
1. `config.py` 为 pydantic-settings + `extra="ignore"`,新增 `Settings.writer` 字段无 yaml 时取默认值,零风险
2. 中文模板锚点 idsection:introduction/function_list/screen_list/report_list/db_design/if_definition/batch_list)已从真实日文模板解析确认,镜像后映射/注入器零改动
3. `_format_impact` 无测试断言"受影响"字样,标签本地化安全;`[新規]/[変更]/[削除]/[警告]` 中日通用保留
4. `WriterAgent.max_retries` 既有测试均显式传参(2 或 1),默认值 1→2 不破坏测试
5. prompt 段落名【语言约束】保留、内容改引变量 → 既有模板断言兼容
### 评审修正(2 处)
| # | 原计划 | 修正后 | 依据 |
|---|--------|--------|------|
| R1 | C 维度"歧义记 unverifiable 跳过" | **不可验证 = 满分通过(score 1.0)**,维度恒参与均值;detail 注明 "unverifiable" | 实测 `test_eval_scorer.py L154` / `test_phase5_scorer.py L82` 断言 `total_score == 1.0`——若跳过维度或给非满分,既有用例总分断言即破裂;恒定维度数 + 满分兜底可让多数既有断言不变 |
| R2 | `ChapterArtifact.title: str = ""``expected_language` 两版表述并存 | 统一为 **仅加 `expected_language: str = ""`**,scorer 完全不自行推导 | 单一事实来源原则,避免 scorer 内再出现一套标题猜测逻辑 |
### 评审补充(3 处说明)
| # | 说明 |
|---|------|
| N1 | `validate_doc` 增加**可选** kwarg `expected_language=None`qa_loop.py L55/L69 与 test_phase5_report/test_phase5_validator 共 4 处调用方零破坏);None 时该章按 unverifiable=满分处理 |
| N2 | `max_retries` 默认 2 后,orchestrator.py L72 与 qa_loop.py L28 两处真实管线在失败时会多一次 LLM 调用——与 QALoopController 外层循环叠加的成本上界 = 章 × 2 × QA轮次,可接受;实施时在 WriterAgent docstring 标注 |
| N3 | 运维注意事项写入 README/计划:zh 场景规则文档仍为日文 → auto 回落会判 ja,**中文输出必须显式 `--output-language zh`** |
### 遗留风险(接受)
- auto 模式对纯汉字日文正式文档(无假名标题+无假名规则)无法证明语言——保守跳过强制,由 C 维度报告层呈现 unverifiable
Binary file not shown.
+82
View File
@@ -0,0 +1,82 @@
"""生成中文镜像模板(步骤 0)。
镜像既有日文概要设计书模板的结构与锚点 id(section:xxx 原样保留),
仅将 H1/H2 标题与封面字段翻译为中文,产物 samples/概要设计书模板_中文.docx 入库。
锚点段落文本保持不变是硬约束:Writer 子系统(template_mapper / docx_injector
依赖这些占位符定位章节,任何改动都会破坏映射与注入。
"""
from __future__ import annotations
import shutil
from pathlib import Path
from docx import Document
# H1/H2 标题日文 → 中文(保持编号前缀一一对应)
HEADING_TRANSLATIONS: dict[str, str] = {
"1. はじめに": "1. 前言",
"2. 機能一覧": "2. 功能一览",
"2.1 機能一覧表": "2.1 功能一览表",
"2.2 機能詳細": "2.2 功能详细",
"3. 画面一覧": "3. 画面一览",
"3.1 画面遷移図": "3.1 画面迁移图",
"3.2 画面一覧表": "3.2 画面一览表",
"4. 帳票一覧": "4. 报表一览",
"5. DB設計": "5. DB设计",
"5.1 テーブル一覧": "5.1 表一览",
"5.2 ER図": "5.2 ER图",
"6. IF定義": "6. 接口定义",
"7. バッチ一覧": "7. 批处理一览",
}
# 封面字段(Normal 段落)日文 → 中文
COVER_TRANSLATIONS: dict[str, str] = {
"概要設計書": "概要设计书",
"ドキュメント名: {{doc_title}}": "文档名: {{doc_title}}",
"バージョン: {{version}}": "版本: {{version}}",
"作成日: {{created_at}}": "创建日期: {{created_at}}",
"テンプレート開始": "模板开始",
}
def _translate(text: str) -> str:
"""锚点段落({{section:...}})原样保留;其余按字典翻译,未命中则原样。"""
if text.startswith("{{section:"):
return text
return HEADING_TRANSLATIONS.get(text, COVER_TRANSLATIONS.get(text, text))
def build_zh_template(src: str, dst: str) -> str:
"""镜像 src(日文模板)为 dst(中文模板),返回 dst 路径。"""
src_path = Path(src)
dst_path = Path(dst)
if not src_path.exists():
raise FileNotFoundError(f"源模板不存在: {src_path}")
dst_path.parent.mkdir(parents=True, exist_ok=True)
# 先物理复制(保留样式/编号定义等所有非文本属性),再逐段改写文本
shutil.copyfile(str(src_path), str(dst_path))
doc = Document(str(dst_path))
for para in doc.paragraphs:
new_text = _translate(para.text)
if new_text != para.text:
para.text = new_text
doc.save(str(dst_path))
return str(dst_path)
def main() -> None:
import argparse
parser = argparse.ArgumentParser(description="生成中文镜像模板")
default_src = str(Path(__file__).resolve().parents[1] / "samples" / "概要設計書テンプレート.docx")
default_dst = str(Path(__file__).resolve().parents[1] / "samples" / "概要设计书模板_中文.docx")
parser.add_argument("--src", default=default_src, help="源日文模板")
parser.add_argument("--dst", default=default_dst, help="输出中文模板")
args = parser.parse_args()
out = build_zh_template(args.src, args.dst)
print(f"中文模板已生成: {out}")
if __name__ == "__main__":
main()
+3
View File
@@ -53,6 +53,8 @@ def _parse_args(argv: list[str] | None) -> argparse.Namespace:
p.add_argument("--existing-system", default=str(ROOT / "samples" / "existing-system"),
help="既有系统源码目录(追加/改修场景)")
p.add_argument("--language", default="java", help="既有系统开发语言(默认 java,None=自动探测)")
p.add_argument("--output-language", default="auto",
choices=["auto", "zh", "ja"], help="输出语言:auto=与标题一致,zh=简体中文,ja=日文")
p.add_argument("--samples-dir", default=str(ROOT / "samples"), help="样本资产根目录")
p.add_argument("--output", default=str(ROOT / "output" / "output.docx"), help="概要设计书输出路径")
p.add_argument("--impact-report", default=str(ROOT / "output" / "impact-report.json"),
@@ -86,6 +88,7 @@ def main(argv: list[str] | None = None) -> dict:
samples_dir=args.samples_dir,
engine=engine,
template_path=str(Path(args.template)),
output_language=args.output_language,
)
report = ss.impact_report
+15 -1
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import os
from pathlib import Path
from typing import Any
from typing import Any, Literal
import yaml
from pydantic import BaseModel, Field
@@ -127,6 +127,19 @@ class RagConfig(BaseModel):
rerank: RerankConfig = Field(default_factory=RerankConfig)
class WriterConfig(BaseModel):
"""Writer 子系统配置(步骤 1:输出语言参数)。
output_language: 生成概要设计书正文的自然语言
- "auto":与章节标题所用语言保持一致(默认,向后兼容既有日文文档)
- "zh":强制简体中文
- "ja":强制日文
表格数据始终照抄源 Excel 原文(不翻译),见 design.md §7.2。
"""
output_language: Literal["auto", "zh", "ja"] = "auto"
# ---------- 加载辅助 ----------
def _expand_env(data: Any) -> Any:
@@ -192,6 +205,7 @@ class Settings(BaseSettings):
app: AppConfig = Field(default_factory=AppConfig)
inference: InferenceConfig = Field(default_factory=InferenceConfig)
rag: RagConfig = Field(default_factory=RagConfig)
writer: WriterConfig = Field(default_factory=WriterConfig)
@classmethod
def from_dir(cls, config_dir: Path | str) -> "Settings":
+45
View File
@@ -34,6 +34,9 @@ class ChapterArtifact:
text: str
source_uris: list[str]
template_sections_expected: list[str]
expected_language: str = "" # 期望输出语言("zh"/"ja";空=不可验证,维度记满分)
# 块级 (type, text) 列表:供语言一致性维度排除 heading/table(照抄源/跟随模板)
blocks: list[tuple[str, str]] = field(default_factory=list)
# 逐章评估通过阈值(基于逐章总分)
@@ -56,6 +59,7 @@ DEFAULT_THRESHOLDS: dict[str, float] = {
"traceability": 1.0,
"placeholder_residue": 1.0,
"chapter_completeness": 1.0,
"language_consistency": 1.0,
}
@@ -75,6 +79,7 @@ class ChapterScorer:
dimensions.append(self._traceability(chapters, source))
dimensions.append(self._placeholder_residue(chapters))
dimensions.append(self._completeness(chapters))
dimensions.append(self._language_consistency(chapters))
# LLM 语义维度钩子(每个章节独立评,取该维度平均)
for name, fn in self.llm_evaluators.items():
@@ -144,6 +149,45 @@ class ChapterScorer:
detail = " | ".join(s.detail for s in scores) if scores else "no chapters"
return DimensionScore(name, round(avg, 4), all(s.passed for s in scores), detail)
# ---------- 语言一致性维度(步骤 C,确定性)----------
def _language_consistency(self, chapters: list[ChapterArtifact]) -> DimensionScore:
"""第 11 维度:输出语言与期望语言一致(确定性,脚本可验证)。
与 writer.language 共用单一检测事实来源。期望语言为空(auto/不可验证)
→ 记满分 1.0 通过(评审 R1:不拉低总分,避免误伤既有断言)。
仅检正文块(heading/table 不检,表格照抄源、标题跟随模板)。
"""
from genesis.writer.language import find_language_violations
if not chapters:
return DimensionScore("language_consistency", 1.0, True, "no chapters")
per: list[DimensionScore] = []
for ch in chapters:
expected = ch.expected_language
if not expected:
per.append(DimensionScore(
"language_consistency", 1.0, True, "unverifiable (no expected language)"))
continue
from genesis.writer.models import ContentBlock
if ch.blocks:
# 优先用块级信息(可排除 heading/table
blocks = [ContentBlock(block_id=str(i), type=t, text=tx)
for i, (t, tx) in enumerate(ch.blocks)]
else:
# 回退:整段正文作为单个 paragraph 块
blocks = [ContentBlock(block_id="0", type="paragraph", text=ch.text or "")]
viol = find_language_violations(blocks, expected)
score = 0.0 if viol else 1.0
passed = score >= self.thresholds["language_consistency"]
per.append(DimensionScore(
"language_consistency", score, passed,
f"期望 {expected},违规 {len(viol)}" if viol else f"期望 {expected},一致"))
avg = sum(p.score for p in per) / len(per)
passed = all(p.passed for p in per)
detail = " | ".join(p.detail for p in per)
return DimensionScore("language_consistency", round(avg, 4), passed, detail)
# ---------- 逐章评估 ----------
def _score_chapter(self, chapter: ChapterArtifact, source: StructuredSource) -> EvalReport:
@@ -156,6 +200,7 @@ class ChapterScorer:
self._traceability([chapter], source),
self._placeholder_residue([chapter]),
self._adequacy(chapter),
self._language_consistency([chapter]),
]
total = sum(d.score for d in dims) / len(dims) if dims else 0.0
failed = (not all(d.passed for d in dims)) or total < PASS_THRESHOLD
+9 -6
View File
@@ -21,8 +21,8 @@ class QALoop:
def __init__(self, max_rounds: int = DEFAULT_MAX_QA_ROUNDS) -> None:
self.controller = QALoopController(max_rounds=max_rounds)
def _build(self, structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id, only_ids=None, prev=None):
ctxs = build_contexts(structured_source, samples_dir)
def _build(self, structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id, only_ids=None, prev=None, output_language: str = "auto"):
ctxs = build_contexts(structured_source, samples_dir, output_language=output_language)
_warn_unanchored(ctxs)
state = WriterState([c.chapter_id for c in ctxs])
agent = WriterAgent(session_id=session_id, engine=engine, prompt_registry=prompt_registry, state=state)
@@ -47,12 +47,14 @@ class QALoop:
doc.save(output_path)
return [contents_map[cid] for cid in order]
def run(self, structured_source, output_path, session_id="writer", samples_dir="samples", engine=None, prompt_registry=None, template_path=None) -> QAReport:
def run(self, structured_source, output_path, session_id="writer", samples_dir="samples", engine=None, prompt_registry=None, template_path=None, output_language: str = "auto") -> QAReport:
engine = engine or build_inference_engine()
prompt_registry = prompt_registry or PromptRegistry()
validator = QAValidator()
contents = self._build(structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id)
report = validator.validate_doc(contents, structured_source)
# auto 不可推导期望语言 → 语言维度记满分(unverifiable);zh/ja 显式强制
expected = output_language if output_language in ("zh", "ja") else ""
contents = self._build(structured_source, samples_dir, engine, prompt_registry, template_path, output_path, session_id, output_language=output_language)
report = validator.validate_doc(contents, structured_source, expected_language=expected)
while self.controller.can_continue() and report.failed_chapters:
self.controller.advance()
contents = self._build(
@@ -65,6 +67,7 @@ class QALoop:
session_id,
only_ids=set(report.failed_chapters),
prev={c.chapter_id: c for c in contents},
output_language=output_language,
)
report = validator.validate_doc(contents, structured_source)
report = validator.validate_doc(contents, structured_source, expected_language=expected)
return report
+20 -8
View File
@@ -10,8 +10,12 @@ class QAValidator:
def __init__(self, scorer: ChapterScorer | None = None) -> None:
self.scorer = scorer or ChapterScorer()
def _to_artifact(self, content: ChapterContent) -> ChapterArtifact:
"""把 ChapterContent 转换为评分器所需的 ChapterArtifact(聚合正文与来源 URI)。"""
def _to_artifact(self, content: ChapterContent, expected_language: str = "") -> ChapterArtifact:
"""把 ChapterContent 转换为评分器所需的 ChapterArtifact(聚合正文与来源 URI)。
expected_language:期望输出语言("zh"/"ja";空=不可验证,维度记满分)。
blocks 保留 (type, text) 供语言维度排除 heading/table。
"""
text = "".join(b.text or "" for b in content.blocks)
source_uris: list[str] = []
for b in content.blocks:
@@ -21,18 +25,26 @@ class QAValidator:
text=text,
source_uris=source_uris,
template_sections_expected=[],
expected_language=expected_language,
blocks=[(b.type, b.text or "") for b in content.blocks],
)
def validate_chapter(self, content: ChapterContent, structured_source) -> tuple[bool, EvalReport]:
report = self.scorer.score([self._to_artifact(content)], structured_source)
def validate_chapter(
self, content: ChapterContent, structured_source, expected_language: str = ""
) -> tuple[bool, EvalReport]:
report = self.scorer.score([self._to_artifact(content, expected_language)], structured_source)
passed = content.chapter_id not in report.failed_chapters
return passed, report
def validate_document(self, contents: list[ChapterContent], structured_source) -> list[tuple[bool, EvalReport]]:
return [self.validate_chapter(c, structured_source) for c in contents]
def validate_document(
self, contents: list[ChapterContent], structured_source, expected_language: str = ""
) -> list[tuple[bool, EvalReport]]:
return [self.validate_chapter(c, structured_source, expected_language) for c in contents]
def validate_doc(self, contents: list[ChapterContent], structured_source) -> QAReport:
results = self.validate_document(contents, structured_source)
def validate_doc(
self, contents: list[ChapterContent], structured_source, expected_language: str = ""
) -> QAReport:
results = self.validate_document(contents, structured_source, expected_language)
passed_flags = [ok for ok, _ in results]
reports = [rep for _, rep in results]
failed = [c.chapter_id for c, ok in zip(contents, passed_flags) if not ok]
+6 -1
View File
@@ -7,7 +7,11 @@ from genesis.writer.models import ChapterSpec, GenerationContext
from genesis.writer.template_mapper import map_template
def build_contexts(structured_source: StructuredSource, samples_dir: str = "samples") -> list[GenerationContext]:
def build_contexts(
structured_source: StructuredSource,
samples_dir: str = "samples",
output_language: str = "auto",
) -> list[GenerationContext]:
specs: list[ChapterSpec] = map_template(structured_source.template)
rag = CannedRagService(samples_dir=samples_dir)
out: list[GenerationContext] = []
@@ -27,6 +31,7 @@ def build_contexts(structured_source: StructuredSource, samples_dir: str = "samp
template_styles=set(used),
prior_state=None,
impact_report=getattr(structured_source, "impact_report", None),
output_language=output_language,
)
)
return out
+104
View File
@@ -0,0 +1,104 @@
"""输出语言确定性检测与强制(步骤 A)。
设计要点:
- 日文标题多为纯汉字(如「DB設計」无假名),仅凭标题无法判定期望语言;
故 resolve_expected_language 采用两级推导:显式 > 标题假名 > 规则文档主导脚本。
- 检测仅基于「是否含日文假名」:CJK 汉字零假名视为中文(日文不可能不含假名地
使用汉字),反之中日混排含假名判日文。这是确定可机器验证的唯一稳健信号。
- find_language_violations 仅检正文类块(paragraph/note/list);heading 跟随模板、
table 照抄源 Excel 原文,二者均不检(design.md §7.2 内容准确性/可追溯性)。
- 短文本(<12 字)不误杀(如专有术语),阈值见 MIN_VIOLATION_LEN。
"""
from __future__ import annotations
from genesis.writer.models import ContentBlock
# 日文假名 Unicode 区间
_HIRAGANA = (0x3040, 0x309F)
_KATAKANA = (0x30A0, 0x30FF)
# 中日韩统一表意文字(CJK 汉字)
_CJK = (0x4E00, 0x9FFF)
# 受检的正文块类型(heading/table 不检)
_CHECKED_BLOCK_TYPES = {"paragraph", "note", "list"}
# 触发违规判定的最小正文长度(防短术语误杀)
MIN_VIOLATION_LEN = 12
def _in_range(ch: str, lo: int, hi: int) -> bool:
cp = ord(ch)
return lo <= cp <= hi
def has_kana(text: str) -> bool:
"""文本是否含日文假名(平假名/片假名)。"""
return any(_in_range(c, *_HIRAGANA) or _in_range(c, *_KATAKANA) for c in text)
def has_cjk(text: str) -> bool:
"""文本是否含 CJK 汉字。"""
return any(_in_range(c, *_CJK) for c in text)
def detect_script(text: str) -> str | None:
"""检测文本主导自然语言。
含假名 → "ja";含 CJK 汉字但零假名 → "zh";二者皆非(纯 ASCII 等)→ None。
"""
if not text:
return None
if has_kana(text):
return "ja"
if has_cjk(text):
return "zh"
return None
def resolve_expected_language(
explicit: str,
title: str = "",
fallback_texts: tuple[str, ...] | list[str] = (),
) -> str:
"""推导本章期望输出语言(单一事实来源,A 的重试校验与 C 的 QA 维度共用)。
- explicit 为 "zh"/"ja" → 直接采用(用户显式选择优先)
- 否则看标题是否含假名(仅假名可可靠判为日文;纯汉字标题对中/日均可能,不可信)
- 否则看 fallback_texts(如影响调查书/章节数据,通常日文)的主导脚本
- 均无法推导 → 返回 ""(不可验证,交由上层按 unverifiable 处理)
"""
if explicit in ("zh", "ja"):
return explicit
# 标题仅当含假名时可靠指示日文;纯汉字/ASCII 标题跳过,改看 fallback
if has_kana(title or ""):
return "ja"
for text in fallback_texts:
s = detect_script(text or "")
if s:
return s
return ""
def find_language_violations(blocks: list[ContentBlock], expected_language: str) -> list[str]:
"""返回违规正文块文本片段(期望语言非空时才有意义)。
违规判定:
- 期望 "ja":正文块含 CJK 汉字且零假名(即纯中文)且长度 ≥ 阈值
- 期望 "zh":正文块含日文假名
heading/table 块始终跳过(跟随模板 / 照抄源)。
"""
if expected_language not in ("zh", "ja"):
return []
violations: list[str] = []
for b in blocks:
if b.type not in _CHECKED_BLOCK_TYPES:
continue
text = b.text or ""
if len(text) < MIN_VIOLATION_LEN:
continue
if expected_language == "ja":
if has_cjk(text) and not has_kana(text):
violations.append(text)
else: # zh
if has_kana(text):
violations.append(text)
return violations
+40 -5
View File
@@ -109,6 +109,19 @@ class GenerationContext:
template_styles: set[str]
prior_state: object | None = None # WriterState,避免循环 import 用 object
impact_report: object | None = None # ImpactReport 影响调查书(生成主上下文)
output_language: str = "auto" # "auto" | "zh" | "ja"(步骤 1:用户可选输出语言)
def _language_instruction(self) -> str:
"""根据 output_language 生成【语言约束】段的具体指令(步骤 1)。"""
if self.output_language == "zh":
return "必须使用简体中文撰写(标题、正文与所有说明一律中文)。"
if self.output_language == "ja":
return "必ず日本語で記述すること(タイトル・本文・すべての説明は日本語)。"
# auto:沿用与标题语言一致的旧语义(向后兼容既有日文文档)
return (
f"必须与章节标题「{self.title}」所用自然语言保持一致:"
"标题为日文则用日文撰写,为中文则用中文撰写,依此类推。"
)
def to_vars(self) -> dict:
"""返回供 prompt 渲染的变量字典。"""
@@ -125,6 +138,7 @@ class GenerationContext:
f"- {h}" for h in (getattr(tm, "sub_headings", None) or [])
),
"prior_state": str(self.prior_state) if self.prior_state is not None else "",
"language_instruction": self._language_instruction(),
"data": _format_chapter_data(
self.structured_source,
CHAPTER_SHEET_TYPES.get(self.chapter_id, []),
@@ -132,6 +146,7 @@ class GenerationContext:
"impact": _format_impact(
self.impact_report,
CHAPTER_IMPACT_ELEMENT.get(self.chapter_id, None),
self.output_language,
),
}
@@ -177,11 +192,30 @@ def _format_chapter_data(structured_source: object | None, sheet_types: list[She
return "\n".join(lines).rstrip()
def _format_impact(report: object | None, element_type: ElementType | None = None) -> str:
# 影响调查标签本地化(步骤 B):auto/ja 默认日文,zh 中文
# 注:方括号标记 [..] 保留(中日通用),仅标签词本地化
_IMPACT_LABELS: dict[str, dict[str, str]] = {
"zh": {
"new": "新建", "modified": "变更", "deleted": "删除", "warning": "警告",
"affected": "受影响",
},
"ja": {
"new": "新規", "modified": "変更", "deleted": "削除", "warning": "警告",
"affected": "受影响",
},
}
def _format_impact(
report: object | None,
element_type: ElementType | None = None,
output_language: str = "auto",
) -> str:
"""将影响调查书格式化为 prompt 可读文本(无报告/无分析时为空串)。
element_type 非 None 时仅保留该类型要素(章节级定向,design.md §6.8 ①);
警告始终保留(不依赖要素类型)。
output_language 控制标签语言(步骤 B);auto 回落到 ja 标签。
"""
if report is None:
return ""
@@ -192,6 +226,7 @@ def _format_impact(report: object | None, element_type: ElementType | None = Non
def keep(el) -> bool:
return element_type is None or el.element_type == element_type.value
lab = _IMPACT_LABELS.get(output_language, _IMPACT_LABELS["ja"])
summary = getattr(report, "summary", {}) or {}
lines = [f"project_type={getattr(ca, 'project_type', '')}"]
lines.append(
@@ -204,15 +239,15 @@ def _format_impact(report: object | None, element_type: ElementType | None = Non
)
for el in getattr(ca, "new_elements", []) or []:
if keep(el):
lines.append(f"[新規] {el.element_id} {el.element_type} {el.name}")
lines.append(f"[{lab['new']}] {el.element_id} {el.element_type} {el.name}")
for el in getattr(ca, "modified_elements", []) or []:
if keep(el):
impacted = ", ".join(el.impacted_existing) or "-"
lines.append(f"[変更] {el.element_id} {el.element_type} {el.name}受影响: {impacted}")
lines.append(f"[{lab['modified']}] {el.element_id} {el.element_type} {el.name}{lab['affected']}: {impacted}")
for el in getattr(ca, "deleted_elements", []) or []:
if keep(el):
impacted = ", ".join(el.impacted_existing) or "-"
lines.append(f"[削除] {el.element_id} {el.element_type} {el.name}受影响: {impacted}")
lines.append(f"[{lab['deleted']}] {el.element_id} {el.element_type} {el.name}{lab['affected']}: {impacted}")
for w in getattr(ca, "warnings", []) or []:
lines.append(f"[警告] {w.element_id}: {w.issue}")
lines.append(f"[{lab['warning']}] {w.element_id}: {w.issue}")
return "\n".join(lines)
+2 -1
View File
@@ -56,6 +56,7 @@ class WriteOrchestrator:
template_path: str | None = None,
impact_report=None,
meta: dict | None = None,
output_language: str = "auto",
) -> list[ChapterContent]:
engine = engine or build_inference_engine()
prompt_registry = prompt_registry or PromptRegistry()
@@ -66,7 +67,7 @@ class WriteOrchestrator:
if impact_report is not None:
# 回填 structured_source,便于 QA/日志/后续下载
structured_source.impact_report = impact_report
ctxs = build_contexts(structured_source, samples_dir)
ctxs = build_contexts(structured_source, samples_dir, output_language=output_language)
_warn_unanchored(ctxs)
state = WriterState([c.chapter_id for c in ctxs])
agent = WriterAgent(session_id=session_id, engine=engine, prompt_registry=prompt_registry, state=state)
+26 -5
View File
@@ -13,6 +13,10 @@ from genesis.inference.types import Prompt, StructuredResult
from genesis.writer.models import ChapterContent, GenerationContext
from genesis.writer.writer_state import WriterState
from genesis.writer.exceptions import WriterGenerationError
from genesis.writer.language import (
find_language_violations,
resolve_expected_language,
)
WRITER_PROMPT_TEMPLATE = (
@@ -33,10 +37,9 @@ WRITER_PROMPT_TEMPLATE = (
"严格以「参考资料(本章对应数据)」中的要件定义数据和「影响调查上下文」为核心依据;"
"禁止输出与本章无关的系统整体架构、通用设计说明等内容,禁止套用其他章节的主题。"
"若本章数据为空,则基于规则与影响调查上下文简要撰写,不得虚构数据。\n"
"【语言约束】章节正文(所有 block 的 text 字段)所使用的自然语言"
"必须与章节标题「{{title}}」所用语言保持一致:标题为日文则用日文撰写,"
"为中文则用中文撰写,依此类推。"
)
"【语言约束】章节正文(所有 block 的 text 字段)所使用的自然语言{{language_instruction}}\n"
)
CHAPTER_OUTPUT_SCHEMA = {
"type": "object",
@@ -70,7 +73,7 @@ class WriterAgent:
engine: InferenceEngine,
prompt_registry: PromptRegistry,
state: WriterState,
max_retries: int = 1,
max_retries: int = 2,
) -> None:
self.session_id = session_id
self.engine = engine
@@ -132,6 +135,15 @@ class WriterAgent:
def generate_chapter(self, context: GenerationContext) -> ChapterContent:
self._chunk_source(context.structured_source) # 分块可用性验证(真实拼回留待后续)
# 步骤 A:推导本章期望输出语言(仅依赖 context,循环前置,稳定)
v = context.to_vars()
expected = resolve_expected_language(
explicit=context.output_language,
title=context.title,
fallback_texts=[v.get("impact", "") or "", v.get("data", "") or ""],
)
last_err: Exception | None = None
for _ in range(max(1, self.max_retries)):
try:
@@ -148,6 +160,15 @@ class WriterAgent:
# heading 块(prompt 约束为尽力而为,此处程序化强制)
if not (getattr(context.template_marker, "sub_headings", None) or []):
content.blocks = [b for b in content.blocks if b.type != "heading"]
# 步骤 A:输出语言一致性强制(期望语言可推导时,违规按失败重试)
if expected:
viol = find_language_violations(content.blocks, expected)
if viol:
last_err = WriterGenerationError(
f"章节 {context.chapter_id} 语言不一致(期望 {expected}"
f"发现 {len(viol)} 处违规正文)"
)
continue
self.state.record_success(content)
return content
raise WriterGenerationError(f"章节 {context.chapter_id} 重试耗尽: {last_err}")
+102
View File
@@ -0,0 +1,102 @@
"""步骤 A/C 边界覆盖测试(恢复覆盖率至 ≥99%)。"""
from __future__ import annotations
from pathlib import Path
from types import SimpleNamespace
from genesis.writer.language import find_language_violations
from genesis.writer.models import ChapterContent, ContentBlock, GenerationContext, ChapterSpec
from genesis.writer.writer_agent import WriterAgent
from genesis.writer.writer_state import WriterState
from genesis.writer.exceptions import WriterGenerationError
from genesis.qa.qa_loop import QALoop
from genesis.data_models import StructuredSource
from genesis.parsers.word_template_parser import WordTemplateParser
class _Reg:
@staticmethod
def get_or_create(name, template):
return SimpleNamespace(name=name, version="1", template=template)
def test_find_violations_unexpected_language_code_returns_empty():
blocks = [ContentBlock(block_id="0", type="paragraph", text="中文正文内容充分长度足够")]
# 非 zh/ja 的期望语言 → 不强制,返回空
assert find_language_violations(blocks, "xx") == []
def test_writer_agent_zh_expected_rejects_japanese():
class JpEngine:
def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2):
return SimpleNamespace(
data={"title": variables["title"],
"blocks": [{"type": "paragraph",
"text": "本機能は注文処理を行う画面であり、詳細は以下の通り。"}]},
status="ok",
)
agent = WriterAgent(session_id="s", engine=JpEngine(), prompt_registry=_Reg(),
state=WriterState(["db_design"]), max_retries=1)
ctx = GenerationContext(chapter_id="db_design", title="DB 設計",
template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"),
structured_source=None, write_rules=[], design_rules=[],
template_styles=set(), output_language="zh")
try:
agent.generate_chapter(ctx)
assert False, "应抛 WriterGenerationError"
except WriterGenerationError:
pass
def test_qa_loop_passes_with_explicit_zh_on_chinese_fake(tmp_path):
"""显式 zh + 中文 Fake → 语言维度通过(验证透传链路)。"""
template = Path("samples/phase5-slice/template.docx")
if not template.exists():
import pytest
pytest.skip("样本模板缺失")
parsed = WordTemplateParser().parse(str(template))
ss = StructuredSource(template=parsed, tables=[], rule_docs=[], image_analyses=[],
existing_system=None, comments=[])
out = tmp_path / "o.docx"
class ZhEngine:
def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2):
return SimpleNamespace(
data={"title": variables["title"],
"blocks": [{"type": "paragraph",
"text": "由 FakeLLM 生成的充分说明内容,满足写入规则要求。"}]},
status="ok",
)
loop = QALoop(max_rounds=1)
report = loop.run(ss, str(out), samples_dir="nonexistent_dir_xyz",
engine=ZhEngine(), template_path=str(template), output_language="zh")
assert report.passed is True
def test_qa_loop_fails_with_explicit_ja_on_chinese_fake(tmp_path):
"""显式 ja + 中文 Fake → 语言维度判失败。"""
template = Path("samples/phase5-slice/template.docx")
if not template.exists():
import pytest
pytest.skip("样本模板缺失")
parsed = WordTemplateParser().parse(str(template))
ss = StructuredSource(template=parsed, tables=[], rule_docs=[], image_analyses=[],
existing_system=None, comments=[])
out = tmp_path / "o.docx"
class ZhEngine:
def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2):
return SimpleNamespace(
data={"title": variables["title"],
"blocks": [{"type": "paragraph",
"text": "由 FakeLLM 生成的充分说明内容,满足写入规则要求。"}]},
status="ok",
)
loop = QALoop(max_rounds=1)
# 显式 ja + 中文 Fake:writer 生成阶段即因语言不一致硬失败(设计语义)
try:
loop.run(ss, str(out), samples_dir="nonexistent_dir_xyz",
engine=ZhEngine(), template_path=str(template), output_language="ja")
assert False, "应抛出 WriterGenerationError"
except WriterGenerationError:
pass
+93
View File
@@ -0,0 +1,93 @@
"""步骤 1:语言参数贯通测试(config → GenerationContext → to_vars → context_builder → run_trial)。"""
from __future__ import annotations
from genesis.data_models import (
CellValue, ExcelTable, ParsedTemplate, Provenance, SheetType, StructuredSource,
)
from genesis.writer.context_builder import build_contexts
from genesis.writer.models import ChapterSpec, GenerationContext
def _src() -> StructuredSource:
return StructuredSource(
tables=[],
template=ParsedTemplate("t.docx", [__import__("genesis.data_models", fromlist=["ChapterMarker"]).ChapterMarker("heading", "1. はじめに", 1)], {}, {"used": []}),
rule_docs=[], image_analyses=[], existing_system=None, comments=[],
)
def test_generation_context_default_output_language_is_auto():
ctx = GenerationContext(
chapter_id="db_design", title="DB 設計",
template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"),
structured_source=None, write_rules=[], design_rules=[], template_styles=set(),
)
assert ctx.output_language == "auto"
def test_to_vars_language_instruction_explicit_zh():
ctx = GenerationContext(
chapter_id="db_design", title="DB 设計",
template_marker=ChapterSpec(chapter_id="db_design", title="DB 设計"),
structured_source=None, write_rules=[], design_rules=[], template_styles=set(),
output_language="zh",
)
v = ctx.to_vars()
assert "language_instruction" in v
assert "简体中文" in v["language_instruction"]
def test_to_vars_language_instruction_explicit_ja():
ctx = GenerationContext(
chapter_id="db_design", title="DB 設計",
template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"),
structured_source=None, write_rules=[], design_rules=[], template_styles=set(),
output_language="ja",
)
v = ctx.to_vars()
assert "日本語" in v["language_instruction"]
def test_to_vars_language_instruction_auto_follows_title():
ctx = GenerationContext(
chapter_id="db_design", title="DB 設計",
template_marker=ChapterSpec(chapter_id="db_design", title="DB 設計"),
structured_source=None, write_rules=[], design_rules=[], template_styles=set(),
output_language="auto",
)
v = ctx.to_vars()
assert "标题" in v["language_instruction"] and "一致" in v["language_instruction"]
def test_context_builder_passes_output_language():
src = _src()
ctxs = build_contexts(src, output_language="zh")
assert all(c.output_language == "zh" for c in ctxs)
# 回归:默认仍为 auto
ctxs2 = build_contexts(src)
assert all(c.output_language == "auto" for c in ctxs2)
def test_format_impact_zh_labels_localized():
"""步骤 Boutput_language=zh 时影响调查标签本地化为中文,且不影响 ja 默认。"""
from genesis.data_models import (
ChangeAnalysis, ChangeElement, ChangeType, ImpactReport, ImpactWarning,
)
from genesis.writer.models import _format_impact
ca = ChangeAnalysis(
project_type="enhancement",
new_elements=[ChangeElement("F001", "機能", "止损风控", ChangeType.NEW)],
modified_elements=[ChangeElement("F002", "機能", "下单流程", ChangeType.MODIFIED)],
deleted_elements=[ChangeElement("F030", "機能", "旧功能", ChangeType.DELETED)],
unchanged_elements=[],
warnings=[ImpactWarning("F030", "缺少既存対応")],
)
report = ImpactReport(metadata={}, change_analysis=ca, summary={})
zh = _format_impact(report, None, "zh")
assert "[新建]" in zh and "[变更]" in zh and "[删除]" in zh and "[警告]" in zh
assert "受影响" in zh
# ja / auto 回落仍用日文标签
ja = _format_impact(report, None, "ja")
assert "[新規]" in ja and "[削除]" in ja
+5 -4
View File
@@ -12,13 +12,14 @@ from genesis.qa.qa_loop import QALoop
class FakeEngine:
def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2):
cid = variables.get("title", "x")
# 日文模板章节(标题含假名)→ 返回日文正文,满足语言一致性强制
return SimpleNamespace(
data={
"title": cid,
"blocks": [
{"type": "heading", "level": 2, "text": f"{cid}"},
{"type": "paragraph", "text": "FakeLLM 生成的充分说明内容,满足写入规则要求"},
{"type": "table", "caption": "示例表", "rows": [["列1", "列2"], ["1", "2"]]},
{"type": "heading", "level": 2, "text": f"{cid}"},
{"type": "paragraph", "text": "本機能はFakeLLMにより生成された十分な説明内容であり、書込規則を満たす"},
{"type": "table", "caption": "示例表", "rows": [["列1", "列2"], ["1", "2"]]},
],
},
status="ok",
@@ -50,5 +51,5 @@ def test_phase5_e2e_fake_llm(tmp_path):
assert report.passed is True
loaded = Document(str(out))
text = _all_text(loaded)
assert "FakeLLM 生成的充分说明内容" in text
assert "本機能はFakeLLMにより生成された十分な説明内容" in text
assert "示例表" in text
+92
View File
@@ -0,0 +1,92 @@
"""步骤 CQA 第 11 维度 language_consistency 测试。
评审 R1:不可验证(expected_language 为空/auto)→ 维度满分 1.0 通过,不拉低总分,
使既有 test_eval_scorer / test_phase5_scorer 的 total_score==1.0 断言继续成立。
"""
from __future__ import annotations
from genesis.data_models import (
CellValue, ExcelTable, ParsedTemplate, Provenance, SheetType, StructuredSource,
)
from genesis.eval.scorer import ChapterArtifact, ChapterScorer, EvalReport
def _dim(report: EvalReport, name: str):
for d in report.dimensions:
if d.name == name:
return d
raise AssertionError(f"维度未找到: {name}")
def _source() -> StructuredSource:
cell = CellValue(
value="登録",
provenance=Provenance(file_name="f.xlsx", sheet_name="機能一覧", row=3, column="C", column_header="x"),
)
table = ExcelTable(
name="機能一覧", detected_type=SheetType.FUNCTION,
extraction_method="structured", headers=["v"], rows=[{"v": cell}],
)
return StructuredSource(
tables=[table],
template=ParsedTemplate(file_name="t.docx", sections=[], placeholders={}, styles={}),
rule_docs=[], image_analyses=[], existing_system=None, comments=[],
)
def test_language_dim_unverifiable_is_full_score():
"""expected_language 为空(auto/不可验证)→ 维度满分通过,不拉低总分。"""
src = _source()
artifact = ChapterArtifact(
chapter_id="ch3", text="本機能は注文処理を行う画面である。",
source_uris=[], template_sections_expected=["ch3"],
expected_language="",
)
report = ChapterScorer().score([artifact], src)
dim = _dim(report, "language_consistency")
assert dim.score == 1.0
assert dim.passed is True
# 既有总分断言仍成立(全部确定性维度满分)
assert report.total_score == 1.0
def test_language_dim_ja_expected_but_chinese_fails():
src = _source()
artifact = ChapterArtifact(
chapter_id="ch3", text="这是一段纯中文的章节正文内容,应当判定为语言违规。",
source_uris=[], template_sections_expected=["ch3"],
expected_language="ja",
)
report = ChapterScorer().score([artifact], src)
dim = _dim(report, "language_consistency")
assert dim.score == 0.0
assert dim.passed is False
assert report.total_score < 1.0
def test_language_dim_zh_expected_japanese_fails():
src = _source()
artifact = ChapterArtifact(
chapter_id="ch3", text="本機能は注文処理を行う画面であり、詳細は以下の通り。",
source_uris=[], template_sections_expected=["ch3"],
expected_language="zh",
)
report = ChapterScorer().score([artifact], src)
dim = _dim(report, "language_consistency")
assert dim.score == 0.0
def test_language_dim_threshold_in_defaults():
assert ChapterScorer().thresholds["language_consistency"] == 1.0
def test_dimension_count_increases_by_one():
src = _source()
artifact = ChapterArtifact(
chapter_id="ch3", text="正常内容", source_uris=[], template_sections_expected=["ch3"],
)
before = len(ChapterScorer(thresholds={}, llm_evaluators={}).score([artifact], src).dimensions)
# 默认 scorer 含 language_consistency;对比一个不含该维度的基线不可行,
# 此处仅断言维度中包含 language_consistency 名称
report = ChapterScorer().score([artifact], src)
assert any(d.name == "language_consistency" for d in report.dimensions)
+172
View File
@@ -0,0 +1,172 @@
"""步骤 A:确定性语言检测 + 输出语言强制测试。
- detect_script:含假名→jaCJK 零假名→zh,否则 None
- resolve_expected_language:显式 > 标题假名 > 规则文档主导脚本 > ""(不可验证)
- find_language_violations:仅检正文块(heading/table 不检),长度≥12 防误杀
- WriterAgent.generate_chapter:期望语言非空时违规即重试,耗尽→硬失败
"""
from __future__ import annotations
from genesis.writer.language import (
detect_script,
find_language_violations,
resolve_expected_language,
)
from genesis.writer.models import ChapterContent, ContentBlock, GenerationContext, ChapterSpec
from genesis.writer.writer_agent import WriterAgent
from genesis.writer.writer_state import WriterState
from genesis.writer.exceptions import WriterGenerationError
# ---------- detect_script ----------
def test_detect_script_kana_is_ja():
assert detect_script("機能一覧の説明を記述します。") == "ja"
def test_detect_script_cjk_no_kana_is_zh():
assert detect_script("功能一览的说明内容。") == "zh"
def test_detect_script_ascii_is_none():
assert detect_script("Hello world 123") is None
def test_detect_script_mixed_cjk_kana_is_ja():
# 含假名即判 ja,即使混有汉字(中文不可能含日文假名)
assert detect_script("DB設計の概要を説明する。") == "ja"
# ---------- resolve_expected_language ----------
def test_resolve_explicit_overrides():
assert resolve_expected_language("zh", "DB 設計", ["日本語の影響"]) == "zh"
assert resolve_expected_language("ja", "DB 设計", ["中文影响"]) == "ja"
def test_resolve_falls_back_to_title_kana():
# 标题含假名→ja(注意:纯汉字日文标题如「DB設計」无假名,会落入 fallback)
assert resolve_expected_language("auto", "機能一覧の説明", []) == "ja"
def test_resolve_falls_back_to_fallback_text():
# 标题纯汉字无假名(中文式)→ 看 fallback(日文影响书)→ ja
assert resolve_expected_language("auto", "DB 設計", ["影響調査の結果(日本語)"]) == "ja"
def test_resolve_unverifiable_when_no_hint():
assert resolve_expected_language("auto", "DB 設計", ["123 abc"]) == ""
# ---------- find_language_violations ----------
def _blocks(*specs):
out = []
for i, (t, text) in enumerate(specs):
out.append(ContentBlock(block_id=str(i), type=t, text=text))
return out
def test_violation_ja_expected_but_pure_chinese_paragraph():
blocks = _blocks(("paragraph", "这是一段纯中文的章节正文内容。"))
viol = find_language_violations(blocks, "ja")
assert viol # 期望日文却为中文 → 违规
def test_no_violation_ja_expected_japanese_paragraph():
blocks = _blocks(("paragraph", "本機能は注文処理を行う画面であり、詳細は以下の通り。"))
assert not find_language_violations(blocks, "ja")
def test_heading_and_table_blocks_excluded():
# heading 与 table(照抄原文)即使含中文也不算违规
blocks = _blocks(
("heading", "機能一覧表"),
("table", "機能ID 機能名"), # 表格不检
("paragraph", "本機能は注文処理を行う。"),
)
assert not find_language_violations(blocks, "ja")
def test_short_chinese_not_flagged_under_threshold():
# 长度<12 的短术语不误杀
blocks = _blocks(("paragraph", "中文术语"))
assert not find_language_violations(blocks, "ja")
def test_violation_zh_expected_but_kana_present():
blocks = _blocks(("paragraph", "本機能は注文処理を行う画面である。"))
assert find_language_violations(blocks, "zh")
# ---------- WriterAgent 强制 ----------
class MixedLanguageEngine:
"""返回中英混杂(期望日文时违规)的章节内容。"""
def __init__(self, texts):
self._texts = list(texts)
self._i = 0
def chat_structured(self, *, session_id, prompt, variables, schema, retry_count=2):
from types import SimpleNamespace
text = self._texts[min(self._i, len(self._texts) - 1)]
self._i += 1
return SimpleNamespace(
data={"title": variables["title"], "blocks": [{"type": "paragraph", "text": text}]},
status="ok",
)
class FakePromptRegistry:
@staticmethod
def get_or_create(name, template):
from types import SimpleNamespace
return SimpleNamespace(name=name, version="1", template=template)
def _ctx_with_lang(cid, title, lang):
return GenerationContext(
chapter_id=cid, title=title,
template_marker=ChapterSpec(chapter_id=cid, title=title),
structured_source=None, write_rules=[], design_rules=[],
template_styles=set(), output_language=lang,
)
def test_generate_chapter_retries_on_language_violation_then_passes():
# 第一次返回中文(ja 期望→违规),第二次返回日文(通过)
engine = MixedLanguageEngine([
"这是一段纯中文的章节正文内容,应当被判定为语言违规。",
"本機能は注文処理を行う画面であり、詳細は以下の通り記述する。",
])
agent = WriterAgent(session_id="s", engine=engine,
prompt_registry=FakePromptRegistry(),
state=WriterState(["db_design"]), max_retries=3)
content = agent.generate_chapter(_ctx_with_lang("db_design", "DB 設計", "ja"))
assert content is not None
assert "本機能" in content.blocks[0].text
def test_generate_chapter_hard_fails_when_violation_persists():
# 始终返回中文(ja 期望),重试耗尽→硬失败
engine = MixedLanguageEngine([
"这是一段纯中文的章节正文内容,应当被判定为语言违规。",
"还是一段纯中文的章节正文内容,依旧违规。",
])
agent = WriterAgent(session_id="s", engine=engine,
prompt_registry=FakePromptRegistry(),
state=WriterState(["db_design"]), max_retries=2)
try:
agent.generate_chapter(_ctx_with_lang("db_design", "DB 設計", "ja"))
assert False, "应抛出 WriterGenerationError"
except WriterGenerationError as e:
assert "语言" in str(e) or "language" in str(e).lower()
def test_generate_chapter_auto_unverifiable_not_enforced():
# auto 且无法推导期望语言(标题纯汉字无假名、无日文 fallback)→ 不强制,直接通过
engine = MixedLanguageEngine(["这是中文正文但 auto 不可验证所以放行。"])
agent = WriterAgent(session_id="s", engine=engine,
prompt_registry=FakePromptRegistry(),
state=WriterState(["db_design"]), max_retries=1)
content = agent.generate_chapter(_ctx_with_lang("db_design", "DB 設計", "auto"))
assert content is not None
+77
View File
@@ -0,0 +1,77 @@
"""中文样本模板镜像测试(步骤 0)。
断言 zh 模板与 ja 模板章节结构一致:7 个 H1 章、锚点 id(section:xxx)逐一对应,
且锚点段落文本原样保留(映射/注入器零改动)。
"""
from __future__ import annotations
import shutil
from pathlib import Path
import pytest
from docx import Document
from genesis.parsers.word_template_parser import WordTemplateParser
from genesis.writer.template_mapper import map_template
_SAMPLES = Path(r"D:\00_project\Genesis\samples")
_JA = _SAMPLES / "概要設計書テンプレート.docx"
_ZH = _SAMPLES / "概要设计书模板_中文.docx"
EXPECTED_ANCHORS = [
"introduction", "function_list", "screen_list", "report_list",
"db_design", "if_definition", "batch_list",
]
@pytest.fixture
def zh_template(tmp_path):
"""生成 zh 模板到临时目录并返回路径(不污染 samples/)。"""
from scripts.make_zh_template import build_zh_template
dst = tmp_path / "概要设计书模板_中文.docx"
build_zh_template(str(_JA), str(dst))
return dst
def _parse(path: Path):
return WordTemplateParser().parse(str(path))
def test_zh_template_has_same_seven_chapter_anchors(zh_template):
ja = _parse(_JA)
zh = _parse(zh_template)
ja_ids = [ph for ph in ja.placeholders if ph.startswith("section:")]
zh_ids = [ph for ph in zh.placeholders if ph.startswith("section:")]
assert sorted(zh_ids) == sorted(ja_ids)
assert zh_ids == [f"section:{a}" for a in EXPECTED_ANCHORS]
def test_zh_template_anchor_paragraphs_unchanged(zh_template):
"""锚点段落文本在 zh 模板中必须与 ja 完全一致(映射/注入器依赖它)。"""
ja_doc = Document(str(_JA))
zh_doc = Document(str(zh_template))
ja_anchors = [p.text for p in ja_doc.paragraphs if p.text.startswith("{{section:")]
zh_anchors = [p.text for p in zh_doc.paragraphs if p.text.startswith("{{section:")]
assert zh_anchors == ja_anchors
def test_zh_template_headings_translated(zh_template):
zh_doc = Document(str(zh_template))
texts = {p.text for p in zh_doc.paragraphs if p.style.name.startswith("Heading")}
assert "1. 前言" in texts
assert "2. 功能一览" in texts
assert "5. DB设计" in texts
assert "7. 批处理一览" in texts
def test_zh_template_maps_to_same_seven_chapters(zh_template):
"""template_mapper 对 zh 模板产出 7 个章节,锚点与 ja 对应。"""
pt = _parse(zh_template)
specs = map_template(pt)
assert len(specs) == 7
assert [s.section_placeholder for s in specs] == [
f"section:{a}" for a in EXPECTED_ANCHORS
]