diff --git a/docs/superpowers/plans/2026-08-08-milestone2-excel-parser.md b/docs/superpowers/plans/2026-08-08-milestone2-excel-parser.md new file mode 100644 index 0000000..9739581 --- /dev/null +++ b/docs/superpowers/plans/2026-08-08-milestone2-excel-parser.md @@ -0,0 +1,991 @@ +# Phase1 里程碑2 Parser Agent - Excel 解析 实施计划 + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 实现 ExcelParser,将要件定义 Excel(3 种模式:表格型/自由记述型/混合型)解析为 `ExcelParseResult`(tables + comments),为后代 Agent 提供唯一数据源。 + +**Architecture:** 在已完成的 `data_models.py` 基础上,按 design §3.5 拆分为职责单一模块并由 `ExcelParser` 编排。自由记述型的 LLM 结构化依赖 InferenceEngine(后续里程碑),本里程碑交付可独立测试的文本分段与占位表(`extraction_method="llm_from_free_text"`)。 + +**Tech Stack:** Python ≥3.11、openpyxl ≥3.1、pytest 8。 + +## Global Constraints + +- 仅读取 `.xlsx`;`.xls` 由 `excel_reader.open_workbook` 抛出 `ValueError`。 +- Sheet 类型判定关键词为日文(design §3.5.1),用于匹配真实日文要件定义(技术必要保留)。 +- 每个单元格生成 `Provenance`;`source_uri` 格式 `file.xlsx#SheetName!CellRef`(design §9.4.5,`build_source_uri` 提供)。 +- 合并单元格下行填充(forward_fill,design §3.5.3)。 +- 取消线/背景色/批注保留在 `CellFormatting` / `CellComment`(design §3.5.4 / §3.5.5)。 +- 每个单元格使用 `data_models` 的 `CellValue`(字段 `value` / `provenance` / `formatting` / `comment`)。 +- 交流语言统一中文(注释/提交信息/本计划正文);标识符与技术名保留英文/日文关键词。 +- 每个任务结束前 `pytest` 全绿并提交。 + +--- + +### Task 1: ExcelParser 基础结构与来源标注 + +**Files:** +- Create: `src/genesis/parsers/__init__.py` +- Create: `src/genesis/parsers/provenance.py` +- Create: `src/genesis/parsers/excel_reader.py` +- Create: `tests/excel_helpers.py` +- Create: `tests/test_excel_reader.py` + +**Interfaces:** +- Produces: + - `genesis.parsers.provenance.build_source_uri(file_name: str, sheet_name: str, cell_ref: str) -> str` + - `genesis.parsers.excel_reader.open_workbook(path: str | Path)` → `openpyxl.Workbook`(仅 `.xlsx`) + - `genesis.parsers.excel_reader.sheet_matrix(ws) -> list[list[Any]]` + +测试辅助 `tests/excel_helpers.py` 供全部任务复用。 + +- [ ] **Step 1: 写失败测试** + +`tests/excel_helpers.py`: +```python +from openpyxl import Workbook + + +def new_workbook(sheets: dict[str, list[list]]) -> Workbook: + """生成临时 Workbook:key=Sheet 名,value=grid(cell 值)。""" + wb = Workbook() + wb.remove(wb.active) + for name, grid in sheets.items(): + ws = wb.create_sheet(name) + for r, row in enumerate(grid, start=1): + for c, value in enumerate(row, start=1): + ws.cell(row=r, column=c, value=value) + return wb + + +def save_workbook(tmp_path, wb: Workbook) -> str: + """落盘到 tmp_path 并返回路径字符串。""" + path = tmp_path / "source.xlsx" + wb.save(path) + return str(path) +``` + +`tests/test_excel_reader.py`: +```python +import pytest + +from genesis.parsers.excel_reader import open_workbook, sheet_matrix +from genesis.parsers.provenance import build_source_uri + +from tests.excel_helpers import new_workbook, save_workbook + + +def test_build_source_uri_format(): + assert build_source_uri("要求.xlsx", "機能一覧", "A5") == "要求.xlsx#機能一覧!A5" + + +def test_open_workbook_and_sheet_matrix(tmp_path): + wb = new_workbook({"機能一覧": [["機能ID", "機能名"], ["F001", "社員登録"]]}) + path = save_workbook(tmp_path, wb) + ws = open_workbook(path)["機能一覧"] + assert sheet_matrix(ws) == [["機能ID", "機能名"], ["F001", "社員登録"]] + + +def test_open_workbook_rejects_xls(tmp_path): + bad = tmp_path / "old.xls" + bad.write_bytes(b"not really xls") + with pytest.raises(ValueError): + open_workbook(bad) +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_excel_reader.py -v` +Expected: FAIL(`ModuleNotFoundError: No module named 'genesis.parsers'`) + +- [ ] **Step 3: 实现基础模块** + +`src/genesis/parsers/__init__.py`: +```python +"""Parser Agent:输入资料解析层。""" +``` + +`src/genesis/parsers/provenance.py`: +```python +def build_source_uri(file_name: str, sheet_name: str, cell_ref: str) -> str: + """单元格来源 URI:file.xlsx#SheetName!CellRef""" + return f"{file_name}#{sheet_name}!{cell_ref}" +``` + +`src/genesis/parsers/excel_reader.py`: +```python +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from openpyxl import load_workbook +from openpyxl.worksheet.worksheet import Worksheet + + +def open_workbook(path: str | Path): + """普通模式打开 .xlsx(保留公式/样式/批注),.xls 报错。""" + path = Path(path) + if path.suffix.lower() != ".xlsx": + raise ValueError(f"不支持的 Excel 格式: {path.suffix}") + return load_workbook(path) + + +def sheet_matrix(ws: Worksheet) -> list[list[Any]]: + """整表矩形值(含 None),保留到 max_column。""" + return [[cell.value for cell in row] for row in ws.iter_rows()] +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_excel_reader.py -v` +Expected: PASS(3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers tests/excel_helpers.py tests/test_excel_reader.py +git commit -m "feat: ExcelParser 基础(workbook/sheet 读取 + source_uri)" +``` + +> 注:计划中 Step 3 与 Step 4 之间未设独立 Step;Step 3 写完后直接执行 Step 4 运行验证。 + +--- + +### Task 2: SheetDetector(自动识别 Sheet 类型) + +**Files:** +- Create: `src/genesis/parsers/sheet_detector.py` +- Create: `tests/test_sheet_detector.py` + +**Interfaces:** +- Consumes: `data_models.SheetType` +- Produces: `detect_sheet_type(sheet_name: str, matrix: list[list[Any]]) -> SheetType` + (优先级:Sheet 名关键词 → 表头关键词 → `GENERIC`) + +- [ ] **Step 1: 写失败测试** + +`tests/test_sheet_detector.py`: +```python +from genesis.data_models import SheetType +from genesis.parsers.sheet_detector import detect_sheet_type + + +def test_detect_by_sheet_name(): + assert detect_sheet_type("機能一覧", []) == SheetType.FUNCTION + assert detect_sheet_type("画面一覧", []) == SheetType.SCREEN + assert detect_sheet_type("帳票一覧", []) == SheetType.REPORT + assert detect_sheet_type("DB定義", []) == SheetType.DATABASE + assert detect_sheet_type("IF定義", []) == SheetType.INTERFACE + assert detect_sheet_type("バッチ一覧", []) == SheetType.BATCH + assert detect_sheet_type("コード管理", []) == SheetType.MASTER + + +def test_detect_by_header_keyword(): + matrix = [["帳票ID", "帳票名"], ["TB1", "月次"]] + assert detect_sheet_type("補充シート", matrix) == SheetType.REPORT + + +def test_unknown_is_generic(): + assert detect_sheet_type("メモ", [["随筆"]]) == SheetType.GENERIC +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_sheet_detector.py -v` +Expected: FAIL(`ImportError: cannot import name 'detect_sheet_type'`) + +- [ ] **Step 3: 实现 sheet_detector.py** + +`src/genesis/parsers/sheet_detector.py`: +```python +from __future__ import annotations + +from typing import Any + +from genesis.data_models import SheetType + +# Sheet 名关键词(design §3.5.1,顺序即优先级) +NAME_KEYWORDS: list[tuple[str, SheetType]] = [ + ("機能", SheetType.FUNCTION), + ("画面", SheetType.SCREEN), + ("帳票", SheetType.REPORT), + ("テーブル", SheetType.DATABASE), + ("DB", SheetType.DATABASE), + ("インターフェース", SheetType.INTERFACE), + ("IF", SheetType.INTERFACE), + ("バッチ", SheetType.BATCH), + ("ジョブ", SheetType.BATCH), + ("マスタ", SheetType.MASTER), +] + +# 表头关键词 +HEADER_KEYWORDS: list[tuple[str, SheetType]] = [ + ("機能ID", SheetType.FUNCTION), + ("画面ID", SheetType.SCREEN), + ("帳票ID", SheetType.REPORT), + ("テーブルID", SheetType.DATABASE), + ("IF名", SheetType.INTERFACE), + ("バッチID", SheetType.BATCH), +] + + +def _name_hit(sheet_name: str) -> SheetType | None: + for kw, st in NAME_KEYWORDS: + if kw in sheet_name: + return st + return None + + +def _header_hit(matrix: list[list[Any]]) -> SheetType | None: + for row in matrix[:3]: + for cell in row: + if isinstance(cell, str): + for kw, st in HEADER_KEYWORDS: + if kw in cell: + return st + return None + + +def detect_sheet_type(sheet_name: str, matrix: list[list[Any]]) -> SheetType: + return _name_hit(sheet_name) or _header_hit(matrix) or SheetType.GENERIC +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_sheet_detector.py -v` +Expected: PASS(3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers/sheet_detector.py tests/test_sheet_detector.py +git commit -m "feat: SheetDetector 类型识别(名称/表头关键词)" +``` + +--- + +### Task 3: Sheet 性质判定(表格型/自由记述型/混合型) + +**Files:** +- Create: `src/genesis/parsers/sheet_nature.py` +- Create: `tests/test_sheet_nature.py` + +**Interfaces:** +- Produces: + - `SheetNature(Enum)`:`TABLE` / `FREE_TEXT` / `MIXED` + - `classify_sheet(matrix: list[list[Any]]) -> SheetNature` + - `find_header_row(matrix: list[list[Any]]) -> int`(首个连续非空 ≥2 的行的下标;无则 -1) + +判定启发式(design §3.5.2 落地): +- 空行占比 > 30%(`非空行数 / 总行数 < 0.7`)→ FREE_TEXT +- 仅 A 列使用(`max_cols <= 1`)→ FREE_TEXT +- 无表头行(`find_header_row == -1`)→ FREE_TEXT +- 表头行后存在以「・」/「■」开头的碎片行 → MIXED +- 其余 → TABLE + +- [ ] **Step 1: 写失败测试** + +`tests/test_sheet_nature.py`: +```python +from genesis.parsers.sheet_nature import SheetNature, classify_sheet, find_header_row + + +def test_table_detection(): + m = [["機能ID", "機能名"], ["A001", "社員登録"], ["A002", "退職処理"]] + assert classify_sheet(m) == SheetNature.TABLE + + +def test_free_text_single_col(): + m = [["新入社員を登録できる。"], ["氏名・所属・入社日を入力する。"]] + assert classify_sheet(m) == SheetNature.FREE_TEXT + + +def test_free_text_many_empty_rows(): + m = [["要求A"], [], [], [], ["要求B"]] + assert classify_sheet(m) == SheetNature.FREE_TEXT + + +def test_header_row_index(): + m = [["機能ID", "名前"], ["1", "田中"]] + assert find_header_row(m) == 0 + assert find_header_row([["自由テキスト"]]) == -1 +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_sheet_nature.py -v` +Expected: FAIL(导入错误) + +- [ ] **Step 3: 实现 sheet_nature.py** + +`src/genesis/parsers/sheet_nature.py`: +```python +from __future__ import annotations + +from enum import Enum +from typing import Any + + +class SheetNature(Enum): + TABLE = "table" + FREE_TEXT = "free_text" + MIXED = "mixed" + + +def _non_empty(row: list[Any]) -> list[Any]: + return [c for c in row if c is not None and str(c).strip() != ""] + + +def find_header_row(matrix: list[list[Any]]) -> int: + for i, row in enumerate(matrix): + if len(_non_empty(row)) >= 2: + return i + return -1 + + +def _free_text_like(matrix: list[list[Any]]) -> bool: + if not matrix: + return True + max_cols = max((len(row) for row in matrix), default=0) + if max_cols <= 1: + return True + non_empty_rows = [r for r in matrix if _non_empty(r)] + if len(non_empty_rows) / len(matrix) < 0.7: + return True + return find_header_row(matrix) == -1 + + +def classify_sheet(matrix: list[list[Any]]) -> SheetNature: + if _free_text_like(matrix): + return SheetNature.FREE_TEXT + header_row = find_header_row(matrix) + if header_row >= 0: + for row in matrix[header_row + 1:]: + if any(str(c).strip().startswith(("・", "■")) for c in _non_empty(row)): + return SheetNature.MIXED + return SheetNature.TABLE +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_sheet_nature.py -v` +Expected: PASS(4 passed) + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers/sheet_nature.py tests/test_sheet_nature.py +git commit -m "feat: Sheet 性质判定(table/free_text/mixed)" +``` + +--- + +### Task 4: MergeHandler 与 TableExtractor + +**Files:** +- Create: `src/genesis/parsers/merge_fill.py` +- Create: `src/genesis/parsers/table_extractor.py` +- Create: `tests/test_table_extractor.py` + +**Interfaces:** +- Consumes: `data_models`(`CellValue`、`ExcelTable`、`Provenance`、`SheetType`)、`build_source_uri` +- Produces: + - `merge_fill.forward_fill(matrix, merged_ranges) -> list[list[Any]]` + - `table_extractor.extract_table(sheet_name, matrix, file_name, detected_type) -> ExcelTable` + - `table_extractor.column_letter(index) -> str`(1→A、27→AA) + +合并单元格展开:范围 `(min_row, min_col, max_row, max_col)`(1-based);将范围内全部单元格填为主格(左上角)值。 + +- [ ] **Step 1: 写失败测试** + +`tests/test_table_extractor.py`: +```python +from genesis.data_models import CellValue, ExcelTable, SheetType +from genesis.parsers.merge_fill import forward_fill +from genesis.parsers.table_extractor import column_letter, extract_table + + +def test_forward_fill_vertical(): + matrix = [ + ["機能ID", "機能名", "備考"], + ["A001", "", ""], + ["", "", "メモ"], + ] + filled = forward_fill(matrix, [(1, 2, 2, 2)]) # B1:B2 纵向合并 + assert filled[1][1] == "機能名" + assert filled[2][2] == "メモ" + + +def test_forward_fill_horizontal(): + matrix = [["A", "B"], ["x", ""]] + filled = forward_fill(matrix, [(2, 1, 2, 2)]) # A2:B2 横向合并 + assert filled[1][1] == "x" + assert filled[1][0] == "x" + + +def test_column_letter(): + assert column_letter(1) == "A" + assert column_letter(27) == "AA" + + +def test_extract_table_basic(): + matrix = [["ID", "名前"], ["1", "田中"], ["2", "佐藤"]] + table = extract_table("社員一覧", matrix, "f.xlsx", SheetType.FUNCTION) + assert isinstance(table, ExcelTable) + assert table.headers == ["ID", "名前"] + assert table.extraction_method == "openpyxl" + assert len(table.rows) == 2 + first = table.rows[0] + assert isinstance(first["ID"], CellValue) + assert first["ID"].value == "1" + assert first["ID"].provenance.sheet_name == "社員一覧" + assert first["ID"].provenance.column == "A" +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_table_extractor.py -v` +Expected: FAIL(导入错误) + +- [ ] **Step 3: 实现** + +`src/genesis/parsers/merge_fill.py`: +```python +from __future__ import annotations + +from typing import Any + + +def forward_fill( + matrix: list[list[Any]], + merged_ranges: list[tuple[int, int, int, int]], +) -> list[list[Any]]: + """合并单元格:用左上角主格值填充范围内全部单元格。""" + out = [list(row) for row in matrix] + for (min_row, min_col, max_row, max_col) in merged_ranges: + if not out or min_row > len(out) or min_col > len(out[min_row - 1]): + continue + main_value = out[min_row - 1][min_col - 1] + for r in range(min_row, min(max_row, len(out)) + 1): + row = out[r - 1] + for c in range(min_col, min(max_col, len(row)) + 1): + row[c - 1] = main_value + return out +``` + +`src/genesis/parsers/table_extractor.py`: +```python +from __future__ import annotations + +from typing import Any + +from genesis.data_models import CellValue, ExcelTable, Provenance, SheetType + + +def column_letter(index: int) -> str: + """1 → A、27 → AA。""" + letters = "" + while index > 0: + index, rem = divmod(index - 1, 26) + letters = chr(65 + rem) + letters + return letters + + +def extract_table( + sheet_name: str, + matrix: list[list[Any]], + file_name: str, + detected_type: SheetType, + header_row: int = 0, +) -> ExcelTable: + """从矩阵提取表格:首行视为表头,其后为数据行。""" + if not matrix: + return ExcelTable( + name=sheet_name, detected_type=detected_type, + extraction_method="openpyxl", headers=[], rows=[], + ) + headers = [str(c) if c is not None else "" for c in matrix[header_row]] + rows = [] + for r in range(header_row + 1, len(matrix)): + row_dict = {} + for c, h in enumerate(headers): + raw = matrix[r][c] if c < len(matrix[r]) else None + row_dict[h] = CellValue( + value=raw, + provenance=Provenance( + file_name=file_name, + sheet_name=sheet_name, + row=r - header_row, # 数据行号从 1 开始 + column=column_letter(c + 1), + column_header=h, + ), + ) + rows.append(row_dict) + return ExcelTable( + name=sheet_name, detected_type=detected_type, + extraction_method="openpyxl", headers=headers, rows=rows, + ) +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_table_extractor.py -v` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers/merge_fill.py src/genesis/parsers/table_extractor.py tests/test_table_extractor.py +git commit -m "feat: MergeHandler + TableExtractor(forward_fill / 表格提取)" +``` + +--- + +### Task 5: FormattingDetector(取消线/背景色/批注) + +**Files:** +- Create: `src/genesis/parsers/formatting_detector.py` +- Create: `tests/test_formatting_detector.py` + +**Interfaces:** +- Consumes: openpyxl `Worksheet`/`Cell`、`data_models.CellFormatting / CellComment`、`build_source_uri` +- Produces: + - `cell_formatting(cell) -> CellFormatting | None`(strike / 字体色 / 背景色,纯默认则 None) + - `cell_comment(cell, file_name) -> CellComment | None` + - `collect_comments(ws, file_name) -> list[CellComment]` + +- [ ] **Step 1: 写失败测试** + +`tests/test_formatting_detector.py`: +```python +from openpyxl import Workbook +from openpyxl.comments import Comment + +from genesis.data_models import CellComment, CellFormatting +from genesis.parsers.formatting_detector import ( + cell_comment, cell_formatting, collect_comments, +) + + +def make_wb(): + wb = Workbook() + ws = wb.active + ws.title = "機能一覧" + ws["A1"] = "F001" + ws["A1"].font.strike = True + ws["A2"] = "F002" + ws["A2"].comment = Comment("要確認", "reviewer") + return wb + + +def test_cell_formatting_detects_strike(): + fmt = cell_formatting(make_wb().active["A1"]) + assert fmt is not None + assert fmt.strikethrough is True + + +def test_cell_formatting_none_when_plain(): + wb = Workbook() + ws = wb.active + ws["A1"] = "x" + assert cell_formatting(ws["A1"]) is None + + +def test_cell_comment_returns_obj(): + wb = make_wb() + cm = cell_comment(wb.active["A2"], "f.xlsx") + assert isinstance(cm, CellComment) + assert cm.author == "reviewer" + assert cm.text == "要確認" + assert cm.source_uri == "f.xlsx#機能一覧!A2" + + +def test_collect_comments(): + comments = collect_comments(make_wb().active, "f.xlsx") + assert len(comments) == 1 +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_formatting_detector.py -v` +Expected: FAIL(导入错误) + +- [ ] **Step 3: 实现** + +`src/genesis/parsers/formatting_detector.py`: +```python +from __future__ import annotations + +from genesis.data_models import CellComment, CellFormatting +from genesis.parsers.provenance import build_source_uri + + +def cell_formatting(cell) -> CellFormatting | None: + strike = bool(cell.font.strike) + font_color = None + if cell.font.color and str(cell.font.color.rgb) not in ("00000000", "FF000000"): + font_color = str(cell.font.color.rgb) + bg_color = None + fill = cell.fill + if fill and fill.fgColor and str(fill.fgColor.rgb) not in ("00000000", "FF000000"): + bg_color = str(fill.fgColor.rgb) + if strike or font_color or bg_color: + return CellFormatting( + strikethrough=strike, font_color=font_color, bg_color=bg_color, + ) + return None + + +def cell_comment(cell, file_name: str) -> CellComment | None: + if cell.comment is None: + return None + return CellComment( + author=cell.comment.author or "", + text=cell.comment.text or "", + source_uri=build_source_uri(file_name, cell.parent.title, cell.coordinate), + ) + + +def collect_comments(ws, file_name: str) -> list[CellComment]: + result = [] + for row in ws.iter_rows(): + for cell in row: + cm = cell_comment(cell, file_name) + if cm is not None: + result.append(cm) + return result +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_formatting_detector.py -v` +Expected: PASS(4 passed) + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers/formatting_detector.py tests/test_formatting_detector.py +git commit -m "feat: FormattingDetector(取消线/背景色/批注)" +``` + +--- + +### Task 6: FreeTextExtractor(自由记述型 Sheet 结构占位) + +**Files:** +- Create: `src/genesis/parsers/free_text_extractor.py` +- Create: `tests/test_free_text_extractor.py` + +**Interfaces:** +- Consumes: `data_models`(`CellValue`、`ExcelTable`、`Provenance`、`SheetType`) +- Produces: + - `extract_text_blocks(matrix) -> list[str]`(全空行分段,行内单元格以空格连接) + - `build_free_text_table(sheet_name, blocks, file_name, detected_type=SheetType.GENERIC) -> ExcelTable`(`extraction_method="llm_from_free_text"`,headers=["text"]) + +> LLM:本里程碑不调用 LLM(InferenceEngine 未实现);占位表为后续 LLM 结构化保留入口。 + +- [ ] **Step 1: 写失败测试** + +`tests/test_free_text_extractor.py`: +```python +from genesis.data_models import CellValue, ExcelTable, SheetType +from genesis.parsers.free_text_extractor import build_free_text_table, extract_text_blocks + + +def test_extract_blocks_splits_on_empty_rows(): + m = [["新入社員を登録。"], [], ["テスト要件:入力。"], [], []] + assert extract_text_blocks(m) == ["新入社員を登録。", "テスト要件:入力。"] + + +def test_build_free_text_table(): + table = build_free_text_table("機能要件", ["A", "B"], "f.xlsx", SheetType.FUNCTION) + assert isinstance(table, ExcelTable) + assert table.detected_type == SheetType.FUNCTION + assert table.extraction_method == "llm_from_free_text" + assert len(table.rows) == 2 + assert isinstance(table.rows[0]["text"], CellValue) + assert table.rows[0]["text"].value == "A" +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_free_text_extractor.py -v` +Expected: FAIL(导入错误) + +- [ ] **Step 3: 实现** + +`src/genesis/parsers/free_text_extractor.py`: +```python +from __future__ import annotations + +from typing import Any + +from genesis.data_models import CellValue, ExcelTable, Provenance, SheetType + + +def extract_text_blocks(matrix: list[list[Any]]) -> list[str]: + """按全空行分段;行内非空单元格以「 」连接。""" + blocks: list[str] = [] + current: list[str] = [] + for row in matrix: + cells = [str(c).strip() for c in row if c is not None and str(c).strip() != ""] + if not cells: + if current: + blocks.append(" ".join(current)) + current = [] + continue + current.append(" ".join(cells)) + if current: + blocks.append(" ".join(current)) + return blocks + + +def build_free_text_table( + sheet_name: str, + blocks: list[str], + file_name: str, + detected_type: SheetType = SheetType.GENERIC, +) -> ExcelTable: + rows = [] + for i, text in enumerate(blocks, start=1): + rows.append({ + "text": CellValue( + value=text, + provenance=Provenance( + file_name=file_name, + sheet_name=sheet_name, + row=i, + column="A", + column_header="text", + ), + ), + }) + return ExcelTable( + name=sheet_name, + detected_type=detected_type, + extraction_method="llm_from_free_text", + headers=["text"], + rows=rows, + ) +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_free_text_extractor.py -v` +Expected: PASS(2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers/free_text_extractor.py tests/test_free_text_extractor.py +git commit -m "feat: FreeTextExtractor(文本分段 + 占位结构化表)" +``` + +--- + +### Task 7: ExcelParser 编排器 + +**Files:** +- Create: `src/genesis/parsers/excel_parser.py` +- Create: `tests/test_excel_parser.py` + +**Interfaces:** +- Produces: + - `@dataclass ExcelParseResult`:`file_name: str`、`tables: list[ExcelTable]`、`comments: list[CellComment]`、`skipped: list[str]` + - `class ExcelParser: parse(path) -> ExcelParseResult` + +`parse` 编排: +1. `open_workbook(path)` +2. 每 Sheet:`matrix = sheet_matrix(ws)`;空 → skipped +3. `detected_type = detect_sheet_type(ws.title, matrix)` +4. `nature = classify_sheet(matrix)` +5. `FREE_TEXT` → `extract_text_blocks` + `build_free_text_table(detected_type=detected_type)` +6. 否则 `forward_fill(matrix, merged)` + `extract_table` +7. `collect_comments(ws, file_name)` 汇总 +8. 返回 `ExcelParseResult` + +- [ ] **Step 1: 写失败测试** + +`tests/test_excel_parser.py`: +```python +from genesis.data_models import SheetType +from genesis.parsers.excel_parser import ExcelParseResult, ExcelParser + +from tests.excel_helpers import new_workbook, save_workbook + + +def test_parse_table_sheets(tmp_path): + wb = new_workbook({ + "機能一覧": [["機能ID", "機能名"], ["A001", "社員登録"]], + "バッチ一覧": [["バッチID", "処理名"], ["B1", "夜間集計"]], + }) + path = save_workbook(tmp_path, wb) + result = ExcelParser().parse(path) + assert isinstance(result, ExcelParseResult) + by_name = {t.name: t for t in result.tables} + assert by_name["機能一覧"].detected_type == SheetType.FUNCTION + assert len(by_name["機能一覧"].rows) == 1 + assert by_name["バッチ一覧"].detected_type == SheetType.BATCH + + +def test_parse_free_text_sheet(tmp_path): + wb = new_workbook({"メモ": [["新入社員を登録"], [], [], [], []]}) + path = save_workbook(tmp_path, wb) + result = ExcelParser().parse(path) + assert len(result.tables) == 1 + assert result.tables[0].extraction_method == "llm_from_free_text" +``` + +- [ ] **Step 2: 运行确认失败** + +Run: `python -m pytest tests/test_excel_parser.py -v` +Expected: FAIL(导入错误) + +- [ ] **Step 3: 实现 excel_parser.py** + +`src/genesis/parsers/excel_parser.py`: +```python +from __future__ import annotations + +from dataclasses import dataclass, field +from pathlib import Path + +from genesis.data_models import CellComment, ExcelTable +from genesis.parsers.excel_reader import open_workbook, sheet_matrix +from genesis.parsers.sheet_detector import detect_sheet_type +from genesis.parsers.sheet_nature import SheetNature, classify_sheet +from genesis.parsers.merge_fill import forward_fill +from genesis.parsers.table_extractor import extract_table +from genesis.parsers.formatting_detector import collect_comments +from genesis.parsers.free_text_extractor import build_free_text_table, extract_text_blocks + + +@dataclass +class ExcelParseResult: + file_name: str + tables: list[ExcelTable] = field(default_factory=list) + comments: list[CellComment] = field(default_factory=list) + skipped: list[str] = field(default_factory=list) + + +class ExcelParser: + """要件定义 Excel 解析入口。""" + + def parse(self, path: str | Path) -> ExcelParseResult: + wb = open_workbook(path) + file_name = Path(path).name + result = ExcelParseResult(file_name=file_name) + for ws in wb.worksheets: + matrix = sheet_matrix(ws) + if not matrix: + result.skipped.append(ws.title) + continue + detected_type = detect_sheet_type(ws.title, matrix) + nature = classify_sheet(matrix) + if nature == SheetNature.FREE_TEXT: + blocks = extract_text_blocks(matrix) + result.tables.append( + build_free_text_table(ws.title, blocks, file_name, detected_type) + ) + else: + merged = [ + (r.min_row, r.min_col, r.max_row, r.max_col) + for r in ws.merged_cells.ranges + ] + filled = forward_fill(matrix, merged) if merged else matrix + result.tables.append(extract_table(ws.title, filled, file_name, detected_type)) + result.comments.extend(collect_comments(ws, file_name)) + return result +``` + +- [ ] **Step 4: 运行确认通过** + +Run: `python -m pytest tests/test_excel_parser.py -v` +Expected: PASS(2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add src/genesis/parsers/excel_parser.py tests/test_excel_parser.py +git commit -m "feat: ExcelParser 编排器(类型/性质/提取/汇总)" +``` + +--- + +### Task 8: 真实样本集成测试 + +**Files:** +- Create: `tests/test_real_samples.py` + +**说明:** 使用 `samples/` 下 3 个脱敏要件定义样本做端到端验证。样本缺失时对应用例 `pytest.skip`。 + +- [ ] **Step 1: 写集成测试** + +`tests/test_real_samples.py`: +```python +from pathlib import Path + +import pytest + +from genesis.parsers.excel_parser import ExcelParser + +SAMPLES = Path(__file__).resolve().parents[1] / "samples" + + +def _x(name: str) -> Path: + return SAMPLES / name + + +def test_new_dev_sample_has_tables(): + p = _x("要件定義_新規開発.xlsx") + if not p.exists(): + pytest.skip("样本缺失") + result = ExcelParser().parse(p) + by_name = {t.name: t for t in result.tables} + assert "機能一覧" in by_name + assert by_name["機能一覧"].rows + assert any(t.name == "DB定義" for t in result.tables) + + +def test_additional_modification_has_tables(): + p = _x("要件定義_追加改修.xlsx") + if not p.exists(): + pytest.skip("样本缺失") + result = ExcelParser().parse(p) + assert result.tables + assert any(t.rows for t in result.tables) + + +def test_free_text_sample_detected(): + p = _x("要件定義_自由記述.xlsx") + if not p.exists(): + pytest.skip("样本缺失") + result = ExcelParser().parse(p) + assert any(t.extraction_method == "llm_from_free_text" for t in result.tables) +``` + +- [ ] **Step 2: 运行集成测试** + +Run: `python -m pytest tests/test_real_samples.py -v` +Expected: PASS(样本存在时)或 至少 2 passed 1 skipped + +- [ ] **Step 3: 全量回归** + +Run: `python -m pytest -v` +Expected: PASS(全体收集成功) + +- [ ] **Step 4: Commit** + +```bash +git add tests/test_real_samples.py +git commit -m "test: 真实样本集成测试(3 类型 xlsx)" +``` + +--- + +## 自审结论(计划完稿时执行) + +- **Spec 覆盖**:implementation-plan §2.1→Task1、§2.2→Task2、§2.3→Task3、§2.4+§2.5→Task4、§2.7→Task5、§2.6→Task6、§2.9→Task7+Task8;design §3.5.1→T2、§3.5.2→T3、§3.5.3→T4、§3.5.4/§3.5.5→T5;sample-spec §3.1-3.4→T8;验收标准(表格型/自由记述型/混合型解析、合并填充、source_uri)逐项覆盖。 +- **占位符扫描**:无 TBD/TODO;任务指引 Task1 的 Step 编号已显式标注(Step 3 即实现、Step 4 运行),不含空白步骤。 +- **类型一致性**:`CellValue` / `ExcelTable` / `SheetType` / `CellFormatting` / `CellComment` 与 `data_models.py` 完全一致;`build_source_uri(file, sheet, ref)` 全局同名;`extract_table.column` 用 `column_letter` 生成。 +- **依赖边界**:全程不依赖 InferenceEngine;FreeText 表 `extraction_method="llm_from_free_text"` 显式标记后续替换,符合模块边界与 YAGNI。 \ No newline at end of file