2026Technology-Competition/docs/superpowers/plans/2026-08-09-mixed-paragraph-parsing.md

MIXED 完整段落解析 实施计划

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: 将 MIXED（混合型）Sheet 落地 design §3.5.2「段落分割→各段落最优解析」，显式表达段落边界与类型。

Architecture: 新增纯函数段落分割器 paragraph_splitter.py（以空行为界的通用分割）；扩展 data_models.py 追加 MixedParagraph/MixedSheet 容器与 ExcelParseResult.mixed 字段；excel_parser.py 中为 MIXED 性质新增装配分支（分割→每段 classify→表格段 extract_table/自由文本段 build_free_text_table）；新增混合型样本 samples/要件定義_混合型.xlsx 驱动端到端。

Tech Stack: Python 3.11+，openpyxl 3.1.5，pytest，dataclasses，typing。

Global Constraints

• 项目为中文交流（注释中文、标识符英文），Windows/PowerShell 环境
• data_models.py 的既有类与字段不得修改或删除（仅追加新类与带默认值的新字段）
• 既有 TABLE / FREE_TEXT 解析路径不得改行为（回归保持 41 passed）
• 测试命令：python -m pytest tests/<file> -v；全量回归：python -m pytest -v
• 提交消息风格：feat: / test: / docs:（简中文描述）
• 每次修改后按项目规则追加 _AI_USAGE_LOG.md 记录（范式步骤列：Agent 实现 或 测试验证）

---

Task 9: 段落分割器 split_paragraphs

Files:

• Create: src/genesis/parsers/paragraph_splitter.py
• Create: tests/test_paragraph_splitter.py

Interfaces:

• Consumes: 无（纯函数，仅类型 Any）

• Produces: split_paragraphs(matrix: list[list[Any]]) -> list[tuple[int, int]] — 以全空行为界的段落序号区间（含行区间端点**，矩阵 0-based；空矩阵 → []）

• Step 1: 写失败测试

tests/test_paragraph_splitter.py:

from genesis.parsers.paragraph_splitter import split_paragraphs


def test_empty_matrix():
    assert split_paragraphs([]) == []


def test_single_paragraph_no_empty_rows():
    m = [["a", "b"], ["c", "d"]]
    assert split_paragraphs(m) == [(0, 1)]


def test_split_on_middle_empty_row():
    m = [["a"], [], ["b"], ["c"], []]
    assert split_paragraphs(m) == [(0, 0), (2, 3)]


def test_trailing_empty_rows_no_extra_paragraph():
    m = [["a"], [], [], []]
    assert split_paragraphs(m) == [(0, 0)]


def test_leading_empty_rows_start_at_first_nonempty():
    m = [[], ["a"], [], ["b"]]
    assert split_paragraphs(m) == [(1, 1), (3, 3)]

• Step 2: 运行确认失败

Run: python -m pytest tests/test_paragraph_splitter.py -v Expected: FAIL（import 错误 ModuleNotFoundError: No module named 'genesis.parsers.paragraph_splitter'）

• Step 3: 实现

src/genesis/parsers/paragraph_splitter.py:

from __future__ import annotations

from typing import Any


def _is_blank_row(row: list[Any]) -> bool:
    return all(c is None or str(c).strip() == "" for c in row)


def split_paragraphs(matrix: list[list[Any]]) -> list[tuple[int, int]]:
    """以全空行为界的通用段落分割；返回 (start_row, end_row)（含端，0-based）。"""
    paragraphs: list[tuple[int, int]] = []
    start: int | None = None
    for i, row in enumerate(matrix):
        if not _is_blank_row(row):
            if start is None:
                start = i
        else:
            if start is not None:
                paragraphs.append((start, i - 1))
                start = None
    if start is not None:
        paragraphs.append((start, len(matrix) - 1))
    return paragraphs

• Step 4: 运行确认通过

Run: python -m pytest tests/test_paragraph_splitter.py -v Expected: PASS（5 passed）

• Step 5: 提交

git add src/genesis/parsers/paragraph_splitter.py tests/test_paragraph_splitter.py
git commit -m "feat: 段落分割 split_paragraphs（通用空行分词）"

---

Task 10: data_models 扩展（MixedParagraph / MixedSheet）

Files:

• Modify: src/genesis/data_models.py（在文件末尾追加，不修改既有类）

Interfaces:

• Consumes: typing.Literal、ExcelTable、SheetType（已有）

• Produces:

  • MixedParagraph(kind: Literal["table","free_text"], matrix: list[list[Any]] | None = None, table: ExcelTable | None = None, text: str | None = None, source_range: tuple[int, int] | None = None)
  • MixedSheet(name: str, paragraphs: list[MixedParagraph] = field(default_factory=list))
  • ExcelParseResult.mixed: list[MixedSheet] 新增字段（带默认值）

• Step 1: 写失败测试

在 tests/test_excel_parser.py 末尾追加（先测字段存在性与默认行为）：

from genesis.data_models import MixedParagraph, MixedSheet
from genesis.parsers.excel_parser import ExcelParseResult


def test_excel_parse_result_has_mixed_default():
    r = ExcelParseResult(file_name="f.xlsx")
    assert r.mixed == []


def test_mixed_paragraph_defaults():
    p = MixedParagraph(kind="table")
    assert p.table is None
    assert p.text is None
    assert p.source_range is None


def test_mixed_sheet_holds_paragraphs():
    p1 = MixedParagraph(kind="table")
    p2 = MixedParagraph(kind="free_text", text="备注")
    ms = MixedSheet(name="混合", paragraphs=[p1, p2])
    assert ms.name == "混合"
    assert [p.kind for p in ms.paragraphs] == ["table", "free_text"]

• Step 2: 运行确认失败

Run: python -m pytest tests/test_excel_parser.py -v Expected: FAIL（ImportError: cannot import name 'MixedParagraph'）

• Step 3: 实现（data_models.py 末尾追加）

@dataclass
class MixedParagraph:
    """混合 sheet 的一个段落（表格或自由文本）"""
    kind: Literal["table", "free_text"]
    matrix: list[list[Any]] | None = None    # 该段原始矩阵（调试/重现）
    table: ExcelTable | None = None           # kind="table" 时填充
    text: str | None = None                   # kind="free_text" 时填充（段全文）
    source_range: tuple[int, int] | None = None  # (first_row, last_row) 矩阵 0-based


@dataclass
class MixedSheet:
    """混合 sheet 的段落集合"""
    name: str
    paragraphs: list[MixedParagraph] = field(default_factory=list)

（field 已在文件顶部导入；Literal 需在 from typing import Any 处加 Literal）

• Step 4: 运行确认通过

Run: python -m pytest tests/test_excel_parser.py -v Expected: PASS（新增 3 passed 全绿）

• Step 5: 修改 ExcelParseResult（excel_parser.py）

在 src/genesis/parsers/excel_parser.py 的 ExcelParseResult 中加字段 mixed：

@dataclass
class ExcelParseResult:
    file_name: str
    tables: list[ExcelTable] = field(default_factory=list)
    comments: list[CellComment] = field(default_factory=list)
    skipped: list[str] = field(default_factory=list)
    mixed: list[MixedSheet] = field(default_factory=list)

并更新导入 from genesis.data_models import CellComment, ExcelTable, MixedSheet。

• Step 6: 全量回归

Run: python -m pytest -v Expected: PASS（41 + 3 = 44 passed）

• Step 7: 提交

git add src/genesis/data_models.py src/genesis/parsers/excel_parser.py tests/test_excel_parser.py
git commit -m "feat: data_models 扩展 MixedParagraph/MixedSheet（段落容器）"

---

Task 11: MIXED 装配（excel_parser 分段解析）

Files:

• Modify: src/genesis/parsers/excel_parser.py
• Create: 无新文件
• Test: tests/test_excel_parser.py（追加）

Interfaces:

• Consumes: split_paragraphs（Task9）、MixedParagraph/MixedSheet（Task10）、既有 classify_sheet / forward_fill / extract_table / extract_text_blocks / build_free_text_table、cell_formatting / collect_comments
• Produces: ExcelParser.parse 对 MIXED 性质产生 result.mixed 段落集合 + 追加表到 result.tables

关键实现约定（来自 spec 3.4 修正）：合并单元格对整 sheet 先 forward_fill 再按段切片，避免坐标换算错误；段内 extract_table 用 header_row=0（段首行为表头）。

• Step 1: 写失败测试

tests/test_excel_parser.py 追加（构造混合矩阵：表格段 + 空行 + 碎片段，落盘解析）：

def test_parse_mixed_sheet_segmented(tmp_path):
    wb = new_workbook({
        "混合": [
            ["機能ID", "機能名"],
            ["F101", "社員登録"],
            ["F102", "退職処理"],
            [],
            ["・改修ポイント：F102 追加バリデーション"],
            ["■対象画面：SC001"],
        ],
    })
    path = save_workbook(tmp_path, wb)
    result = ExcelParser().parse(path)
    assert result.mixed, "混合 sheet 应产产出 mixed 段落"
    ms = result.mixed[0]
    assert ms.name == "混合"
    kinds = [p.kind for p in ms.paragraphs]
    assert "table" in kinds and "free_text" in kinds
    # 表格段无碎片污染：table 段应含 2 数据行，机能ID 首行为 F101
    tbl = [p.table for p in ms.paragraphs if p.kind == "table"][0]
    assert tbl is not None and len(tbl.rows) == 2
    assert tbl.rows[0]["機能ID"].value == "F101"
    # 自由文本段捕获碎片
    ft = [p for p in ms.paragraphs if p.kind == "free_text"][0]
    assert "改修ポイント" in (ft.text or "")


def test_parse_mixed_sheet_formatting_in_mid_segment(tmp_path):
    # 表格段不在物理行 0（自由文本段在前），数据行 F101 设取消线 —— 锁住 formatting_map 坐标错位
    from copy import copy
    from openpyxl import Workbook
    wb = Workbook()
    ws = wb.active
    ws.title = "混合"
    data = [
        ["■はじめに"], ["前提説明"], [],
        ["機能ID", "機能名"], ["F101", "社員登録"], ["F102", "退職処理"],
        [], ["・改修ポイント"],
    ]
    for r, row in enumerate(data, start=1):
        for c, v in enumerate(row, start=1):
            if v:
                ws.cell(row=r, column=c, value=v)
    font = copy(ws.cell(row=5, column=1).font)  # F101 所在物理行（第 5 行）
    font.strike = True
    ws.cell(row=5, column=1).font = font
    path = save_workbook(tmp_path, wb)
    result = ExcelParser().parse(path)
    ms = result.mixed[0]
    tbl = [p.table for p in ms.paragraphs if p.kind == "table"][0]
    cv = tbl.rows[0]["機能ID"]  # F101
    assert cv.formatting is not None
    assert cv.formatting.strikethrough is True

• Step 2: 运行确认失败

Run: python -m pytest tests/test_excel_parser.py::test_parse_mixed_sheet_segmented -v Expected: FAIL（当前 MIXED 折叠进表格路径，result.mixed 为空）

• Step 3: 实现装配分支（excel_parser.py）

在 parse 的 if nature == SheetNature.FREE_TEXT 之后插入 MIXED 分支，或将 nature == SheetNature.MIXED 单独处理：

            if nature == SheetNature.MIXED:
                # 合并单元格：整 sheet 先填充再按段切片
                merged = [
                    (r.min_row, r.min_col, r.max_row, r.max_col)
                    for r in ws.merged_cells.ranges
                ]
                filled_all = forward_fill(matrix, merged) if merged else matrix
                # 整 sheet 构建格式映射一次（按物理坐标）
                fmt_map = {}
                for row in ws.iter_rows():
                    for cell in row:
                        fmt = cell_formatting(cell)
                        if fmt is not None:
                            fmt_map[(cell.row - 1, cell.column - 1)] = fmt
                mixed_sheet = MixedSheet(name=ws.title)
                for (s, e) in split_paragraphs(matrix):
                    seg = filled_all[s:e + 1]
                    seg_nature = classify_sheet(seg)
                    if seg_nature == SheetNature.TABLE:
                        header_row = find_header_row(seg)
                        if header_row < 0:
                            header_row = 0
                        # extract_table 的 formatting_map 用段内矩阵坐标 (r,c)；
                        # 从整 sheet 物理 map 抽出段内子 map（物理行 pr → 段内 pr-s）
                        seg_fmt_map = {}
                        for (pr, pc), fmt in fmt_map.items():
                            if s <= pr <= e:
                                seg_fmt_map[(pr - s, pc)] = fmt
                        table = extract_table(
                            ws.title, seg, file_name, detected_type,
                            header_row=header_row, formatting_map=seg_fmt_map,
                        )
                        result.tables.append(table)
                        mixed_sheet.paragraphs.append(MixedParagraph(
                            kind="table", matrix=seg, table=table,
                            source_range=(s, e),
                        ))
                    else:
                        blocks = extract_text_blocks(seg)
                        table = build_free_text_table(ws.title, blocks, file_name, detected_type)
                        result.tables.append(table)
                        mixed_sheet.paragraphs.append(MixedParagraph(
                            kind="free_text", matrix=seg,
                            text="\n".join(blocks), source_range=(s, e),
                        ))
                result.mixed.append(mixed_sheet)
            elif nature == SheetNature.FREE_TEXT:
                blocks = extract_text_blocks(matrix)
                result.tables.append(
                    build_free_text_table(ws.title, blocks, file_name, detected_type)
                )
            else:
                # 现有 TABLE 路径（含 MIXED 旧折叠）

注：原 else 分支现在是 TABLE 专用；MIXED 已独立。fmt_map 对整 sheet 按物理坐标构建一次；表格段经 seg_fmt_map 抽为段内坐标后传给 extract_table（避免段内相对索引与物理坐标错位）。自由文本段无表格 CellValue，不使用格式化。

• Step 4: 运行确认通过

Run: python -m pytest tests/test_excel_parser.py -v Expected: PASS（含新 MIXED 用例）

• Step 5: 全量回归

Run: python -m pytest -v Expected: PASS（44 + 2 = 46 passed）

• Step 6: 提交

git add src/genesis/parsers/excel_parser.py tests/test_excel_parser.py
git commit -m "feat: MIXED 完整段落解析（分割→每段最优解析）"

---

Task 12: 混合型样本 + 端到端验证

Files:

• Create: samples/要件定義_混合型.xlsx
• Modify: tests/test_real_samples.py（追加用例）

Interfaces:

• Consumes: ExcelParser.parse（含 MIXED 装配）

• Produces: 混合样本（表格段 + 碎片段）端到端用例

• Step 1: 生成样本

用 Python 脚本生成 samples/要件定義_混合型.xlsx（落盘）：

from openpyxl import Workbook
wb = Workbook()
ws = wb.active
ws.title = "機能一覧"
rows = [
    ["機能ID", "機能名", "画面ID"],
    ["F101", "社員登録", "SC001"],
    ["F102", "退職処理", "SC002"],
    ["F103", "給与計算", "SC003"],
    [],
    ["・改修ポイント：F103 に年末調整バッチ連携を追加する。"],
    ["■対象期間：2026年度下半期"],
]
for r, row in enumerate(rows, start=1):
    for c, v in enumerate(row, start=1):
        if v is not None:
            ws.cell(row=r, column=c, value=v)
wb.save(r"samples\要件定義_混合型.xlsx")
print("saved")

（实际执行时用 PowerShell 运行；「・」与「■」行连续无空行，split_paragraphs 将其合并为一段碎片段，返回 [(0,3),(5,6)] 两段：表格段 + 碎片段。）

• Step 2: 写端到端测试

tests/test_real_samples.py 追加：

def test_mixed_sample_segments_detected():
    p = _x("要件定義_混合型.xlsx")
    if not p.exists():
        pytest.skip("样本缺失")
    result = ExcelParser().parse(p)
    by_name = {t.name: t for t in result.tables}
    assert "機能一覧" in by_name
    assert result.mixed, "混合样本应产产出段落"
    mixed = result.mixed[0]
    assert len(mixed.paragraphs) == 2          # 表格段 + 碎片段（・/■ 连续）
    assert [p.kind for p in mixed.paragraphs] == ["table", "free_text"]
    # 表格段无碎片污染
    table = [p.table for p in mixed.paragraphs if p.kind == "table"][0]
    assert table.rows[0]["機能ID"].value == "F101"
    assert len(table.rows) == 3
    # 碎片段含 ・ 与 ■ 两行文本
    ft = [p for p in mixed.paragraphs if p.kind == "free_text"][0]
    assert "改修ポイント" in (ft.text or "")
    assert "対象期間" in (ft.text or "")

• Step 3: 运行端到端

Run: python -m pytest tests/test_real_samples.py -v Expected: PASS（4 passed，无 skip）

• Step 4: 全量回归

Run: python -m pytest -v Expected: PASS（46 + 1 = 47 passed）

• Step 5: 提交

git add samples/要件定義_混合型.xlsx tests/test_real_samples.py
git commit -m "test: MIXED 混合样本 + 端到端段落验证"

---

Task 13: 遗留清理收尾（枚举/注释/边界/测试补齐）

Files:

• Modify: src/genesis/parsers/free_text_extractor.py（extraction_method 用枚举值同源）
• Modify: src/genesis/parsers/table_extractor.py（extraction_method 用枚举值同源；header_row 越界防御）
• Modify: src/genesis/parsers/merge_fill.py（min_row 非法值防御）
• Modify: src/genesis/data_models.py（Provenance.row 语义注释）
• Modify: src/genesis/parsers/sheet_nature.py（「・/■」MIXED 用例已有，不追加代码）
• Modify: tests/test_sheet_nature.py（补 MIXED 正向/边界用例）
• Modify: tests/test_formatting_detector.py（补字体色/背景色正向断言）

Interfaces:

• Consumes: ExtractionMethod 枚举（data_models）

• Produces: 一致的枚举同源与防御性实现

• Step 1: 写失败测试（MIXED 断言 + formatting 正向）

tests/test_sheet_nature.py 追加（补 MIXED 正向用例，test_table_detection 已覆盖 TABLE 无需重复）：

def test_classify_sheet_mixed_with_bullet_line():
    m = [["ID", "名前"], ["1", "田中"], ["・備考行"]]
    assert classify_sheet(m) == SheetNature.MIXED


def test_classify_sheet_mixed_with_square_line():
    m = [["ID", "名前"], ["1", "田中"], ["■備考行"]]
    assert classify_sheet(m) == SheetNature.MIXED

tests/test_table_extractor.py 追加（锁枚举同源）：

from genesis.data_models import ExtractionMethod


def test_extract_table_uses_enum_value():
    matrix = [["ID"], ["1"]]
    table = extract_table("社員", matrix, "f.xlsx", SheetType.FUNCTION)
    assert table.extraction_method == ExtractionMethod.OPENPYXL.value

tests/test_free_text_extractor.py 追加：

from genesis.data_models import ExtractionMethod


def test_free_text_uses_enum_value():
    table = build_free_text_table("メモ", ["A"], "f.xlsx")
    assert table.extraction_method == ExtractionMethod.LLM_FROM_FREE_TEXT.value

tests/test_formatting_detector.py 追加（复用现有 make_wb 复制字体技巧）：

def test_cell_formatting_detects_font_color():
    from openpyxl.styles import Font
    wb = Workbook()
    ws = wb.active
    ws["A1"] = "x"
    f = copy(ws["A1"].font)
    f.color = Font(color="FF0000FF").color
    ws["A1"].font = f
    fmt = cell_formatting(ws["A1"])
    assert fmt is not None
    assert fmt.font_color is not None

注意 openpyxl 颜色格式：不硬编码具体 hex 断言，只断言 font_color is not None，避免格式漂移；同时确认纯默认单元格仍返回 None（已有用例覆盖）。

• Step 2: 实现（枚举同源 + 防御）

free_text_extractor.py：导入并改用：

from genesis.data_models import ExtractionMethod
# build_free_text_table 中：
extraction_method=ExtractionMethod.LLM_FROM_FREE_TEXT.value,

table_extractor.py：

from genesis.data_models import ExtractionMethod
# extract_table 中：
extraction_method=ExtractionMethod.OPENPYXL.value,
# 空矩阵分支同样同源

table_extractor.py 的 header_row 防御：

    if not matrix:
        return ExcelTable(...)
    if header_row < 0 or header_row >= len(matrix):
        header_row = 0

merge_fill.py 防御非法入参（完整替换 forward_fill）：

def forward_fill(
    matrix: list[list[Any]],
    merged_ranges: list[tuple[int, int, int, int]],
) -> list[list[Any]]:
    """合并单元格：用左上角主格值填充范围内全部单元格。"""
    out = [list(row) for row in matrix]
    for (min_row, min_col, max_row, max_col) in merged_ranges:
        # 防御：合并范围 1-based，非法（<1）或越界时跳过该范围
        if not out or min_row < 1 or min_col < 1:
            continue
        if min_row > len(out) or min_col > len(out[min_row - 1]):
            continue
        main_value = out[min_row - 1][min_col - 1]
        for r in range(min_row, min(max_row, len(out)) + 1):
            row = out[r - 1]
            for c in range(min_col, min(max_col, len(row)) + 1):
                row[c - 1] = main_value
    return out

tests/test_table_extractor.py 追加（验证防御）：

def test_forward_fill_ignores_invalid_range():
    matrix = [["A"], ["B"]]
    assert forward_fill(matrix, [(0, 1, 2, 1)]) == matrix  # min_row=0 非法 → 不崩溃不改数据
    assert forward_fill(matrix, [(9, 9, 9, 9)]) == matrix  # 越界 → 不崩溃

data_models.py 注释（Provenance.row）：

@dataclass
class Provenance:
    file_name: str
    sheet_name: str
    row: int          # 数据行号（从 1 起：表格为物理行-表头行；自由文本为块序）
    column: str
    column_header: str

• Step 3: 运行全部新增/修改测试

Run: python -m pytest tests/test_sheet_nature.py tests/test_formatting_detector.py tests/test_free_text_extractor.py tests/test_table_extractor.py -v Expected: PASS（新增 2 MIXED + 1 formatting + 1 自由文本枚举 + 2 表提取枚举/防御 = 6 个新用例）

• Step 4: 全量回归

Run: python -m pytest -v Expected: PASS（47 + 6 = 53 passed）

• Step 5: 提交

git add src/genesis/parsers/free_text_extractor.py src/genesis/parsers/table_extractor.py src/genesis/parsers/merge_fill.py src/genesis/data_models.py tests/test_sheet_nature.py tests/test_formatting_detector.py tests/test_free_text_extractor.py tests/test_table_extractor.py
git commit -m "fix: 遗留清理（枚举同源/边界防御/断言补强）"

---

Self-Review

1. Spec 覆盖：§3.1→Task9、§3.3→Task10、§3.4→Task11、§3.5→Task12、§3.6/§4→Task13 + 各任务内测试。✓ 2. 占位符检查：无 TODO/TBD；修正了拼写错误（「占位」→实际代码）。✓ 3. 类型一致性：split_paragraphs 签名在 Task9 定义、Task11/12 使用一致；MixedParagraph.kind 用 Literal["table","free_text"] 一致。✓ 4. 兼容性：Task10 只追加字段（默认值）；ExcelParseResult 既有消费（tests、下游）不受破坏。✓