7fb9304212
merge local cobol_testgen improvements into v3 shared modules
...
- cond.py: SQLCODE/SQLSTATE handling, alphanumeric >/< boundary fix
- output.py: termination tracking, db_input support, _is_field_assigned filter
- coverage.py: mark_from_gcov, THRU support, KeyError protection
- gcov.py: new file (dependency for coverage.py)
- grammar.lark: multi-segment PIC support
- read.py: SQL INCLUDE resolution, DECLARE TABLE parsing, * comment fix
- core.py: SQL parsing, blocked_names, keyword list
- design.py: multi-sentinel, THRU ranges, PERFORM VARYING last iteration
- __init__.py: local main() + v3 API functions, guarded imports
All 6 ZAN programs verified passing through v3 pipeline
2026-06-23 22:38:17 +08:00
e5ab3baa46
提升：37/37基准程序全量解析+O(N)路径枚举+运行时gcov验证
...
## 核心变更
### 1. 新PROCEDURE DIVISION解析器（procedure_parser.py）
- 行级状态机替换旧的BrParser regex解析器
- 覆盖：IF/ELSE/END-IF（嵌套）、EVALUATE/WHEN/ALSO、
PERFORM UNTIL/VARYING、READ/AT END/NOT AT END、
SORT/MERGE、GO TO DEPENDING ON
- 之前：3/37程序有分支检测 → 现在：37/37全部有分支
- 速度：~20ms/程序，纯规则引擎
### 2. 桥接层（pipeline_bridge.py）
- 新解析器为主，旧解析器3秒超时兜底
- 自动选取分支数更多的结果
### 3. 线性路径枚举（design_mcdc.py）
- 替换旧的Cartesian积路径枚举（O(2^N)）为每决策点独立枚举（O(N)）
- 28-sysin: 162分支仅163条路径（之前需截断到60DP）
- 消除了500路径硬上限和60DP截断
### 4. 条件解析修复（cond.py）
- NOT运算符规范化：X NOT = 5 → X <> 5
- 88-level反向：NOT WS-EOF-Y → parent <> value
- 裸字段引用：NOT WS-EOF → WS-EOF <> 'Y'
- 验证：1182个IF条件中0个NOT污染
### 5. 约束字段过滤（__init__.py）
- OF限定词剥离：STD-KEY OF MASTER-REC → STD-KEY
- 下标字段解析：WS-ITEM(SUB) → WS-ITEM
- 跳过不在fields_dict中的字段（group item/伪影）
### 6. 预处理器增强（read.py）
- VALUE ALL剥离（VALUE ALL '*' → VALUE '*'）
- &续行合并（COBOL多行字符串拼接）
- PIC小数点点→V转换（Z(9)9.99. → Z(9)9V99.）
- 缺少点号补全
### 7. Grammar修复（grammar.lark）
- OCCURS 1 TIME支持（原只认TIMES）
- USAGE IS COMP支持（可选IS）
- $符号在PICTURE_STRING中
- 无NAME条款支持（clause+）
### 8. Flatfile写入（flatfile.py）
- 多记录FD支持（选字段最多的记录）
- Path类型强制转换
- 回退零值记录
### 9. Bug修复
- trace_to_root空列表保护（core.py）
### 10. 测试套件（S16-S21）
- S16: 全量基准程序端到端
- S17: gcov运行时对比
- S18/S19: 桥接器验证
- S20: DISPLAY插桩运行时验证+gcov分支覆盖率
- S21: 条件解析修复验证
- 全部17/17回归测试通过
Co-Authored-By: Claude <noreply@anthropic.com >
2026-06-22 23:41:22 +08:00
0e7472598d
fix: 跨文件KEY约束 + PERFORM分支统计 + 平面文件写入
...
1. 跨文件KEY约束(修复)
匹配型程的M-KEY与D-KEY值不同导致匹配0条。
修复: generate_data后处理检测IF KEY比较,
前半记录对齐KEY值(8条匹配),后半保待差异(9条不匹配).
实际cobc运行验证: MATCHED=8, PASS.
2. extract_structure PERFORM分支统计(修复)
_walk函数未添加BrPerform决策点, total_branches缺失.
修复: 为PERFORM UNTIL/VARYING决策点添加2分支(Enter/Skip).
之前total_branches=0,现在=2.
3. flatfile.py(新增)
COBOL固定长平面文件写入器.
- analyze_fd_layout(): 从COBOL源码自动解析文件布局
- write_flat_file(): 生成为COBOL可直接读取的二进制格式
Co-Authored-By: Claude <noreply@anthropic.com >
2026-06-22 13:52:56 +08:00
0b0a013f51
fix: 3 critical parsing bugs found through statement benchmark testing
...
Bug 1: ELSE IF breaks IF false_seq parsing (core.py)
- _parse_if checked self.clean() == 'ELSE' which fails on 'ELSE IF ...'
- Fix: use startswith('ELSE'), reinsert IF portion for recursive parse
- Impact: ALL ELSE IF chains were silently dropped (huge branch loss)
Bug 2: READ skip loop greedily consumes subsequent statements (core.py)
- READ's AT END / NOT AT END skip loop used bare advance() with no
statement boundary detection
- Fix: add _stmt_boundary regex that stops on IF/PERFORM/READ/etc.
- Impact: everything after first READ was consumed as 'AT END' lines
Bug 3: _walk() in extract_structure doesn't descend into BrPerform (__init__.py)
- Branch counting _walk() only handled BrIf/BrEval/BrSeq
- IF statements inside PERFORM bodies were never counted
- Fix: add BrPerform.body_seq and BrSearch descent
Combined impact: matching programs (MT01-33) now correctly report
their branches instead of 0. Full regression: 749 passed (unchanged).
2026-06-21 12:52:04 +08:00
bc1d56d1a4
feat: Phase 2 complete — 13 Phases of COBOL type classification and test benchmark
...
P0.6: gcov infrastructure
P1: extract_structure output expansion (11 new feature fields)
P2: Confusion group rule engine (8 pairs + contradiction + backtrack)
P3: 4-factor confidence calculation + quality gate update
P4: 33+2 COBOL program type test samples (22 files, 7 categories)
P5: parametrized/ test data generation engine
P6: japanese_data.py lookup tables
P7-10: Type-specific test suites (~159 parametrized tests)
P11: Full classification pipeline (classify_program) + orchestrator integration
P12: Documentation (module-interfaces, test-plan v3.0, coverage-matrix)
Architecture decisions:
- classification_pipeline/ merged to hina/pipeline/
- parametrized/ as independent module
- japanese_data.py as root-level file
- hina/__all__ only exports classify_program()
Co-Authored-By: Claude <noreply@anthropic.com >
2026-06-19 23:51:55 +08:00
097530b036
feat: Phase 1 - cobol_testgen API + quality fields + retry handler
2026-06-18 15:47:35 +08:00
hangshuo652
NB-076
hangshuo652