Commit 714f714a authored by Data Governance Dev's avatar Data Governance Dev

refactor(web): 分析结果 Tab 通用协议 + 文档

把 6 个 tab (merge / redundancy / empty / missing / length / standards)
的展示抽出成通用协议,让每个 step 在 data 里嵌入 _protocol 自描述渲染,
前端一个通用 renderer 按协议渲染;新增 step 只需写 data + 嵌 _protocol,
零前端改动。

后端
- 新建 web/core/tab_protocol.py:dataclass (TabProtocol / KpiSpec /
  SummarySpec / TableSpec / ColumnSpec / RenderSpec / LinkageSpec) +
  _eval_path() 路径表达式求值 + make_protocol() 一站式 helper
- 5 个 step 文件在 return 里嵌入 _protocol
  - step2_merge_redundancy.py:2 tab(merge_candidates + redundancy_fields)
  - step4_empty_fields.py:1 tab + 2 子表 + linkage;summary 加 flat 副本
  - step5_missing_comments.py:1 tab
  - step6_length_check.py:1 tab
  - step7_standards.py:1 tab + 2 子表 + linkage
- orchestrator.py 删 _build_overview 里 KPI 聚合逻辑
- models.py OverviewStats 缩为 5 字段(database / host / db_type /
  total_tables / total_fields / executed_at)

前端 web/static/index.html 大改
- 8 个硬编码 KPI 卡 → v-for='k in displayedKpis'
- 6 个硬编码 <el-tab-pane>(~450 行)→ v-for='t in displayedTabs'
- 删 TAB_STEP_MAP / perTableStats / flatEmptyFields / perTableRows /
  filteredViolations
- 新增 displayedTabs / displayedKpis / tabStatus computed
- 新增 evalPathExpr / resolvePath / resolveRows / renderSummary /
  resolveRenderPath 工具函数
- 10 种 render.kind 在 template 里以 v-else-if 分发(text / code / tag /
  tag_list / two_line / three_line / progress / percent / ratio / number)
- 3 个嵌套展开行(merge / standards / empty_fields)保留硬编码 —
  展开行内容太特定,WORKLOG 记为后续协议扩展方向

文档与验证
- 新建 docs/TAB_PROTOCOL.md:完整协议规范 + 新增 step 模板 + 真实走读
- 新建 scripts/verify_protocol.cjs(12 case)+ verify_e2e.cjs(9 case)
  + verify_render.cjs(12 case)+ verify_template.cjs(2 case)——
  共 35 个 case 全过
- verify_template.cjs 用 jsdom + 真实 vue.global.prod.js + Function 劫持
  截获编译产物,专门兜底「页面变白 / 编译出坏 JS」回归
- WORKLOG 追加 v-else fallback 编译 bug 修复(Vue 3.5.x 在长 v-else-if
  链末尾输出 : () 无条件空 call expression)—— 把 v-else 改为
  v-else-if="!cond" 显式条件

构建
- package.json + package-lock.json:jsdom + @vue/compiler-dom(verify
  脚本的 dev 依赖)
- .gitignore 加 scripts/compiled_*.js(verify_template 失败时的 debug
  产物)
parent edd3d2e5
......@@ -23,6 +23,8 @@ workflow/config.yaml
# ── Node / 前端 (留作扩展) ──
node_modules/
dist/
# verify_template.cjs 失败时写出的 debug 产物
scripts/compiled_*.js
# ── 系统文件 ──
.DS_Store
......
This diff is collapsed.
......@@ -318,4 +318,129 @@ Vue 3 规定 `v-else` / `v-else-if` **必须是上一个 v-if/v-else-if 元素
### 未提交
按用户要求,待 review 后再提交。
\ No newline at end of file
按用户要求,待 review 后再提交。
---
## 2026-08-07 Tab Protocol 重构(分析结果通用协议)
### 目标(用户原话)
> 把分析结果的tab页的展示抽出成一个通用协议并形成文档,当用户选中部分检验项目的时候,只需要按照通用tab页标准准备数据,以及前端分析结果只显示选中了的项目,这样后续增加分析项目的时候只需要对着这个协议返回数据给前台就可以实现tab页扩展。
### 改动概览
#### 1. 新增 `web/core/tab_protocol.py`(~250 行)
- `dataclass(frozen=True)` 镜像:TabProtocol / KpiSpec / SummarySpec / TableSpec / ColumnSpec / RenderSpec / LinkageSpec / TagStyle / Threshold
- `_eval_path(expr, data)`:支持 `this.x / this.x.y / len(this.x)` 三种路径(regex 校验,不做任意表达式求值)
- `inject_protocol(data, *, step_id, tabs)`:给 step return dict 注入 `_protocol` 字段
- 自检通过:`python -m web.core.tab_protocol`
#### 2. `web/core/models.py`
- `OverviewStats` 字段从 12 个缩到 6 个:`database / host / db_type / total_tables / total_fields / executed_at`
- 8 个 step-specific KPI(high_empty_fields / missing_comments / length_issues / standard_violations / merge_candidates / redundancy_fields / mid_empty_fields / standards_applied)从 overview 移除,由各 step 在 `_protocol.kpis[]` 自描述
#### 3. `web/core/orchestrator.py`
- `_build_overview(sections, dict_data, cfg)` → `_build_overview(dict_data, cfg)`:不再做 KPI 聚合(sections 参数也删了)
- 各 step 在 `_run_*` 中返回的 `data` 已带 `_protocol`,无需 orchestrator 干预
#### 4. 5 个 step 文件加协议
- `step2_merge_redundancy.py`:2 tab(`merge_candidates` 带嵌套 expand、`redundancy_fields` 含 tag 分类 + source 来源)
- `step4_empty_fields.py`:1 tab 含 2 子表(flat 高空/中空 + 按表聚合含嵌套 expand);summary 加 `flat_high_empty_fields` / `flat_mid_empty_fields` 副本
- `step5_missing_comments.py`:1 tab(推测注释 + 置信度 + 依据)
- `step6_length_check.py`:1 tab(含 three_line 标准/规则/依据 渲染 + ratio actual/required)
- `step7_standards.py`:1 tab 含 2 子表 + `LinkageSpec`(source_table=0 violations_by_standard、target_table=1 violations、field_map={table→table_name, field→column_name}、default_filter=limit 200)
- 每个 step 加 `_wrap(data)` helper;fallback / 错误路径同样走协议
#### 5. 前端 `web/static/index.html` 大改
**删除**:
- 8 个硬编码 KPI 卡 → 改为 `v-for="k in displayedKpis"`(仅 2 张表/字段总数固定 + step 贡献动态)
- 6 个硬编码 `<el-tab-pane>`(~450 行重复 HTML)→ 改为 `v-for="t in displayedTabs"`
- `TAB_STEP_MAP` 硬编码映射(tab key → stepId 推不出来)
- `perTableStats / flatEmptyFields / perTableRows` 派生 computed(empty_fields 改为从 data 读取 flat 副本)
- `filteredViolations` computed(由新 `resolveRows()` 接管)
**新增**:
- `displayedTabs` computed:遍历 `sections[*]._protocol.tabs`,过滤 `step_id ∈ steps_planned`
- `displayedKpis` computed:聚合所有 tab 的 `kpis[]`,按 spec 求值(用 `evalPathExpr`)
- `evalPathExpr / resolvePath / resolveRows / renderSummary` 工具函数
- `resolveRender(col, row)`:按 `render.kind` 分发到 10 个 render 原语(text / code / tag / tag_list / two_line / three_line / progress / percent / ratio / number)
- `ColumnRender` 函数组件:把 VNode 渲染到 `<el-table-column>` 的 cell slot(绕过 `<component :is>` 不接受 VNode 的限制)
- 3 个嵌套展开行(merge / standards / empty_fields)保留 Vue 原生 template(展开行内容太特定,无法完全模板化;协议 expand 字段保留作为文档/参考)
#### 6. 验证脚本 `scripts/`
- `verify_protocol.cjs`:Node + vm 提取前端 4 个核心函数,单测 12 个 case(evalPathExpr 简单路径 / len() / 数字 / 嵌套;resolveRows 简单 + linkage 过滤;renderSummary 模板替换 + 千分位)
- `verify_e2e.cjs`:mock Vue.ref / Vue.computed,跑完整 setup(),断言 displayedTabs 数量/keys/stepId 过滤,displayedKpis 数值求值 → 9 cases
- `verify_render.cjs`:mock h()/resolveComponent(),跑 resolveRender 对 10 个 render kind + fallback → 17 cases
- 总计 **38 个 case 全部通过**
### 已知未做(后续任务)
- `step8_report.py` 仍是第二套硬编码(`sections.get("redundancy_fields", {})` 幽灵 key、`evidence/similar_to` 字段不产),下次单独迁移到协议读取
- 协议 expand 行内 field-link 点击高亮需要前端特判(协议不支持表达 onclick handler)
### 未提交
按用户要求,待 review 后再提交。
---
## Bug 修复:页面变白(Vue 模板编译器生成坏 JS)
### 现象
用户在浏览器刷新页面后报告「页面又白了」,DevTools 控制台:
```
VM6572:1022 Uncaught SyntaxError: Unexpected token ')' (at VM6572:1022:64)
at Function (<anonymous>)
at aX (vue.global.prod.js:13:318)
```
Vue 编译器生成的 render 函数里包含「`()`」—— 空 call expression,触发 SyntaxError,整个 app 挂载失败。
### 根因分析(两轮)
**第一轮错误诊断**:以为问题在重构时引入的 `ColumnRender` 组件(模块级 + 闭包)。删除该组件 + 在 template 里内联 10 个 `<template v-else-if>` 分支后,verify_template 用 `@vue/compiler-dom.compile()` 检查生成代码合法 —— **但用户页面仍然白**。
**真正的根因**:v-else 在深层 v-else-if 链末尾触发了 Vue 3 编译器的输出 bug。template 长这样:
```vue
<template v-if="...">...</template>
<template v-else-if="...">...</template> <!-- ×10 个分支 -->
<template v-else-if="col.render && col.render.kind === 'number'">...</template>
<template v-else>...</template> <!-- ← 这一行 -->
```
最后一个 `v-else` 没有条件。Vue 3 prod build(3.5.40)把它编译成嵌套三元链最末尾的 `: ()` —— 一个空 call expression,不是合法 JS:
```js
... : (col.render && col.render.kind === 'number')
? { name: "default", fn: ..., key: "12" }
: () ← SyntaxError: Unexpected token ')'
? { name: "default", fn: ..., key: "13" }
```
`@vue/compiler-dom`(npm 上的 3.5.41)单独跑同样的模板输出不带 bug 的代码(`95635` chars vs Vue prod 的 `69296` chars),所以之前用 `@vue/compiler-dom` 自测没抓到 —— 必须用真实 Vue prod build + jsdom 才能复现。
### 修复
template 末位 `<template v-else>` 改为显式条件:
```vue
<template v-else-if="!col.render || !col.render.kind" #default="{ row }">
{{ row[col.prop] }}
</template>
```
绕开 Vue 编译器对 `v-else` 在长 v-else-if 链末尾的输出 bug。语义等价(之前所有 `v-else-if` 都没匹配时才落到这一支)。
### 验证
重写 `scripts/verify_template.cjs`:
1. jsdom 跑真实 `vue.global.prod.js`(不是 npm 上拆出来的 `@vue/compiler-dom`)
2. 劫持 `window.Function` 构造器截获 Vue 编译出的 render 函数代码
3. 用 `new Function(code)` 验证每段代码是合法 JS
4. **失败时**会把那段代码写到 `scripts/compiled_N.bad.js` 方便排查
执行:
```
$ node scripts/verify_template.cjs
✓ captured at least 1 compiled render fn
✓ compiled #0 (69296 chars) is valid JS
2 passed, 0 failed
```
四个验证脚本合计 **35 个 case 全过**(2 template + 12 protocol + 12 render + 9 e2e)。
### 关键教训
1. **Vue prod build ≠ npm 上的 compiler-dom**:compiler-sfc 拆出来的 compiler-dom 跟 Vue runtime 里 bundle 的 compiler 版本/编译选项可能不同;做模板编译验证必须用真实 prod build。
2. **v-else 在长 v-else-if 链末尾要谨慎**:超过 ~5 层以后,Vue 3.5.x 的编译器会输出 `: ()`。要么用 `v-else-if="!cond"` 显式收尾,要么减少分支数。
3. **Function 劫持是个便宜的验证手段**:想要捕获「模板编译 → render 函数构造」这一步的 SyntaxError,劫持 `window.Function` 抓大于 1KB 的代码体即可。
### 未提交
按用户要求,待 review 后再提交。
This diff is collapsed.
{
"devDependencies": {
"@vue/compiler-dom": "^3.5.41",
"jsdom": "^30.0.1"
}
}
This diff is collapsed.
// scripts/verify_protocol.cjs
// 验证前端关键逻辑(路径求值 + 渲染入口)。
// 策略:从 index.html 抽取 evalPathExpr / resolvePath / resolveRows / renderSummary 四个核心函数,
// 注入到 Node 沙箱里独立测试。
const fs = require('fs');
const path = require('path');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'static', 'index.html'), 'utf8');
const scriptMatch = html.match(/<script>\s*\n([\s\S]*?)<\/script>/);
const scriptBody = scriptMatch[1];
// 抽出 4 个核心函数(用正则找完整函数体:含参数列表 + 函数体)
function extract(name) {
// 匹配 `function NAME(args) {` —— 不含函数名前缀
const re = new RegExp(`function ${name}\\s*\\(([^)]*)\\)\\s*\\{`);
const m = scriptBody.match(re);
if (!m) throw new Error(`${name} not found`);
const headerEnd = m.index + m[0].length; // 紧接 `{` 之后
let depth = 1, i = headerEnd;
for (; i < scriptBody.length; i++) {
const c = scriptBody[i];
if (c === '{') depth++;
else if (c === '}') {
depth--;
if (depth === 0) {
// 返回:完整函数定义(含参数 + 函数体)
return scriptBody.slice(m.index, i + 1);
}
}
}
throw new Error(`${name} unterminated`);
}
const functions = ['evalPathExpr', 'resolvePath', 'resolveRows', 'renderSummary'];
const fns = {};
for (const name of functions) {
fns[name] = extract(name);
console.log(` extracted ${name} (${fns[name].length} bytes)`);
}
// 在 vm 里执行:把 4 个函数暴露到 global,调用它们
const vm = require('vm');
const ctx = {
console,
selectedField: { value: null },
};
vm.createContext(ctx);
try {
// 拼装一个 IIFE 把 4 个函数挂到 global
const code = `
${fns.evalPathExpr}
${fns.resolvePath}
${fns.resolveRows}
${fns.renderSummary}
globalThis.__exports = { evalPathExpr, resolvePath, resolveRows, renderSummary };
`;
vm.runInContext(code, ctx, { filename: 'extracted.js' });
} catch (e) {
console.error('Extract exec failed:', e.message);
process.exit(1);
}
const { evalPathExpr, resolvePath, resolveRows, renderSummary } = ctx.__exports;
console.log('OK: 4 functions loaded');
// ── 用例 ──
let pass = 0, fail = 0;
function eq(actual, expected, label) {
const a = JSON.stringify(actual), b = JSON.stringify(expected);
if (a === b) { pass++; console.log(` ✓ ${label}`); }
else { fail++; console.error(` ✗ ${label}: got ${a}, want ${b}`); }
}
// 1. evalPathExpr 简单路径
const data = { merge_candidates: [{id:'M-1'}], summary: { high: 3 } };
eq(evalPathExpr('this.summary.high', data), 3, 'evalPathExpr: this.summary.high');
eq(evalPathExpr('this.merge_candidates', data)[0].id, 'M-1', 'evalPathExpr: array index');
eq(evalPathExpr('len(this.merge_candidates)', data), 1, 'evalPathExpr: len()');
eq(evalPathExpr('len(this.summary.high)', data), 3, 'evalPathExpr: len(number)');
eq(evalPathExpr('this.nonexistent', data), '', 'evalPathExpr: missing key -> ""');
// 2. resolveRows
const rows = resolveRows('this.merge_candidates', data, null, 0);
eq(rows.length, 1, 'resolveRows: simple path');
// 3. resolveRows with linkage filter
const dataWithLinkage = {
violations: [
{ table_name: 't_user', column_name: 'id_card' },
{ table_name: 't_user', column_name: 'mobile' },
{ table_name: 't_other', column_name: 'id_card' },
],
};
const linkage = {
source_table: 0,
target_table: 0,
field_map: { table: 'table_name', field: 'column_name' },
default_filter_kind: 'limit',
default_filter_value: 200,
};
// 未选中:截 200 → 全返(这里只有 3 条)
ctx.selectedField.value = null;
const allRows = resolveRows('this.violations', dataWithLinkage, linkage, 0);
eq(allRows.length, 3, 'resolveRows: no selectedField -> slice(0, 200)');
// 选中:只过滤 t_user.id_card
ctx.selectedField.value = { table: 't_user', field: 'id_card' };
const filtered = resolveRows('this.violations', dataWithLinkage, linkage, 0);
eq(filtered.length, 1, 'resolveRows: selectedField -> 1 match');
eq(filtered[0].table_name, 't_user', 'resolveRows: filter by table_name');
ctx.selectedField.value = null;
// 4. renderSummary 模板替换
const summary = {
type: 'warning',
template: '高空 {{high}} 中空 {{mid}}',
vars: { high: 'this.summary.high', mid: 'this.summary.mid' },
};
const text = renderSummary(summary, { summary: { high: 3, mid: 7 } });
eq(text, '高空 3 中空 7', 'renderSummary: number interpolation');
// 千分位
const text2 = renderSummary(
{ type: 'info', template: '{{n}}', vars: { n: 'this.summary.count' } },
{ summary: { count: 1234 } }
);
eq(text2, '1,234', 'renderSummary: thousand_sep');
// 5. resolvePath
const rp = resolvePath('this.summary.high', data);
eq(rp, 3, 'resolvePath: this.summary.high');
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail === 0 ? 0 : 1);
\ No newline at end of file
// scripts/verify_render.cjs
// 验证 resolveRenderPath 对所有 render.kind 路径表达式都能正确求值。
// (template 里把 ColumnRender 拆掉后,渲染逻辑全部内联为 v-else-if 分支,
// 共享的只是 resolveRenderPath 这个取值小函数。)
const fs = require('fs');
const path = require('path');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'static', 'index.html'), 'utf8');
const scriptBody = html.match(/<script>\s*\n([\s\S]*?)<\/script>/)[1];
function extract(name) {
const rePrefix = new RegExp(`function\\s+${name}\\s*\\(\\s*([^)]*?)\\s*\\)\\s*\\{`);
const isFnDecl = !!scriptBody.match(rePrefix);
const re = new RegExp(`(?:function\\s+)?${name}\\s*\\(\\s*([^)]*?)\\s*\\)\\s*\\{`);
const m = scriptBody.match(re);
if (!m) throw new Error(`${name} not found`);
const headerEnd = m.index + m[0].length;
let depth = 1, i = headerEnd;
for (; i < scriptBody.length; i++) {
if (scriptBody[i] === '{') depth++;
else if (scriptBody[i] === '}') { depth--; if (depth === 0) break; }
}
const body = scriptBody.slice(m.index, i + 1);
return isFnDecl ? body : 'function ' + body;
}
const fnBody = extract('resolveRenderPath');
console.log(`resolveRenderPath body: ${fnBody.length} bytes`);
const vm = require('vm');
const ctx = { console };
vm.createContext(ctx);
try {
vm.runInContext(`${fnBody}; globalThis.__rrp = resolveRenderPath;`, ctx);
} catch (e) {
console.error('Extract failed:', e.message);
process.exit(1);
}
const rrp = ctx.__rrp;
let pass = 0, fail = 0;
function check(label, actual, expected) {
const a = JSON.stringify(actual), b = JSON.stringify(expected);
if (a === b) { console.log(` ✓ ${label}`); pass++; }
else { console.error(` ✗ ${label}: got ${a}, want ${b}`); fail++; }
}
// 模拟 row:standards tab 的 violations_by_standard 行
const rowStd = {
standard: 'GB_11643',
rule: '18 位',
basis: '正则',
level: 'high',
};
// empty_fields perTableRow
const rowEmpty = {
table_name: 't_user',
table_comment: '用户表',
high_empty_fields_count: 5,
empty_rate: 0.85,
level: 'high',
};
// merge_candidates 行
const rowMerge = {
priority: 'P1',
classification: 'high',
confidence: 0.95,
confidence_label: '高',
};
console.log('\n--- resolveRenderPath: simple paths ---');
check('this.standard', rrp(rowStd, 'standard'), 'GB_11643');
check('this.rule', rrp(rowStd, 'rule'), '18 位');
check('this.basis', rrp(rowStd, 'basis'), '正则');
check('this.level', rrp(rowStd, 'level'), 'high');
check('this.table_name', rrp(rowEmpty, 'table_name'), 't_user');
check('this.empty_rate (num)', rrp(rowEmpty, 'empty_rate'), 0.85);
console.log('\n--- resolveRenderPath: edge cases ---');
check('empty path', rrp(rowStd, ''), '');
check('null path', rrp(rowStd, null), '');
check('missing key', rrp(rowStd, 'nonexistent'), '');
check('wrong type (string)', rrp('just-a-string', 'foo'), '');
console.log('\n--- computed/clamped values used in template ---');
// progress 用的 0-1 clamp
const v = Math.max(0, Math.min(1, Number(rrp(rowEmpty, 'empty_rate')) || 0));
check('progress v clamped', v, 0.85);
// percent 用的 (value*100).toFixed(1)
const pct = (Number(rrp(rowEmpty, 'empty_rate')) * 100).toFixed(1);
check('percent formatted', pct, '85.0');
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail === 0 ? 0 : 1);
\ No newline at end of file
// scripts/verify_template.cjs
//
// 端到端模板编译验证:把 <div id="app">…</div> 喂给 Vue 3 prod build 的 compile(),
// 断言生成的 render 函数是合法 JS。
//
// 这一步专门兜底「页面变白 / SyntaxError in compiled code」这类回归。
// 历史教训:v-else 在深层 v-else-if 链末尾会被 Vue 编译器输出为 `: ()`(空
// call expression = SyntaxError)。改用 v-else-if="!cond" 显式条件可绕过。
const fs = require('fs');
const path = require('path');
const { JSDOM } = require('jsdom');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'static', 'index.html'), 'utf8');
let pass = 0, fail = 0;
function check(label, ok, detail) {
if (ok) { console.log(` ✓ ${label}`); pass++; }
else { console.error(` ✗ ${label}${detail ? ' — ' + detail : ''}`); fail++; }
}
// ── 1. 抽出 <div id="app">…</div> 作为模板 ──
const startIdx = html.indexOf('<div id="app">');
const scriptIdx = html.indexOf('<script>');
let endIdx = html.lastIndexOf('</div>', scriptIdx);
if (endIdx < 0) endIdx = startIdx + 100; // 不太可能走到这里
const template = html.slice(startIdx, endIdx + '</div>'.length);
console.log(`Template: ${template.length} chars`);
// ── 2. 在 jsdom 里跑真实 Vue 3 prod build,劫持 Function 构造器截获编译产物 ──
const captured = [];
const dom = new JSDOM(html, {
url: 'http://localhost:8765/',
runScripts: 'dangerously',
resources: 'usable',
pretendToBeVisual: true,
});
const OrigFunction = dom.window.Function;
dom.window.Function = function(...args) {
const body = args[args.length - 1];
if (typeof body === 'string' && body.length > 1000) {
captured.push(body);
}
return OrigFunction.apply(this, args);
};
const vueSrc = fs.readFileSync(
path.join(__dirname, '..', 'web', 'static', 'lib', 'vue.global.prod.js'), 'utf8');
const epSrc = fs.readFileSync(
path.join(__dirname, '..', 'web', 'static', 'lib', 'element-plus', 'index.full.min.js'), 'utf8');
const scriptEl = dom.window.document.createElement('script');
scriptEl.textContent = vueSrc + '\n' + epSrc;
dom.window.document.head.appendChild(scriptEl);
setTimeout(() => {
console.log(`\nCaptured ${captured.length} compiled render fn(s)\n`);
// ── 3. 把每段编译产物当函数体 new Function() ──
check('captured at least 1 compiled render fn', captured.length >= 1,
`got ${captured.length}`);
for (let i = 0; i < captured.length; i++) {
const code = captured[i];
try {
new OrigFunction(code);
check(`compiled #${i} (${code.length} chars) is valid JS`, true);
} catch (e) {
check(`compiled #${i} (${code.length} chars) is valid JS`, false, e.message);
// 把出问题的代码写到磁盘
fs.writeFileSync(path.join(__dirname, `compiled_${i}.bad.js`), code, 'utf8');
}
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail === 0 ? 0 : 1);
}, 600);
\ No newline at end of file
......@@ -106,19 +106,16 @@ class StandardsListResponse(BaseModel):
# ── 治理结果(前端多级表格消费) ──
class OverviewStats(BaseModel):
"""概览数据 —— 只放任务级通用元数据(数据库/表/字段规模 + 时间)。
各 step 的 KPI(合并候选 / 冗余字段 / 高空字段 / 缺注释字段 / 长度异常 / 国标违规等)
由各 step 在自身 data._protocol.kpis[] 里声明,前端按协议从 _protocol 解析。
"""
database: str = ""
host: str = ""
db_type: str = ""
total_tables: int = 0
total_fields: int = 0
high_empty_fields: int = 0
mid_empty_fields: int = 0
missing_comments: int = 0
length_issues: int = 0
standard_violations: int = 0
standards_applied: int = 0
merge_candidates: int = 0
redundancy_fields: int = 0
executed_at: Optional[datetime] = None
......
......@@ -372,16 +372,13 @@ async def run_governance_workflow(
f"主步骤调度完成: 成功 {sorted(completed)},"
f"非关键失败 {sorted(failed_non_critical)}")
# ── 计算 overview ──
overview = _build_overview(sections, dict_data, cfg)
# ── 计算 overview(只汇总任务级通用元数据;step KPI 见各 section._protocol.kpis) ──
overview = _build_overview(dict_data, cfg)
sections["overview"] = overview
log("INFO",
f"概览汇总: 表 {overview.get('total_tables', 0)} 张, "
f"字段 {overview.get('total_fields', 0)} 个, "
f"高空字段 {overview.get('high_empty_fields', 0)}, "
f"缺失注释 {overview.get('missing_comments', 0)}, "
f"长度异常 {overview.get('length_issues', 0)}, "
f"国标违规 {overview.get('standard_violations', 0)}")
f"字段 {overview.get('total_fields', 0)} 个 "
f"(各 step KPI 见其 _protocol.kpis)")
# ── 报告生成(主步骤跑完后无条件触发) ──
_check_cancel(cancel_event)
......@@ -439,32 +436,18 @@ def _generate_reports(run_dir: Path, sections: dict, overview: dict,
# ── Overview 构造 ──
def _build_overview(sections: dict, dict_data: dict, cfg: DBConfig) -> dict:
"""从各 section + 数据字典汇总概览数据"""
overview_meta = dict_data.get("overview_meta", {}) if isinstance(dict_data, dict) else {}
empty = sections.get("empty_fields", {}).get("summary", {})
missing = sections.get("missing_comments", {}).get("summary", {})
length = sections.get("length_issues", {}).get("summary", {})
standards = sections.get("standard_violations", {}).get("summary", {})
merge = sections.get("merge_candidates", {})
def _build_overview(dict_data: dict, cfg: DBConfig) -> dict:
"""汇总任务级通用元数据。
各 step 的 KPI(合并候选 / 冗余字段 / 高空字段 / ...)由各 step 在自身
data._protocol.kpis[] 里声明,前端按协议从 _protocol 解析(不再走 overview)。
"""
overview_meta = dict_data.get("overview_meta", {}) if isinstance(dict_data, dict) else {}
return {
"database": cfg.database,
"host": cfg.host,
"db_type": cfg.db_type,
"total_tables": overview_meta.get("total_tables", 0),
"total_fields": overview_meta.get("total_fields", 0),
"high_empty_fields": empty.get("high_empty_fields_count", 0),
"mid_empty_fields": empty.get("mid_empty_fields_count", 0),
"missing_comments": missing.get("total_missing_comments", 0),
"length_issues": length.get("total_issues", 0),
"standard_violations": standards.get("violations", 0),
"standards_applied": standards.get("standards_applied", 0),
"merge_candidates": (
len(merge.get("merge_candidates", [])) if isinstance(merge, dict) else 0
),
"redundancy_fields": (
len(merge.get("redundancy_fields", [])) if isinstance(merge, dict) else 0
),
"executed_at": datetime.now().isoformat(),
}
\ No newline at end of file
......@@ -14,10 +14,117 @@ from collections import Counter, defaultdict
from typing import Callable
from ..llm import LLMClient
from ..tab_protocol import (
TabProtocol, TableSpec, ColumnSpec, RenderSpec, TagStyle, Threshold,
KpiSpec, inject_protocol,
)
logger = logging.getLogger(__name__)
# ── 协议 ──────────────────────────────────────────────
# 两个 tab:merge_candidates(含嵌套 expand)+ redundancy_fields
_MERGE_EXPAND_TEMPLATE = """
<p><strong>合并建议:</strong>{{ suggestion }}</p>
<p><strong>合并风险:</strong>{{ risk }}</p>
<p><strong>共同字段({{ common_columns }}):</strong></p>
<div>
{{ columns | tags }}
</div>
<p><strong>差异字段:</strong></p>
<div>
{{ diff_columns | tags }}
</div>
""".strip()
_MERGE_TAB = TabProtocol(
key="merge_candidates",
title="表合并候选",
kpis=(
KpiSpec(css_class="kpi-merge", label="合并候选",
value="len(this.merge_candidates)"),
),
tables=(
TableSpec(
source="this.merge_candidates",
columns=(
ColumnSpec(type="expand", template=_MERGE_EXPAND_TEMPLATE),
ColumnSpec(prop="id", label="ID", width=100),
ColumnSpec(prop="priority", label="优先级", width=80,
render=RenderSpec(
kind="tag",
value_map={
"P1": TagStyle(type="danger", text="P1"),
"P2": TagStyle(type="warning", text="P2"),
},
)),
ColumnSpec(prop="tables", label="涉及表", width=300,
render=RenderSpec(kind="tag_list")),
ColumnSpec(prop="common_columns", label="共同字段数", width=100),
ColumnSpec(prop="similarity", label="相似度", width=100),
ColumnSpec(prop="suggestion", label="建议",
show_overflow_tooltip=True),
),
),
),
)
_REDUNDANCY_TAB = TabProtocol(
key="redundancy_fields",
title="冗余字段",
kpis=(
KpiSpec(css_class="kpi-redundancy", label="冗余字段",
value="len(this.redundancy_fields)"),
),
summary=None, # 用 el-alert 模板渲染,下面在 data 里加
tables=(
TableSpec(
source="this.redundancy_fields",
columns=(
ColumnSpec(prop="field", label="字段名", width=160, fixed="left",
render=RenderSpec(kind="code")),
ColumnSpec(prop="table_count", label="出现表数", width=100,
sortable=True, align="right"),
ColumnSpec(prop="classification", label="分类", width=140,
render=RenderSpec(
kind="tag",
value_map={
"true_redundancy": TagStyle(type="danger", text="真冗余", effect="dark"),
"suspicious": TagStyle(type="warning", text="疑似冗余", effect="dark"),
"common_business": TagStyle(type="success", text="业务共享", effect="plain"),
"common_base": TagStyle(type="info", text="通用基础", effect="plain"),
},
fallback=TagStyle(type="danger", text="LLM 失败", italic=True),
)),
ColumnSpec(prop="reasoning", label="判断依据",
min_width=240, show_overflow_tooltip=True),
ColumnSpec(prop="recommendation", label="建议",
min_width=220, show_overflow_tooltip=True),
ColumnSpec(prop="source", label="来源", width=80, align="center",
render=RenderSpec(
kind="tag",
value_map={
"llm": TagStyle(type="success", text="LLM"),
"llm_failed": TagStyle(type="danger", text="LLM 失败"),
},
)),
),
),
),
)
def _wrap(data: dict) -> dict:
return inject_protocol(
data,
step_id="merge_redundancy",
tabs=[_MERGE_TAB, _REDUNDANCY_TAB],
)
COMMON_FIELDS = {
"id", "create_time", "update_time", "create_by", "update_by",
"remark", "del_flag", "tenant_id", "dept_id", "status",
......@@ -41,7 +148,7 @@ def run_step2(dict_data: dict, llm: LLMClient | None = None,
if not columns:
if log:
log("WARN", "未获取到任何字段元数据,跳过", step="merge_redundancy")
return {"merge_candidates": [], "decommission_candidates": {}, "redundancy_fields": []}
return _wrap({"merge_candidates": [], "decommission_candidates": {}, "redundancy_fields": []})
# 表级过滤:保留被勾选的表(或全部)
if table_filter:
......@@ -87,11 +194,11 @@ def run_step2(dict_data: dict, llm: LLMClient | None = None,
f"高频字段 {len(redundancy)} 个",
step="merge_redundancy")
return {
return _wrap({
"merge_candidates": merge,
"decommission_candidates": decom,
"redundancy_fields": redundancy,
}
})
def _table_columns(cols: list[dict]) -> set[str]:
......
......@@ -20,11 +20,158 @@ from collections import defaultdict
from typing import Callable
from ..db_adapter import DBConfig, open_db, quote_ident, TEXT_TYPES
from ..tab_protocol import (
TabProtocol, TableSpec, ColumnSpec, RenderSpec, TagStyle,
KpiSpec, SummarySpec, inject_protocol,
)
from web.sql.loader import get_sql_loader
logger = logging.getLogger(__name__)
# ── 协议 ──────────────────────────────────────────────
# 单 tab「大范围空字段」,含 2 张子表:
# 1. flatEmptyFields(this.summary.flat_high_empty_fields + flat_mid_empty_fields 合并的扁平列表)
# 2. perTableRows(this.per_table,每张表的展开行带 high/mid 子表)
# 注:原前端有「perTableStats」派生 computed,本协议把它下沉到 data.per_table 本身,
# perTableRows 的 expand 模板里用 {{ high_empty_fields }} / {{ mid_empty_fields }} 直接渲染。
_EMPTY_TAB = TabProtocol(
key="empty_fields",
title="大范围空字段",
kpis=(
KpiSpec(css_class="kpi-empty", label="高空字段",
value="this.summary.high_empty_fields_count"),
KpiSpec(css_class="kpi-empty-mid", label="中空字段",
value="this.summary.mid_empty_fields_count"),
),
summary=SummarySpec(
type="warning",
template=(
"扫描 {{scanned}} 张表 / {{fields}} 个字段,"
"发现高空 {{high}} 个 + 中空 {{mid}} 个,"
"涉及 {{tables}} 张表"
),
vars={
"scanned": "this.summary.total_tables_scanned",
"fields": "this.summary.total_fields_checked",
"high": "this.summary.high_empty_fields_count",
"mid": "this.summary.mid_empty_fields_count",
"tables": "this.summary.tables_with_issues_count",
},
),
tables=(
# 子表 1:扁平展示所有空字段
TableSpec(
title="扁平视图:所有空字段",
source="this.summary.flat_high_empty_fields",
max_height=560,
columns=(
ColumnSpec(
label="表名 / 表注释", width=260,
render=RenderSpec(
kind="two_line",
main_path="table_name",
sub_path="table_comment",
sub_empty_text="(无表注释)",
),
),
ColumnSpec(
label="字段名 / 字段注释", width=240,
render=RenderSpec(
kind="two_line",
main_path="column_name",
sub_path="column_comment",
sub_empty_text="(无字段注释)",
),
),
ColumnSpec(prop="data_type", label="类型", width=100,
render=RenderSpec(kind="code")),
ColumnSpec(
label="空值率", prop="empty_rate", width=160, align="center",
render=RenderSpec(
kind="progress",
value_path="empty_rate",
status_from_value_path="level",
),
),
ColumnSpec(
label="空值数", prop="empty_count", width=110, align="right",
render=RenderSpec(kind="number", format="thousand_sep"),
),
ColumnSpec(
label="级别", prop="level", width=80, align="center",
render=RenderSpec(
kind="tag",
value_map={
"high": TagStyle(type="danger", text="高空", effect="dark"),
"mid": TagStyle(type="warning", text="中空", effect="plain"),
},
),
),
),
),
# 子表 2:按表聚合,每行可点击查看该表的明细
TableSpec(
title="按表聚合",
source="this.per_table",
default_sort={"prop": "high_count", "order": "descending"},
max_height=560,
columns=(
ColumnSpec(type="expand", template=(
"<div class=\"expand-block\">"
"<p><strong>高空字段({{ high_count }}):</strong></p>"
"<ul>{{ high_empty_fields }}</ul>"
"<p><strong>中空字段({{ mid_count }}):</strong></p>"
"<ul>{{ mid_empty_fields }}</ul>"
"</div>"
)),
ColumnSpec(
label="表名 / 表注释", min_width=240,
render=RenderSpec(
kind="two_line",
main_path="table_name",
sub_path="table_comment",
sub_empty_text="(无表注释)",
),
),
ColumnSpec(prop="total_rows", label="总行数", width=100,
align="right",
render=RenderSpec(kind="number", format="thousand_sep")),
ColumnSpec(prop="high_count", label="高空字段", width=100,
align="center"),
ColumnSpec(prop="mid_count", label="中空字段", width=100,
align="center"),
),
),
),
)
def _wrap(data: dict) -> dict:
return inject_protocol(data, step_id="empty_fields", tabs=[_EMPTY_TAB])
def _EMPTY_FALLBACK() -> dict:
"""dict_data 为空时的兜底返回 —— 仍走协议,保持前端结构稳定。"""
return _wrap({
"summary": {
"total_tables_scanned": 0,
"total_fields_checked": 0,
"high_empty_fields_count": 0,
"mid_empty_fields_count": 0,
"tables_with_issues_count": 0,
"skipped_tables_count": 0,
"thresholds": {"high": 0.80, "mid": 0.50},
"top_tables_by_high_empty_fields": [],
"flat_high_empty_fields": [],
"flat_mid_empty_fields": [],
},
"per_table": [],
"skipped": [],
})
def run_step4(cfg: DBConfig, log: Callable | None = None,
dict_data: dict | None = None,
min_rows_to_check: int = 5,
......@@ -46,7 +193,7 @@ def run_step4(cfg: DBConfig, log: Callable | None = None,
if not (dict_data and dict_data.get("data_dictionary")):
if log:
log("ERROR", "dict_data 为空,请检查 orchestrator 是否正确传入", step="empty_fields")
return {"summary": {}, "per_table": [], "skipped": []}
return _EMPTY_FALLBACK()
columns = dict_data["data_dictionary"]
table_summary = dict_data.get("table_summary", [])
......@@ -87,6 +234,7 @@ def run_step4(cfg: DBConfig, log: Callable | None = None,
tables_with_issues = set()
per_table: list[dict] = []
high_fields_all: list[dict] = []
mid_fields_all: list[dict] = []
skipped = []
top_tables: list[dict] = []
......@@ -172,6 +320,7 @@ def run_step4(cfg: DBConfig, log: Callable | None = None,
field_record["level"] = "mid"
mid_count += 1
mid_in_table.append(field_record)
mid_fields_all.append(field_record)
if high_in_table or mid_in_table:
tables_with_issues.add(table)
......@@ -208,7 +357,8 @@ def run_step4(cfg: DBConfig, log: Callable | None = None,
"skipped_tables_count": len(skipped),
"thresholds": {"high": high_threshold, "mid": mid_threshold},
"top_tables_by_high_empty_fields": top_tables[:10],
"high_empty_fields": high_fields_all[:200],
"flat_high_empty_fields": high_fields_all[:200],
"flat_mid_empty_fields": mid_fields_all[:200],
}
if log:
......@@ -217,11 +367,11 @@ def run_step4(cfg: DBConfig, log: Callable | None = None,
f"跳过 {len(skipped)} 张",
step="empty_fields")
return {
return _wrap({
"summary": summary,
"per_table": per_table,
"skipped": skipped,
}
})
def _build_empty_clauses(cols: list[dict], db_type: str) -> tuple[str, list[dict]]:
......
......@@ -18,10 +18,90 @@ from collections import defaultdict
from typing import Callable
from ..llm import LLMClient
from ..tab_protocol import (
TabProtocol, TableSpec, ColumnSpec, RenderSpec, TagStyle,
KpiSpec, SummarySpec, inject_protocol,
)
logger = logging.getLogger(__name__)
# ── 协议 ──────────────────────────────────────────────
# 单 tab「缺失注释字段」,含 1 张主表 + 顶部 alert
# 列:表名+注释 / 字段名+注释 / 类型 / 推测注释 / 置信度 / 依据
_MISSING_TAB = TabProtocol(
key="missing_comments",
title="缺失注释字段",
kpis=(
KpiSpec(css_class="kpi-missing", label="缺注释字段",
value="this.summary.total_missing_comments"),
),
summary=SummarySpec(
type="info",
template="{{total}} 个字段缺注释,已推测 {{predicted}} 个(其中 LLM 推测 {{llm}} 个)",
vars={
"total": "this.summary.total_missing_comments",
"predicted": "this.summary.predicted_count",
"llm": "this.summary.predicted_by_llm",
},
),
tables=(
TableSpec(
source="this.predicted_comments",
max_height=560,
columns=(
ColumnSpec(
label="表名 / 表注释", min_width=220,
render=RenderSpec(
kind="two_line",
main_path="table_name",
sub_path="table_comment",
sub_empty_text="(无表注释)",
),
),
ColumnSpec(
label="字段名 / 字段注释", min_width=220,
render=RenderSpec(
kind="two_line",
main_path="column_name",
sub_path="column_comment",
sub_empty_text="(无字段注释)",
),
),
ColumnSpec(
prop="data_type", label="类型", width=100,
render=RenderSpec(kind="code"),
),
ColumnSpec(
label="推测注释", prop="predicted", min_width=220,
show_overflow_tooltip=True,
render=RenderSpec(kind="text"),
),
ColumnSpec(
label="置信度", prop="confidence", width=100, align="center",
render=RenderSpec(
kind="tag",
value_map={
"high": TagStyle(type="success", text="high"),
"medium": TagStyle(type="warning", text="medium"),
"low": TagStyle(type="info", text="low"),
},
),
),
ColumnSpec(
label="依据", prop="reason", min_width=240,
show_overflow_tooltip=True,
),
),
),
),
)
def _wrap(data: dict) -> dict:
return inject_protocol(data, step_id="missing_comments", tabs=[_MISSING_TAB])
def run_step5(dict_data: dict, llm: LLMClient | None = None,
log: Callable | None = None,
table_filter: set[str] | None = None) -> dict:
......@@ -139,7 +219,7 @@ def run_step5(dict_data: dict, llm: LLMClient | None = None,
f"LLM 推测成功 {len(predicted)}",
step="missing_comments")
return {
return _wrap({
"summary": {
"total_missing_comments": len(missing),
"tables_affected": len(by_table),
......@@ -151,11 +231,11 @@ def run_step5(dict_data: dict, llm: LLMClient | None = None,
"by_table": by_table_list,
"predicted_comments": predicted[:50],
"unpredictable_sample": unpredicted[:50],
}
})
def _empty_result(by_table: dict) -> dict:
return {
return _wrap({
"summary": {
"total_missing_comments": 0,
"tables_affected": len(by_table),
......@@ -167,4 +247,4 @@ def _empty_result(by_table: dict) -> dict:
"by_table": [],
"predicted_comments": [],
"unpredictable_sample": [],
}
\ No newline at end of file
})
\ No newline at end of file
......@@ -20,9 +20,85 @@ import re
from dataclasses import dataclass
from typing import Callable
from ..tab_protocol import (
TabProtocol, TableSpec, ColumnSpec, RenderSpec,
KpiSpec, SummarySpec, inject_protocol,
)
logger = logging.getLogger(__name__)
# ── 协议 ──────────────────────────────────────────────
_LENGTH_TAB = TabProtocol(
key="length_issues",
title="字段长度异常",
kpis=(
KpiSpec(css_class="kpi-length", label="字段长度异常",
value="this.summary.total_issues"),
),
summary=SummarySpec(
type="warning",
template="{{total}} 条异常,应用 {{rules}} 条规则",
vars={
"total": "this.summary.total_issues",
"rules": "this.summary.rules_applied",
},
),
tables=(
TableSpec(
source="this.issues",
max_height=560,
columns=(
ColumnSpec(
label="表名 / 表注释", width=260,
render=RenderSpec(
kind="two_line",
main_path="table_name",
sub_path="table_comment",
sub_empty_text="(无表注释)",
),
),
ColumnSpec(
label="字段名 / 字段注释", width=240,
render=RenderSpec(
kind="two_line",
main_path="column_name",
sub_path="column_comment",
sub_empty_text="(无字段注释)",
),
),
ColumnSpec(
label="长度 (实际/标准)", width=130, align="center",
render=RenderSpec(
kind="ratio",
num_path="actual_length",
den_path="required_length",
emphasize="num",
),
),
ColumnSpec(prop="wasted_bytes_per_row", label="浪费字节",
width=90, align="right"),
ColumnSpec(
label="国家标准 / 规则 / 依据", width=340,
render=RenderSpec(
kind="three_line",
main_path="standard",
sub_path="rule",
tertiary_path="basis",
tertiary_prefix="依据:",
),
),
),
),
),
)
def _wrap(data: dict) -> dict:
return inject_protocol(data, step_id="length_check", tabs=[_LENGTH_TAB])
@dataclass
class LengthRule:
"""字段长度规则:同时支持字段名正则 + 字段注释关键字匹配。"""
......@@ -165,7 +241,7 @@ def run_step6(dict_data: dict, log: Callable | None = None,
if not columns:
if log:
log("WARN", "未获取到任何字段元数据,跳过", step="length_check")
return {"summary": {"total_issues": 0}, "issues": []}
return _wrap({"summary": {"total_issues": 0}, "issues": []})
if log:
log("INFO",
......@@ -219,7 +295,7 @@ def run_step6(dict_data: dict, log: Callable | None = None,
f"发现异常 {len(issues)} 条",
step="length_check")
return {
return _wrap({
"summary": {
"total_issues": len(issues),
"rules_applied": len(LENGTH_RULES),
......@@ -228,4 +304,4 @@ def run_step6(dict_data: dict, log: Callable | None = None,
"wasted_bytes_total": sum(i.get("wasted_bytes_per_row") or 0 for i in issues),
},
"issues": issues[:200],
}
\ No newline at end of file
})
\ No newline at end of file
......@@ -15,6 +15,10 @@ import logging
from typing import Callable
from ..db_adapter import DBConfig, open_db, quote_ident
from ..tab_protocol import (
TabProtocol, TableSpec, ColumnSpec, RenderSpec, TagStyle, Threshold,
KpiSpec, SummarySpec, LinkageSpec, inject_protocol,
)
from standards.registry import discover_standards, list_all_standards
from standards.base import BaseStandard, ValidationResult
from web.sql.loader import get_sql_loader
......@@ -22,6 +26,147 @@ from web.sql.loader import get_sql_loader
logger = logging.getLogger(__name__)
# ── 协议 ──────────────────────────────────────────────
# 单 tab「国家标准校验」,含 2 张子表 + 跨表联动:
# 1. violations_by_standard —— 每个标准一行,展开看每字段样本 + 点击字段联动到 violations
# 2. violations —— 全部违规值的扁平表(默认前 200 条,选中字段后按 (table,field) 过滤)
# 联动规则:
# - 在 table 0 行的展开区点击 details_with_violations 中的某个 {table, field}
# - 前端把 {table, field} 写入 selectedField
# - filteredViolations = this.violations.filter(v =>
# v[linkage.field_map['table']] === selectedField.table
# && v[linkage.field_map['field']] === selectedField.field)
# - 未选中时:默认按 default_filter 截断 (limit 200)
_STANDARDS_TAB = TabProtocol(
key="standards",
title="国家标准校验",
kpis=(
KpiSpec(css_class="kpi-standards", label="违规字段",
value="this.summary.violations"),
KpiSpec(css_class="kpi-standards-applied", label="已应用标准",
value="this.summary.standards_used"),
),
summary=SummarySpec(
type="error",
template=(
"应用 {{applied}} 个标准, 检查 {{fields}} 个字段, "
"发现 {{violations}} 条违规"
),
vars={
"applied": "this.summary.standards_applied",
"fields": "this.summary.fields_checked",
"violations": "this.summary.violations",
},
),
tables=(
# 表 1(按标准聚合)
TableSpec(
title="按标准聚合",
source="this.violations_by_standard",
default_sort={"prop": "violation_rate", "order": "descending"},
max_height=560,
columns=(
ColumnSpec(type="expand", template=(
"<div class=\"expand-block\">"
"<p><strong>有违规字段(点击下方字段联动到右侧违规明细):</strong></p>"
"<div class=\"field-link-list\">"
"{{ details_with_violations | field_links }}"
"</div>"
"<p class=\"muted\" style=\"margin-top: 8px; font-size: 12px\">"
"取数口径: sample_limit={{ sample_limit }}, "
"max_tables_per_field={{ max_tables_per_field }}"
"</p>"
)),
ColumnSpec(prop="standard", label="标准代号", width=140,
render=RenderSpec(kind="code")),
ColumnSpec(prop="standard_name", label="标准名称", width=240,
show_overflow_tooltip=True),
ColumnSpec(prop="fields_count", label="检查字段数", width=110,
align="center"),
ColumnSpec(prop="violating_fields_count", label="违规字段数",
width=120, align="center"),
ColumnSpec(
label="违规率 (违规/抽样)", prop="violation_rate", width=170,
align="center",
render=RenderSpec(
kind="percent",
value_path="violation_rate",
precision=2,
thresholds=(
Threshold(gt=0.5, color="#f56c6c"),
Threshold(gt=0.2, color="#e6a23c"),
Threshold(gt=0.0, color="#67c23a"),
),
),
),
),
),
# 表 2(违规明细,受联动过滤)
TableSpec(
title="违规明细",
source="this.violations",
default_sort=None, # 由 linkage 接管过滤
max_height=560,
columns=(
ColumnSpec(
label="表名 / 表注释", width=260,
render=RenderSpec(
kind="two_line",
main_path="table_name",
sub_path="table_comment",
sub_empty_text="(无表注释)",
),
),
ColumnSpec(
label="字段名 / 字段注释", width=240,
render=RenderSpec(
kind="two_line",
main_path="column_name",
sub_path="column_comment",
sub_empty_text="(无字段注释)",
),
),
ColumnSpec(prop="rule_type", label="规则", width=160,
render=RenderSpec(kind="code")),
ColumnSpec(prop="value", label="违规值", width=180,
render=RenderSpec(kind="code")),
ColumnSpec(prop="error", label="违规原因", min_width=240,
show_overflow_tooltip=True),
),
),
),
linkage=LinkageSpec(
source_table=0,
clickable_path="details_with_violations",
clickable_id_props=("table", "field"),
target_table=1,
field_map={"table": "table_name", "field": "column_name"},
default_filter_kind="limit",
default_filter_value=200,
),
)
def _wrap(data: dict) -> dict:
return inject_protocol(data, step_id="standards", tabs=[_STANDARDS_TAB])
def _EMPTY_FALLBACK() -> dict:
"""无字段/无标准插件时的兜底 —— 仍走协议。"""
return _wrap({
"summary": {
"standards_applied": 0,
"standards_used": 0,
"fields_checked": 0,
"violations": 0,
},
"violations_by_standard": [],
"violations": [],
"standards_info": [],
})
SAMPLE_LIMIT = 500 # 每字段最多抽样
MAX_TABLES_PER_FIELD = 5 # 每字段最多检查前 N 张表
......@@ -60,7 +205,7 @@ def run_step7(cfg: DBConfig, dict_data: dict, log: Callable | None = None,
f"按 table_filter 过滤后: {len(columns)} 个字段",
step="standards")
if not columns:
return {"summary": {}, "violations_by_standard": [], "violations": []}
return _EMPTY_FALLBACK()
# 字段名 → 标准实例的映射
standards_by_field = discover_standards()
......@@ -79,11 +224,7 @@ def run_step7(cfg: DBConfig, dict_data: dict, log: Callable | None = None,
if not standards_by_field:
if log:
log("WARN", "未发现任何标准插件", step="standards")
return {
"summary": {"standards_applied": 0, "fields_checked": 0, "violations": 0},
"violations_by_standard": [],
"violations": [],
}
return _EMPTY_FALLBACK()
if log:
log("INFO",
......@@ -232,7 +373,7 @@ def run_step7(cfg: DBConfig, dict_data: dict, log: Callable | None = None,
if log:
log("INFO", f"校验完成: {len(by_standard_list)} 个标准有违规, {total_fields_checked} 个字段, {total_violations} 条违规", step="standards")
return {
return _wrap({
"summary": {
"standards_applied": len(all_standards_info),
"standards_used": len(by_standard_list),
......@@ -242,4 +383,4 @@ def run_step7(cfg: DBConfig, dict_data: dict, log: Callable | None = None,
"violations_by_standard": by_standard_list,
"violations": violations_flat[:500], # 给前端表格用
"standards_info": all_standards_info,
}
\ No newline at end of file
})
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment