Skip to content

D
db-tools
Commit 54429b71 authored by wangteng's avatar wangteng
Browse files
Options

整理优化完善

parent bd79c9e5
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 1269 additions and 2152 deletions
.gitignore
View file @ 54429b71
......@@ -5,6 +5,8 @@ work-logs
# ── 数据库相关 ──
national_standards
# Oracle Instant Client 为平台二进制依赖，体积大且应从官方渠道获取。
oracle_client/
# ── 运行产出（不纳入版本控制） ──
outputs/
......@@ -15,9 +17,13 @@ web/outputs/
web/configs/llm.yaml
web/configs/db_defaults.yaml
web/backend/configs/llm.yaml
web/backend/configs/database.test.yaml
web/backend/configs/database.production.yaml
workflow/config.yaml
backend-deploy.tgz
frontend-deploy.tgz
db-tool-*-deploy.tgz
release/
# ── web 运行时数据 ──
web/data/
......
CLAUDE.md
View file @ 54429b71
# 说明
- 这是一个数据库工具，当前代码统一位于 `./web/`
- 这是一个数据库工具，当前业务代码统一位于 `./web/`
# 总体要求
- 每做一个任务就在工作记录中记录一次
# 服务器发版本运行
- deploy_server.py
- `deploy/deploy_server.py`
# 目录约定
- `web/`：前端、后端及产品文档，是日常业务开发的唯一入口。
- `deploy/`：构建发布包、远程部署、systemd 与 Nginx 配置。
- `scripts/verify/`：前端组件和协议验证脚本。
- `scripts/maintenance/`：本地维护脚本，例如停止开发服务、整理历史数据。
- `scripts/diagnostics/`：一次性环境与数据库诊断脚本。
- `release/`：构建生成的发布包，不纳入版本控制。
README.md 0 → 100644
View file @ 54429b71
# 数据库工具
项目的日常开发入口是 `web/`，前端使用 Vue 3 + Vite，后端使用 FastAPI + SQLAlchemy。当前业务数据保存于 MySQL，旧 SQLite 文件仅用于历史数据迁移。
## 目录
- `web/`：产品代码、运行配置和产品文档。
- `deploy/`：发布构建、远程部署、systemd 与 Nginx 配置。
- `scripts/`：验证、维护和诊断脚本。
- `oracle_client/`：本机可选的 Oracle Instant Client 运行文件，不纳入版本控制。
- `standards/`：标准校验规则与基础数据。
- `workflow/`：离线数据治理工作流。
- `data_dictionary/`：数据字典采集和检查工具。
- `release/`：本地生成的发布包，不纳入版本控制。
## 本地运行
Windows 可执行 `web/start.bat` 同时启动前后端，也可分别运行：
```powershell
python -m web.backend.start
cd web
npm install
npm run dev
```
后端默认地址为 `http://localhost:8767`，前端默认地址为 `http://localhost:5175`。详细配置与功能说明见 [web/README.md](web/README.md)。
## 验证与发布
```powershell
cd web
npm run build
cd ..
node scripts/verify/verify_tree.cjs
python deploy/build_release.py
```
发布产物生成到 `release/`，部署步骤见 [deploy/README.md](deploy/README.md)。
deploy/README.md 0 → 100644
View file @ 54429b71
# 发布与运维
本目录只保存部署相关文件，业务代码不应放入此处。
1. 在项目根目录执行 `python deploy/build_release.py`，构建产物生成到 `release/`。
2. 设置 `DBTOOL_PW` 后执行 `python deploy/deploy_server.py`，上传发布包并重启服务。
3. 使用 `python deploy/remote_shell.py "命令"` 执行临时远程检查；密码不会写入脚本或仓库。
`db-tool.service` 是 systemd 服务定义，`db-tool-nginx.conf` 是 Nginx 反向代理示例。
部署脚本默认使用现有服务器地址和 `root` 用户，也可通过 `DBTOOL_HOST`、`DBTOOL_USER` 覆盖；SSH 密码只从 `DBTOOL_PW` 读取。
如生产环境需要 Oracle Thick 模式，请在构建前从 Oracle 官方获取 Linux Instant Client，并解压到项目根目录的 `oracle_client/linux/`。该目录是本机构建依赖，不纳入版本控制。
deploy/build_release.py 0 → 100644
View file @ 54429b71
#!/usr/bin/env python3
"""构建 db-tools 正式发布包。
发布文件统一输出到项目根目录的 ``release/``，避免和源码、部署配置混放。
"""
from __future__ import annotations
import os
import subprocess
import tarfile
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
WEB = ROOT / "web"
RELEASE_DIR = ROOT / "release"
def _skip(path: Path) -> bool:
return (
"logs" in path.parts
or path.name in {"__pycache__", "database.test.yaml"}
or path.suffix in {".pyc", ".pyo"}
or ("oracle_client" in path.parts and path.suffix.lower() == ".zip")
)
def _add_tree(archive: tarfile.TarFile, root: Path, arcname: str) -> None:
for path in root.rglob("*"):
if _skip(path) or not path.is_file():
continue
archive.add(path, arcname=f"{arcname}/{path.relative_to(root).as_posix()}")
def main() -> None:
npm = "npm.cmd" if os.name == "nt" else "npm"
subprocess.run([npm, "run", "build"], cwd=WEB, check=True)
RELEASE_DIR.mkdir(exist_ok=True)
with tarfile.open(RELEASE_DIR / "backend-deploy.tgz", "w:gz") as archive:
_add_tree(archive, WEB / "backend", "backend")
# 部署目标是 Linux，只带原生 Linux Instant Client，避免把本地 Windows DLL 一并发布。
oracle_client = ROOT / "oracle_client" / "linux"
if oracle_client.is_dir():
_add_tree(archive, oracle_client, "oracle_client/linux")
with tarfile.open(RELEASE_DIR / "frontend-deploy.tgz", "w:gz") as archive:
_add_tree(archive, WEB / "dist", ".")
print(f"发布包已生成：{RELEASE_DIR}")
if __name__ == "__main__":
main()
db-tool.service → deploy/db-tool.service
View file @ 54429b71
......@@ -7,7 +7,7 @@ Type=simple
User=root
WorkingDirectory=/opt/app/db-tool
Environment=PYTHONPATH=/opt/app/db-tool
Environment=WEB_DB_PATH=/opt/app/db-tool/web/data/web.db
Environment=APP_ENV=production
Environment=WEB_HOST=127.0.0.1
Environment=WEB_PORT=8767
ExecStart=/opt/app/db-tool/.venv/bin/python -m web.backend.start
......
deploy_server.py → deploy/deploy_server.py
View file @ 54429b71
#!/usr/bin/env python3
"""db-tools 发版：上传 → 备份旧版 → 解压 → 重启 → 验证。
不动 /opt/app/db-tool/web/data/（web.db、backups、backup-config.json、sources）
后端发布包包含私有 configs/llm.yaml，部署后无需再配置服务器环境变量。"""
"""db-tools 正式发版：上传、迁移旧 SQLite、切换 MySQL、重启并验证。"""
from __future__ import annotations
import os
import sys
from pathlib import Path
import paramiko
HOST, USER = "192.168.20.175", "root"
HOST = os.environ.get("DBTOOL_HOST", "192.168.20.175")
USER = os.environ.get("DBTOOL_USER", "root")
ROOT = Path(__file__).resolve().parent.parent
RELEASE_DIR = ROOT / "release"
SERVICE_FILE = Path(__file__).with_name("db-tool.service")
pw = os.environ.get("DBTOOL_PW")
if not pw:
sys.exit("set DBTOOL_PW first")
......@@ -26,35 +31,44 @@ def run(cmd, must=True):
return o
sftp = c.open_sftp()
sftp.put("backend-deploy.tgz", "/tmp/dbtool-backend.tgz")
sftp.put("frontend-deploy.tgz", "/tmp/dbtool-frontend.tgz")
sftp.put(str(RELEASE_DIR / "backend-deploy.tgz"), "/tmp/dbtool-backend.tgz")
sftp.put(str(RELEASE_DIR / "frontend-deploy.tgz"), "/tmp/dbtool-frontend.tgz")
sftp.put(str(SERVICE_FILE), "/tmp/db-tool.service")
sftp.close()
print("uploaded both deployment packages\n")
# 0) 部署前快照（证明 data/ 没被碰）
run("md5sum /opt/app/db-tool/web/data/web.db; ls /opt/app/db-tool/web/data/backups | wc -l")
# 0) 部署前快照（旧 SQLite 只作为首次 MySQL 迁移源）
run("test -f /opt/app/db-tool/web/data/web.db && md5sum /opt/app/db-tool/web/data/web.db || true")
# 1) 备份当前版本
run("TS=$(date +%Y%m%d-%H%M%S) && mkdir -p /opt/app/db-tool/releases/$TS && "
"cp -a /opt/app/db-tool/web/backend /opt/app/db-tool/releases/$TS/backend && "
"cp -a /opt/web/db-tool /opt/app/db-tool/releases/$TS/frontend && echo backup at $TS")
# 2) 解压后端（不动 data/、logs/），并锁定包内私有 LLM 配置权限
# 2) 停止旧服务并备份其 SQLite，保证迁移读取一致快照
run("systemctl stop db-tool && TS=$(date +%Y%m%d-%H%M%S) && "
"mkdir -p /opt/app/db-tool/releases/$TS && "
"cp -a /opt/app/db-tool/web/data/web.db /opt/app/db-tool/releases/$TS/web.db")
# 3) 解压后端（不动 data/、logs/），并锁定私有配置权限
run("tar xzf /tmp/dbtool-backend.tgz -C /opt/app/db-tool/web/ && "
"find /opt/app/db-tool/web/backend -name __pycache__ -type d -exec rm -rf {} + ; "
"test -f /opt/app/db-tool/web/backend/configs/llm.yaml && "
"chmod 600 /opt/app/db-tool/web/backend/configs/llm.yaml && "
"sed -E -e 's/^(api_key:[[:space:]]*).*/\\1[REDACTED]/' "
"/opt/app/db-tool/web/backend/configs/llm.yaml")
"chmod 600 /opt/app/db-tool/web/backend/configs/llm.yaml /opt/app/db-tool/web/backend/configs/database.production.yaml")
# 4) 首次发布时迁移旧 SQLite 到正式 MySQL；已有 MySQL 数据时不覆盖
run("cd /opt/app/db-tool && APP_ENV=production PYTHONPATH=/opt/app/db-tool "
"/opt/app/db-tool/.venv/bin/python -m web.backend.db.migrate_sqlite_to_mysql "
"--source /opt/app/db-tool/web/data/web.db --skip-if-populated")
# 3) 前端：清旧 assets 再解压（避免旧 hash 文件堆积）
# 5) 安装正式 systemd 配置并解压前端
run("install -m 644 /tmp/db-tool.service /etc/systemd/system/db-tool.service && systemctl daemon-reload")
run("rm -rf /opt/web/db-tool/assets && tar xzf /tmp/dbtool-frontend.tgz -C /opt/web/db-tool/ && ls /opt/web/db-tool/ /opt/web/db-tool/assets/")
# 4) 重启
# 6) 重启
run("systemctl restart db-tool && sleep 4 && systemctl is-active db-tool && ss -tlnp | grep 8767")
# 5) 部署后核对 data/ 未变 + 日志干净
run("md5sum /opt/app/db-tool/web/data/web.db; ls /opt/app/db-tool/web/data/backups | wc -l; journalctl -u db-tool -n 5 --no-pager | tail -5")
# 7) 部署后核对
run("curl -fsS http://127.0.0.1:8767/api/health && journalctl -u db-tool -n 12 --no-pager | tail -12")
c.close()
print("DEPLOY DONE")
rssh.py → deploy/remote_shell.py
View file @ 54429b71
#!/usr/bin/env python3
"""db-tools 服务器远程命令助手 — python rssh.py "命令" (密码不落盘, 从环境变量 DBTOOL_PW 读)"""
import sys, os, paramiko
"""服务器远程命令助手；密码从 ``DBTOOL_PW`` 环境变量读取。"""
from __future__ import annotations
HOST, USER = "192.168.20.175", "root"
pw = os.environ.get("DBTOOL_PW")
if not pw:
import os
import sys
import paramiko
HOST = os.environ.get("DBTOOL_HOST", "192.168.20.175")
USER = os.environ.get("DBTOOL_USER", "root")
password = os.environ.get("DBTOOL_PW")
if not password:
sys.exit("set DBTOOL_PW first")
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
c.connect(HOST, username=USER, password=pw, timeout=10, look_for_keys=False, allow_agent=False)
c.connect(HOST, username=USER, password=password, timeout=10, look_for_keys=False, allow_agent=False)
cmd = sys.argv[1] if len(sys.argv) > 1 else "echo no-cmd"
_, out, err = c.exec_command(cmd, timeout=560)
sys.stdout.write(out.read().decode("utf-8", "replace"))
......
package-lock.json deleted 100644 → 0
View file @ bd79c9e5
This diff is collapsed.
package.json deleted 100644 → 0
View file @ bd79c9e5
{
"devDependencies": {
"@vue/compiler-dom": "^3.5.41",
"jsdom": "^30.0.1"
}
}
scripts/README.md 0 → 100644
View file @ 54429b71
# 脚本目录
脚本按用途分类，避免开发、验证和运维文件混在同一层。
- `verify/`：验证当前 Vue 页面使用的树形勾选逻辑。请从项目根目录执行 `node scripts/verify/verify_tree.cjs`。
- 旧版静态页面验证脚本已删除；它们依赖不存在的 `web/static` 构建产物，不能代表当前 Vite 应用。
- `maintenance/`：本地维护工具。`kill_web.bat` 与 `kill_web.sh` 用于释放开发端口；`reformat_reasons.py` 用于整理历史原因文本。
- `diagnostics/`：环境或数据库探针，不属于日常业务流程。
scripts/_probe_dm.py → scripts/diagnostics/probe_dm.py
View file @ 54429b71
......@@ -3,9 +3,9 @@
"""达梦连接独立诊断脚本
用法：
python scripts/_probe_dm.py
python scripts/diagnostics/probe_dm.py
# 或带参：
python scripts/_probe_dm.py --host 192.168.20.24 --port 5236 \
python scripts/diagnostics/probe_dm.py --host 192.168.20.24 --port 5236 \
--user HGWXZJ --password *** --schema HGWXZJ
为什么需要它：
......
scripts/kill_web.bat → scripts/maintenance/kill_web.bat
View file @ 54429b71
......@@ -3,8 +3,8 @@ REM ── 杀掉 web 服务（端口 8765）──
REM 适用：Windows 原生 cmd / PowerShell
REM
REM 用法：
REM scripts\kill_web.bat REM 默认杀 8765
REM scripts\kill_web.bat 9000 REM 自定义端口
REM scripts\maintenance\kill_web.bat REM 默认杀 8765
REM scripts\maintenance\kill_web.bat 9000 REM 自定义端口
REM
REM 退出码（通过 errorlevel）：
REM 0 杀掉占用进程成功 / 端口空闲
......
scripts/kill_web.sh → scripts/maintenance/kill_web.sh
View file @ 54429b71
......@@ -4,8 +4,8 @@
# Linux/macOS（lsof + kill）—— 已自动分支
#
# 用法：
# bash scripts/kill_web.sh # 默认杀 8765
# bash scripts/kill_web.sh 9000 # 自定义端口
# bash scripts/maintenance/kill_web.sh # 默认杀 8765
# bash scripts/maintenance/kill_web.sh 9000 # 自定义端口
#
# 退出码：
# 0 端口空闲 / 已杀掉占用进程
......
scripts/reformat_reasons.py → scripts/maintenance/reformat_reasons.py
View file @ 54429b71
......@@ -12,7 +12,7 @@
本脚本幂等：再次运行不会重复插入。
用法：
python scripts/reformat_reasons.py
python scripts/maintenance/reformat_reasons.py
"""
from __future__ import annotations
......@@ -142,7 +142,8 @@ def transform_source(source: str) -> tuple[str, int]:
def main():
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
project_root = Path(__file__).resolve().parent.parent
# 当前脚本位于 scripts/maintenance/，向上三级才是项目根目录。
project_root = Path(__file__).resolve().parent.parent.parent
all_files = IND_FILES + STEP_IMPL_FILES
grand_total = 0
print(f"{'File':<60} {'Insertions':>10}")
......
scripts/verify_caret_span.cjs deleted 100644 → 0
View file @ bd79c9e5
// 验证 caret 改成 span+unicode 后渲染正常
const fs = require('fs');
const { JSDOM } = require('jsdom');
const dom = new JSDOM('<!DOCTYPE html>', { runScripts: 'dangerously', url: 'http://localhost:8765/' });
const win = dom.window;
const vue = fs.readFileSync('web/static/lib/vue.global.prod.js', 'utf8');
win.eval(vue);
const Vue = win.Vue;
const tpl = `
<div class="checks-tree-table">
<div v-for="row in rows" :key="row.key" class="tree-row" :class="row.is_group ? 'tree-row--group' : 'tree-row--leaf'">
<template v-if="row.is_group">
<span
class="tree-caret"
role="button"
tabindex="0"
:aria-expanded="row._expanded"
@click="onClick(row)"
@keydown.enter.prevent="onClick(row)"
>{{ row._expanded ? '▼' : '▶' }}</span>
<el-checkbox :model-value="row._checked" :indeterminate="row._indeterminate" @change="onGroupCheck(row, $event)">
<span class="group-title">{{ row.title }}</span>
<span class="group-count">({{ row._checked_count }}/{{ row._total_count }})</span>
<span v-if="row.description" class="group-description" :title="row.description">{{ row.description }}</span>
</el-checkbox>
</template>
<template v-else>
<span class="tree-caret-spacer" />
<el-checkbox :model-value="row.s" :disabled="row.r" @change="onLeafCheck(row, $event)">
<span>{{ row.title }}</span>
</el-checkbox>
</template>
</div>
</div>
`;
const rows = [
{key:'g1', is_group:true, _expanded:true, _checked:true, _indeterminate:false, _checked_count:2, _total_count:3, title:'基础检查', description:'基础数据质量检查'},
{key:'l1', is_group:false, s:true, r:false, title:'合并表'},
{key:'g2', is_group:true, _expanded:false, _checked:false, _indeterminate:false, _checked_count:0, _total_count:2, title:'国标', description:'国标校验'},
];
const ElCheckbox = {
props: ['modelValue', 'indeterminate', 'disabled'],
emits: ['change', 'update:modelValue'],
template: '<label class="el-checkbox"><input type="checkbox" /><span><slot /></span></label>',
};
const App = {
data() { return { rows }; },
methods: { onClick:()=>{}, onGroupCheck:()=>{}, onLeafCheck:()=>{} },
template: tpl,
};
const vueApp = Vue.createApp(App);
vueApp.component('el-checkbox', ElCheckbox);
const container = win.document.createElement('div');
win.document.body.appendChild(container);
vueApp.mount(container);
console.log('=== DOM ===');
console.log(container.innerHTML);
const carets = container.querySelectorAll('.tree-caret');
console.log(`\ncaret 数: ${carets.length}`);
carets.forEach((c, i) => console.log(` [${i}] text="${c.textContent}" role="${c.getAttribute('role')}" aria-expanded=${c.getAttribute('aria-expanded')}`));
\ No newline at end of file
scripts/verify_detail_passthrough.cjs deleted 100644 → 0
View file @ bd79c9e5
// 简化：直接传对象给 Vue data，Vue 会把它包成 reactive proxy
const fs = require('fs');
const { JSDOM } = require('jsdom');
const dom = new JSDOM('<!DOCTYPE html>', {
runScripts: 'dangerously',
resources: 'usable',
url: 'http://localhost:8765/',
});
const win = dom.window;
const css = fs.readFileSync('web/static/lib/element-plus/index.css', 'utf8');
const style = win.document.createElement('style');
style.textContent = css;
win.document.head.appendChild(style);
const vue = fs.readFileSync('web/static/lib/vue.global.prod.js', 'utf8');
win.eval(vue);
const ep = fs.readFileSync('web/static/lib/element-plus/index.full.min.js', 'utf8');
win.eval(ep);
function waitFor(pred, name, tries = 200) {
return new Promise((resolve, reject) => {
let n = 0;
const t = () => {
if (pred()) return resolve();
if (++n > tries) return reject(new Error(name + ' 未加载'));
setTimeout(t, 50);
};
t();
});
}
(async () => {
await waitFor(() => win.Vue && win.ElementPlus, 'Vue + ElementPlus');
const Vue = win.Vue;
const ElementPlus = win.ElementPlus;
const analysisTree = {
groups: [
{
key: 'g1', title: '基础检查', default_expand: true, description: 'desc1',
checks: [
{ step_id: 'merge_redundancy', title: '表合并与冗余字段分析',
llm_mode: 'required', required: false,
detail: { purpose: '找出可合并表', target: '全部表', format: 'LLM' } },
{ step_id: 'empty_fields', title: '大范围空字段扫描',
llm_mode: 'none', required: false,
detail: { purpose: '扫描空字段', target: '全部字段', format: '阈值 80%' } },
],
},
],
};
const app = Vue.createApp({
data() { return { analysisTree, form: { steps: [] }, expandedGroups: ['g1'] }; },
computed: {
flatCheckRows() {
const out = [];
for (const g of this.analysisTree.groups || []) {
out.push({ key: g.key, _level: 0, is_group: true, title: g.title });
if (this.expandedGroups.includes(g.key)) {
for (const c of (g.checks || [])) {
out.push({
key: c.step_id, _level: 1, is_group: false,
step_id: c.step_id, title: c.title,
llm_mode: c.llm_mode, required: c.required,
detail: c.detail || null,
});
}
}
}
return out;
},
},
methods: { toggleStep() {} },
template: `
<div class="checks-tree-table">
<div v-for="row in flatCheckRows" :key="row.key" class="tree-row"
:class="row.is_group ? 'tree-row--group' : 'tree-row--leaf'">
<template v-if="!row.is_group">
<span class="tree-caret-spacer"></span>
<el-checkbox :model-value="false">
<span>{{ row.title }}</span>
</el-checkbox>
<el-tooltip v-if="row.detail" placement="top" effect="dark">
<template #content>
<div class="step-detail">
<div v-if="row.detail.purpose" class="step-detail-row">
<span class="step-detail-label">作用</span>
<span>{{ row.detail.purpose }}</span>
</div>
<div v-if="row.detail.target" class="step-detail-row">
<span class="step-detail-label">查什么</span>
<span>{{ row.detail.target }}</span>
</div>
<div v-if="row.detail.format" class="step-detail-row">
<span class="step-detail-label">格式</span>
<span>{{ row.detail.format }}</span>
</div>
</div>
</template>
<span class="info-icon" tabindex="0" role="button">i</span>
</el-tooltip>
</template>
</div>
</div>
`,
});
app.use(ElementPlus);
const container = win.document.createElement('div');
win.document.body.appendChild(container);
app.mount(container);
await new Promise(r => setTimeout(r, 200));
const infoIcons = container.querySelectorAll('.info-icon');
const tooltips = container.querySelectorAll('.el-tooltip');
const detailRows = container.querySelectorAll('.step-detail-row');
const labels = container.querySelectorAll('.el-checkbox__label');
console.log('=== 验证结果 ===');
console.log('info-icon 数:', infoIcons.length, '(预期 2)');
console.log('el-tooltip 数:', tooltips.length);
console.log('step-detail-row 数:', detailRows.length, '(预期 6)');
console.log('el-checkbox__label 数:', labels.length, '(预期 2)');
if (labels.length) {
console.log(' label[0]:', JSON.stringify(labels[0].textContent.trim()));
}
})();
\ No newline at end of file
scripts/verify_e2e.cjs deleted 100644 → 0
View file @ bd79c9e5
This diff is collapsed.
scripts/verify_info_icon.cjs deleted 100644 → 0
View file @ bd79c9e5
// 验证 info icon + tooltip 渲染
const fs = require('fs');
const { JSDOM } = require('jsdom');
const dom = new JSDOM('<!DOCTYPE html>', { runScripts: 'dangerously', url: 'http://localhost:8765/' });
const win = dom.window;
const vue = fs.readFileSync('web/static/lib/vue.global.prod.js', 'utf8');
win.eval(vue);
const Vue = win.Vue;
// 拉一次真服务器的 HTML 拿实际模板
const { execSync } = require('child_process');
const html = execSync('curl -s http://localhost:8765/static/index.html', { encoding: 'utf8' });
// 抽出 leaf 模板段
const start = html.indexOf('<span class="tree-caret-spacer"');
const end = html.indexOf('</template>\n </div>', start);
const tpl = html.substring(start, end + '</template>'.length);
console.log('=== 截取到的 leaf 模板 (前 500 字符) ===');
console.log(tpl.substring(0, 500));
console.log('\n=== 含 info-icon 关键字 ===', tpl.includes('info-icon') ? 'YES' : 'NO');
console.log('=== 含 step-detail 关键字 ===', tpl.includes('step-detail') ? 'YES' : 'NO');
console.log('=== 含 row.detail ===', tpl.includes('row.detail') ? 'YES' : 'NO');
console.log('=== 含 el-tooltip ===', (tpl.match(/el-tooltip/g) || []).length, '处');
const ElCheckbox = {
props: ['modelValue', 'indeterminate', 'disabled'],
emits: ['change', 'update:modelValue'],
template: '<label class="el-checkbox"><input type="checkbox" /><span><slot /></span></label>',
};
const ElTooltip = {
props: ['placement', 'effect'],
template: '<span class="el-tooltip"><slot /><div class="el-tooltip__popper"><slot name="content" /></div></span>',
};
const row = {
key: 'std_ind_001_a',
step_id: 'std_ind_001_a',
title: 'IND-001-a · GB 11643-1999 身份证号格式',
llm_mode: 'none',
required: false,
detail: {
purpose: '长度 = 18；6 位地址 + 8 位出生日期 + 3 位顺序码 + 1 位校验位',
target: '字段名含 id_card / id_card_no / id_number / identity_card',
format: 'GB 11643-1999',
},
};
const App = {
data() { return { row, form: { steps: ['std_ind_001_a'] } }; },
methods: { toggleStep:()=>{} },
template: '<div>' + tpl + '</div>',
};
const app = Vue.createApp(App);
app.component('el-checkbox', ElCheckbox);
app.component('el-tooltip', ElTooltip);
const container = win.document.createElement('div');
win.document.body.appendChild(container);
app.mount(container);
const infoIcon = container.querySelector('.info-icon');
const tooltipRows = container.querySelectorAll('.step-detail-row');
const tooltipLabels = container.querySelectorAll('.step-detail-label');
console.log('\n=== 渲染结果 ===');
console.log('info-icon 数:', infoIcon ? 1 : 0);
console.log('step-detail-row 数:', tooltipRows.length);
tooltipLabels.forEach((l, i) => console.log(` [${i}] label: "${l.textContent.trim()}"`));
if (tooltipRows.length) {
console.log('\n=== tooltip 第一行内容 ===');
console.log(tooltipRows[0].textContent.trim());
console.log('\n=== tooltip 第二行内容 ===');
console.log(tooltipRows[1].textContent.trim());
console.log('\n=== tooltip 第三行内容 ===');
console.log(tooltipRows[2].textContent.trim());
}
\ No newline at end of file
scripts/verify_leaf_real.cjs deleted 100644 → 0
View file @ bd79c9e5
// 用 prod Vue 渲染真实 leaf 模板（不加任何 mock el-checkbox）
// 模拟 Element Plus 的 el-checkbox 真实 DOM 结构
const fs = require('fs');
const { JSDOM } = require('jsdom');
const dom = new JSDOM('<!DOCTYPE html>', { runScripts: 'dangerously', url: 'http://localhost:8765/' });
const win = dom.window;
const vue = fs.readFileSync('web/static/lib/vue.global.prod.js', 'utf8');
win.eval(vue);
const Vue = win.Vue;
// 拉真服务器 HTML 取模板
const { execSync } = require('child_process');
const html = execSync('curl -s http://localhost:8765/static/index.html', { encoding: 'utf8' });
const start = html.indexOf('<span class="tree-caret-spacer"');
const end = html.indexOf('</template>\n </div>', start);
const tpl = html.substring(start, end + '</template>'.length);
const row = {
key: 'std_ind_001_a',
step_id: 'std_ind_001_a',
title: 'IND-001-a · GB 11643-1999 身份证号格式',
llm_mode: 'none',
required: false,
detail: {
purpose: '长度 = 18',
target: '字段名含 id_card',
format: 'GB 11643-1999',
},
};
// 模拟 Element Plus el-checkbox 真实 DOM 结构
const ElCheckbox = {
name: 'ElCheckbox',
props: ['modelValue', 'indeterminate', 'disabled', 'label'],
emits: ['change', 'update:modelValue'],
template: `
<label class="el-checkbox" :class="{ 'is-disabled': disabled, 'is-checked': modelValue, 'is-indeterminate': indeterminate }">
<span class="el-checkbox__input">
<input type="checkbox" :disabled="disabled" :checked="modelValue" class="el-checkbox__original" />
<span class="el-checkbox__inner"></span>
</span>
<span class="el-checkbox__label">
<slot />
</span>
</label>
`,
};
const ElTooltip = {
name: 'ElTooltip',
props: ['placement', 'effect', 'content'],
template: '<span class="el-tooltip"><slot /><slot name="content" /></span>',
};
const ElTag = {
name: 'ElTag',
props: ['size', 'type', 'effect'],
template: '<span class="el-tag" :class="type ? `el-tag--${type}` : \'\'"><slot /></span>',
};
const App = {
data() { return { row, form: { steps: ['std_ind_001_a'] } }; },
methods: { toggleStep:()=>{} },
template: '<div>' + tpl + '</div>',
};
const app = Vue.createApp(App);
app.component('el-checkbox', ElCheckbox);
app.component('el-tooltip', ElTooltip);
app.component('el-tag', ElTag);
const container = win.document.createElement('div');
win.document.body.appendChild(container);
try {
app.mount(container);
console.log('=== 渲染结果 ===');
console.log(container.innerHTML);
console.log('\n=== 关键元素 ===');
console.log('el-checkbox 数:', container.querySelectorAll('.el-checkbox').length);
console.log('el-checkbox__label 数:', container.querySelectorAll('.el-checkbox__label').length);
const label = container.querySelector('.el-checkbox__label');
if (label) console.log('label textContent:', JSON.stringify(label.textContent.trim()));
console.log('info-icon 数:', container.querySelectorAll('.info-icon').length);
} catch (e) {
console.error('mount ERROR:', e.message);
console.error(e.stack);
}
\ No newline at end of file
scripts/verify_leaf_real_elplus.cjs deleted 100644 → 0
View file @ bd79c9e5
// 用真 Element Plus 跑完整模板，看 el-checkbox 实际怎么渲染
const fs = require('fs');
const { JSDOM } = require('jsdom');
const dom = new JSDOM('<!DOCTYPE html>', {
runScripts: 'dangerously',
resources: 'usable',
pretendToBeVisual: true,
url: 'http://localhost:8765/',
});
const win = dom.window;
// 注入 Element Plus CSS（jsdom 模拟）
const css = fs.readFileSync('web/static/lib/element-plus/index.css', 'utf8');
const style = win.document.createElement('style');
style.textContent = css;
win.document.head.appendChild(style);
// 加载 Vue + Element Plus
const vue = fs.readFileSync('web/static/lib/vue.global.prod.js', 'utf8');
win.eval(vue);
const ep = fs.readFileSync('web/static/lib/element-plus/index.full.min.js', 'utf8');
win.eval(ep);
function waitFor(predicate, name, maxTries = 200) {
return new Promise((resolve, reject) => {
let tries = 0;
const tick = () => {
if (predicate()) return resolve();
if (++tries > maxTries) return reject(new Error(`${name} 未加载`));
setTimeout(tick, 50);
};
tick();
});
}
(async () => {
try {
await waitFor(() => win.Vue && win.ElementPlus, 'Vue + ElementPlus');
console.log('Vue + ElementPlus loaded');
} catch (e) {
console.error(e.message);
process.exit(1);
}
const Vue = win.Vue;
const ElementPlus = win.ElementPlus;
// 拉真服务器 HTML 取模板
const { execSync } = require('child_process');
const html = execSync('curl -s http://localhost:8765/static/index.html', { encoding: 'utf8' });
const start = html.indexOf('<span class="tree-caret-spacer"');
const end = html.indexOf('</template>\n </div>', start);
const tpl = html.substring(start, end + '</template>'.length);
const row = {
key: 'std_ind_001_a',
step_id: 'std_ind_001_a',
title: 'IND-001-a · GB 11643-1999 身份证号格式',
llm_mode: 'none',
required: false,
detail: {
purpose: '长度 = 18',
target: '字段名含 id_card',
format: 'GB 11643-1999',
},
};
const App = {
data() { return { row, form: { steps: ['std_ind_001_a'] } }; },
methods: { toggleStep:()=>{} },
template: '<div>' + tpl + '</div>',
};
const app = Vue.createApp(App);
app.use(ElementPlus);
const container = win.document.createElement('div');
win.document.body.appendChild(container);
try {
app.mount(container);
// 等一帧让 popper 之类 ready
await new Promise(r => setTimeout(r, 300));
console.log('=== 渲染结果 ===');
console.log(container.innerHTML.substring(0, 2000));
console.log('\n=== 关键元素 ===');
console.log('el-checkbox 数:', container.querySelectorAll('.el-checkbox').length);
console.log('el-checkbox__label 数:', container.querySelectorAll('.el-checkbox__label').length);
const labels = container.querySelectorAll('.el-checkbox__label');
labels.forEach((l, i) => {
console.log(` label[${i}] textContent: ${JSON.stringify(l.textContent.trim())}`);
});
console.log('info-icon 数:', container.querySelectorAll('.info-icon').length);
} catch (e) {
console.error('mount ERROR:', e.message);
console.error(e.stack);
}
})();
\ No newline at end of file
scripts/verify_prod_tree.cjs deleted 100644 → 0
View file @ bd79c9e5
// 用真实 Vue 3 prod build 编译 template，把 render 函数喂给 Function 看是否合法
// 用 el-checkbox 而不是 span，看 slot 内容能否正常 render
const fs = require('fs');
const { JSDOM } = require('jsdom');
const dom = new JSDOM('<!DOCTYPE html>', { runScripts: 'dangerously', url: 'http://localhost:8765/' });
const win = dom.window;
// 先加载 Vue
const vue = fs.readFileSync('web/static/lib/vue.global.prod.js', 'utf8');
win.eval(vue);
const Vue = win.Vue;
// 用 el-checkbox 组件名（Element Plus 提供的）跑完整 template
const tpl = `
<div class="checks-tree-table">
<div v-for="row in rows" :key="row.key" class="tree-row" :class="row.is_group ? 'tree-row--group' : 'tree-row--leaf'" :style="{ paddingLeft: row._level * 24 + 'px' }">
<template v-if="row.is_group">
<button type="button" class="tree-caret" :aria-expanded="row._expanded" @click="onClick(row)">
<span v-if="row._expanded">▼</span>
<span v-else>▶</span>
</button>
<el-checkbox :model-value="row._checked" :indeterminate="row._indeterminate" @change="onGroupCheck(row, $event)">
<span class="group-title">{{ row.title }}</span>
<span class="group-count">({{ row._checked_count }}/{{ row._total_count }})</span>
<span v-if="row.description" class="group-description" :title="row.description">{{ row.description }}</span>
</el-checkbox>
</template>
<template v-else>
<span class="tree-caret-spacer" />
<el-checkbox :model-value="row.s" :disabled="row.r" @change="onLeafCheck(row, $event)">
<span>{{ row.title }}</span>
<span v-if="row.llm_mode === 'required'" class="badge">需 LLM</span>
</el-checkbox>
</template>
</div>
</div>
`;
const rows = [
{key:'g1', is_group:true, _level:0, _expanded:true, _checked:true, _indeterminate:false, _checked_count:2, _total_count:3, title:'基础检查', description:'基础数据质量检查'},
{key:'l1', is_group:false, _level:1, s:true, r:false, title:'合并表', llm_mode:'none'},
{key:'g2', is_group:true, _level:0, _expanded:false, _checked:false, _indeterminate:false, _checked_count:0, _total_count:2, title:'国标', description:'国标校验'},
];
// 注册一个 mock el-checkbox 组件（只关心 children 是否 render）
const ElCheckbox = {
props: ['modelValue', 'indeterminate', 'disabled'],
emits: ['change', 'update:modelValue'],
template: '<label class="el-checkbox"><input type="checkbox" /><span><slot /></span></label>',
};
const App = {
data() { return { rows }; },
methods: {
onClick: () => console.log('caret click'),
onGroupCheck: (row, v) => console.log('group check', row.key, v),
onLeafCheck: (row, v) => console.log('leaf check', row.key, v),
},
template: tpl,
};
const vueApp = Vue.createApp(App);
vueApp.component('el-checkbox', ElCheckbox);
vueApp.component('el-icon', { template: '<i><slot /></i>' });
// 劫持 Function 抓 render
const captured = [];
const origFunction = win.Function;
win.Function = function(...args) {
const body = args[args.length - 1];
if (typeof body === 'string' && body.length > 2000) {
captured.push({ body, args: args.slice(0, -1) });
}
return new origFunction(...args);
};
const container = win.document.createElement('div');
win.document.body.appendChild(container);
try {
vueApp.mount(container);
console.log('=== DOM ===');
console.log(container.innerHTML);
console.log('\n=== render fn 数 ===', captured.length);
if (captured[0]) {
console.log('=== render fn #0 中是否含 el-checkbox / group-title ===');
const c = captured[0].body;
console.log(' el-checkbox:', (c.match(/el-checkbox/g) || []).length);
console.log(' group-title:', (c.match(/group-title/g) || []).length);
console.log(' group-description:', (c.match(/group-description/g) || []).length);
console.log(' tree-caret:', (c.match(/tree-caret/g) || []).length);
console.log(' onGroupCheck:', (c.match(/onGroupCheck/g) || []).length);
console.log(' createBlock:', (c.match(/createBlock/g) || []).length);
// 把产物写出来供查
fs.writeFileSync('scripts/compiled_tree.js', c);
console.log(' → scripts/compiled_tree.js');
}
} catch (e) {
console.error('mount err:', e.message);
console.error(e.stack);
}
\ No newline at end of file
scripts/verify_protocol.cjs deleted 100644 → 0
View file @ bd79c9e5
// scripts/verify_protocol.cjs
// 验证前端关键逻辑（路径求值 + 渲染入口）。
// 策略：从 index.html 抽取 evalPathExpr / resolvePath / resolveRows / renderSummary 四个核心函数，
// 注入到 Node 沙箱里独立测试。
const fs = require('fs');
const path = require('path');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'static', 'index.html'), 'utf8');
const scriptMatch = html.match(/<script>\s*\n([\s\S]*?)<\/script>/);
const scriptBody = scriptMatch[1];
// 抽出 4 个核心函数（用正则找完整函数体：含参数列表 + 函数体）
function extract(name) {
// 匹配 `function NAME(args) {` —— 不含函数名前缀
const re = new RegExp(`function ${name}\\s*\\(([^)]*)\\)\\s*\\{`);
const m = scriptBody.match(re);
if (!m) throw new Error(`${name} not found`);
const headerEnd = m.index + m[0].length; // 紧接 `{` 之后
let depth = 1, i = headerEnd;
for (; i < scriptBody.length; i++) {
const c = scriptBody[i];
if (c === '{') depth++;
else if (c === '}') {
depth--;
if (depth === 0) {
// 返回：完整函数定义（含参数 + 函数体）
return scriptBody.slice(m.index, i + 1);
}
}
}
throw new Error(`${name} unterminated`);
}
const functions = ['evalPathExpr', 'resolvePath', 'resolveRows', 'renderSummary'];
const fns = {};
for (const name of functions) {
fns[name] = extract(name);
console.log(` extracted ${name} (${fns[name].length} bytes)`);
}
// 在 vm 里执行：把 4 个函数暴露到 global，调用它们
const vm = require('vm');
const ctx = {
console,
selectedField: { value: null },
};
vm.createContext(ctx);
try {
// 拼装一个 IIFE 把 4 个函数挂到 global
const code = `
${fns.evalPathExpr}
${fns.resolvePath}
${fns.resolveRows}
${fns.renderSummary}
globalThis.__exports = { evalPathExpr, resolvePath, resolveRows, renderSummary };
`;
vm.runInContext(code, ctx, { filename: 'extracted.js' });
} catch (e) {
console.error('Extract exec failed:', e.message);
process.exit(1);
}
const { evalPathExpr, resolvePath, resolveRows, renderSummary } = ctx.__exports;
console.log('OK: 4 functions loaded');
// ── 用例 ──
let pass = 0, fail = 0;
function eq(actual, expected, label) {
const a = JSON.stringify(actual), b = JSON.stringify(expected);
if (a === b) { pass++; console.log(` ✓ ${label}`); }
else { fail++; console.error(` ✗ ${label}: got ${a}, want ${b}`); }
}
// 1. evalPathExpr 简单路径
const data = { merge_candidates: [{id:'M-1'}], summary: { high: 3 } };
eq(evalPathExpr('this.summary.high', data), 3, 'evalPathExpr: this.summary.high');
eq(evalPathExpr('this.merge_candidates', data)[0].id, 'M-1', 'evalPathExpr: array index');
eq(evalPathExpr('len(this.merge_candidates)', data), 1, 'evalPathExpr: len()');
eq(evalPathExpr('len(this.summary.high)', data), 3, 'evalPathExpr: len(number)');
eq(evalPathExpr('this.nonexistent', data), '', 'evalPathExpr: missing key -> ""');
// 2. resolveRows
const rows = resolveRows('this.merge_candidates', data, null, 0);
eq(rows.length, 1, 'resolveRows: simple path');
// 3. resolveRows with linkage filter
const dataWithLinkage = {
violations: [
{ table_name: 't_user', column_name: 'id_card' },
{ table_name: 't_user', column_name: 'mobile' },
{ table_name: 't_other', column_name: 'id_card' },
],
};
const linkage = {
source_table: 0,
target_table: 0,
field_map: { table: 'table_name', field: 'column_name' },
default_filter_kind: 'limit',
default_filter_value: 200,
};
// 未选中：截 200 → 全返（这里只有 3 条）
ctx.selectedField.value = null;
const allRows = resolveRows('this.violations', dataWithLinkage, linkage, 0);
eq(allRows.length, 3, 'resolveRows: no selectedField -> slice(0, 200)');
// 选中：只过滤 t_user.id_card
ctx.selectedField.value = { table: 't_user', field: 'id_card' };
const filtered = resolveRows('this.violations', dataWithLinkage, linkage, 0);
eq(filtered.length, 1, 'resolveRows: selectedField -> 1 match');
eq(filtered[0].table_name, 't_user', 'resolveRows: filter by table_name');
ctx.selectedField.value = null;
// 4. renderSummary 模板替换
const summary = {
type: 'warning',
template: '高空 {{high}} 中空 {{mid}}',
vars: { high: 'this.summary.high', mid: 'this.summary.mid' },
};
const text = renderSummary(summary, { summary: { high: 3, mid: 7 } });
eq(text, '高空 3 中空 7', 'renderSummary: number interpolation');
// 千分位
const text2 = renderSummary(
{ type: 'info', template: '{{n}}', vars: { n: 'this.summary.count' } },
{ summary: { count: 1234 } }
);
eq(text2, '1,234', 'renderSummary: thousand_sep');
// 5. resolvePath
const rp = resolvePath('this.summary.high', data);
eq(rp, 3, 'resolvePath: this.summary.high');
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail === 0 ? 0 : 1);
\ No newline at end of file
scripts/verify_render.cjs deleted 100644 → 0
View file @ bd79c9e5
// scripts/verify_render.cjs
// 验证 resolveRenderPath 对所有 render.kind 路径表达式都能正确求值。
// （template 里把 ColumnRender 拆掉后，渲染逻辑全部内联为 v-else-if 分支，
// 共享的只是 resolveRenderPath 这个取值小函数。）
const fs = require('fs');
const path = require('path');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'static', 'index.html'), 'utf8');
const scriptBody = html.match(/<script>\s*\n([\s\S]*?)<\/script>/)[1];
function extract(name) {
const rePrefix = new RegExp(`function\\s+${name}\\s*\\(\\s*([^)]*?)\\s*\\)\\s*\\{`);
const isFnDecl = !!scriptBody.match(rePrefix);
const re = new RegExp(`(?:function\\s+)?${name}\\s*\\(\\s*([^)]*?)\\s*\\)\\s*\\{`);
const m = scriptBody.match(re);
if (!m) throw new Error(`${name} not found`);
const headerEnd = m.index + m[0].length;
let depth = 1, i = headerEnd;
for (; i < scriptBody.length; i++) {
if (scriptBody[i] === '{') depth++;
else if (scriptBody[i] === '}') { depth--; if (depth === 0) break; }
}
const body = scriptBody.slice(m.index, i + 1);
return isFnDecl ? body : 'function ' + body;
}
const fnBody = extract('resolveRenderPath');
console.log(`resolveRenderPath body: ${fnBody.length} bytes`);
const vm = require('vm');
const ctx = { console };
vm.createContext(ctx);
try {
vm.runInContext(`${fnBody}; globalThis.__rrp = resolveRenderPath;`, ctx);
} catch (e) {
console.error('Extract failed:', e.message);
process.exit(1);
}
const rrp = ctx.__rrp;
let pass = 0, fail = 0;
function check(label, actual, expected) {
const a = JSON.stringify(actual), b = JSON.stringify(expected);
if (a === b) { console.log(` ✓ ${label}`); pass++; }
else { console.error(` ✗ ${label}: got ${a}, want ${b}`); fail++; }
}
// 模拟 row：standards tab 的 violations_by_standard 行
const rowStd = {
standard: 'GB_11643',
rule: '18 位',
basis: '正则',
level: 'high',
};
// empty_fields perTableRow
const rowEmpty = {
table_name: 't_user',
table_comment: '用户表',
high_empty_fields_count: 5,
empty_rate: 0.85,
level: 'high',
};
// merge_candidates 行
const rowMerge = {
priority: 'P1',
classification: 'high',
confidence: 0.95,
confidence_label: '高',
};
console.log('\n--- resolveRenderPath: simple paths ---');
check('this.standard', rrp(rowStd, 'standard'), 'GB_11643');
check('this.rule', rrp(rowStd, 'rule'), '18 位');
check('this.basis', rrp(rowStd, 'basis'), '正则');
check('this.level', rrp(rowStd, 'level'), 'high');
check('this.table_name', rrp(rowEmpty, 'table_name'), 't_user');
check('this.empty_rate (num)', rrp(rowEmpty, 'empty_rate'), 0.85);
console.log('\n--- resolveRenderPath: edge cases ---');
check('empty path', rrp(rowStd, ''), '');
check('null path', rrp(rowStd, null), '');
check('missing key', rrp(rowStd, 'nonexistent'), '');
check('wrong type (string)', rrp('just-a-string', 'foo'), '');
console.log('\n--- computed/clamped values used in template ---');
// progress 用的 0-1 clamp
const v = Math.max(0, Math.min(1, Number(rrp(rowEmpty, 'empty_rate')) || 0));
check('progress v clamped', v, 0.85);
// percent 用的 (value*100).toFixed(1)
const pct = (Number(rrp(rowEmpty, 'empty_rate')) * 100).toFixed(1);
check('percent formatted', pct, '85.0');
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail === 0 ? 0 : 1);
\ No newline at end of file
scripts/verify_render_tree.cjs deleted 100644 → 0
View file @ bd79c9e5
// 验证：用 jsdom 实际跑一遍最新版 setup()，检查渲染后 DOM 里 tree-row 是不是真有
// group-title / el-checkbox / group-description。
//
// 为什么不能用之前的脚手架 eval：Vue 是外部 <script src="..."> 加载的全局，
// jsdom 默认不执行外部脚本。我们改用 vm 跑一遍。
const fs = require('fs');
const { JSDOM } = require('jsdom');
const html = fs.readFileSync('web/static/index.html', 'utf8');
const dom = new JSDOM(html, {
runScripts: 'dangerously',
pretendToBeVisual: true,
url: 'http://localhost:8765/',
resources: 'usable',
});
// 把 window 暴露给 vm context —— Vue + Element Plus 已经在 window 上
// 因为 runScripts=dangerously + resources=usable，会去拉 /static/lib/vue / element-plus
// 但 jsdom 没 HTTP 服务（localhost:8765 是用户的服务，我们可以访问），应该能拉
// 等 window 上有 Vue 后再继续
function waitForVue() {
return new Promise((resolve, reject) => {
let tries = 0;
const tick = () => {
if (dom.window.Vue && dom.window.ElementPlus) return resolve();
if (++tries > 200) return reject(new Error('Vue / ElementPlus 没加载'));
setTimeout(tick, 50);
};
tick();
});
}
(async () => {
try {
await waitForVue();
console.log('Vue loaded; Vue =', typeof dom.window.Vue, ', ElementPlus =', typeof dom.window.ElementPlus);
} catch (e) {
console.error(e.message);
process.exit(1);
}
// 抽出 inner script 并把 fetch 调用打补丁（jsdom 有 fetch 是 window 的，但 fetch target 是真实服务器，会失败不能影响 setup）
const m = html.match(/<script>([\s\S]*?)<\/script>/);
let code = m[1];
// 给 code 套一层 wrapper，使用 window 上下文（在 jsdom 里 window 是 global）
// 真实 index.html 的 inline script 直接调用了 createApp，但没 try/catch fetch —— 我们让它跑
try {
dom.window.eval(code);
console.log('setup() ran');
} catch (e) {
console.error('setup() ERROR:', e.message);
if (e.stack) console.error(e.stack.split('\n').slice(0, 5).join('\n'));
}
// 等一帧让 v-for 渲染
await new Promise(r => setTimeout(r, 500));
const doc = dom.window.document;
const treeTable = doc.querySelector('.checks-tree-table');
const treeRows = doc.querySelectorAll('.tree-row');
const groupTitles = doc.querySelectorAll('.group-title');
const groupDescriptions = doc.querySelectorAll('.group-description');
const groupCheckboxes = doc.querySelectorAll('.tree-row--group .el-checkbox');
const carets = doc.querySelectorAll('.tree-caret');
const leafCheckboxes = doc.querySelectorAll('.tree-row--leaf .el-checkbox');
console.log('\n=== 渲染结果 ===');
console.log('tree-table container:', !!treeTable);
console.log('tree-rows count:', treeRows.length);
console.log('carets:', carets.length);
console.log('group-titles:', groupTitles.length);
console.log('group-descriptions:', groupDescriptions.length);
console.log('group rows with .el-checkbox:', groupCheckboxes.length);
console.log('leaf rows with .el-checkbox:', leafCheckboxes.length);
if (treeRows.length > 0) {
console.log('\n--- 第 1 个 tree-row HTML ---');
console.log(treeRows[0].outerHTML.substring(0, 600));
}
if (groupTitles.length > 0) {
console.log('\n--- 第 1 个 group-title ---');
console.log(groupTitles[0].textContent);
}
process.exit(0);
})();
scripts/verify_template.cjs deleted 100644 → 0
View file @ bd79c9e5
// scripts/verify_template.cjs
//
// 端到端模板编译验证：把 <div id="app">…</div> 喂给 Vue 3 prod build 的 compile()，
// 断言生成的 render 函数是合法 JS。
//
// 这一步专门兜底「页面变白 / SyntaxError in compiled code」这类回归。
// 历史教训：v-else 在深层 v-else-if 链末尾会被 Vue 编译器输出为 `: ()`（空
// call expression = SyntaxError）。改用 v-else-if="!cond" 显式条件可绕过。
const fs = require('fs');
const path = require('path');
const { JSDOM } = require('jsdom');
const html = fs.readFileSync(path.join(__dirname, '..', 'web', 'static', 'index.html'), 'utf8');
let pass = 0, fail = 0;
function check(label, ok, detail) {
if (ok) { console.log(` ✓ ${label}`); pass++; }
else { console.error(` ✗ ${label}${detail ? ' — ' + detail : ''}`); fail++; }
}
// ── 1. 抽出 <div id="app">…</div> 作为模板 ──
const startIdx = html.indexOf('<div id="app">');
const scriptIdx = html.indexOf('<script>');
let endIdx = html.lastIndexOf('</div>', scriptIdx);
if (endIdx < 0) endIdx = startIdx + 100; // 不太可能走到这里
const template = html.slice(startIdx, endIdx + '</div>'.length);
console.log(`Template: ${template.length} chars`);
// ── 2. 在 jsdom 里跑真实 Vue 3 prod build，劫持 Function 构造器截获编译产物 ──
const captured = [];
const dom = new JSDOM(html, {
url: 'http://localhost:8765/',
runScripts: 'dangerously',
resources: 'usable',
pretendToBeVisual: true,
});
const OrigFunction = dom.window.Function;
dom.window.Function = function(...args) {
const body = args[args.length - 1];
if (typeof body === 'string' && body.length > 1000) {
captured.push(body);
}
return OrigFunction.apply(this, args);
};
const vueSrc = fs.readFileSync(
path.join(__dirname, '..', 'web', 'static', 'lib', 'vue.global.prod.js'), 'utf8');
const epSrc = fs.readFileSync(
path.join(__dirname, '..', 'web', 'static', 'lib', 'element-plus', 'index.full.min.js'), 'utf8');
const scriptEl = dom.window.document.createElement('script');
scriptEl.textContent = vueSrc + '\n' + epSrc;
dom.window.document.head.appendChild(scriptEl);
setTimeout(() => {
console.log(`\nCaptured ${captured.length} compiled render fn(s)\n`);
// ── 3. 把每段编译产物当函数体 new Function() ──
check('captured at least 1 compiled render fn', captured.length >= 1,
`got ${captured.length}`);
for (let i = 0; i < captured.length; i++) {
const code = captured[i];
try {
new OrigFunction(code);
check(`compiled #${i} (${code.length} chars) is valid JS`, true);
} catch (e) {
check(`compiled #${i} (${code.length} chars) is valid JS`, false, e.message);
// 把出问题的代码写到磁盘
fs.writeFileSync(path.join(__dirname, `compiled_${i}.bad.js`), code, 'utf8');
}
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail === 0 ? 0 : 1);
}, 600);
\ No newline at end of file
web/README.md
View file @ 54429b71
......@@ -33,7 +33,7 @@ python -m web.backend.start
## 数据存储
- 任务、字段、规则和数据源保存于 `web/data/web.db`。
- 任务、字段、规则、数据源、系统配置和模型配置保存于 MySQL；本机默认读取 `database.test.yaml`，正式服务读取 `database.production.yaml`。
- 数据源密码目前按内部工具模式保存；部署到共享或生产环境前，应替换为受保护的凭据方案。
- 备份、还原、清表、加密、解密的操作记录保存于 `web/data/web.db` 的 `operation_record` 表；浏览器缓存清理、换浏览器或换电脑不会丢失记录。
- 实际备份文件仍保存于系统配置指定的备份目录，操作记录保存文件名和执行结果，不保存加密密码。
......
web/backend/README.md 0 → 100644
View file @ 54429b71
# 后端目录
后端按职责分层，新增功能请放入对应目录，避免把业务逻辑继续堆到入口文件：
- `app.py`、`start.py`：FastAPI 应用入口和启动适配。
- `config.py`、`_logging.py`：运行参数与日志基础设施。
- `routers/`：HTTP 接口层，只负责参数校验、权限边界和调用领域服务。
- `models/`：SQLAlchemy 数据模型；新增模型后同步加入 `models/__init__.py`，确保建表时被加载。
- `db/`：数据库引擎、初始化、迁移和种子数据。
- `core/`：数据库适配、规则执行、LLM、备份等可复用业务能力。
- `configs/`：配置模板和本地/生产私有配置，真实凭据不提交到仓库。
接口命名和数据结构优先复用已有路由、模型与 `core` 工具，避免在多个路由中复制连接、序列化和错误处理逻辑。
首次建库如需为历史连接预设注入密码，可设置 `DBTOOL_SEED_MYSQL_PASSWORD`、`DBTOOL_SEED_DAMENG_PASSWORD` 和 `DBTOOL_SEED_ORACLE_PASSWORD`。未设置时预设密码留空，可在页面中补录。
web/backend/app.py
View file @ 54429b71
......@@ -30,10 +30,10 @@ logger.info(f"Python: {sys.version.split()[0]}, 平台: {sys.platform}")
logger.info(f"工作目录: {PROJECT_ROOT}")
from web.backend.db.database import init_db, DB_PATH # noqa: E402
from web.backend.db.database import engine, init_db # noqa: E402
from web.backend import models # noqa: F401, E402 (必须 import 所有 ORM 才能被 Base.metadata 知道)
init_db()
logger.info(f"DB: {DB_PATH} (存在={DB_PATH.exists()})")
logger.info("DB: %s", engine.url.render_as_string(hide_password=True))
app = FastAPI(
......@@ -61,6 +61,7 @@ from web.backend.routers.data_sources import router as data_sources_router # no
from web.backend.routers.operations import router as operations_router # noqa: E402
from web.backend.routers.rules import router as rules_router # noqa: E402
from web.backend.routers.table_design import router as table_design_router # noqa: E402
from web.backend.routers.llm_config import router as llm_config_router # noqa: E402
app.include_router(task_groups_router, prefix="/api")
app.include_router(tasks_router, prefix="/api")
app.include_router(db_router, prefix="/api")
......@@ -71,6 +72,7 @@ app.include_router(data_sources_router, prefix="/api")
app.include_router(operations_router, prefix="/api")
app.include_router(rules_router, prefix="/api")
app.include_router(table_design_router, prefix="/api")
app.include_router(llm_config_router, prefix="/api")
logger.info("已注册路由：/api/task-groups, /api/tasks, /api/data-sources, /api/connect/*, /api/connection-presets, /api/ai/*, /api/queries/*")
......
web/backend/config.py
View file @ 54429b71
"""web 后端路径与运行参数常量
- WEB_DB_PATH：主业务 SQLite 文件（任务/字段/规则/不合规）
- WEB_SOURCES_DIR：用户数据源 SQLite 所在目录（运行时只读扫描）
- WEB_PORT / WEB_HOST：FastAPI 监听端口（默认 8767）
"""
"""web 后端路径、运行参数及测试/正式 MySQL 配置。"""
from __future__ import annotations
import os
from pathlib import Path
import yaml
from sqlalchemy import URL
# backend/ 的父目录 = web/
WEB_DIR = Path(__file__).resolve().parent
# web 的父目录 = 项目根（含 data/）
......@@ -20,7 +18,37 @@ DATA_DIR.mkdir(parents=True, exist_ok=True)
SOURCES_DIR = DATA_DIR / "sources"
SOURCES_DIR.mkdir(parents=True, exist_ok=True)
DB_PATH = Path(os.environ.get("WEB_DB_PATH") or (DATA_DIR / "web.db"))
APP_ENV = os.environ.get("APP_ENV", "test").strip().lower()
if APP_ENV not in {"test", "production"}:
raise RuntimeError("APP_ENV 只支持 test 或 production")
def _database_settings() -> dict:
config_file = Path(os.environ.get("DB_CONFIG_FILE") or (WEB_DIR / "configs" / f"database.{APP_ENV}.yaml"))
if not config_file.exists():
raise RuntimeError(f"未找到数据库配置文件：{config_file}")
try:
data = yaml.safe_load(config_file.read_text(encoding="utf-8")) or {}
except (OSError, yaml.YAMLError) as exc:
raise RuntimeError(f"读取数据库配置失败：{config_file}") from exc
required = ("driver", "host", "port", "database", "username", "password")
missing = [key for key in required if not data.get(key)]
if missing:
raise RuntimeError(f"数据库配置缺少字段：{', '.join(missing)}")
return data
DATABASE_SETTINGS = _database_settings()
DATABASE_URL = URL.create(
str(DATABASE_SETTINGS["driver"]),
username=str(DATABASE_SETTINGS["username"]),
password=str(DATABASE_SETTINGS["password"]),
host=str(DATABASE_SETTINGS["host"]),
port=int(DATABASE_SETTINGS["port"]),
database=str(DATABASE_SETTINGS["database"]),
query={"charset": "utf8mb4"},
)
DATABASE_CONNECT_ARGS = {"ssl": {}} if bool(DATABASE_SETTINGS.get("ssl")) else {}
PORT = int(os.environ.get("WEB_PORT", "8767"))
HOST = os.environ.get("WEB_HOST", "0.0.0.0")
......
web/backend/configs/database.example.yaml 0 → 100644
View file @ 54429b71
# Copy as database.test.yaml or database.production.yaml. Do not commit real passwords.
driver: mysql+pymysql
host: 127.0.0.1
port: 3306
database: db-tool
username: root
password: replace-me
ssl: false
web/backend/core/backup_storage.py
View file @ 54429b71
"""本地备份文件存储与路径配置。"""
from __future__ import annotations
import json
from pathlib import Path
from web.backend.config import DATA_DIR
from web.backend.db.database import SessionLocal
from web.backend.models.system_config import SystemConfig
DEFAULT_BACKUP_DIR = DATA_DIR / "backups"
CONFIG_FILE = DATA_DIR / "backup-config.json"
def get_backup_dir() -> Path:
path = DEFAULT_BACKUP_DIR
def _configured_backup_dir() -> str:
"""从业务数据库读取备份目录；数据库未初始化时使用默认值。"""
try:
data = json.loads(CONFIG_FILE.read_text(encoding="utf-8"))
configured = data.get("backup_dir")
if configured:
path = Path(configured).expanduser()
except (OSError, ValueError, TypeError):
pass
with SessionLocal() as db:
record = db.get(SystemConfig, "backup_dir")
return str(record.setting_value or "") if record else ""
except Exception:
return ""
def get_backup_dir() -> Path:
configured = _configured_backup_dir()
path = Path(configured).expanduser() if configured else DEFAULT_BACKUP_DIR
path = path.resolve()
path.mkdir(parents=True, exist_ok=True)
return path
......@@ -29,7 +33,14 @@ def set_backup_dir(value: str) -> Path:
raise ValueError("备份目录不能为空")
path = Path(value.strip()).expanduser().resolve()
path.mkdir(parents=True, exist_ok=True)
CONFIG_FILE.write_text(json.dumps({"backup_dir": str(path)}, ensure_ascii=False, indent=2), encoding="utf-8")
with SessionLocal() as db:
record = db.get(SystemConfig, "backup_dir")
if record is None:
record = SystemConfig(setting_key="backup_dir", setting_value=str(path))
db.add(record)
else:
record.setting_value = str(path)
db.commit()
return path
......
web/backend/core/db_adapter.py
View file @ 54429b71
......@@ -20,10 +20,13 @@
from __future__ import annotations
import logging
import os
import re
import sys
import time
from contextlib import contextmanager
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterator, Optional
import pymysql
......@@ -78,7 +81,7 @@ class DBConfig:
- 不要在 connect 时传 ``connect_timeout``：dmPython 2.5.x 对这个 kwarg 的
支持视 wheel 而异，不传最安全（超时由 OS TCP 重传决定）
参考：scripts/_probe_dm.py + 同环境已验证可连的 export_all_tables.py 工作脚本
参考：scripts/diagnostics/probe_dm.py + 同环境已验证可连的 export_all_tables.py 工作脚本
"""
return {
"user": self.user,
......@@ -246,6 +249,77 @@ def _render_health_sql(dialect: str) -> str:
_oracle_client_initialized: bool = False
def _oracle_client_library_names() -> tuple[str, ...]:
"""Return platform-specific Oracle Instant Client library names."""
if os.name == "nt":
return ("oci.dll",)
if sys.platform == "darwin":
return ("libclntsh.dylib",)
return ("libclntsh.so",)
def _looks_like_oracle_client(path: Path) -> bool:
"""Whether *path* contains the native library needed by python-oracledb."""
if not path.is_dir():
return False
names = _oracle_client_library_names()
if os.name == "nt":
return (path / names[0]).is_file()
# Linux distributions often ship a versioned libclntsh.so.* and a symlink
# may be omitted from a deployment archive, so accept either form.
return any((path / name).is_file() for name in names) or any(
p.is_file() and p.name.startswith("libclntsh.so.") for p in path.iterdir()
)
def _bundled_oracle_client_dir(explicit: Optional[str]) -> Optional[str]:
"""Find a bundled Instant Client without requiring host installation.
Search order is explicit config, environment override, then directories
shipped with the project. The returned directory is only accepted when
its native client library is present.
"""
candidates: list[Path] = []
if explicit:
candidates.append(Path(explicit).expanduser())
for env_name in ("DBTOOLS_ORACLE_CLIENT_DIR", "WEB_ORACLE_CLIENT_DIR"):
value = os.environ.get(env_name)
if value:
candidates.append(Path(value).expanduser())
# db_adapter.py lives in <project>/web/backend/core/. Keep the candidate
# paths stable so the same backend archive works on Windows and Linux.
module_path = Path(__file__).resolve()
bundled_roots = [
module_path.parent.parent / "oracle_client", # web/backend/oracle_client
module_path.parents[2] / "oracle_client", # web/oracle_client
module_path.parents[3] / "oracle_client", # project/oracle_client
]
# 开发机为 Windows、部署机为 Linux 时可同时把对应客户端放入项目：
# oracle_client/windows/oci.dll 和 oracle_client/linux/libclntsh.so。
# 当前平台优先加载自己的目录，随后保留旧的平铺目录兼容性。
platform_dir = "windows" if os.name == "nt" else "linux" if sys.platform.startswith("linux") else None
if platform_dir:
platform_roots = [root / platform_dir for root in bundled_roots]
candidates.extend(platform_roots)
# Oracle 官方压缩包会额外包一层 instantclient_<version> 目录。
# 同时兼容手动平铺和保持官方目录结构的两种放置方式。
for root in platform_roots:
try:
candidates.extend(path for path in root.iterdir() if path.is_dir())
except OSError:
continue
candidates.extend(bundled_roots)
for candidate in candidates:
try:
resolved = candidate.resolve()
except OSError:
continue
if _looks_like_oracle_client(resolved):
return str(resolved)
return None
def _ensure_oracle_client(lib_dir: Optional[str] = None) -> str:
"""尝试加载 Oracle Instant Client，返回实际生效的连接模式
......@@ -262,12 +336,14 @@ def _ensure_oracle_client(lib_dir: Optional[str] = None) -> str:
return "thick"
try:
import oracledb
if lib_dir:
oracledb.init_oracle_client(lib_dir=lib_dir)
bundled_dir = _bundled_oracle_client_dir(lib_dir)
if bundled_dir:
oracledb.init_oracle_client(lib_dir=bundled_dir)
logger.info(f"[DB] Oracle Instant Client 已加载（项目内置）lib_dir={bundled_dir}")
else:
oracledb.init_oracle_client() # 默认搜索 PATH / 注册表 / ORACLE_HOME
logger.info("[DB] Oracle Instant Client 已加载（系统搜索路径）")
_oracle_client_initialized = True
logger.info(f"[DB] Oracle Instant Client 已加载（thick 模式）{f'lib_dir={lib_dir}' if lib_dir else '（默认搜索路径）'}")
return "thick"
except Exception as e:
err_str = str(e)
......@@ -466,7 +542,7 @@ class DBConnection:
" 1) dmPython 与本机达梦客户端版本不匹配（dmPython 包版本 vs libdmdpi.so / dmdpi.dll）\n"
" 2) charset 不被达梦支持（达梦常见 GB18030 / UTF-8，MySQL 风格的 utf8mb4 / utf8 不一定可用）\n"
" 3) 主机/端口/服务名/账号密码错（真实 ORA-like 错误被 C 层吃了）\n"
"调试: 在终端单独跑 scripts/_probe_dm.py 看真实错误"
"调试: 在终端单独跑 scripts/diagnostics/probe_dm.py 看真实错误"
)
logger.error(
f"[DB] dmPython 连接失败（C 扩展吞了真实异常）: {e}\n"
......
web/backend/core/llm.py
View file @ 54429b71
......@@ -87,7 +87,14 @@ class LLMConfig:
if v is not None and hasattr(cfg, k):
setattr(cfg, k, v)
# 2. 环境变量（最高）
# 2. 业务数据库：页面化模型配置的持久化来源，高于旧 YAML。
database_cfg = _read_database_config()
if database_cfg:
for k, v in database_cfg.items():
if v is not None and hasattr(cfg, k):
setattr(cfg, k, v)
# 3. 环境变量（最高）
env_overrides: dict[str, Any] = {
"api_key": os.environ.get("LLM_API_KEY"),
"base_url": os.environ.get("LLM_BASE_URL"),
......@@ -135,6 +142,25 @@ def _read_yaml_config(path: Path) -> Optional[dict]:
return None
def _read_database_config() -> Optional[dict]:
"""读取当前环境 MySQL 中的模型配置，建表前安全返回空。"""
try:
from sqlalchemy import create_engine, text
from web.backend.config import DATABASE_CONNECT_ARGS, DATABASE_URL
db_engine = create_engine(DATABASE_URL, connect_args=DATABASE_CONNECT_ARGS, pool_pre_ping=True)
with db_engine.connect() as conn:
row = conn.execute(text(
"SELECT api_key, base_url, model, max_tokens, timeout FROM llm_config WHERE id = 1"
)).fetchone()
db_engine.dispose()
if not row:
return None
return dict(zip(("api_key", "base_url", "model", "max_tokens", "timeout"), row))
except Exception:
return None
# ── 客户端基类 ────────────────────────────────────────────
class LLMUnavailable(Exception):
"""LLM 不可用（无 Key / 网络/解析失败）"""
......
web/backend/db/apply_mysql_comments.py 0 → 100644
View file @ 54429b71
"""为当前 APP_ENV 的 MySQL 业务表补齐中文注释。"""
from __future__ import annotations
from sqlalchemy import text
from web.backend.db.database import engine
from web.backend.db.schema_comments import SCHEMA_COMMENTS
def _quote(identifier: str) -> str:
return f"`{identifier.replace('`', '``')}`"
def _sql_string(value: object) -> str:
return "'" + str(value).replace("\\", "\\\\").replace("'", "\\'") + "'"
def _default_sql(value: object) -> str:
text_value = str(value)
if text_value.upper().startswith("CURRENT_TIMESTAMP"):
return text_value.upper()
if text_value.upper() == "NULL":
return "NULL"
return _sql_string(value)
def _column_definition(column: dict, comment: str) -> str:
parts = [_quote(column["COLUMN_NAME"]), column["COLUMN_TYPE"]]
parts.append("NULL" if column["IS_NULLABLE"] == "YES" else "NOT NULL")
if column["COLUMN_DEFAULT"] is not None:
parts.extend(("DEFAULT", _default_sql(column["COLUMN_DEFAULT"])))
extra = str(column["EXTRA"] or "")
if "auto_increment" in extra.lower():
parts.append("AUTO_INCREMENT")
if "on update" in extra.lower():
parts.append("ON UPDATE CURRENT_TIMESTAMP")
parts.extend(("COMMENT", _sql_string(comment)))
return " ".join(parts)
def apply_mysql_comments() -> int:
"""只修改缺失或不一致的注释，不改数据、索引或字段语义。"""
if engine.dialect.name != "mysql":
return 0
changed = 0
schema = engine.url.database
with engine.begin() as conn:
table_rows = conn.execute(text("""
SELECT TABLE_NAME, TABLE_COMMENT
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = :schema
"""), {"schema": schema}).mappings().all()
table_comments = {row["TABLE_NAME"]: row["TABLE_COMMENT"] or "" for row in table_rows}
columns = conn.execute(text("""
SELECT TABLE_NAME, COLUMN_NAME, COLUMN_TYPE, IS_NULLABLE, COLUMN_DEFAULT, EXTRA, COLUMN_COMMENT
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = :schema
ORDER BY ORDINAL_POSITION
"""), {"schema": schema}).mappings().all()
columns_by_table = {}
for column in columns:
columns_by_table.setdefault(column["TABLE_NAME"], {})[column["COLUMN_NAME"]] = column
for table, spec in SCHEMA_COMMENTS.items():
if table not in table_comments:
continue
if table_comments[table] != spec["table"]:
conn.execute(text(f"ALTER TABLE {_quote(table)} COMMENT = {_sql_string(spec['table'])}"))
changed += 1
actual_columns = columns_by_table.get(table, {})
for name, comment in spec["columns"].items():
column = actual_columns.get(name)
if column and (column["COLUMN_COMMENT"] or "") != comment:
definition = _column_definition(column, comment)
conn.execute(text(f"ALTER TABLE {_quote(table)} MODIFY COLUMN {definition}"))
changed += 1
return changed
if __name__ == "__main__":
print(f"已更新 {apply_mysql_comments()} 项 MySQL 注释")
web/backend/db/database.py
View file @ 54429b71
"""SQLAlchemy 引擎与会话
- 单例 engine（sqlite + check_same_thread=False，FastAPI 跨线程访问需要）
- SessionLocal：每次请求拿一个新 session
- init_db()：首次启动时若 DB 文件不存在，跑 schema.sql 建表 + seed 灌入种子数据
"""
"""SQLAlchemy MySQL 引擎、会话与跨环境建表迁移。"""
from __future__ import annotations
from contextlib import contextmanager
import json
from pathlib import Path
from sqlalchemy import create_engine, event
from sqlalchemy.orm import sessionmaker, Session, declarative_base
from web.backend.config import DB_PATH
from web.backend.config import APP_ENV, DATA_DIR, DATABASE_CONNECT_ARGS, DATABASE_URL
from web.backend._logging import get_logger
logger = get_logger("backend.db")
......@@ -22,18 +18,22 @@ Base = declarative_base()
# ── engine ──
engine = create_engine(
f"sqlite:///{DB_PATH}",
DATABASE_URL,
echo=False,
connect_args={"check_same_thread": False},
connect_args=DATABASE_CONNECT_ARGS,
pool_pre_ping=True,
pool_recycle=1800,
)
IS_SQLITE = engine.dialect.name == "sqlite"
# SQLite 外键约束默认是关的，打开它
@event.listens_for(engine, "connect")
def _enable_sqlite_fk(dbapi_conn, _):
cur = dbapi_conn.cursor()
cur.execute("PRAGMA foreign_keys = ON")
cur.close()
if IS_SQLITE:
@event.listens_for(engine, "connect")
def _enable_sqlite_fk(dbapi_conn, _):
cur = dbapi_conn.cursor()
cur.execute("PRAGMA foreign_keys = ON")
cur.close()
SessionLocal = sessionmaker(bind=engine, autocommit=False, autoflush=False)
......@@ -63,24 +63,10 @@ def session_scope() -> Session:
# ── 首次启动初始化 ──
def init_db() -> None:
"""DB 不存在 → 跑 schema.sql 全量建表 + seed。
DB 已存在 → 用 SQLAlchemy 补齐缺失的表（增量迁移，不破坏已有数据），
然后 seed 各码表（每个 seed 自己判重，幂等）。
"""
"""初始化当前环境的 MySQL 表，并保留 SQLite 旧库升级兼容。"""
from web.backend.db import seed
if not DB_PATH.exists():
# 首次建库：跑 schema.sql 全量
schema = Path(__file__).parent / "schema.sql"
sql = schema.read_text(encoding="utf-8")
with engine.begin() as conn:
for stmt in [s.strip() for s in sql.split(";") if s.strip()]:
conn.exec_driver_sql(stmt)
logger.info(f"[init_db] 首次建库完成：{DB_PATH}")
else:
# 已有 DB：增量补齐 ORM 里声明但 DB 里尚未存在的表（如新加的 connection_preset）
# Base.metadata.create_all 只会创建缺失的表，已存在的表不动
Base.metadata.create_all(engine)
Base.metadata.create_all(engine)
if IS_SQLITE:
# 一次性迁移：补齐 connection_preset 的 UNIQUE(db_type, name) 约束
# 历史背景：connection_preset 最初是手动跑 schema.sql 片段建的（仅 7 张主表里没有它），
# 然后通过 Base.metadata.create_all 增量建出来时只采纳了 ORM 列，丢了 SQL 里的 UNIQUE。
......@@ -106,12 +92,20 @@ def init_db() -> None:
_migrate_task_data_source_id()
_migrate_rule_library_ai_prompt()
_migrate_legacy_rules_to_library()
logger.info(f"[init_db] DB 已存在，已执行增量建表检查：{DB_PATH}")
logger.info("[init_db] SQLite 兼容迁移完成")
else:
logger.info("[init_db] %s MySQL 表结构已就绪", APP_ENV)
from web.backend.db.apply_mysql_comments import apply_mysql_comments
changed = apply_mysql_comments()
if changed:
logger.info("[init_db] 已更新 %s 项 MySQL 中文注释", changed)
# 灌种子（每个 seed 函数内部判重，可重复调用）
seed.seed_task_groups()
seed.seed_connection_presets()
seed.seed_data_sources()
print(f"[init_db] 初始化完成：{DB_PATH}")
_migrate_llm_config_from_file()
_migrate_backup_config_to_database()
print(f"[init_db] 初始化完成：{engine.url.render_as_string(hide_password=True)}")
def _migrate_connection_preset_unique() -> None:
......@@ -310,3 +304,49 @@ def _migrate_legacy_rules_to_library() -> None:
if added:
db.commit()
logger.info(f"[migrate] 已将 {added} 条历史规则登记到规则库")
def _migrate_llm_config_from_file() -> None:
"""首次升级时将旧项目 YAML 配置迁入业务数据库。"""
from web.backend.core.llm import _read_yaml_config, default_config_path
from web.backend.models.llm_config import LLMConfigRecord
with SessionLocal() as db:
if db.get(LLMConfigRecord, 1):
return
legacy = _read_yaml_config(default_config_path()) or {}
api_key = str(legacy.get("api_key") or "").strip()
base_url = str(legacy.get("base_url") or "").strip().rstrip("/")
model = str(legacy.get("model") or "").strip()
if not api_key or not base_url or not model:
return
db.add(LLMConfigRecord(
id=1,
api_key=api_key,
base_url=base_url,
model=model,
max_tokens=int(legacy.get("max_tokens") or 1024),
timeout=int(legacy.get("timeout") or 30),
))
db.commit()
logger.info("[migrate] 已将项目 LLM 配置迁移到业务数据库")
def _migrate_backup_config_to_database() -> None:
"""将旧备份目录 JSON 迁入 system_config，之后备份模块只读取数据库。"""
from web.backend.models.system_config import SystemConfig
with SessionLocal() as db:
if db.get(SystemConfig, "backup_dir"):
return
configured = ""
legacy_path = DATA_DIR / "backup-config.json"
try:
data = json.loads(legacy_path.read_text(encoding="utf-8"))
configured = str(data.get("backup_dir") or "").strip()
except (OSError, ValueError, TypeError):
pass
backup_dir = Path(configured).expanduser() if configured else DATA_DIR / "backups"
db.add(SystemConfig(setting_key="backup_dir", setting_value=str(backup_dir.resolve())))
db.commit()
logger.info("[migrate] 已将系统备份目录迁移到业务数据库")
web/backend/db/migrate_sqlite_to_mysql.py 0 → 100644
View file @ 54429b71
"""将旧 SQLite 业务库安全迁移至当前 APP_ENV 的 MySQL。"""
from __future__ import annotations
import argparse
import re
import sqlite3
from pathlib import Path
from sqlalchemy import URL, create_engine, text
from web.backend.config import DATABASE_CONNECT_ARGS, DATABASE_SETTINGS, DATABASE_URL
from web.backend.db.database import Base, engine
from web.backend import models # noqa: F401 注册全部 ORM 表
TABLE_ORDER = (
"task_group", "connection_preset", "data_source", "task", "field", "rule",
"rule_library", "field_rule", "validation_run", "violation", "operation_record",
"table_design_config", "llm_config", "system_config",
)
def _ensure_database() -> None:
name = str(DATABASE_SETTINGS["database"])
if not re.fullmatch(r"[A-Za-z0-9_-]+", name):
raise ValueError("数据库名称只能包含字母、数字、下划线或短横线")
admin_url = URL.create(
str(DATABASE_SETTINGS["driver"]),
username=str(DATABASE_SETTINGS["username"]),
password=str(DATABASE_SETTINGS["password"]),
host=str(DATABASE_SETTINGS["host"]),
port=int(DATABASE_SETTINGS["port"]),
query={"charset": "utf8mb4"},
)
admin = create_engine(admin_url, connect_args=DATABASE_CONNECT_ARGS, pool_pre_ping=True)
try:
with admin.begin() as conn:
conn.execute(text(f"CREATE DATABASE IF NOT EXISTS `{name}` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"))
finally:
admin.dispose()
def _source_rows(source: sqlite3.Connection, table: str, allowed: set[str]) -> list[dict]:
available = {row[1] for row in source.execute(f"PRAGMA table_info('{table}')")}
columns = sorted(allowed & available)
if not columns:
return []
quoted = ", ".join(f'"{column}"' for column in columns)
return [dict(row) for row in source.execute(f'SELECT {quoted} FROM "{table}"')]
def migrate(source_path: Path, *, skip_if_populated: bool = False) -> dict[str, int]:
if not source_path.is_file():
raise FileNotFoundError(f"SQLite 源文件不存在：{source_path}")
_ensure_database()
Base.metadata.create_all(engine)
with engine.connect() as target:
existing_rows = sum(
target.execute(text(f"SELECT COUNT(*) FROM `{name}`")).scalar_one()
for name in TABLE_ORDER if name in Base.metadata.tables
)
if existing_rows and skip_if_populated:
print("目标 MySQL 已有业务数据，跳过迁移")
return {}
with sqlite3.connect(source_path) as source:
source.row_factory = sqlite3.Row
source_tables = {row[0] for row in source.execute("SELECT name FROM sqlite_master WHERE type='table'")}
copied: dict[str, int] = {}
with engine.begin() as target:
for name in TABLE_ORDER:
if name not in source_tables or name not in Base.metadata.tables:
continue
table = Base.metadata.tables[name]
existing = target.execute(text(f"SELECT COUNT(*) FROM `{name}`")).scalar_one()
if existing:
raise RuntimeError(f"目标表 {name} 已有 {existing} 条数据，拒绝覆盖")
rows = _source_rows(source, name, set(table.columns.keys()))
if rows:
target.execute(table.insert(), rows)
copied[name] = len(rows)
with engine.connect() as target:
for name, expected in copied.items():
actual = target.execute(text(f"SELECT COUNT(*) FROM `{name}`")).scalar_one()
if actual != expected:
raise RuntimeError(f"表 {name} 迁移校验失败：源 {expected}，目标 {actual}")
return copied
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--source", required=True, type=Path, help="旧 SQLite web.db 的绝对或相对路径")
parser.add_argument("--skip-if-populated", action="store_true", help="目标库已有数据时跳过，供后续发布使用")
args = parser.parse_args()
copied = migrate(args.source.resolve(), skip_if_populated=args.skip_if_populated)
print("SQLite -> MySQL 迁移完成")
for name, count in copied.items():
print(f"{name}: {count}")
print(f"目标：{DATABASE_URL.render_as_string(hide_password=True)}")
if __name__ == "__main__":
main()
web/backend/db/schema.sql
View file @ 54429b71
-- ============================================================
-- web 主业务库 · schema.sql
-- web 主业务库 · SQLite 历史兼容脚本
-- ============================================================
-- 数据库：data/web.db（SQLite）
-- 维护方式：DB 文件首次生成时由 db/database.py::init_db() 全量执行
-- 注意：本文件是「从零建表」脚本，已存在 DB 时不会执行。
-- 表结构变更后续走迁移（plan §11 待办）。
-- 用途：保留旧 SQLite 数据库迁移和本地兼容场景，不是当前 MySQL 初始化入口。
-- 当前业务库：由 SQLAlchemy ORM 在 database.py::init_db() 中创建和增量维护。
-- 注意：本文件中的 AUTOINCREMENT 等语法仅适用于 SQLite。
-- ============================================================
-- ─────────────────────────────────────────────────────────────
......@@ -226,3 +225,21 @@ CREATE TABLE table_design_config (
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX idx_table_design_config_source ON table_design_config(source_id);
-- 11) llm_config · 模型服务配置（仅保留 id=1 的单例记录）
CREATE TABLE llm_config (
id INTEGER PRIMARY KEY CHECK (id = 1),
base_url TEXT NOT NULL,
model TEXT NOT NULL,
api_key TEXT NOT NULL,
max_tokens INTEGER NOT NULL DEFAULT 1024,
timeout INTEGER NOT NULL DEFAULT 30,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- 12) system_config · 系统级键值配置
CREATE TABLE system_config (
setting_key TEXT PRIMARY KEY,
setting_value TEXT NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
web/backend/db/schema_comments.py 0 → 100644
View file @ 54429b71
"""db-tool MySQL 表及字段中文注释。"""
from __future__ import annotations
from typing import Dict
SCHEMA_COMMENTS: Dict[str, dict] = {
"task_group": {
"table": "数据质量校验任务分组",
"columns": {"id": "主键", "name": "分组名称", "ord": "排序号", "created_at": "创建时间"},
},
"task": {
"table": "数据质量校验任务",
"columns": {
"id": "主键", "name": "任务名称", "group_id": "所属任务分组ID", "data_source_path": "历史数据源文件路径",
"data_source_id": "数据源ID", "source_table": "源数据表名", "status": "任务状态", "description": "任务说明",
"db_type": "源数据库类型", "conn_json": "历史连接配置JSON", "created_at": "创建时间", "updated_at": "更新时间",
"checked_at": "最近校验时间", "table_comment": "源数据表中文注释",
},
},
"field": {
"table": "校验任务字段",
"columns": {"id": "主键", "task_id": "所属任务ID", "field_key": "数据库字段名", "show_default": "是否默认展示", "ord": "排序号"},
},
"rule": {
"table": "历史字段校验规则",
"columns": {"id": "主键", "field_id": "所属字段ID", "desc": "规则说明", "name": "规则名称", "regex": "正则表达式", "rule_type": "规则类型", "code": "Python校验代码", "skip_null": "是否跳过空值", "ord": "排序号"},
},
"rule_library": {
"table": "可复用校验规则库",
"columns": {"id": "主键", "name": "规则名称", "ai_prompt": "AI生成提示词", "description": "规则说明", "rule_type": "规则类型", "regex": "正则表达式", "code": "Python校验代码", "skip_null": "是否跳过空值", "created_at": "创建时间", "updated_at": "更新时间"},
},
"field_rule": {
"table": "字段与规则库关联",
"columns": {"id": "主键", "field_id": "字段ID", "rule_library_id": "规则库ID", "ord": "排序号"},
},
"connection_preset": {
"table": "历史数据库连接预设",
"columns": {"id": "主键", "db_type": "数据库类型", "name": "预设标识", "display": "展示名称", "host": "数据库地址", "port": "数据库端口", "user": "登录用户名", "password": "登录密码", "database": "默认数据库名", "oracle_client_dir": "Oracle客户端目录", "ord": "排序号", "created_at": "创建时间"},
},
"data_source": {
"table": "可复用数据源",
"columns": {"id": "主键", "name": "数据源名称", "db_type": "数据库类型", "host": "数据库地址", "port": "数据库端口", "database": "默认数据库名", "schema": "Schema名称", "user": "登录用户名", "password": "登录密码", "oracle_client_dir": "Oracle客户端目录", "status": "数据源状态", "last_tested_at": "最近测试时间", "last_test_ok": "最近测试是否成功", "created_at": "创建时间", "updated_at": "更新时间"},
},
"operation_record": {
"table": "数据库操作记录",
"columns": {"id": "主键", "operation": "操作类型", "source_id": "数据源ID", "source_name": "数据源名称", "database_name": "数据库名称", "scope": "操作范围", "tables_json": "数据表列表JSON", "filename": "备份文件名", "execute_setting": "执行设置", "status": "执行状态", "created_at": "创建时间"},
},
"table_design_config": {
"table": "表设计校验配置",
"columns": {"id": "主键", "name": "配置名称", "source_id": "数据源ID", "schema_name": "Schema名称", "scope": "校验范围", "tables_json": "指定数据表JSON", "check_table_comment": "是否校验表注释", "check_column_comment": "是否校验字段注释", "checked_at": "最近校验时间", "last_result_json": "最近校验结果JSON", "created_at": "创建时间", "updated_at": "更新时间"},
},
"llm_config": {
"table": "AI模型服务配置",
"columns": {"id": "固定单例主键", "base_url": "模型服务地址", "model": "模型名称", "api_key": "模型服务密钥", "max_tokens": "最大输出Token数", "timeout": "请求超时秒数", "updated_at": "更新时间"},
},
"system_config": {
"table": "系统参数配置",
"columns": {"setting_key": "配置参数键", "setting_value": "配置参数值", "updated_at": "更新时间"},
},
}
web/backend/db/seed.py
View file @ 54429b71
"""种子数据 —— 首次建库后灌入基础数据
- seed_task_groups()：6 个分组（来自 mock 需求里的左侧菜单「校验任务」）
- seed_connection_presets()：13 条历史连接（来自 web2/configs/db_defaults.yaml，
硬编码避免引入 pyyaml 依赖；改完需要重启 web 后端生效）
- seed_task_groups()：当前默认任务分组
- seed_connection_presets()：13 条历史连接；密码仅从环境变量读取，不写入源码。
"""
from __future__ import annotations
import logging
import os
from web.backend.db.database import session_scope
from web.backend.models.connection_preset import ConnectionPreset
......@@ -17,7 +17,7 @@ logger = logging.getLogger(__name__)
# ── DML ─────────────────────────────────────────────────────
# 来自 mock web/requirement/数据质量检测结果.html 里左侧菜单「校验任务」下的 6 个分组
# 参考原型保存在 web/docs/reference/，当前产品统一使用一个「校验」分组。
# 注意：不要把「任务配置」加进来 —— 它是导航项不是分组
DEFAULT_TASK_GROUPS = [("校验", 10)]
......@@ -38,31 +38,37 @@ def seed_task_groups() -> None:
# db_type 内部小写（mysql / dameng / oracle）
# name yaml 的 id 字段（同一 db_type 内唯一，UI 选中后存 task.conn_json 时也带回去）
# display yaml 的 display 字段（UI 下拉展示用）
# host / port / user / password / database / oracle_client_dir
# yaml 对应字段
# host / port / user / database / oracle_client_dir
# 历史配置对应字段；密码在首次建库时从环境变量注入
# ord 同 db_type 内的显示顺序（10/20/... 一组；跨 db_type 各自递增）
DEFAULT_CONNECTION_PRESETS: list[dict] = [
# ── MySQL（11 条）──
{"db_type": "mysql", "name": "smart-build", "display": "智能建造", "host": "192.168.20.10", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "smart-build", "ord": 10},
{"db_type": "mysql", "name": "cd-zhfw-jiangsu", "display": "江苏房屋", "host": "192.168.20.10", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "cd-zhfw-jiangsu", "ord": 20},
{"db_type": "mysql", "name": "cd-sewage", "display": "湖北污水", "host": "192.168.20.11", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "cd-sewage", "ord": 30},
{"db_type": "mysql", "name": "city-sewage", "display": "垃圾", "host": "192.168.20.11", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "city-sewage", "ord": 40},
{"db_type": "mysql", "name": "farmhouse", "display": "农房工匠", "host": "192.168.20.11", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "farmhouse", "ord": 50},
{"db_type": "mysql", "name": "hbzdmhxm", "display": "重大项目", "host": "192.168.20.11", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "hbzdmhxm", "ord": 60},
{"db_type": "mysql", "name": "old_community", "display": "老旧小区", "host": "192.168.20.11", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "old_community", "ord": 70},
{"db_type": "mysql", "name": "traditional-villages","display": "传统村落", "host": "192.168.20.11", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "traditional-villages","ord": 80},
{"db_type": "mysql", "name": "community-embedded", "display": "智慧社区", "host": "192.168.20.10", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "community-embedded", "ord": 90},
{"db_type": "mysql", "name": "construction", "display": "建筑业", "host": "192.168.20.10", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "construction", "ord": 100},
{"db_type": "mysql", "name": "retailers", "display": "楚凤家政", "host": "192.168.20.10", "port": 3306, "user": "root", "password": "Cd@85617711", "database": "retailers", "ord": 110},
{"db_type": "mysql", "name": "smart-build", "display": "智能建造", "host": "192.168.20.10", "port": 3306, "user": "root", "database": "smart-build", "ord": 10},
{"db_type": "mysql", "name": "cd-zhfw-jiangsu", "display": "江苏房屋", "host": "192.168.20.10", "port": 3306, "user": "root", "database": "cd-zhfw-jiangsu", "ord": 20},
{"db_type": "mysql", "name": "cd-sewage", "display": "湖北污水", "host": "192.168.20.11", "port": 3306, "user": "root", "database": "cd-sewage", "ord": 30},
{"db_type": "mysql", "name": "city-sewage", "display": "垃圾", "host": "192.168.20.11", "port": 3306, "user": "root", "database": "city-sewage", "ord": 40},
{"db_type": "mysql", "name": "farmhouse", "display": "农房工匠", "host": "192.168.20.11", "port": 3306, "user": "root", "database": "farmhouse", "ord": 50},
{"db_type": "mysql", "name": "hbzdmhxm", "display": "重大项目", "host": "192.168.20.11", "port": 3306, "user": "root", "database": "hbzdmhxm", "ord": 60},
{"db_type": "mysql", "name": "old_community", "display": "老旧小区", "host": "192.168.20.11", "port": 3306, "user": "root", "database": "old_community", "ord": 70},
{"db_type": "mysql", "name": "traditional-villages","display": "传统村落", "host": "192.168.20.11", "port": 3306, "user": "root", "database": "traditional-villages","ord": 80},
{"db_type": "mysql", "name": "community-embedded", "display": "智慧社区", "host": "192.168.20.10", "port": 3306, "user": "root", "database": "community-embedded", "ord": 90},
{"db_type": "mysql", "name": "construction", "display": "建筑业", "host": "192.168.20.10", "port": 3306, "user": "root", "database": "construction", "ord": 100},
{"db_type": "mysql", "name": "retailers", "display": "楚凤家政", "host": "192.168.20.10", "port": 3306, "user": "root", "database": "retailers", "ord": 110},
# ── 达梦 DM（1 条）──
{"db_type": "dameng", "name": "sanjin", "display": "三金系统", "host": "192.168.20.24", "port": 5236, "user": "HGWXZJ", "password": "HGwxzj2026", "database": "HGWXZJ", "ord": 10},
{"db_type": "dameng", "name": "sanjin", "display": "三金系统", "host": "192.168.20.24", "port": 5236, "user": "HGWXZJ", "database": "HGWXZJ", "ord": 10},
# ── Oracle（1 条）──
{"db_type": "oracle", "name": "huanggang-gjj", "display": "黄冈公积金", "host": "192.168.20.196", "port": 1521, "user": "GJJ80", "password": "GJJ80_Hg41RG", "database": "ZFGJJ", "ord": 10,
{"db_type": "oracle", "name": "huanggang-gjj", "display": "黄冈公积金", "host": "192.168.20.196", "port": 1521, "user": "GJJ80", "database": "ZFGJJ", "ord": 10,
"oracle_client_dir": "D:\\Oracle\\instantclient-basic-windows.x64-12.2.0.1.0\\instantclient_12_2"},
]
PRESET_PASSWORD_ENV = {
"mysql": "DBTOOL_SEED_MYSQL_PASSWORD",
"dameng": "DBTOOL_SEED_DAMENG_PASSWORD",
"oracle": "DBTOOL_SEED_ORACLE_PASSWORD",
}
def seed_connection_presets() -> None:
"""首次建库时插入 13 条历史连接；已存在则跳过（按 db_type + name 判重）。"""
......@@ -79,7 +85,7 @@ def seed_connection_presets() -> None:
host=cfg["host"],
port=int(cfg["port"]),
user=cfg["user"],
password=cfg["password"],
password=os.environ.get(PRESET_PASSWORD_ENV[cfg["db_type"]], ""),
database=cfg["database"],
oracle_client_dir=cfg.get("oracle_client_dir"),
ord=cfg["ord"],
......@@ -94,7 +100,6 @@ DEFAULT_DATA_SOURCES = [
{"name": "Oracle 示例", "db_type": "Oracle", "host": "127.0.0.1", "port": 1521, "database": "ORCL", "schema": "DEMO", "user": "demo", "password": ""},
]
def seed_data_sources() -> None:
"""插入示例数据源；按名称判重，用户已有数据不受影响。"""
with session_scope() as db:
......
web/backend/models/__init__.py
View file @ 54429b71
......@@ -14,5 +14,7 @@ from .task_group import TaskGroup
from .data_source import DataSource
from .operation_record import OperationRecord
from .table_design_config import TableDesignConfig
from .llm_config import LLMConfigRecord
from .system_config import SystemConfig
__all__ = ["ConnectionPreset", "Field", "Rule", "RuleLibrary", "FieldRule", "Task", "TaskGroup", "DataSource", "OperationRecord", "TableDesignConfig"]
__all__ = ["ConnectionPreset", "Field", "Rule", "RuleLibrary", "FieldRule", "Task", "TaskGroup", "DataSource", "OperationRecord", "TableDesignConfig", "LLMConfigRecord", "SystemConfig"]
web/backend/models/connection_preset.py
View file @ 54429b71
......@@ -30,15 +30,15 @@ class ConnectionPreset(Base):
)
id = Column(Integer, primary_key=True, autoincrement=True)
db_type = Column(String, nullable=False) # mysql / dameng / oracle
name = Column(String, nullable=False) # yaml 的 id 字段
display = Column(String, nullable=True) # UI 展示名（yaml 的 display 字段）
host = Column(String, nullable=False)
db_type = Column(String(30), nullable=False) # mysql / dameng / oracle
name = Column(String(100), nullable=False) # yaml 的 id 字段
display = Column(String(200), nullable=True) # UI 展示名（yaml 的 display 字段）
host = Column(String(255), nullable=False)
port = Column(Integer, nullable=False)
user = Column(String, nullable=False)
password = Column(String, nullable=False, default="") # 与 task.conn_json 同策略：明文存（内部工具）
database = Column(String, nullable=False, default="")
oracle_client_dir = Column(String, nullable=True) # 仅 Oracle 使用
user = Column(String(128), nullable=False)
password = Column(String(1000), nullable=False, default="") # 与 task.conn_json 同策略：明文存（内部工具）
database = Column(String(255), nullable=False, default="")
oracle_client_dir = Column(String(512), nullable=True) # 仅 Oracle 使用
ord = Column(Integer, nullable=False, default=0) # 显示顺序
created_at = Column(DateTime, nullable=False, server_default=func.current_timestamp())
......
web/backend/models/field.py
View file @ 54429b71
......@@ -28,7 +28,7 @@ class Field(Base):
id = Column(Integer, primary_key=True, autoincrement=True)
task_id = Column(Integer, ForeignKey("task.id", ondelete="CASCADE"), nullable=False)
field_key = Column(String, nullable=False) # 英文字段名（与数据源对齐）
field_key = Column(String(255), nullable=False) # 英文字段名（与数据源对齐）
show_default = Column(Integer, nullable=False, default=1) # 0/1：结果表默认展示
ord = Column(Integer, nullable=False, default=0)
......@@ -49,4 +49,4 @@ class Field(Base):
}
def __repr__(self) -> str:
return f"<Field id={self.id} task_id={self.task_id} field_key={self.field_key!r}>"
\ No newline at end of file
return f"<Field id={self.id} task_id={self.task_id} field_key={self.field_key!r}>"
web/backend/models/llm_config.py 0 → 100644
View file @ 54429b71
"""模型服务配置（单例记录，保存于当前环境的业务数据库）。"""
from __future__ import annotations
from sqlalchemy import Column, DateTime, Integer, String
from sqlalchemy.sql import func
from web.backend.db.database import Base
class LLMConfigRecord(Base):
__tablename__ = "llm_config"
id = Column(Integer, primary_key=True, default=1)
base_url = Column(String(500), nullable=False)
model = Column(String(200), nullable=False)
api_key = Column(String(1000), nullable=False)
max_tokens = Column(Integer, nullable=False, default=1024)
timeout = Column(Integer, nullable=False, default=30)
updated_at = Column(DateTime, nullable=False, server_default=func.current_timestamp(), onupdate=func.current_timestamp())
web/backend/models/rule.py
View file @ 54429b71
......@@ -43,10 +43,10 @@ class Rule(Base):
field_id = Column(Integer, ForeignKey("field.id", ondelete="CASCADE"), nullable=False)
desc = Column(Text, nullable=False) # 规则说明（自然语言）
# 2026-08-24：规则名称，给前端展示「违反哪条规则」用；老规则默认空串（迁移兼容）
name = Column(Text, nullable=False, default="", server_default="")
name = Column(String(255), nullable=False, default="", server_default="")
regex = Column(Text, nullable=True) # Python 正则（regex 类型用）
rule_type = Column( # 校验类型（2026-08-21 扩展）
String, nullable=False, default=DEFAULT_RULE_TYPE,
String(20), nullable=False, default=DEFAULT_RULE_TYPE,
)
code = Column(Text, nullable=True) # Python 函数源码（number/date/string 用）
# 2026-08-21：值为空（None/空 str）时跳过该规则，不走 check 直接判 True
......@@ -67,4 +67,4 @@ class Rule(Base):
}
def __repr__(self) -> str:
return f"<Rule id={self.id} field_id={self.field_id} rule_type={self.rule_type!r}>"
\ No newline at end of file
return f"<Rule id={self.id} field_id={self.field_id} rule_type={self.rule_type!r}>"
web/backend/models/system_config.py 0 → 100644
View file @ 54429b71
"""系统级键值配置，存放于当前环境的业务数据库。"""
from __future__ import annotations
from sqlalchemy import Column, DateTime, String, Text
from sqlalchemy.sql import func
from web.backend.db.database import Base
class SystemConfig(Base):
__tablename__ = "system_config"
setting_key = Column(String(100), primary_key=True)
setting_value = Column(Text, nullable=False)
updated_at = Column(DateTime, nullable=False, server_default=func.current_timestamp(), onupdate=func.current_timestamp())
web/backend/models/task.py
View file @ 54429b71
......@@ -27,14 +27,14 @@ class Task(Base):
__tablename__ = "task"
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String, nullable=False, unique=True)
name = Column(String(255), nullable=False, unique=True)
group_id = Column(Integer, ForeignKey("task_group.id"), nullable=False)
data_source_path = Column(Text, nullable=True) # 例如 sources/customer_db/customer.sqlite
data_source_id = Column(Integer, ForeignKey("data_source.id"), nullable=True)
source_table = Column(String, nullable=True) # 例如 t_user_info
status = Column(String, nullable=False, default="启用") # 启用 / 停用
source_table = Column(String(255), nullable=True) # 例如 t_user_info
status = Column(String(20), nullable=False, default="启用") # 启用 / 停用
description = Column(Text, nullable=True)
db_type = Column(String, nullable=False, default="MySQL") # UI 展示冗余
db_type = Column(String(30), nullable=False, default="MySQL") # UI 展示冗余
conn_json = Column(Text, nullable=True) # UI 表单 JSON
created_at = Column(DateTime, nullable=False, server_default=func.current_timestamp())
updated_at = Column(DateTime, nullable=False,
......
web/backend/models/task_group.py
View file @ 54429b71
......@@ -19,7 +19,7 @@ class TaskGroup(Base):
__tablename__ = "task_group"
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String, nullable=False, unique=True)
name = Column(String(100), nullable=False, unique=True)
ord = Column(Integer, nullable=False, default=0)
created_at = Column(DateTime, nullable=False, server_default=func.current_timestamp())
......@@ -32,4 +32,4 @@ class TaskGroup(Base):
}
def __repr__(self) -> str:
return f"<TaskGroup id={self.id} name={self.name!r} ord={self.ord}>"
\ No newline at end of file
return f"<TaskGroup id={self.id} name={self.name!r} ord={self.ord}>"
web/backend/routers/ai.py
View file @ 54429b71
......@@ -69,10 +69,13 @@ async def ai_gen_rule(req: GenRuleRequest):
try:
# 两项输出都只依赖用户的原始需求，可以并行请求，避免用户等待两次模型响应。
source_desc = (req.prompt or req.desc or req.name).strip()
description_task = asyncio.to_thread(
gen_rule_description, req.name, req.desc, req.rule_type, req.prompt
# asyncio.to_thread 仅在 Python 3.9+ 可用；正式环境仍为 Python 3.8。
# run_in_executor 同样将同步 SDK 调用放入线程池，并支持两项请求并行执行。
loop = asyncio.get_running_loop()
description_task = loop.run_in_executor(
None, gen_rule_description, req.name, req.desc, req.rule_type, req.prompt
)
code_task = asyncio.to_thread(gen_rule, source_desc, req.rule_type)
code_task = loop.run_in_executor(None, gen_rule, source_desc, req.rule_type)
(description, desc_note), (code, note) = await asyncio.gather(
description_task, code_task
)
......
web/backend/routers/connection_presets.py
View file @ 54429b71
......@@ -16,7 +16,7 @@
"""
from __future__ import annotations
from typing import Optional
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
......@@ -74,7 +74,7 @@ class ConnectionPresetUpdate(BaseModel):
# ── 路由 ──
@router.get("", response_model=list[ConnectionPresetOut], summary="列出历史连接预设")
@router.get("", response_model=List[ConnectionPresetOut], summary="列出历史连接预设")
def list_connection_presets(
db_type: Optional[str] = Query(None, description="按 db_type 过滤：mysql/dameng/oracle"),
db: Session = Depends(get_session),
......
web/backend/routers/data_sources.py
View file @ 54429b71
......@@ -13,6 +13,7 @@ from sqlalchemy.orm import Session
from web.backend.db.database import get_session
from web.backend.models.data_source import DataSource
from web.backend.core.db_adapter import DBConfig, DBConnection, quote_ident
from web.backend.core.sql_loader import get_sql_loader
from web.backend.core.backup_storage import get_backup_dir
from web.backend.core.operation_history import record_operation
......@@ -132,9 +133,13 @@ def delete_data_source(source_id: int, db: Session = Depends(get_session)):
def test_data_source(source_id: int, db: Session = Depends(get_session)):
row = db.get(DataSource, source_id)
if not row: raise HTTPException(404, "数据源不存在")
cfg = _cfg(row)
try:
with DBConnection(_cfg(row)) as conn:
conn.fetch_scalar("SELECT 1")
with DBConnection(cfg) as conn:
# Oracle requires a FROM clause (`dual`); use the shared
# dialect-specific health-check template.
health_sql = get_sql_loader().render("health/check_connection", dialect=cfg.db_type)
conn.fetch_scalar(health_sql)
row.last_test_ok = 1; row.last_tested_at = datetime.utcnow(); db.commit()
return {"ok": True, "message": "连接成功"}
except Exception as e:
......
web/backend/routers/llm_config.py 0 → 100644
View file @ 54429b71
"""项目内 LLM 配置管理接口。"""
from __future__ import annotations
import os
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from web.backend.core.llm import LLMConfig, reset_llm_client
from web.backend.db.database import get_session
from web.backend.models.llm_config import LLMConfigRecord
from web.backend._logging import get_logger
router = APIRouter(prefix="/llm-config", tags=["llm-config"])
logger = get_logger("backend.routers.llm_config")
class LLMConfigUpdate(BaseModel):
base_url: str = Field(..., min_length=1, max_length=500)
model: str = Field(..., min_length=1, max_length=200)
api_key: str = Field("", max_length=1000)
max_tokens: int = Field(1024, ge=256, le=16384)
timeout: int = Field(30, ge=5, le=300)
def _environment_override_names() -> list[str]:
keys = (
"LLM_API_KEY", "LLM_BASE_URL", "LLM_MODEL", "LLM_MAX_TOKENS",
"ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_BASE_URL", "ANTHROPIC_MODEL",
)
return [key for key in keys if os.environ.get(key)]
def _masked_api_key(configured: bool) -> str:
"""用于页面默认展示，固定掩码而非真实密钥或其长度。"""
return "********" if configured else ""
@router.get("")
def get_llm_config(db: Session = Depends(get_session)):
"""返回当前生效配置，密钥只返回是否已配置。"""
cfg = LLMConfig.load()
stored = db.get(LLMConfigRecord, 1)
return {
"base_url": cfg.effective_base_url,
"model": cfg.effective_model,
"max_tokens": cfg.max_tokens,
"timeout": cfg.timeout,
"api_key_configured": bool(cfg.api_key),
"api_key_masked": _masked_api_key(bool(cfg.api_key)),
"environment_overrides": _environment_override_names(),
"stored": bool(stored),
}
@router.post("/secret")
def reveal_llm_api_key(db: Session = Depends(get_session)):
"""仅在配置页用户主动点击查看时返回数据库中保存的密钥。"""
record = db.get(LLMConfigRecord, 1)
if not record or not record.api_key:
raise HTTPException(404, "数据库中未保存 API 密钥")
return {"api_key": record.api_key}
@router.put("")
def update_llm_config(payload: LLMConfigUpdate, db: Session = Depends(get_session)):
"""保存数据库配置，并让下一次 AI 请求重新创建客户端。"""
record = db.get(LLMConfigRecord, 1)
api_key = payload.api_key.strip() or (record.api_key.strip() if record else "")
if not api_key:
raise HTTPException(400, "请填写 API 密钥")
base_url = payload.base_url.strip().rstrip("/")
model = payload.model.strip()
if not base_url or not model:
raise HTTPException(400, "服务地址和模型名称不能为空")
if record is None:
record = LLMConfigRecord(id=1)
db.add(record)
record.api_key = api_key
record.base_url = base_url
record.model = model
record.max_tokens = payload.max_tokens
record.timeout = payload.timeout
db.commit()
reset_llm_client()
logger.info("LLM 数据库配置已更新: base_url=%s model=%s", base_url, model)
return {
"base_url": base_url,
"model": model,
"max_tokens": payload.max_tokens,
"timeout": payload.timeout,
"api_key_configured": True,
"api_key_masked": _masked_api_key(True),
"message": "模型配置已保存，下一次 AI 请求将使用新配置",
}
web/backend/routers/operations.py
View file @ 54429b71
......@@ -93,11 +93,11 @@ def _key(password: str, salt: bytes) -> bytes:
return base64.urlsafe_b64encode(kdf.derive(password.encode("utf-8")))
def _placeholders(db_type: str, count: int) -> list[str]:
def _placeholders(db_type: str, count: int, start: int = 1) -> list[str]:
if db_type == "mysql":
return ["%s"] * count
if db_type == "oracle":
return [f":{index}" for index in range(1, count + 1)]
return [f":{index}" for index in range(start, start + count)]
return ["?"] * count
......@@ -227,7 +227,10 @@ def _data_crypto(payload: DataCryptoRequest, db: Session, decrypt: bool = False)
if isinstance(old, str) and old.startswith("DBTOOLS$1$"):
continue
new = _crypto_pack_with_password(old, payload.password)
updates.append(f"{quote_ident(name, cfg.db_type)} = {_placeholders(cfg.db_type, 1)[0]}")
updates.append(
f"{quote_ident(name, cfg.db_type)} = "
f"{_placeholders(cfg.db_type, 1, start=len(update_values) + 1)[0]}"
)
update_values.append(new)
if not updates:
continue
......@@ -239,7 +242,10 @@ def _data_crypto(payload: DataCryptoRequest, db: Session, decrypt: bool = False)
if value is None:
where.append(f"{qname} IS NULL")
else:
where.append(f"{qname} = {_placeholders(cfg.db_type, 1)[0]}")
where.append(
f"{qname} = "
f"{_placeholders(cfg.db_type, 1, start=len(update_values) + len(where_values) + 1)[0]}"
)
where_values.append(value)
sql = f"UPDATE {qualified} SET {', '.join(updates)} WHERE {' AND '.join(where)}"
conn.execute(sql, tuple(update_values + where_values))
......
web/backend/routers/task_groups.py
View file @ 54429b71
......@@ -11,6 +11,8 @@
"""
from __future__ import annotations
from typing import List
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from sqlalchemy.exc import IntegrityError
......@@ -39,7 +41,7 @@ class TaskGroupCreate(BaseModel):
# ── 路由 ──
@router.get("", response_model=list[TaskGroupOut], summary="列出所有任务分组")
@router.get("", response_model=List[TaskGroupOut], summary="列出所有任务分组")
def list_task_groups(db: Session = Depends(get_session)) -> list[TaskGroupOut]:
rows = db.query(TaskGroup).order_by(TaskGroup.ord, TaskGroup.id).all()
return [TaskGroupOut(**g.to_dict()) for g in rows]
......@@ -55,4 +57,4 @@ def create_task_group(payload: TaskGroupCreate, db: Session = Depends(get_sessio
db.rollback()
raise HTTPException(status_code=409, detail=f"分组名已存在：{payload.name}")
db.refresh(g)
return TaskGroupOut(**g.to_dict())
\ No newline at end of file
return TaskGroupOut(**g.to_dict())
web/backend/routers/tasks.py
View file @ 54429b71
......@@ -19,7 +19,7 @@
from __future__ import annotations
import json
from typing import Optional
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field as PydField
......@@ -410,7 +410,7 @@ def _row_to_out(db: Session, t: Task, *, include_field_list: bool = False) -> Ta
# ── 路由 ──────────────────────────────────────────────────────────
@router.get("", response_model=list[TaskOut], summary="任务列表（支持 group/keyword 筛选）")
@router.get("", response_model=List[TaskOut], summary="任务列表（支持 group/keyword 筛选）")
def list_tasks(
group: Optional[str] = Query(None, description="按分组名过滤"),
keyword: Optional[str] = Query(None, description="按名称 / 表名模糊匹配"),
......
web/docs/db-schema.md
View file @ 54429b71
# web 数据库 DDL · 完整记录
# web 数据库结构说明
> **数据库文件**：`web/data/web.db`（SQLite）
> **当前业务库**：MySQL（由 `APP_ENV` 和 `web/backend/configs/database.*.yaml` 决定）。
> **历史兼容源**：旧版本的 `web/data/web.db` SQLite 文件，仅用于一次性迁移。
> **DDL 源文件**：[web/backend/db/schema.sql](../backend/db/schema.sql)
> **ORM 模型**：[web/backend/models/](../backend/models/)（`task.py` / `task_group.py`）
> **首次初始化**：`db/database.py::init_db()` —— 文件不存在时跑 schema.sql + seed
> **初始化入口**：`web/backend/db/database.py::init_db()` —— ORM 建表、兼容迁移、种子数据和中文注释同步。
---
......@@ -12,18 +13,17 @@
```
python -m web.backend.start
└─ app.py:init_db()
├─ 存在 web.db？ → 直接返回
└─ 不存在？
├─ 读 schema.sql 全文，按 ; 切语句，依次 exec_driver_sql 建表
└─ seed.seed_task_groups() 灌 6 个默认分组
├─ ORM Base.metadata.create_all() 补齐缺失表
├─ MySQL：补齐表注释并执行幂等种子
└─ SQLite：仅用于旧库兼容迁移
```
⚠️ **当前无 Alembic**：`init_db()` 只在 DB 文件不存在时建表；**已存在的 DB 文件不会被改**。
后续 schema 变更流程：
⚠️ **当前无 Alembic**：结构变更由 `database.py` 中的兼容迁移和 ORM `create_all()` 处理；生产环境不要直接删除 MySQL 数据库。
历史 SQLite 本地库如需迁移：
1. 删 `web/data/web.db`（开发环境直接重来）
2. 改 `schema.sql` / `seed.py` / ORM 模型
3. 重启后端 → 重建
1. 先备份 `web/data/web.db`。
2. 使用 `python -m web.backend.db.migrate_sqlite_to_mysql --source ... --skip-if-populated`。
3. 核对目标 MySQL 行数后再切换服务。
迁移机制后面接 Alembic 时再补。
......
web/docs/reference/README.md 0 → 100644
View file @ 54429b71
# 参考原型
这里保存早期需求页面原型，仅用于对照历史交互和默认分组，不参与 Vite 构建，也不作为线上页面入口。
web/plan.md
View file @ 54429b71
......@@ -3,8 +3,8 @@
> **文档状态：历史计划。** 当前功能和运行说明以 [README.md](README.md) 为准。
> 本文保留早期设计与演进记录，部分“待实现”描述已经过期，例如静态 `mockData` 已删除、数据源已提供 CRUD 页面、备份/还原/加密/解密已改为操作记录列表加新增弹框。
> 第三版 = 前端 Vue（已搭好）+ 后端 Python（FastAPI）+ SQLite（本地）+ 进程内异步任务（参考 web2）+ LLM（从 web1 拷贝，不直接引用）
> 数据落地在 `web/data/` 下，与代码同级目录。
> 本文记录早期 SQLite 方案，仅用于了解演进过程；当前运行架构已切换为 MySQL，详见 [README.md](README.md) 和 [docs/db-schema.md](docs/db-schema.md)。
> 早期版本 = 前端 Vue + 后端 Python（FastAPI）+ SQLite（本地）+ 进程内异步任务 + LLM。
> **不** 引入 Redis / Celery —— 单进程内网工具，没必要。
---
......
web/src/api/llmConfig.js 0 → 100644
View file @ 54429b71
import { http } from './client'
export const getLLMConfig = () => http.get('/llm-config')
export const revealLLMApiKey = () => http.post('/llm-config/secret')
export const saveLLMConfig = (payload) => http.put('/llm-config', payload)
web/src/components/TaskModal.vue
View file @ 54429b71
......@@ -163,10 +163,10 @@
<!-- Tab3 校验规则（字段列表） -->
<div class="tab-panel single-page-panel" :class="{ active: activeTab === 'rule' }">
<div v-if="props.task && form.table && fieldList.length > 0" class="form-hint" style="margin-bottom: 12px; color: #606266; font-size: 13px; display: flex; align-items: center; justify-content: flex-end; gap: 12px;">
<div v-if="props.task && form.table" class="form-hint" style="margin-bottom: 12px; color: #606266; font-size: 13px; display: flex; align-items: center; justify-content: flex-end; gap: 12px;">
<!-- 编辑场景：用现有 conn_json 重新拉一次 DB 列元数据，覆盖 cn/type；DB 中途变更后手动刷新 -->
<button
v-if="props.task && form.table && fieldList.length > 0"
v-if="props.task && form.table"
type="button"
class="btn"
style="padding: 3px 10px; font-size: 12px;"
......@@ -346,7 +346,7 @@ const loadingTable = ref(false)
const loadingColumns = ref(false)
// 编辑场景「重新同步字段」按钮专用：与 loadColumns 的 loadingColumns 解耦——
// loadColumns 会把整张表藏掉（listColumns → 重建 fieldList）
// refreshColumnsForEdit 只更新 cn/type，表格保留可见，按钮显示「同步中…」
// refreshColumnsForEdit 原地替换整张字段表，表格保留可见，按钮显示「同步中…」
const refreshingColumns = ref(false)
const form = reactive({
......@@ -547,13 +547,11 @@ watch(() => form.schema, (newVal, oldVal) => {
// 设计依据：cn / type 故意不持久化（plan §4 + 用户原话），编辑时按 source_table 重拉；
// 但用户每次都得手动点「测试连接」体验差，本函数把「打开弹窗」就当成一次隐式刷新。
// 行为约定：
// - 不动 stored rules（按 field_key 索引合并），只覆盖 cn / type / isNullable
// - DB 中已不存在的字段 → 保留 stored 状态（cn='' type=''），顶部 warning
// - DB 中新出现的字段（没有 stored rules）→ 不主动加，避免污染用户视角；
// 真要加新字段，用户改 form.table 走 loadColumns 那条路会自然带上
// - 以数据库返回的列为准重建字段列表，确保编辑场景和新增场景展示整张表
// - 按 field_key 合并已保存的 rules / ruleIds / showDefault，避免同步时丢配置
// - DB 中已不存在的字段不再继续展示；保存时也不会把已删除字段写回任务
// - API 失败（密码空 / DB 不通 / 表被删）→ 静默退化（用户没主动触发，自动同步失败不该打扰）
// - 与 loadColumns 解耦：loadColumns 是「重建 fieldList（rules 备份后回填）」，
// 本函数是「in-place 覆盖 cn/type」，共存无冲突
// - 与 loadColumns 解耦：loadColumns 用于用户切换表，本函数用于编辑场景同步当前表
async function refreshColumnsForEdit() {
if (!props.task?.source_table) return // 新建任务 / 旧任务没存表名 → 跳过
// 2026-09-03：MySQL 的 schema 就是 database。老任务 conn_json 可能只存了 schema（如 db-test）
......@@ -572,27 +570,22 @@ async function refreshColumnsForEdit() {
// 表已删 / DB 不通 / 列表为空 → 静默（不影响旧数据展示）
return
}
// 按 column_name 索引，方便 O(1) 查找 merge
const colByKey = new Map(r.columns.map((c) => [c.column_name, c]))
let missingCount = 0
fieldList.value = fieldList.value.map((f) => {
const c = colByKey.get(f.key)
if (!c) {
missingCount++
return f // stored 字段已不在 DB → 原样保留
}
// 按字段名合并旧配置。数据库驱动可能改变字段名大小写，因此同时保留小写索引。
const savedByKey = new Map(fieldList.value.map((f) => [String(f.key).toLowerCase(), f]))
fieldList.value = r.columns.map((c) => {
const key = c.column_name
const saved = savedByKey.get(String(key).toLowerCase())
return {
...f,
cn: c.column_comment || '', // 字段注释（无 → 空串）
type: c.column_type || c.data_type || '', // 数据类型显示（VARCHAR(50) 等）
isNullable: c.is_nullable, // Y/N（后续 P4 必填校验可能用）
key,
en: key,
cn: c.column_comment || '',
type: c.column_type || c.data_type || '',
isNullable: c.is_nullable,
rules: saved?.rules || [],
ruleIds: saved?.ruleIds || [],
showDefault: saved ? saved.showDefault !== false : true,
}
})
if (missingCount > 0) {
ElMessage.warning(
`有 ${missingCount} 个字段在数据库「${props.task.source_table}」中已不存在，请检查（字段注释/类型暂留空）`
)
}
} catch (e) {
// 静默：不弹错误（自动同步失败不该打扰；用户可手动去「数据连接」Tab 测试）
console.warn('[TaskModal] 自动同步字段元数据失败：', e)
......@@ -777,7 +770,7 @@ watch(() => props.open, async (v) => {
isInitializing.value = false
// 2026-09-03：详情在初始化期间到达（field_list watch 里挂起的那次同步）→
// form 填完后再补跑，保证 refreshColumnsForEdit 用的是本次任务的完整连接信息
if (pendingColumnSync.value) {
if (pendingColumnSync.value || (props.task && form.table)) {
pendingColumnSync.value = false
refreshColumnsForEdit()
}
......
web/src/layouts/MainLayout.vue
View file @ 54429b71
......@@ -43,6 +43,7 @@
<el-menu-item index="/config/data-sources"><template #title>数据源</template></el-menu-item>
<el-menu-item index="/config/rule-config"><template #title>规则设置</template></el-menu-item>
<el-menu-item index="/config/system"><template #title>系统配置</template></el-menu-item>
<el-menu-item index="/config/model"><template #title>模型配置</template></el-menu-item>
</el-sub-menu>
</el-menu>
<div class="sidebar-footer">
......@@ -105,6 +106,7 @@ const currentTitle = computed(() => {
if (route.path === '/config/rule-config') return '规则设置'
if (route.path === '/config/data-sources') return '数据源'
if (route.path === '/config/system') return '系统配置'
if (route.path === '/config/model') return '模型配置'
if (route.path === '/backup') return '备份'
if (route.path === '/restore') return '还原'
if (route.path === '/clear-tables') return '清表'
......
web/src/router/index.js
View file @ 54429b71
......@@ -7,6 +7,7 @@ import DataSourceView from '@/views/DataSourceView.vue'
import OperationView from '@/views/OperationView.vue'
import ClearTablesView from '@/views/ClearTablesView.vue'
import SystemConfigView from '@/views/SystemConfigView.vue'
import ModelConfigView from '@/views/ModelConfigView.vue'
import RuleLibraryView from '@/views/RuleLibraryView.vue'
import TableDesignView from '@/views/TableDesignView.vue'
import TableDesignResultsView from '@/views/TableDesignResultsView.vue'
......@@ -37,6 +38,7 @@ const routes = [
},
{ path: 'config/data-sources', name: 'DataSources', component: DataSourceView, meta: { title: '数据源' } },
{ path: 'config/system', name: 'SystemConfig', component: SystemConfigView, meta: { title: '系统配置' } },
{ path: 'config/model', name: 'ModelConfig', component: ModelConfigView, meta: { title: '模型配置' } },
{ path: 'backup', name: 'Backup', component: OperationView, props: { type: 'backup' }, meta: { title: '备份' } },
{ path: 'restore', name: 'Restore', component: OperationView, props: { type: 'restore' }, meta: { title: '还原' } },
{ path: 'clear-tables', name: 'ClearTables', component: ClearTablesView, meta: { title: '清表' } },
......
web/src/utils/excel.js
View file @ 54429b71
......@@ -2,23 +2,19 @@
// 通用 Excel 导出工具（web2 IdCardTable 抽出来）
//
// 设计要点：
// - 复用 web2 IdCardTable.vue 里的「注释行 + 字段名行 + 数据」三行表头格式
// - 导出内容与查询列表一致：一行字段表头 + 数据行
// - 用 xlsx-js-style（支持 cell.s 边框 + 字体 + 背景 + cell.c 备注）
// - 列定义兼容两种来源：
// ① 直接传 [{ key, cn, comment? }]（resultFields 直接 map 即可）
// ② 传 string[]（仅字段名，无注释行）
// - rows 里的字段值走 row[col.key] 抽取，null/undefined → ''（避免 Excel 显示 undefined）
// - rows 里的字段值走 row[col.key] 抽取，空值统一显示为 (NULL)（与页面结果表一致）
//
// 字段命名（comment）约定：每个字段列头显示两行
// 第 1 行：字段注释（column_comment，缺则留空）
// 第 2 行：字段显示名（cn —— 中文表头；用户原始字段名为保留显示给业务人员）
// 这样业务人员打开 Excel 一眼能看懂字段含义，也能看到 DB 里的实际字段名
// 字段表头使用页面查询列表中的字段名；字段注释不再单独导出。
//
// 违规高亮（2026-08-24）：
// - rows 上若带 row.errorCells: [key,...] + row.issues: [{field, name, desc},...]
// 会给对应单元格加浅红背景（与页面上 .cell-error 同色 #FEF0F0）
// 并写 cell note（Review → Note）：列出违反的规则名 + 规则详细
// - 跟页面表现一致 —— 用户在 Excel 里看到红格 + 悬浮 note 就知道这条违规是为什么
// 违规原因统一放在「备注」列，不写 Excel 批注。
// ============================================================================
import * as XLSX from 'xlsx-js-style'
......@@ -64,31 +60,30 @@ export async function exportToExcel(opts) {
// 列归一：拿到 [{ key, cn, comment }]
const cols = normalizeColumns(columns)
// 第 1 行：注释（comment 缺省时留空字符串，不留 null —— Excel 不喜欢 null）
const commentRow = cols.map((c) => c.comment || '')
// 第 2 行：字段显示名（cn —— 中文表头）
// 表头：与页面查询列表一致
const nameRow = cols.map((c) => c.cn || c.key)
// 数据行：按 col.key 抽取，null/undefined → ''（xlsx 把 null 显示成 #NULL!，不可读）
// 数据行：按 col.key 抽取；备注列按页面 ResultTable 的 issues 格式拼接。
// 空值显示为 (NULL)，与页面结果表的 formatCell 保持一致。
const dataRows = rows.map((row) =>
cols.map((c) => {
if (c.key === '__reason') return formatIssues(row.issues)
const v = row[c.key]
return v === null || v === undefined ? '' : v
return v === null || v === undefined || v === '' ? '(NULL)' : v
}),
)
const aoa = [commentRow, nameRow, ...dataRows]
const aoa = [nameRow, ...dataRows]
const ws = XLSX.utils.aoa_to_sheet(aoa)
// 样式：所有 cell 加细边框 + 前两行加粗 + 浅灰背景
// 样式：所有 cell 加细边框 + 表头加粗 + 浅灰背景
applyAllBorders(ws)
applyHeaderStyle(ws, cols.length)
// 违规高亮 + cell note（依赖前两行已经定好，下面用 (r,c) 偏移算到数据行）
// 数据行从第 2 行开始（0-based: 0=commentRow, 1=nameRow, 2=第一条数据）
applyErrorHighlight(ws, rows, cols, 2)
// 违规高亮（数据行从第 1 行开始，0-based）
applyErrorHighlight(ws, rows, cols, 1)
// 冻结前 2 行（滚动时表头固定）
ws['!freeze'] = { xSplit: 0, ySplit: 2 }
// 冻结表头（滚动时表头固定）
ws['!freeze'] = { xSplit: 0, ySplit: 1 }
const wb = XLSX.utils.book_new()
const safeSheetName = sanitizeSheetName(sheetName)
......@@ -175,10 +170,10 @@ function applyAllBorders(ws) {
}
/**
* 给前两行（注释行 + 字段名行）加粗 + 浅灰背景 + 居中 + 自动换行
* 给字段表头加粗 + 浅灰背景 + 居中 + 自动换行
*/
function applyHeaderStyle(ws, cols) {
for (let R = 0; R < 2; R++) {
for (let R = 0; R < 1; R++) {
for (let C = 0; C < cols; C++) {
const addr = XLSX.utils.encode_cell({ r: R, c: C })
const cell = ws[addr]
......@@ -196,13 +191,12 @@ function applyHeaderStyle(ws, cols) {
/**
* 违规高亮（2026-08-24）：
* - 给违反规则的单元格加浅红背景（与页面上 .cell-error 同色 #FEF0F0）
* - 给单元格写 Excel 备注（Review → Note）：列出违反的规则名 + 规则详细
*
* 入参：
* ws: xlsx worksheet
* rows: 原始数据行（与生成 aoa 用同一份），含 row.errorCells / row.issues
* cols: 归一后的列定义 [{ key, cn, comment }]
* startRow: 数据行在 sheet 里的起始行（0-based）；前两行是注释 + 字段名，所以是 2
* startRow: 数据行在 sheet 里的起始行（0-based）；当前表头占 1 行，所以是 1
*
* 行级映射：aoa 第 i 行（i>=startRow）对应 rows[i-startRow]
* 列级映射：col C 对应 cols[C].key，去 rows 找 row.errorCells.includes(cols[C].key)
......@@ -234,19 +228,23 @@ function applyErrorHighlight(ws, rows, cols, startRow) {
font: { ...(cell.s?.font || {}), ...ERROR_FONT },
}
// 写 Excel 备注（Review → Note）
const issues = (row.issues || []).filter((it) => it && it.field === colKey)
if (issues.length > 0) {
const noteText = issues
.map((it) => {
// 跟 ResultTable.errTip 同款：name + desc 都列（note 空间够，比 cell tooltip 宽松）
const tag = it.name && it.name.trim() ? it.name : '(未命名规则)'
const desc = it.desc || ''
return desc ? `${tag}：${desc}` : tag
})
.join('\n')
cell.c = [{ t: noteText, a: '数据治理工具' }]
}
}
}
}
\ No newline at end of file
}
/**
* 与 ResultTable 的备注列保持同一文案：每条违规一行，规则名和说明同时展示。
*/
function formatIssues(issues) {
if (!Array.isArray(issues) || issues.length === 0) return ''
return issues
.map((issue) => {
const name = (issue?.name || '').trim()
const desc = (issue?.desc || '').trim()
const summary = name && desc && name !== desc
? `${name}（${desc}）`
: (name || desc || '该字段不合法')
return `• ${issue?.field || ''}：${summary}`
})
.join('\n')
}
web/src/views/DataQualityView.vue
View file @ 54429b71
......@@ -19,6 +19,9 @@
@click="onQuery"
>{{ runningQuery ? '扫描中…' : '开始扫描' }}</button>
<button class="btn" @click="backToList">返回列表</button>
<button class="btn" :disabled="!rows.length || exporting" @click="onExport">
{{ exporting ? '导出中…' : '导出 Excel' }}
</button>
<!-- 中断按钮：仅查询中可见 -->
<button
v-if="runningQuery"
......@@ -50,21 +53,8 @@
>💡 点击单元格的红色 <b>！</b> 图标进行 AI 分析</span>
</div>
<!-- 结果区：表头操作 + 通用结果表 -->
<!-- 结果区：通用结果表 -->
<div class="q-card">
<div class="q-card__header">
<div class="q-card__title">
不合规数据明细
<span class="count" v-if="lastResult">共 {{ lastResult.bad_rows }} 条</span>
</div>
<div class="q-card__actions">
<button class="btn" @click="fieldDrawerOpen = true">字段配置</button>
<button class="btn" :disabled="!rows.length || exporting" @click="onExport">
{{ exporting ? '导出中…' : '导出 Excel' }}
</button>
</div>
</div>
<!-- 查询失败 / 警告：横幅 -->
<div v-if="queryError" class="query-error">
<el-icon><WarningFilled /></el-icon>
......@@ -74,21 +64,12 @@
<ResultTable
:columns="resultFields"
:rows="rows"
:visible-fields="visibleFields"
:append-mode="true"
:row-key="rowKeyFn"
@explain-issue="onExplainIssue"
/>
</div>
<!-- 字段配置抽屉 -->
<FieldConfigDrawer
v-model:open="fieldDrawerOpen"
:fields="fields"
:visible-fields="visibleFields"
@save="onSaveFields"
/>
<!-- 2026-08-24：点「!」一键问 LLM —— 解释违规原因 + 审查规则 vs 代码一致性 -->
<ExplainIssueDialog
v-model:open="explainDialogOpen"
......@@ -103,7 +84,6 @@ import { ElMessage, ElMessageBox } from 'element-plus'
import { listTasks } from '@/api/tasks'
import { startQuery, fetchPage, cancelQuery } from '@/api/queries'
import { exportToExcel } from '@/utils/excel'
import FieldConfigDrawer from '@/components/FieldConfigDrawer.vue'
import ResultTable from '@/components/ResultTable.vue'
import ExplainIssueDialog from '@/components/ExplainIssueDialog.vue'
......@@ -150,7 +130,7 @@ async function loadTasksForCurrentType() {
// ── 分组快照缓存（2026-08-24）──
// 切走再切回时保留上次结果：
// - taskOptions / selectedTask / rows / resultFields / visibleFields / lastResult / fieldDrawerOpen
// - taskOptions / selectedTask / rows / resultFields / lastResult
// - 不缓存 session / 进度（in-flight 跨分组不连续；用户切回后想拿新结果点「查询」即可）
// - 用普通 Map 而不是 ref，cache 不参与渲染，只在 watch 里读写
const cacheByType = new Map()
......@@ -167,9 +147,7 @@ function takeSnapshot(typeKey) {
taskOptions: taskOptions.value,
rows: rows.value,
resultFields: resultFields.value,
visibleFields: visibleFields.value,
lastResult: lastResult.value || partial,
fieldDrawerOpen: fieldDrawerOpen.value,
// 续跑所需：in-flight 状态下用，新 session 重新拉剩下的页
inFlight: runningQuery.value ? {
taskId: taskOptions.value.find((t) => t.name === selectedTask.value)?.id,
......@@ -194,9 +172,7 @@ function applySnapshot(snap) {
taskOptions.value = []
rows.value = []
resultFields.value = []
visibleFields.value = []
lastResult.value = null
fieldDrawerOpen.value = false
return
}
// 切回来：恢复上次离开时的状态
......@@ -204,9 +180,7 @@ function applySnapshot(snap) {
taskOptions.value = snap.taskOptions
rows.value = snap.rows
resultFields.value = snap.resultFields
visibleFields.value = snap.visibleFields
lastResult.value = snap.lastResult
fieldDrawerOpen.value = snap.fieldDrawerOpen
// 恢复 in-flight 进度（fire-and-forget 启动续跑）
if (snap.inFlight) {
receivedPages.value = new Map(snap.inFlight.receivedPages)
......@@ -291,25 +265,6 @@ const queryError = ref('')
const rows = ref([]) // 喂给 ResultTable 的行
const resultFields = ref([])
// 字段展示开关 —— 跟 resultFields 联动
// 「字段配置」抽屉列出全部 + 默认勾上 show_default=true 的列；
// 用户手动改勾选后写入 visibleFields，ResultTable 按它过滤（优先于 show_default）
// —— 必须在 resultFields 声明之后定义，否则 TDZ 报错（computed/watch 会立即执行）
const fields = computed(() => resultFields.value)
const visibleFields = ref([])
// 按 resultFields 当前内容，重置 visibleFields 为 showDefault=true 的列
// 2026-08-24：原来是 watch(resultFields) 自动触发，但切换分组恢复快照时
// applySnapshot 把 resultFields.value 设回旧值，watch 会在 microtask 阶段把
// visibleFields 重置成「showDefault 过滤」，覆盖用户保存的选择。
// 改成显式调用：onQuery 拉完字段后调一次；applySnapshot 不调。
function resetVisibleFields() {
visibleFields.value = (resultFields.value || [])
.filter((f) => f.showDefault !== false)
.map((f) => f.key)
}
resetVisibleFields() // 首次：resultFields 还是 []，得到 []
// 行 key：用后端注入的全局行号（append 模式下保证 Vue DOM 复用正确）
const rowKeyFn = (row) => row.__row_index
......@@ -421,8 +376,6 @@ async function onQuery() {
ruleList: f.rule_list || [],
showDefault: f.show_default !== false,
}))
resetVisibleFields() // 按新字段的 show_default 重置可见列
if (start.total_pages === 0) {
// COUNT=0：表里没数据
lastResult.value = { scanned: 0, bad_rows: 0, rows: [] }
......@@ -492,8 +445,7 @@ function onExplainIssue(payload) {
}
// ── 导出 Excel ──
// 字段范围：优先用 FieldConfigDrawer 选定的 visibleFields；
// 为空（用户没保存过 / 默认全勾）时退化为 resultFields 全量
// 字段范围与 ResultTable 一致：使用 show_default=true 的当前结果字段，末尾追加备注列。
const exporting = ref(false)
async function onExport() {
......@@ -501,14 +453,12 @@ async function onExport() {
ElMessage.warning('当前表无数据，无需导出')
return
}
// 决定导出哪些列 —— visibleFields 非空 → 用户勾选；空 → 全部 resultFields
// 过滤掉内部 __reason（不在 resultFields 里，但可见列表里可能含，防止误传）
const fieldSet = new Set(visibleFields.value || [])
const exportCols = resultFields.value.filter(
(c) => c.key !== '__reason' && (fieldSet.size === 0 || fieldSet.has(c.key)),
)
if (!exportCols.length) {
ElMessage.warning('未勾选任何字段，请先在「字段配置」里勾选要导出的列')
const exportCols = [
...resultFields.value.filter((c) => c.key !== '__reason' && c.showDefault !== false),
{ key: '__reason', cn: '备注' },
]
if (exportCols.length <= 1) {
ElMessage.warning('当前没有可导出的字段')
return
}
......@@ -552,13 +502,6 @@ async function onExport() {
}
}
function onSaveFields(list) {
visibleFields.value = list
fieldDrawerOpen.value = false
ElMessage.success('已保存展示字段')
}
const fieldDrawerOpen = ref(false)
// 切换分组（左侧菜单）：
// 0. 如果有 in-flight → 阻止切换 + 弹警告 + router.replace 回老 type（让用户先中断或等待）
// 1. 取消老 in-flight（兜底：用户没 in-flight 时也走一遍）
......@@ -572,8 +515,8 @@ const fieldDrawerOpen = ref(false)
// 业务场景下用户完全可以接受：要么等跑完再切，要么主动点「中断」。
// typeReverting：router.replace 改 route 后 watch 会再触发一次（newType=oldType 短暂回到
// 原值），这个标志位让二次触发直接跳过，避免循环 router.replace
// 注意：放这里是因为 watch 内部引用了 abortInFlight / resultFields / visibleFields /
// fieldDrawerOpen —— 这些都得在 watch 注册前声明完，避免 TDZ
// 注意：放这里是因为 watch 内部引用了 abortInFlight / resultFields，
// 这些都得在 watch 注册前声明完，避免 TDZ
const typeReverting = ref(false)
watch(type, async (newType, oldType) => {
// router.replace 回退触发的二次 watch 跳过
......
web/src/views/DataSourceView.vue
View file @ 54429b71
......@@ -30,7 +30,7 @@
<button class="modal-close" aria-label="关闭" @click="dialog = false">×</button>
</div>
<div class="modal-body data-source-modal__body">
<el-form :model="form" label-width="110px"><el-form-item label="名称" required><el-input v-model="form.name" placeholder="例如：生产库" /></el-form-item><el-form-item label="数据库类型"><el-select v-model="form.db_type" style="width:100%"><el-option label="MySQL" value="MySQL"/><el-option label="Oracle" value="Oracle"/><el-option label="达梦 DM" value="达梦 DM"/></el-select></el-form-item><el-form-item label="主机" required><el-input v-model="form.host"/></el-form-item><el-form-item label="端口" required><el-input v-model.number="form.port" type="number"/></el-form-item><el-form-item label="数据库"><el-input v-model="form.database"/></el-form-item><el-form-item label="Schema"><el-input v-model="form.schema"/></el-form-item><el-form-item label="用户名" required><el-input v-model="form.user"/></el-form-item><el-form-item label="密码"><el-input v-model="form.password" type="password" show-password placeholder="编辑时留空表示保持原密码"/></el-form-item><el-form-item label="状态"><el-switch v-model="enabled" active-text="启用" inactive-text="停用"/></el-form-item></el-form>
<el-form :model="form" label-width="110px"><el-form-item label="名称" required><el-input v-model="form.name" placeholder="例如：生产库" /></el-form-item><el-form-item label="数据库类型"><el-select v-model="form.db_type" style="width:100%"><el-option label="MySQL" value="MySQL"/><el-option label="Oracle" value="Oracle"/><el-option label="达梦 DM" value="达梦 DM"/></el-select></el-form-item><el-form-item label="主机" required><el-input v-model="form.host"/></el-form-item><el-form-item label="端口" required><el-input v-model.number="form.port" type="number"/></el-form-item><el-form-item label="数据库"><el-input v-model="form.database" :placeholder="form.db_type === 'Oracle' ? 'Oracle 服务名，例如 ZFGJJ' : ''"/></el-form-item><el-form-item v-if="form.db_type !== 'MySQL'" label="Schema"><el-input v-model="form.schema" :placeholder="form.db_type === 'Oracle' ? 'Oracle 用户名对应的 Schema' : '模式名'"/></el-form-item><el-form-item label="用户名" required><el-input v-model="form.user"/></el-form-item><el-form-item label="密码"><el-input v-model="form.password" type="password" show-password placeholder="编辑时留空表示保持原密码"/></el-form-item><el-form-item label="状态"><el-switch v-model="enabled" active-text="启用" inactive-text="停用"/></el-form-item></el-form>
</div>
<div class="modal-footer">
<button class="btn" @click="dialog = false">取消</button>
......@@ -49,9 +49,9 @@ const keyword = ref('')
const appliedKeyword = ref('')
const currentPage = ref(1)
const pageSize = ref(10)
const form = reactive({ name:'', db_type:'MySQL', host:'', port:3306, database:'', schema:'', user:'', password:'', status:'启用' })
const form = reactive({ name:'', db_type:'MySQL', host:'', port:3306, database:'', schema:'', oracle_client_dir:'', user:'', password:'', status:'启用' })
const enabled = computed({ get:()=>form.status === '启用', set:v=>form.status = v ? '启用' : '停用' })
function reset(row){ Object.assign(form, row ? {...row, password:''} : {name:'',db_type:'MySQL',host:'',port:3306,database:'',schema:'',user:'',password:'',status:'启用'}) }
function reset(row){ Object.assign(form, row ? {...row, password:''} : {name:'',db_type:'MySQL',host:'',port:3306,database:'',schema:'',oracle_client_dir:'',user:'',password:'',status:'启用'}) }
function openAdd(){ editing.value=null; reset(); dialog.value=true }
function openEdit(row){ editing.value=row; reset(row); dialog.value=true }
const filteredRows = computed(() => {
......
web/src/views/ModelConfigView.vue 0 → 100644
View file @ 54429b71
<template>
<div class="config-page">
<div class="q-card model-panel">
<div class="q-card__header">
<span class="q-card__title">模型配置</span>
<span class="status" :class="configured ? 'ready' : 'missing'">{{ configured ? '已配置' : '未配置密钥' }}</span>
</div>
<div class="config-form">
<div class="config-row">
<label>服务地址</label>
<div class="config-field">
<input v-model.trim="form.base_url" class="text-input" placeholder="http://服务地址:端口" />
<div class="hint">使用 Anthropic Messages 兼容接口的服务根地址。</div>
</div>
</div>
<div class="config-row">
<label>模型名称</label>
<div class="config-field"><input v-model.trim="form.model" class="text-input" placeholder="例如：glm-5.3-flash" /></div>
</div>
<div class="config-row">
<label>API 密钥</label>
<div class="config-field">
<div class="secret-field">
<input :value="secretVisible ? form.api_key : maskedApiKey" class="text-input" :type="secretVisible ? 'text' : 'text'" :readonly="!secretVisible" :placeholder="configured ? '' : '请输入 API 密钥'" autocomplete="new-password" @input="form.api_key = $event.target.value" />
<button class="secret-toggle" type="button" :disabled="revealing" :title="secretVisible ? '隐藏密钥' : '查看已保存密钥'" :aria-label="secretVisible ? '隐藏密钥' : '查看已保存密钥'" @click="toggleSecret"><el-icon><Hide v-if="secretVisible" /><View v-else /></el-icon></button>
</div>
<div class="hint">默认隐藏。点击眼睛图标查看已保存密钥，再次点击立即隐藏。</div>
</div>
</div>
<div class="config-row compact-row">
<label>最大输出 Token</label>
<div class="config-field"><input v-model.number="form.max_tokens" class="text-input number-input" type="number" min="256" max="16384" /></div>
</div>
<div class="config-row compact-row">
<label>请求超时（秒）</label>
<div class="config-field"><input v-model.number="form.timeout" class="text-input number-input" type="number" min="5" max="300" /></div>
</div>
<div v-if="environmentOverrides.length" class="override-note">
检测到服务环境变量覆盖：{{ environmentOverrides.join('、') }}。移除对应环境变量后，页面保存的项目配置才会生效。
</div>
<div class="config-actions"><button class="btn btn-primary" :disabled="saving || loading" @click="save">{{ saving ? '保存中...' : '保存模型配置' }}</button></div>
</div>
</div>
</div>
</template>
<script setup>
import { onMounted, reactive, ref } from 'vue'
import { ElMessage } from 'element-plus'
import { getLLMConfig, revealLLMApiKey, saveLLMConfig } from '@/api/llmConfig'
const form = reactive({ base_url: '', model: '', api_key: '', max_tokens: 1024, timeout: 30 })
const loading = ref(false)
const saving = ref(false)
const configured = ref(false)
const environmentOverrides = ref([])
const secretVisible = ref(false)
const revealing = ref(false)
const maskedApiKey = ref('')
async function load() {
loading.value = true
try {
const data = await getLLMConfig()
form.base_url = data.base_url || ''
form.model = data.model || ''
form.max_tokens = data.max_tokens || 1024
form.timeout = data.timeout || 30
form.api_key = ''
secretVisible.value = false
configured.value = Boolean(data.api_key_configured)
maskedApiKey.value = data.api_key_masked || ''
environmentOverrides.value = data.environment_overrides || []
} catch (error) {
ElMessage.error(`加载模型配置失败：${error.message}`)
} finally {
loading.value = false
}
}
async function toggleSecret() {
if (secretVisible.value) {
secretVisible.value = false
return
}
if (form.api_key) {
secretVisible.value = true
return
}
revealing.value = true
try {
const data = await revealLLMApiKey()
form.api_key = data.api_key || ''
secretVisible.value = true
} catch (error) {
ElMessage.error(`读取密钥失败：${error.message}`)
} finally {
revealing.value = false
}
}
async function save() {
if (!form.base_url || !form.model) {
ElMessage.warning('请填写服务地址和模型名称')
return
}
saving.value = true
try {
const data = await saveLLMConfig({ ...form })
configured.value = Boolean(data.api_key_configured)
form.api_key = ''
secretVisible.value = false
maskedApiKey.value = data.api_key_masked || maskedApiKey.value
ElMessage.success(data.message || '模型配置已保存')
await load()
} catch (error) {
ElMessage.error(`保存失败：${error.message}`)
} finally {
saving.value = false
}
}
onMounted(load)
</script>
<style scoped>
.model-panel { max-width: 940px; }
.q-card__header { display: flex; justify-content: space-between; align-items: center; }
.config-form { padding: 18px 24px 24px; }
.config-row { display: flex; align-items: flex-start; gap: 18px; padding: 14px 0; border-bottom: 1px solid var(--app-border-soft); }
.config-row label { width: 150px; flex: 0 0 150px; line-height: 40px; color: var(--app-text); }
.config-field { flex: 1; min-width: 0; }
.config-field .text-input { width: min(600px, 100%); box-sizing: border-box; }
.secret-field { display: flex; width: min(600px, 100%); }
.secret-field .text-input { width: 100%; padding-right: 46px; }
.secret-field .text-input[readonly] { cursor: default; color: var(--app-text-secondary) !important; letter-spacing: 2px; }
.secret-toggle { width: 40px; height: 40px; margin-left: -40px; padding: 0; display: inline-flex; align-items: center; justify-content: center; color: var(--app-text-secondary); border: 0; background: transparent; cursor: pointer; z-index: 1; }
.secret-toggle:hover:not(:disabled) { color: #fff; }
.secret-toggle:focus-visible { outline: 2px solid var(--app-primary); outline-offset: -3px; }
.secret-toggle:disabled { cursor: wait; opacity: .55; }
.number-input { width: 160px !important; }
.hint { margin-top: 7px; color: var(--app-text-muted); font-size: 12px; line-height: 1.5; }
.compact-row { padding: 12px 0; }
.config-actions { padding-top: 20px; display: flex; justify-content: flex-end; }
.status { font-size: 13px; padding: 3px 9px; border: 1px solid; border-radius: 3px; }
.status.ready { color: #76df96; border-color: rgba(36, 202, 73, .45); background: rgba(36, 202, 73, .12); }
.status.missing { color: #ffd27b; border-color: rgba(247, 185, 64, .45); background: rgba(247, 185, 64, .12); }
.override-note { margin-top: 16px; padding: 10px 12px; color: #ffd27b; background: rgba(247, 185, 64, .12); border: 1px solid rgba(247, 185, 64, .4); font-size: 13px; line-height: 1.5; }
@media (max-width: 680px) { .config-row { display: block; } .config-row label { display: block; width: auto; line-height: 1.4; margin-bottom: 8px; } .config-actions { justify-content: flex-start; } }
</style>
web/src/views/TableDesignResultsView.vue
View file @ 54429b71
......@@ -11,6 +11,9 @@
</select>
<button class="btn btn-primary" :disabled="!selectedConfigId || loading" @click="reload">{{ loading ? '扫描中…' : '开始扫描' }}</button>
<button class="btn" @click="back">返回列表</button>
<button class="btn" :disabled="!result?.results?.length || loading || exporting" @click="onExport">
{{ exporting ? '导出中…' : '导出 Excel' }}
</button>
</div>
</div>
</div>
......@@ -24,7 +27,6 @@
</div>
<div class="q-card">
<div class="q-card__header"><div class="q-card__title">表设计校验结果 <span v-if="result" class="count">共 {{ result.total }} 张表</span></div></div>
<div v-if="error" class="query-error">{{ error }}</div>
<div v-if="loading" class="empty-cell">正在扫描，请稍候…</div>
<div v-else-if="!result" class="empty-cell">请选择校验配置并点击“开始扫描”</div>
......@@ -36,15 +38,46 @@
<script setup>
import { onMounted, ref } from 'vue'
import { useRoute, useRouter } from 'vue-router'
import { ElMessage } from 'element-plus'
import { ElMessage, ElMessageBox } from 'element-plus'
import { checkTableDesign, getTableDesignConfig, listTableDesignConfigs } from '@/api/tableDesign'
import { exportToExcel } from '@/utils/excel'
const route = useRoute(); const router = useRouter()
const configs = ref([]); const selectedConfigId = ref(''); const selectedConfig = ref(null); const result = ref(null); const loading = ref(false); const loadingConfigs = ref(false); const error = ref('')
const configs = ref([]); const selectedConfigId = ref(''); const selectedConfig = ref(null); const result = ref(null); const loading = ref(false); const loadingConfigs = ref(false); const exporting = ref(false); const error = ref('')
function setConfig(id) { selectedConfigId.value = String(id || ''); selectedConfig.value = configs.value.find((x) => String(x.id) === selectedConfigId.value) || null; result.value = selectedConfig.value?.lastResult || null; error.value = '' }
async function onConfigChange() { setConfig(selectedConfigId.value); if (selectedConfig.value?.lastResult) return }
async function reload() { if (!selectedConfig.value) return; loading.value = true; error.value = ''; try { const c = await getTableDesignConfig(selectedConfig.value.id); selectedConfig.value = c; const r = await checkTableDesign({ config_id: c.id, source_id: c.sourceId, schema: c.schema, scope: c.scope, tables: c.tables, check_table_comment: c.checkTableComment, check_column_comment: c.checkColumnComment }); if (!r.ok) throw new Error(r.message); result.value = r; const i = configs.value.findIndex((x) => x.id === c.id); if (i >= 0) configs.value[i] = { ...c, lastResult: r } ; ElMessage.success(`扫描完成：共 ${r.total} 张表，不合格 ${r.failed} 张`) } catch (e) { error.value = e.message; ElMessage.error(`扫描失败：${e.message}`) } finally { loading.value = false } }
function reasonText(row) { const reasons = Array.isArray(row.reasons) ? row.reasons : []; const missing = row.missing_columns?.length ? `字段：${row.missing_columns.join('、')}` : ''; return [...reasons, missing].filter(Boolean).join('\n') || '—' }
async function onExport() {
const rows = result.value?.results || []
if (!rows.length) { ElMessage.warning('当前没有可导出的扫描结果'); return }
const stamp = new Date().toISOString().replace(/[-:T]/g, '').slice(0, 14)
const filename = `${selectedConfig.value?.name || 'table-design'}_${stamp}.xlsx`
exporting.value = true
try {
const summary = await exportToExcel({
rows: rows.map((row) => ({
table_name: row.table_name,
table_comment: row.table_comment || '—',
status: row.ok ? '合格' : '不合格',
reason: reasonText(row),
})),
columns: [
{ key: 'table_name', cn: '数据表' },
{ key: 'table_comment', cn: '表注释' },
{ key: 'status', cn: '状态' },
{ key: 'reason', cn: '不合格原因' },
],
filename,
sheetName: selectedConfig.value?.name || 'table-design',
confirm: async (n) => {
try { await ElMessageBox.confirm(`当前有 ${n.toLocaleString()} 张表，是否继续导出？`, '导出确认', { confirmButtonText: '导出', cancelButtonText: '取消', type: 'warning' }); return true } catch { return false }
},
})
if (summary) ElMessage.success(`已导出 ${summary.rows.toLocaleString()} 行 × ${summary.cols} 列 → ${summary.filename}`)
} catch (e) { ElMessage.error(`导出失败：${e.message || e}`) } finally { exporting.value = false }
}
function back() { router.push({ name: 'TableDesign' }) }
onMounted(async () => { loadingConfigs.value = true; try { configs.value = await listTableDesignConfigs(); const id = route.query.config || configs.value[0]?.id; if (id) { setConfig(id); if (!selectedConfig.value?.lastResult) await reload() } } catch (e) { error.value = e.message } finally { loadingConfigs.value = false } })
</script>
......
web/src/views/TaskConfigView.vue
View file @ 54429b71
......@@ -2,7 +2,7 @@
<!-- 工具栏 -->
<div class="config-toolbar">
<div class="left">
<input type="text" class="text-input" placeholder="任务名称/数据表" style="min-width: 200px;"
<input type="text" name="validation-task-search" class="text-input" placeholder="任务名称/数据表" autocomplete="off" style="min-width: 200px;"
v-model="keyword" @keyup.enter="onQuery" />
<button class="btn btn-primary" @click="onQuery">查询</button>
<button class="btn" @click="onReset">重置</button>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment