Compare commits
No commits in common. "4800a375eea08369eebde5d9e86785761c625bf6" and "29cea8b0aa4e56c822be60a017bfa2d23df6dd4b" have entirely different histories.
4800a375ee
...
29cea8b0aa
101
TODO.md
101
TODO.md
@ -1,101 +0,0 @@
|
||||
# TODO
|
||||
|
||||
- 更新时间:2026-04-02
|
||||
- 进度基线:2026-04-02 已完成一次 MVP 收敛;`npm run test`、`npm run typecheck` 通过;Web/API 实机流程已验证;JD 实时会话导入与 `search/detail/reviews` preview 已实机验证
|
||||
- 关联文档:
|
||||
- `docs/tasks.md`
|
||||
- `docs/DevelopmentPlan.md`
|
||||
- `docs/tdd.md`
|
||||
|
||||
## 维护约定
|
||||
|
||||
- 已完成任务统一使用 `- [x]`
|
||||
- 未完成任务统一使用 `- [ ]`
|
||||
- 进行中任务使用 `- [ ] ...(进行中)`
|
||||
- 阻塞任务使用 `- [ ] ...(阻塞:原因)`
|
||||
- 任务编号必须与 `docs/tasks.md` 对齐;若任务拆分、合并或改号,两个文件必须同步更新
|
||||
|
||||
## 当前主线
|
||||
|
||||
- [x] `S1-06` 会话中心 v1 与全局会话准备后端入口落地(MVP mock 版,支持 24h 会话、清理与回跳)
|
||||
- [x] `S1-07` 新建任务页与全局会话准备入口落地
|
||||
- [x] `S2-01` 首个平台预检查与搜索适配器落地(MVP mock 版)
|
||||
- [x] `S2-05` 标准化 v1 与最小报告快照落地(规则版)
|
||||
- [ ] `S2-06` 单平台执行页闭环与回归包落地(进行中:闭环已可演示,回归包与真实异步执行待补)
|
||||
- [ ] `S3-01` 第二平台 `precheck/search/detail/reviews` 适配器落地(进行中:当前双平台仍以 mock 适配为主)
|
||||
- [ ] `S3-03` 阻塞恢复与 `L3 Browser Recovery` 落地(进行中:恢复页与重试链路已通,真实远程浏览器接管待补)
|
||||
- [ ] `S4-02` AI 结构化报告生成与版本规则落地(进行中:版本规则已落地,真实 AI 生成待接入)
|
||||
- [ ] `S4-05` 留存、删除 API 与联动清理链路落地(进行中:删除 API 与 30/90 天本地清理作业已落地,对象存储联动待补)
|
||||
- [ ] `S4-06` 完整可观测性与审计日志落地(进行中:overview / audit 已有,完整指标体系待补)
|
||||
|
||||
## 阶段快照
|
||||
|
||||
- [ ] `S0` 双平台能力矩阵、fixture/HAR、PoC 验证与 `strategy_attempts` 口径仍未冻结(进行中)
|
||||
- [ ] `S1` 本地 JSON 持久化、API/BFF、会话准备、新建任务页与状态机骨架已可用,但数据库、队列、真实 `SSE` 仍未完成(进行中)
|
||||
- [ ] `S2` 单平台最小闭环和最小报告已可演示,JD `search/detail/reviews` 实时 preview 已验证,但任务执行与标准化主链仍以 mock 数据为主(进行中)
|
||||
- [ ] `S3` 双平台候选确认、执行控制台、恢复页与平台级重试已可用,但第二平台真实适配、`L2` 模板刷新与真实 `L3` 恢复未完成(进行中)
|
||||
- [ ] `S4` 报告版本规则、报告页、历史任务页、版本切换、删除入口与观测概览已落地,但完整聚合、真实 AI、对象存储联动与完整审计仍未完成(进行中)
|
||||
- [ ] `S5` 稳定性、性能、UAT、部署与发布准备尚未进入实施(未开始)
|
||||
|
||||
## `S0`
|
||||
|
||||
- [ ] `S0-01` 冻结双平台能力矩阵(未开始)
|
||||
- [ ] `S0-02` 产出双平台首批 fixture 与 HAR 样本(未开始)
|
||||
- [ ] `S0-03` 验证服务端受控浏览器与会话快照 PoC(进行中)
|
||||
- [ ] `S0-04` 验证至少一个平台的非浏览器主路径 PoC(进行中:JD 已完成授权会话下 `search/detail/reviews` 实时 API 预览验证,待补模板刷新与量化口径)
|
||||
- [x] `S0-05` 搭建 Monorepo 与基础工程骨架
|
||||
- [ ] `S0-06` 冻结 Phase 0 量化评分表、`strategy_attempts` 记录格式与进入开发门槛(未开始)
|
||||
|
||||
## `S1`
|
||||
|
||||
- [x] `S1-01` 共享领域模型与枚举包落地
|
||||
- [ ] `S1-02` 数据库、事件日志与对象存储模型落地(进行中:MVP 先落本地 JSON 持久化,正式数据库与对象存储待补)
|
||||
- [ ] `S1-03` 任务编排、事件持久化与状态机骨架落地(进行中:状态机、事件日志、平台级重试已可用,队列化执行待补)
|
||||
- [ ] `S1-04` API / BFF、平台就绪摘要与 `SSE` 基础接口落地(进行中:REST/BFF 已可用,`SSE` 仍是最小 snapshot 形态)
|
||||
- [x] `S1-05` Web 工作台基础壳层与核心路由落地
|
||||
- [x] `S1-06` 会话中心 v1 与全局会话准备后端入口落地(MVP mock 版)
|
||||
- [x] `S1-07` 新建任务页与全局会话准备入口落地
|
||||
- [ ] `S1-08` TDD 与 CI 基础链路落地(进行中)
|
||||
|
||||
## `S2`
|
||||
|
||||
- [x] `S2-01` 首个平台预检查与搜索适配器落地(MVP mock 版)
|
||||
- [x] `S2-02` 候选确认页与确认 API 落地
|
||||
- [ ] `S2-03` 单平台商品详情抓取链路落地(进行中:JD live detail preview 已接入真实 `pc_detailpage_wareBusiness`,会话导入与解析已验证,待纳入任务执行与标准化主链)
|
||||
- [ ] `S2-04` 单平台评论采集与抽样链路落地(进行中:JD live reviews preview 已接入真实 `getLegoWareDetailComment`,分页参数改写与解析已验证,待纳入任务执行与抽样主链)
|
||||
- [x] `S2-05` 标准化 v1 与最小报告快照落地(规则版)
|
||||
- [ ] `S2-06` 单平台执行页闭环与回归包落地(进行中:新建 -> 确认 -> 执行 -> 报告已打通)
|
||||
|
||||
## `S3`
|
||||
|
||||
- [ ] `S3-01` 第二平台 `precheck/search/detail/reviews` 适配器落地(进行中)
|
||||
- [ ] `S3-02` 模板刷新与 `L2` 路径落地(未开始)
|
||||
- [ ] `S3-03` 阻塞恢复与 `L3 Browser Recovery` 落地(进行中)
|
||||
- [ ] `S3-04` 双平台候选确认与执行控制台落地(进行中:页面与状态展示已具备,真实并发执行待补)
|
||||
- [x] `S3-05` `PartialCompleted`、`Blocked`、`Failed` 汇总规则落地
|
||||
- [ ] `S3-06` 双平台主回归包落地(未开始)
|
||||
|
||||
## `S4`
|
||||
|
||||
- [ ] `S4-01` 完整标准化与三级聚合落地(进行中)
|
||||
- [ ] `S4-02` AI 结构化报告生成与版本规则落地(进行中)
|
||||
- [ ] `S4-03` 报告页、证据抽屉与质量标记落地(进行中:报告页、质量标记与证据索引已落地,证据抽屉待补)
|
||||
- [x] `S4-04` 历史任务页、版本切换与删除入口落地
|
||||
- [ ] `S4-05` 留存、删除 API 与联动清理链路落地(进行中:删除 API 与 30/90 天本地清理作业已落地,对象存储联动待补)
|
||||
- [ ] `S4-06` 完整可观测性与审计日志落地(进行中)
|
||||
|
||||
## `S5`
|
||||
|
||||
- [ ] `S5-01` 平台级定向重试稳定化(进行中)
|
||||
- [ ] `S5-02` 性能与成本优化(未开始)
|
||||
- [ ] `S5-03` UAT 与试运行任务集执行(未开始)
|
||||
- [ ] `S5-04` 部署、值守、排障与热修手册落地(未开始)
|
||||
- [ ] `S5-05` 最终验收与文档同步收口(未开始)
|
||||
|
||||
## 横向任务
|
||||
|
||||
- [ ] `X-01` 上下游文档变更同步(进行中)
|
||||
- [ ] `X-02` 安全与合规检查(未开始)
|
||||
- [ ] `X-03` 测试资产维护(进行中)
|
||||
- [ ] `X-04` 设计一致性与可访问性检查(进行中)
|
||||
- [ ] `X-05` 观测指标复盘(未开始)
|
||||
@ -1,372 +0,0 @@
|
||||
import {
|
||||
parseJdDetailApiResponse,
|
||||
parseJdReviewsApiResponse,
|
||||
parseJdSearchApiResponse,
|
||||
parseJdSearchHtml
|
||||
} from "./parsers";
|
||||
import type {
|
||||
JdDetailPreviewResult,
|
||||
JdLiveService,
|
||||
JdLiveSessionInput,
|
||||
JdLiveSessionSummary,
|
||||
JdReviewsPreviewResult,
|
||||
JdSearchMode,
|
||||
JdSearchPreviewResult,
|
||||
JdTemplateSummary
|
||||
} from "./types";
|
||||
import { firstString, readQueryBody, withUpdatedQueryBody } from "./utils";
|
||||
|
||||
const DEFAULT_JD_USER_AGENT =
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " +
|
||||
"(KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36";
|
||||
|
||||
type StoredJdLiveSession = {
|
||||
cookieHeader: string;
|
||||
importedAt: string;
|
||||
userAgent: string;
|
||||
searchApiTemplateUrl?: string | undefined;
|
||||
detailTemplateUrl?: string | undefined;
|
||||
reviewsTemplateUrl?: string | undefined;
|
||||
searchReferer?: string | undefined;
|
||||
detailReferer?: string | undefined;
|
||||
};
|
||||
|
||||
class JdLiveError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
readonly statusCode: number = 400
|
||||
) {
|
||||
super(message);
|
||||
this.name = "JdLiveError";
|
||||
}
|
||||
}
|
||||
|
||||
function nowIso(): string {
|
||||
return new Date().toISOString();
|
||||
}
|
||||
|
||||
function readEnvSession(): StoredJdLiveSession | null {
|
||||
const cookieHeader = process.env.JD_COOKIE_HEADER?.trim();
|
||||
if (!cookieHeader) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const searchApiTemplateUrl = process.env.JD_SEARCH_API_TEMPLATE_URL?.trim();
|
||||
const detailTemplateUrl = process.env.JD_DETAIL_TEMPLATE_URL?.trim();
|
||||
const reviewsTemplateUrl = process.env.JD_REVIEWS_TEMPLATE_URL?.trim();
|
||||
const searchReferer = process.env.JD_SEARCH_REFERER?.trim();
|
||||
const detailReferer = process.env.JD_DETAIL_REFERER?.trim();
|
||||
|
||||
return {
|
||||
cookieHeader,
|
||||
importedAt: nowIso(),
|
||||
userAgent: process.env.JD_USER_AGENT?.trim() || DEFAULT_JD_USER_AGENT,
|
||||
...(searchApiTemplateUrl ? { searchApiTemplateUrl } : {}),
|
||||
...(detailTemplateUrl ? { detailTemplateUrl } : {}),
|
||||
...(reviewsTemplateUrl ? { reviewsTemplateUrl } : {}),
|
||||
...(searchReferer ? { searchReferer } : {}),
|
||||
...(detailReferer ? { detailReferer } : {})
|
||||
};
|
||||
}
|
||||
|
||||
function requireNonEmptyCookie(cookieHeader: string): string {
|
||||
const normalized = cookieHeader.trim();
|
||||
if (!normalized) {
|
||||
throw new JdLiveError("cookieHeader is required for JD live requests.");
|
||||
}
|
||||
|
||||
return normalized;
|
||||
}
|
||||
|
||||
function extractTemplateSkuId(templateUrl: string | undefined): string | undefined {
|
||||
if (!templateUrl) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const url = new URL(templateUrl);
|
||||
const body = readQueryBody(url);
|
||||
return firstString(body?.skuId, body?.sku) ?? undefined;
|
||||
}
|
||||
|
||||
function buildTemplateSummary(templateUrl: string | undefined): JdTemplateSummary {
|
||||
const skuId = extractTemplateSkuId(templateUrl);
|
||||
|
||||
return {
|
||||
available: Boolean(templateUrl),
|
||||
...(skuId ? { skuId } : {})
|
||||
};
|
||||
}
|
||||
|
||||
function templateMatchesQuery(
|
||||
templateUrl: string | undefined,
|
||||
query: string
|
||||
): boolean {
|
||||
if (!templateUrl) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const templateKeyword = new URL(templateUrl).searchParams.get("keyword");
|
||||
return Boolean(templateKeyword && templateKeyword === query);
|
||||
}
|
||||
|
||||
async function fetchTextOrThrow(
|
||||
url: string,
|
||||
init: RequestInit,
|
||||
sessionExpiredMessage: string
|
||||
): Promise<{ finalUrl: string; text: string }> {
|
||||
let response: Response;
|
||||
|
||||
try {
|
||||
response = await fetch(url, {
|
||||
...init,
|
||||
redirect: "follow"
|
||||
});
|
||||
} catch (error) {
|
||||
throw new JdLiveError(
|
||||
`JD live request failed before receiving a response: ${
|
||||
error instanceof Error ? error.message : "unknown error"
|
||||
}`,
|
||||
502
|
||||
);
|
||||
}
|
||||
|
||||
const text = await response.text();
|
||||
if (response.url.includes("passport.jd.com") || text.includes("passport.jd.com")) {
|
||||
throw new JdLiveError(sessionExpiredMessage, 409);
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
throw new JdLiveError(
|
||||
`JD live request failed with status ${response.status}.`,
|
||||
502
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
finalUrl: response.url,
|
||||
text
|
||||
};
|
||||
}
|
||||
|
||||
export function isJdLiveError(error: unknown): error is Error & { statusCode: number } {
|
||||
return error instanceof Error && "statusCode" in error;
|
||||
}
|
||||
|
||||
export class JdLiveSessionService implements JdLiveService {
|
||||
private session: StoredJdLiveSession | null = readEnvSession();
|
||||
|
||||
getSessionSummary(): JdLiveSessionSummary {
|
||||
return {
|
||||
configured: Boolean(this.session),
|
||||
hasCookie: Boolean(this.session?.cookieHeader),
|
||||
...(this.session?.importedAt ? { importedAt: this.session.importedAt } : {}),
|
||||
...(this.session?.userAgent ? { userAgent: this.session.userAgent } : {}),
|
||||
searchApiTemplate: buildTemplateSummary(this.session?.searchApiTemplateUrl),
|
||||
detailTemplate: buildTemplateSummary(this.session?.detailTemplateUrl),
|
||||
reviewsTemplate: buildTemplateSummary(this.session?.reviewsTemplateUrl)
|
||||
};
|
||||
}
|
||||
|
||||
importSession(input: JdLiveSessionInput): JdLiveSessionSummary {
|
||||
const searchApiTemplateUrl = input.searchApiTemplateUrl?.trim();
|
||||
const detailTemplateUrl = input.detailTemplateUrl?.trim();
|
||||
const reviewsTemplateUrl = input.reviewsTemplateUrl?.trim();
|
||||
const searchReferer = input.searchReferer?.trim();
|
||||
const detailReferer = input.detailReferer?.trim();
|
||||
|
||||
this.session = {
|
||||
cookieHeader: requireNonEmptyCookie(input.cookieHeader),
|
||||
importedAt: nowIso(),
|
||||
userAgent: input.userAgent?.trim() || DEFAULT_JD_USER_AGENT,
|
||||
...(searchApiTemplateUrl ? { searchApiTemplateUrl } : {}),
|
||||
...(detailTemplateUrl ? { detailTemplateUrl } : {}),
|
||||
...(reviewsTemplateUrl ? { reviewsTemplateUrl } : {}),
|
||||
...(searchReferer ? { searchReferer } : {}),
|
||||
...(detailReferer ? { detailReferer } : {})
|
||||
};
|
||||
|
||||
return this.getSessionSummary();
|
||||
}
|
||||
|
||||
clearSession(): void {
|
||||
this.session = readEnvSession();
|
||||
}
|
||||
|
||||
async previewSearch(
|
||||
query: string,
|
||||
mode?: JdSearchMode
|
||||
): Promise<JdSearchPreviewResult> {
|
||||
const session = this.requireSession();
|
||||
const normalizedQuery = query.trim();
|
||||
if (!normalizedQuery) {
|
||||
throw new JdLiveError("query is required for JD live search preview.");
|
||||
}
|
||||
|
||||
const resolvedMode =
|
||||
mode ??
|
||||
(templateMatchesQuery(session.searchApiTemplateUrl, normalizedQuery) ? "api" : "html");
|
||||
|
||||
if (resolvedMode === "api") {
|
||||
if (!session.searchApiTemplateUrl) {
|
||||
throw new JdLiveError(
|
||||
"JD search API template is missing. Import a fresh search request URL or use mode=html."
|
||||
);
|
||||
}
|
||||
|
||||
const templateUrl = new URL(session.searchApiTemplateUrl);
|
||||
const templateKeyword = templateUrl.searchParams.get("keyword");
|
||||
if (templateKeyword && templateKeyword !== normalizedQuery) {
|
||||
throw new JdLiveError(
|
||||
`Imported search API template is locked to query "${templateKeyword}". ` +
|
||||
"Capture a fresh request for the target query or use mode=html."
|
||||
);
|
||||
}
|
||||
|
||||
const response = await fetchTextOrThrow(
|
||||
session.searchApiTemplateUrl,
|
||||
{
|
||||
headers: {
|
||||
Accept: "application/json, text/plain, */*",
|
||||
Cookie: session.cookieHeader,
|
||||
Referer:
|
||||
session.searchReferer ??
|
||||
`https://search.jd.com/Search?keyword=${encodeURIComponent(normalizedQuery)}`,
|
||||
"User-Agent": session.userAgent
|
||||
}
|
||||
},
|
||||
"JD search session appears invalid. Re-login in the browser and re-import the cookie/header."
|
||||
);
|
||||
|
||||
const candidates = parseJdSearchApiResponse(normalizedQuery, { text: response.text });
|
||||
return {
|
||||
query: normalizedQuery,
|
||||
source: "api",
|
||||
candidateCount: candidates.length,
|
||||
candidates
|
||||
};
|
||||
}
|
||||
|
||||
const searchUrl = `https://search.jd.com/Search?keyword=${encodeURIComponent(normalizedQuery)}`;
|
||||
const response = await fetchTextOrThrow(
|
||||
searchUrl,
|
||||
{
|
||||
headers: {
|
||||
Accept:
|
||||
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
|
||||
"Accept-Language": "zh-CN,zh;q=0.9",
|
||||
Cookie: session.cookieHeader,
|
||||
Referer: session.searchReferer ?? "https://www.jd.com/",
|
||||
"User-Agent": session.userAgent
|
||||
}
|
||||
},
|
||||
"JD search session appears invalid. Re-login in the browser and re-import the cookie/header."
|
||||
);
|
||||
|
||||
const candidates = parseJdSearchHtml(normalizedQuery, response.text);
|
||||
return {
|
||||
query: normalizedQuery,
|
||||
source: "html",
|
||||
candidateCount: candidates.length,
|
||||
candidates
|
||||
};
|
||||
}
|
||||
|
||||
async previewDetail(skuId: string): Promise<JdDetailPreviewResult> {
|
||||
const session = this.requireSession();
|
||||
const normalizedSkuId = skuId.trim();
|
||||
if (!normalizedSkuId) {
|
||||
throw new JdLiveError("skuId is required for JD detail preview.");
|
||||
}
|
||||
|
||||
if (!session.detailTemplateUrl) {
|
||||
throw new JdLiveError(
|
||||
"JD detail template is missing. Capture a fresh pc_detailpage_wareBusiness request and import it first."
|
||||
);
|
||||
}
|
||||
|
||||
const templateSkuId = extractTemplateSkuId(session.detailTemplateUrl);
|
||||
if (templateSkuId && templateSkuId !== normalizedSkuId) {
|
||||
throw new JdLiveError(
|
||||
`Imported detail template is bound to sku ${templateSkuId}. Open the matching JD item page and capture a fresh request for sku ${normalizedSkuId}.`
|
||||
);
|
||||
}
|
||||
|
||||
const response = await fetchTextOrThrow(
|
||||
session.detailTemplateUrl,
|
||||
{
|
||||
headers: {
|
||||
Accept: "application/json, text/plain, */*",
|
||||
Cookie: session.cookieHeader,
|
||||
Referer: session.detailReferer ?? `https://item.jd.com/${normalizedSkuId}.html`,
|
||||
"User-Agent": session.userAgent
|
||||
}
|
||||
},
|
||||
"JD detail session appears invalid. Re-login in the browser and re-import the cookie/header."
|
||||
);
|
||||
|
||||
return {
|
||||
skuId: normalizedSkuId,
|
||||
source: "api",
|
||||
detail: parseJdDetailApiResponse(normalizedSkuId, { text: response.text })
|
||||
};
|
||||
}
|
||||
|
||||
async previewReviews(
|
||||
skuId: string,
|
||||
commentCount = 5
|
||||
): Promise<JdReviewsPreviewResult> {
|
||||
const session = this.requireSession();
|
||||
const normalizedSkuId = skuId.trim();
|
||||
if (!normalizedSkuId) {
|
||||
throw new JdLiveError("skuId is required for JD reviews preview.");
|
||||
}
|
||||
|
||||
if (!session.reviewsTemplateUrl) {
|
||||
throw new JdLiveError(
|
||||
"JD reviews template is missing. Capture a fresh getLegoWareDetailComment request and import it first."
|
||||
);
|
||||
}
|
||||
|
||||
const templateSkuId = extractTemplateSkuId(session.reviewsTemplateUrl);
|
||||
if (templateSkuId && templateSkuId !== normalizedSkuId) {
|
||||
throw new JdLiveError(
|
||||
`Imported reviews template is bound to sku ${templateSkuId}. Open the matching JD item page and capture a fresh request for sku ${normalizedSkuId}.`
|
||||
);
|
||||
}
|
||||
|
||||
const templateUrl = new URL(session.reviewsTemplateUrl);
|
||||
const requestUrl = withUpdatedQueryBody(templateUrl, (body) => ({
|
||||
...body,
|
||||
commentNum: commentCount
|
||||
}));
|
||||
|
||||
const response = await fetchTextOrThrow(
|
||||
requestUrl,
|
||||
{
|
||||
headers: {
|
||||
Accept: "application/json, text/plain, */*",
|
||||
Cookie: session.cookieHeader,
|
||||
Referer: session.detailReferer ?? `https://item.jd.com/${normalizedSkuId}.html`,
|
||||
"User-Agent": session.userAgent
|
||||
}
|
||||
},
|
||||
"JD reviews session appears invalid. Re-login in the browser and re-import the cookie/header."
|
||||
);
|
||||
|
||||
return {
|
||||
skuId: normalizedSkuId,
|
||||
source: "api",
|
||||
reviews: parseJdReviewsApiResponse(normalizedSkuId, { text: response.text })
|
||||
};
|
||||
}
|
||||
|
||||
private requireSession(): StoredJdLiveSession {
|
||||
if (!this.session?.cookieHeader) {
|
||||
throw new JdLiveError(
|
||||
"JD live session is not configured. Import a browser cookie/header first."
|
||||
);
|
||||
}
|
||||
|
||||
return this.session;
|
||||
}
|
||||
}
|
||||
@ -1,151 +0,0 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
parseJdDetailApiResponse,
|
||||
parseJdReviewsApiResponse,
|
||||
parseJdSearchApiResponse,
|
||||
parseJdSearchHtml
|
||||
} from "./parsers";
|
||||
|
||||
function readFixture(path: string): unknown {
|
||||
return JSON.parse(readFileSync(new URL(path, import.meta.url), "utf8")) as unknown;
|
||||
}
|
||||
|
||||
describe("JD parsers", () => {
|
||||
it("parses real JD search API fixtures into candidate records", () => {
|
||||
const fixture = readFixture("../../../../../jd-search-json-shape.json") as {
|
||||
firstWare: Record<string, unknown>;
|
||||
};
|
||||
|
||||
const candidates = parseJdSearchApiResponse("iPhone 15", {
|
||||
data: {
|
||||
wareList: [fixture.firstWare]
|
||||
}
|
||||
});
|
||||
|
||||
expect(candidates).toHaveLength(1);
|
||||
expect(candidates[0]).toMatchObject({
|
||||
platform: "jd",
|
||||
title: expect.stringContaining("iPhone"),
|
||||
priceLabel: expect.stringMatching(/^¥/),
|
||||
storeName: expect.any(String)
|
||||
});
|
||||
expect(candidates[0]?.title).not.toContain("<font");
|
||||
});
|
||||
|
||||
it("parses captured JD search card blocks from HTML markup", () => {
|
||||
const fixture = readFixture("../../../../../jd-search-card-blocks.json") as Array<{
|
||||
sku: string;
|
||||
html: string;
|
||||
}>;
|
||||
const html = fixture.map((item) => item.html).join("");
|
||||
|
||||
const candidates = parseJdSearchHtml("iPhone 15", html);
|
||||
|
||||
expect(candidates).toHaveLength(3);
|
||||
expect(candidates).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
candidateId: "jd-100068388533",
|
||||
storeName: "Apple产品京东自营旗舰店",
|
||||
productUrl: "https://item.jd.com/100068388533.html",
|
||||
salesHint: expect.stringContaining("已售500万+")
|
||||
})
|
||||
])
|
||||
);
|
||||
});
|
||||
|
||||
it("parses JD detail payloads from raw API objects", () => {
|
||||
const detail = parseJdDetailApiResponse("100068388533", {
|
||||
wareInfo: {
|
||||
wareInfoMap: {
|
||||
sku_status: 1
|
||||
}
|
||||
},
|
||||
skuHeadVO: {
|
||||
skuTitle: "Apple/苹果 iPhone 15 (A3092) 128GB 绿色 支持移动联通电信5G 双卡双待手机"
|
||||
},
|
||||
price: {
|
||||
p: "4398.00",
|
||||
op: "4599.00"
|
||||
},
|
||||
itemShopInfo: {
|
||||
shopName: "Apple产品京东自营旗舰店"
|
||||
},
|
||||
crumbInfoVO: {
|
||||
crumbs: [
|
||||
{ text: "手机通讯" },
|
||||
{ text: "手机" },
|
||||
{ text: "Apple" }
|
||||
]
|
||||
},
|
||||
stockInfo: {
|
||||
stockStateDesc: "<strong>有货</strong>,仅剩318件"
|
||||
},
|
||||
mainImageVO: {
|
||||
mainImageArea: {
|
||||
imageUrl: "jfs/t1/example.jpg"
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
expect(detail).toMatchObject({
|
||||
skuId: "100068388533",
|
||||
title: "Apple/苹果 iPhone 15 (A3092) 128GB 绿色 支持移动联通电信5G 双卡双待手机",
|
||||
price: "4398.00",
|
||||
originalPrice: "4599.00",
|
||||
shopName: "Apple产品京东自营旗舰店",
|
||||
categoryPath: ["手机通讯", "手机", "Apple"],
|
||||
stockState: "有货,仅剩318件",
|
||||
mainImage: "https://img14.360buyimg.com/n2/jfs/t1/example.jpg"
|
||||
});
|
||||
});
|
||||
|
||||
it("parses JD reviews payloads from raw API objects", () => {
|
||||
const reviews = parseJdReviewsApiResponse("100068388533", {
|
||||
allCnt: "10000",
|
||||
goodRate: "95%",
|
||||
pictureCnt: "500",
|
||||
tagStatisticsinfoList: [
|
||||
{
|
||||
tagId: "tag-1",
|
||||
name: "拍照效果超清晰",
|
||||
count: "9313"
|
||||
},
|
||||
{
|
||||
tagId: "tag-2",
|
||||
name: "手感很舒服",
|
||||
count: "8628"
|
||||
}
|
||||
],
|
||||
commentInfoList: [
|
||||
{
|
||||
commentId: "103893190162198263",
|
||||
commentData: "蓝色 iPhone 15 颜值很高。",
|
||||
commentScore: 5,
|
||||
commentDate: "2026-04-02 19:23:16",
|
||||
userLevelName: "PLUS会员"
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
expect(reviews).toMatchObject({
|
||||
skuId: "100068388533",
|
||||
total: "10000",
|
||||
goodRate: "95%",
|
||||
pictureCount: "500"
|
||||
});
|
||||
expect(reviews.tags[0]).toMatchObject({
|
||||
tagId: "tag-1",
|
||||
name: "拍照效果超清晰",
|
||||
count: "9313"
|
||||
});
|
||||
expect(reviews.comments[0]).toMatchObject({
|
||||
id: "103893190162198263",
|
||||
content: "蓝色 iPhone 15 颜值很高。",
|
||||
score: "5",
|
||||
userLevelName: "PLUS会员"
|
||||
});
|
||||
});
|
||||
});
|
||||
@ -1,361 +0,0 @@
|
||||
import type { CandidateRecord } from "@cross-ai/domain";
|
||||
|
||||
import type {
|
||||
JdProductDetailSnapshot,
|
||||
JdProductReviewsSnapshot,
|
||||
JdReviewCommentSnapshot,
|
||||
JdReviewTagSnapshot
|
||||
} from "./types";
|
||||
import {
|
||||
absolutizeUrl,
|
||||
asArray,
|
||||
asRecord,
|
||||
firstString,
|
||||
normalizeWhitespace,
|
||||
stringFrom,
|
||||
uniqueStrings
|
||||
} from "./utils";
|
||||
|
||||
function unwrapCapturedPayload(input: unknown): unknown {
|
||||
const record = asRecord(input);
|
||||
const text = stringFrom(record?.text);
|
||||
if (!text) {
|
||||
return input;
|
||||
}
|
||||
|
||||
try {
|
||||
return JSON.parse(text) as unknown;
|
||||
} catch {
|
||||
return input;
|
||||
}
|
||||
}
|
||||
|
||||
function extractSpecLabel(title: string): string {
|
||||
const storageMatch = title.match(/\b\d+(?:GB|TB)\b/i);
|
||||
if (storageMatch) {
|
||||
return storageMatch[0].toUpperCase();
|
||||
}
|
||||
|
||||
const colorMatch = title.match(/(黑色|白色|蓝色|粉色|绿色|黄色|紫色|原色|钛金属)/);
|
||||
if (colorMatch) {
|
||||
return colorMatch[0];
|
||||
}
|
||||
|
||||
return "标准版";
|
||||
}
|
||||
|
||||
function normalizePriceText(value: string | null): { value: number; label: string } | null {
|
||||
if (!value) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const stripped = value.replace(/[^\d.]/g, "");
|
||||
if (!stripped) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const parsed = Number.parseFloat(stripped);
|
||||
if (Number.isNaN(parsed)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
value: parsed,
|
||||
label: `¥${parsed.toString()}`
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeInlineText(value: string | null): string | null {
|
||||
if (!value) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const normalized = normalizeWhitespace(value).replace(/\s+([,。;:、])/g, "$1");
|
||||
return normalized || null;
|
||||
}
|
||||
|
||||
function matchFirst(value: string, pattern: RegExp): string | null {
|
||||
const match = pattern.exec(value);
|
||||
return match?.[1] ? normalizeWhitespace(match[1]) : null;
|
||||
}
|
||||
|
||||
function extractSearchCardBlocks(html: string): string[] {
|
||||
const matches = html.matchAll(
|
||||
/<div[^>]*plugin_goodsCardWrapper[^>]*data-sku="[^"]+"[\s\S]*?(?=<div[^>]*plugin_goodsCardWrapper[^>]*data-sku="|$)/g
|
||||
);
|
||||
|
||||
return Array.from(matches, (match) => match[0]);
|
||||
}
|
||||
|
||||
function parseSearchCardBlock(block: string): CandidateRecord | null {
|
||||
const sku = matchFirst(block, /data-sku="([^"]+)"/);
|
||||
const title =
|
||||
matchFirst(
|
||||
block,
|
||||
/_goods_title_container_[^"]*"[\s\S]*?<span[^>]*title="([^"]+)"/
|
||||
) ?? matchFirst(block, /title="([^"]+)"/);
|
||||
const priceText = matchFirst(block, /<span class="[^"]*_price_[^"]*"[^>]*>([\s\S]*?)<\/span>/);
|
||||
const price = normalizePriceText(priceText);
|
||||
const storeName = matchFirst(
|
||||
block,
|
||||
/<span class="[^"]*_name_[^"]*"[\s\S]*?<span>([\s\S]*?)<\/span>/
|
||||
);
|
||||
const imageUrl = absolutizeUrl(
|
||||
matchFirst(block, /<img[^>]+src="([^"]+)"[^>]*alt="">/) ??
|
||||
matchFirst(block, /<img[^>]+data-src="([^"]+)"/)
|
||||
);
|
||||
const soldHint = matchFirst(block, /title="(已售[^"]+)"/);
|
||||
const trendHint = matchFirst(block, /title="(30天[^"]+)"/);
|
||||
const featureMatches = Array.from(
|
||||
block.matchAll(/<span>([^<]+)<\/span>/g),
|
||||
(match) => normalizeWhitespace(match[1] ?? "")
|
||||
).filter(Boolean);
|
||||
const highlights = uniqueStrings(featureMatches).slice(0, 4);
|
||||
|
||||
if (!sku || !title || !price || !storeName) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
candidateId: `jd-${sku}`,
|
||||
platform: "jd",
|
||||
title,
|
||||
price: price.value,
|
||||
priceLabel: price.label,
|
||||
storeName,
|
||||
productUrl: `https://item.jd.com/${sku}.html`,
|
||||
imageUrl: imageUrl ?? "https://placehold.co/640x480?text=JD",
|
||||
salesHint: uniqueStrings([soldHint, trendHint]).join(" · ") || "暂无销量信息",
|
||||
specLabel: extractSpecLabel(title),
|
||||
highlights: highlights.length > 0 ? highlights : ["京东商品卡片已命中"]
|
||||
};
|
||||
}
|
||||
|
||||
export function parseJdSearchHtml(query: string, html: string): CandidateRecord[] {
|
||||
const blocks = extractSearchCardBlocks(html);
|
||||
const seen = new Set<string>();
|
||||
const candidates: CandidateRecord[] = [];
|
||||
|
||||
for (const block of blocks) {
|
||||
const candidate = parseSearchCardBlock(block);
|
||||
if (!candidate || seen.has(candidate.candidateId)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
seen.add(candidate.candidateId);
|
||||
candidates.push(candidate);
|
||||
}
|
||||
|
||||
if (candidates.length > 0) {
|
||||
return candidates;
|
||||
}
|
||||
|
||||
if (html.includes("暂无") || html.includes("很抱歉没有找到")) {
|
||||
return [];
|
||||
}
|
||||
|
||||
return [
|
||||
{
|
||||
candidateId: `jd-fallback-${encodeURIComponent(query)}`,
|
||||
platform: "jd",
|
||||
title: query,
|
||||
price: 0,
|
||||
priceLabel: "¥0",
|
||||
storeName: "京东",
|
||||
productUrl: `https://search.jd.com/Search?keyword=${encodeURIComponent(query)}`,
|
||||
imageUrl: "https://placehold.co/640x480?text=JD",
|
||||
salesHint: "页面已返回,但未解析出稳定商品卡片",
|
||||
specLabel: "待确认",
|
||||
highlights: ["需要刷新搜索模板或调整解析器"]
|
||||
}
|
||||
];
|
||||
}
|
||||
|
||||
export function parseJdSearchApiResponse(query: string, input: unknown): CandidateRecord[] {
|
||||
const payload = asRecord(unwrapCapturedPayload(input));
|
||||
const data = asRecord(payload?.data);
|
||||
const wareList = asArray(data?.wareList);
|
||||
const seen = new Set<string>();
|
||||
const candidates: CandidateRecord[] = [];
|
||||
|
||||
for (const item of wareList) {
|
||||
const ware = asRecord(item);
|
||||
const sku = stringFrom(ware?.skuId);
|
||||
const title = normalizeInlineText(firstString(ware?.wareName, ware?.wname));
|
||||
const priceText = firstString(
|
||||
ware?.jdPrice,
|
||||
asRecord(ware?.finalPrice)?.estimatedPrice,
|
||||
ware?.price
|
||||
);
|
||||
const price = normalizePriceText(priceText);
|
||||
|
||||
if (!sku || !title || !price || seen.has(sku)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const storeName =
|
||||
normalizeInlineText(firstString(ware?.shopName, ware?.storeName, "京东店铺")) ??
|
||||
"京东店铺";
|
||||
const totalSales = stringFrom(ware?.totalSales);
|
||||
const commentFuzzy = firstString(ware?.commentFuzzy, ware?.comment);
|
||||
const highlights = uniqueStrings([
|
||||
stringFrom(ware?.selfSupport) === "1" ? "京东自营" : null,
|
||||
stringFrom(ware?.good),
|
||||
stringFrom(ware?.averageScore) ? `评分 ${stringFrom(ware?.averageScore)}` : null
|
||||
]);
|
||||
|
||||
seen.add(sku);
|
||||
candidates.push({
|
||||
candidateId: `jd-${sku}`,
|
||||
platform: "jd",
|
||||
title,
|
||||
price: price.value,
|
||||
priceLabel: price.label,
|
||||
storeName,
|
||||
productUrl: `https://item.jd.com/${sku}.html`,
|
||||
imageUrl:
|
||||
absolutizeUrl(stringFrom(ware?.imageurl)) ??
|
||||
"https://placehold.co/640x480?text=JD",
|
||||
salesHint:
|
||||
uniqueStrings([
|
||||
totalSales ? `已售${totalSales}` : null,
|
||||
commentFuzzy ? `累计评价 ${commentFuzzy}` : null
|
||||
]).join(" · ") || "暂无销量信息",
|
||||
specLabel: extractSpecLabel(title),
|
||||
highlights: highlights.length > 0 ? highlights : ["京东 API 返回候选"]
|
||||
});
|
||||
}
|
||||
|
||||
return candidates;
|
||||
}
|
||||
|
||||
export function parseJdDetailApiResponse(
|
||||
skuId: string,
|
||||
input: unknown
|
||||
): JdProductDetailSnapshot {
|
||||
const payload = asRecord(unwrapCapturedPayload(input));
|
||||
const wareInfo = asRecord(payload?.wareInfo);
|
||||
const wareInfoMap = asRecord(wareInfo?.wareInfoMap);
|
||||
const price = asRecord(payload?.price);
|
||||
const finalPrice = asRecord(price?.finalPrice);
|
||||
const itemShopInfo = asRecord(payload?.itemShopInfo);
|
||||
const crumbInfo = asRecord(payload?.crumbInfoVO);
|
||||
const stockInfo = asRecord(payload?.stockInfo) ?? asRecord(payload?.stockVO);
|
||||
const mainImage = asRecord(payload?.mainImageVO);
|
||||
const mainImageArea = asRecord(mainImage?.mainImageArea);
|
||||
const skuHead = asRecord(payload?.skuHeadVO);
|
||||
const categoryPath = asArray(crumbInfo?.crumbs)
|
||||
.map((item) => {
|
||||
const crumb = asRecord(item);
|
||||
return firstString(crumb?.text, crumb?.name);
|
||||
})
|
||||
.filter((item): item is string => Boolean(item));
|
||||
|
||||
return {
|
||||
skuId: firstString(wareInfo?.skuId, wareInfoMap?.skuId, skuId) ?? skuId,
|
||||
title: normalizeInlineText(
|
||||
firstString(
|
||||
skuHead?.skuTitle,
|
||||
wareInfo?.wname,
|
||||
wareInfo?.name,
|
||||
wareInfoMap?.wname,
|
||||
wareInfoMap?.name,
|
||||
skuHead?.seoTitle
|
||||
)
|
||||
),
|
||||
price: firstString(price?.p, price?.price),
|
||||
originalPrice: firstString(price?.op, price?.originalPrice),
|
||||
estimatedPrice: firstString(finalPrice?.estimatedPrice, finalPrice?.price),
|
||||
shopName: normalizeInlineText(
|
||||
firstString(itemShopInfo?.shopName, itemShopInfo?.venderName)
|
||||
),
|
||||
vendorId: firstString(itemShopInfo?.venderId, itemShopInfo?.shopId),
|
||||
categoryPath,
|
||||
stockState: normalizeInlineText(
|
||||
firstString(
|
||||
stockInfo?.stockStateDesc,
|
||||
stockInfo?.stockDesc,
|
||||
stockInfo?.stockStateName,
|
||||
stockInfo?.stockState
|
||||
)
|
||||
),
|
||||
mainImage: absolutizeUrl(
|
||||
firstString(
|
||||
mainImageArea?.imageUrl,
|
||||
mainImage?.mainImageUrl,
|
||||
mainImage?.mainImage,
|
||||
asRecord(asArray(mainImage?.carouselArea)[0])?.imageUrl,
|
||||
wareInfo?.imageUrl,
|
||||
wareInfoMap?.imageUrl
|
||||
)
|
||||
),
|
||||
averageScore: firstString(
|
||||
wareInfo?.averageScore,
|
||||
wareInfo?.score,
|
||||
wareInfoMap?.averageScore,
|
||||
wareInfoMap?.score
|
||||
)
|
||||
};
|
||||
}
|
||||
|
||||
function parseReviewTag(input: unknown): JdReviewTagSnapshot | null {
|
||||
const tag = asRecord(input);
|
||||
const name = firstString(tag?.name, tag?.tagName);
|
||||
if (!name) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
tagId: firstString(tag?.tagId, tag?.id),
|
||||
name,
|
||||
count: firstString(tag?.count, tag?.num)
|
||||
};
|
||||
}
|
||||
|
||||
function parseReviewComment(input: unknown): JdReviewCommentSnapshot | null {
|
||||
const comment = asRecord(input);
|
||||
const content = normalizeInlineText(
|
||||
firstString(comment?.content, comment?.commentData, comment?.tagCommentContent)
|
||||
);
|
||||
const id = firstString(comment?.id, comment?.commentId);
|
||||
|
||||
if (!content || !id) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
id,
|
||||
content,
|
||||
score: firstString(comment?.score, comment?.commentScore),
|
||||
creationTime: firstString(
|
||||
comment?.creationTime,
|
||||
comment?.creationDate,
|
||||
comment?.commentDate
|
||||
),
|
||||
userLevelName: normalizeInlineText(
|
||||
firstString(comment?.userLevelName, comment?.userClientShow)
|
||||
)
|
||||
};
|
||||
}
|
||||
|
||||
export function parseJdReviewsApiResponse(
|
||||
skuId: string,
|
||||
input: unknown
|
||||
): JdProductReviewsSnapshot {
|
||||
const payload = asRecord(unwrapCapturedPayload(input));
|
||||
const tags = asArray(payload?.tagStatisticsinfoList)
|
||||
.map((tag) => parseReviewTag(tag))
|
||||
.filter((tag): tag is JdReviewTagSnapshot => Boolean(tag));
|
||||
const comments = asArray(payload?.commentInfoList)
|
||||
.map((comment) => parseReviewComment(comment))
|
||||
.filter((comment): comment is JdReviewCommentSnapshot => Boolean(comment));
|
||||
|
||||
return {
|
||||
skuId,
|
||||
total: firstString(payload?.allCnt, payload?.allCntStr, payload?.goodCnt),
|
||||
goodRate: firstString(payload?.goodRate, payload?.goodRateShow),
|
||||
pictureCount: firstString(payload?.pictureCnt, payload?.showPicCnt),
|
||||
tags,
|
||||
comments
|
||||
};
|
||||
}
|
||||
@ -1,93 +0,0 @@
|
||||
import type { CandidateRecord } from "@cross-ai/domain";
|
||||
|
||||
export type JdSearchMode = "html" | "api";
|
||||
|
||||
export interface JdTemplateSummary {
|
||||
available: boolean;
|
||||
skuId?: string | undefined;
|
||||
}
|
||||
|
||||
export interface JdLiveSessionInput {
|
||||
cookieHeader: string;
|
||||
userAgent?: string | undefined;
|
||||
searchApiTemplateUrl?: string | undefined;
|
||||
detailTemplateUrl?: string | undefined;
|
||||
reviewsTemplateUrl?: string | undefined;
|
||||
searchReferer?: string | undefined;
|
||||
detailReferer?: string | undefined;
|
||||
}
|
||||
|
||||
export interface JdLiveSessionSummary {
|
||||
configured: boolean;
|
||||
importedAt?: string | undefined;
|
||||
hasCookie: boolean;
|
||||
userAgent?: string | undefined;
|
||||
searchApiTemplate: JdTemplateSummary;
|
||||
detailTemplate: JdTemplateSummary;
|
||||
reviewsTemplate: JdTemplateSummary;
|
||||
}
|
||||
|
||||
export interface JdSearchPreviewResult {
|
||||
query: string;
|
||||
source: JdSearchMode;
|
||||
candidateCount: number;
|
||||
candidates: CandidateRecord[];
|
||||
}
|
||||
|
||||
export interface JdProductDetailSnapshot {
|
||||
skuId: string;
|
||||
title: string | null;
|
||||
price: string | null;
|
||||
originalPrice: string | null;
|
||||
estimatedPrice: string | null;
|
||||
shopName: string | null;
|
||||
vendorId: string | null;
|
||||
categoryPath: string[];
|
||||
stockState: string | null;
|
||||
mainImage: string | null;
|
||||
averageScore: string | null;
|
||||
}
|
||||
|
||||
export interface JdReviewTagSnapshot {
|
||||
tagId: string | null;
|
||||
name: string;
|
||||
count: string | null;
|
||||
}
|
||||
|
||||
export interface JdReviewCommentSnapshot {
|
||||
id: string;
|
||||
content: string;
|
||||
score: string | null;
|
||||
creationTime: string | null;
|
||||
userLevelName: string | null;
|
||||
}
|
||||
|
||||
export interface JdProductReviewsSnapshot {
|
||||
skuId: string;
|
||||
total: string | null;
|
||||
goodRate: string | null;
|
||||
pictureCount: string | null;
|
||||
tags: JdReviewTagSnapshot[];
|
||||
comments: JdReviewCommentSnapshot[];
|
||||
}
|
||||
|
||||
export interface JdDetailPreviewResult {
|
||||
skuId: string;
|
||||
source: "api";
|
||||
detail: JdProductDetailSnapshot;
|
||||
}
|
||||
|
||||
export interface JdReviewsPreviewResult {
|
||||
skuId: string;
|
||||
source: "api";
|
||||
reviews: JdProductReviewsSnapshot;
|
||||
}
|
||||
|
||||
export interface JdLiveService {
|
||||
getSessionSummary(): JdLiveSessionSummary;
|
||||
importSession(input: JdLiveSessionInput): JdLiveSessionSummary;
|
||||
clearSession(): void;
|
||||
previewSearch(query: string, mode?: JdSearchMode): Promise<JdSearchPreviewResult>;
|
||||
previewDetail(skuId: string): Promise<JdDetailPreviewResult>;
|
||||
previewReviews(skuId: string, commentCount?: number): Promise<JdReviewsPreviewResult>;
|
||||
}
|
||||
@ -1,115 +0,0 @@
|
||||
export function asRecord(value: unknown): Record<string, unknown> | null {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
|
||||
export function asArray(value: unknown): unknown[] {
|
||||
return Array.isArray(value) ? value : [];
|
||||
}
|
||||
|
||||
export function stringFrom(value: unknown): string | null {
|
||||
if (typeof value === "string") {
|
||||
const trimmed = value.trim();
|
||||
return trimmed.length > 0 ? trimmed : null;
|
||||
}
|
||||
|
||||
if (typeof value === "number" || typeof value === "boolean") {
|
||||
return String(value);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
export function firstString(...values: unknown[]): string | null {
|
||||
for (const value of values) {
|
||||
const resolved = stringFrom(value);
|
||||
if (resolved) {
|
||||
return resolved;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
export function absolutizeUrl(value: string | null | undefined): string | null {
|
||||
if (!value) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (value.startsWith("http://") || value.startsWith("https://")) {
|
||||
return value;
|
||||
}
|
||||
|
||||
if (value.startsWith("//")) {
|
||||
return `https:${value}`;
|
||||
}
|
||||
|
||||
if (value.startsWith("/")) {
|
||||
return `https://www.jd.com${value}`;
|
||||
}
|
||||
|
||||
if (value.startsWith("jfs/") || value.startsWith("t1/") || value.startsWith("t202")) {
|
||||
return `https://img14.360buyimg.com/n2/${value}`;
|
||||
}
|
||||
|
||||
return `https://${value}`;
|
||||
}
|
||||
|
||||
export function decodeHtmlEntities(value: string): string {
|
||||
return value
|
||||
.replace(/ /g, " ")
|
||||
.replace(/&/g, "&")
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, "'")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">");
|
||||
}
|
||||
|
||||
export function stripTags(value: string): string {
|
||||
return value.replace(/<[^>]+>/g, " ");
|
||||
}
|
||||
|
||||
export function normalizeWhitespace(value: string): string {
|
||||
return decodeHtmlEntities(stripTags(value)).replace(/\s+/g, " ").trim();
|
||||
}
|
||||
|
||||
export function uniqueStrings(values: Array<string | null | undefined>): string[] {
|
||||
return Array.from(
|
||||
new Set(
|
||||
values
|
||||
.map((value) => value?.trim())
|
||||
.filter((value): value is string => Boolean(value))
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
export function parseEmbeddedJson(value: string | null): Record<string, unknown> | null {
|
||||
if (!value) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
const parsed = JSON.parse(value) as unknown;
|
||||
return asRecord(parsed);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function readQueryBody(url: URL): Record<string, unknown> | null {
|
||||
return parseEmbeddedJson(url.searchParams.get("body"));
|
||||
}
|
||||
|
||||
export function withUpdatedQueryBody(
|
||||
url: URL,
|
||||
updater: (body: Record<string, unknown>) => Record<string, unknown>
|
||||
): string {
|
||||
const body = readQueryBody(url) ?? {};
|
||||
const next = updater(body);
|
||||
const nextUrl = new URL(url.toString());
|
||||
nextUrl.searchParams.set("body", JSON.stringify(next));
|
||||
return nextUrl.toString();
|
||||
}
|
||||
@ -1,288 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import type {
|
||||
JdDetailPreviewResult,
|
||||
JdLiveService,
|
||||
JdLiveSessionSummary,
|
||||
JdReviewsPreviewResult,
|
||||
JdSearchPreviewResult
|
||||
} from "./platforms/jd/types";
|
||||
import { createServer } from "./server";
|
||||
|
||||
function createJdLiveServiceStub(
|
||||
overrides: Partial<JdLiveService> = {}
|
||||
): JdLiveService {
|
||||
let summary: JdLiveSessionSummary = {
|
||||
configured: false,
|
||||
hasCookie: false,
|
||||
searchApiTemplate: { available: false },
|
||||
detailTemplate: { available: false },
|
||||
reviewsTemplate: { available: false }
|
||||
};
|
||||
|
||||
return {
|
||||
getSessionSummary() {
|
||||
return overrides.getSessionSummary?.() ?? summary;
|
||||
},
|
||||
importSession(input) {
|
||||
if (overrides.importSession) {
|
||||
return overrides.importSession(input);
|
||||
}
|
||||
|
||||
summary = {
|
||||
configured: true,
|
||||
importedAt: "2026-04-02T12:00:00.000Z",
|
||||
hasCookie: true,
|
||||
userAgent: input.userAgent ?? "stub-user-agent",
|
||||
searchApiTemplate: { available: Boolean(input.searchApiTemplateUrl) },
|
||||
detailTemplate: { available: Boolean(input.detailTemplateUrl) },
|
||||
reviewsTemplate: { available: Boolean(input.reviewsTemplateUrl) }
|
||||
};
|
||||
return summary;
|
||||
},
|
||||
clearSession() {
|
||||
if (overrides.clearSession) {
|
||||
overrides.clearSession();
|
||||
return;
|
||||
}
|
||||
|
||||
summary = {
|
||||
configured: false,
|
||||
hasCookie: false,
|
||||
searchApiTemplate: { available: false },
|
||||
detailTemplate: { available: false },
|
||||
reviewsTemplate: { available: false }
|
||||
};
|
||||
},
|
||||
async previewSearch(query, mode) {
|
||||
if (overrides.previewSearch) {
|
||||
return overrides.previewSearch(query, mode);
|
||||
}
|
||||
|
||||
const preview: JdSearchPreviewResult = {
|
||||
query,
|
||||
source: "api",
|
||||
candidateCount: 1,
|
||||
candidates: [
|
||||
{
|
||||
candidateId: "jd-100068388533",
|
||||
platform: "jd",
|
||||
title: "Apple iPhone 15",
|
||||
price: 3898,
|
||||
priceLabel: "CNY 3898",
|
||||
storeName: "JD Self Operated",
|
||||
productUrl: "https://item.jd.com/100068388533.html",
|
||||
imageUrl: "https://img14.360buyimg.com/n2/jfs/t1/example.jpg",
|
||||
salesHint: "sold 500+",
|
||||
specLabel: "128GB",
|
||||
highlights: ["A16"]
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
return preview;
|
||||
},
|
||||
async previewDetail(skuId) {
|
||||
if (overrides.previewDetail) {
|
||||
return overrides.previewDetail(skuId);
|
||||
}
|
||||
|
||||
const preview: JdDetailPreviewResult = {
|
||||
skuId,
|
||||
source: "api",
|
||||
detail: {
|
||||
skuId,
|
||||
title: "Apple iPhone 15",
|
||||
price: "4398.00",
|
||||
originalPrice: "4599.00",
|
||||
estimatedPrice: "3898",
|
||||
shopName: "JD Self Operated",
|
||||
vendorId: null,
|
||||
categoryPath: ["phones", "smartphones", "apple"],
|
||||
stockState: "in stock",
|
||||
mainImage: "https://img14.360buyimg.com/n2/jfs/t1/example.jpg",
|
||||
averageScore: null
|
||||
}
|
||||
};
|
||||
|
||||
return preview;
|
||||
},
|
||||
async previewReviews(skuId, commentCount) {
|
||||
if (overrides.previewReviews) {
|
||||
return overrides.previewReviews(skuId, commentCount);
|
||||
}
|
||||
|
||||
const preview: JdReviewsPreviewResult = {
|
||||
skuId,
|
||||
source: "api",
|
||||
reviews: {
|
||||
skuId,
|
||||
total: "10000",
|
||||
goodRate: "95%",
|
||||
pictureCount: "500",
|
||||
tags: [
|
||||
{
|
||||
tagId: "tag-1",
|
||||
name: "clear camera",
|
||||
count: "9313"
|
||||
}
|
||||
],
|
||||
comments: [
|
||||
{
|
||||
id: "comment-1",
|
||||
content: "smooth system and sharp photos",
|
||||
score: "5",
|
||||
creationTime: "2026-04-02 19:23:16",
|
||||
userLevelName: "PLUS"
|
||||
}
|
||||
]
|
||||
}
|
||||
};
|
||||
|
||||
return preview;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
describe("JD live server endpoints", () => {
|
||||
it("imports and clears a JD live session through dedicated endpoints", async () => {
|
||||
const jdLiveService = createJdLiveServiceStub();
|
||||
const app = createServer({ jdLiveService });
|
||||
await app.ready();
|
||||
|
||||
const importResponse = await app.inject({
|
||||
method: "POST",
|
||||
url: "/api/platforms/jd/live-session",
|
||||
payload: {
|
||||
cookieHeader: "thor=masked; pin=masked;",
|
||||
searchApiTemplateUrl:
|
||||
"https://api.m.jd.com/?functionId=pc_search_searchWare&body=%7B%22keyword%22:%22iphone%2015%22%7D",
|
||||
detailTemplateUrl:
|
||||
"https://api.m.jd.com/?functionId=pc_detailpage_wareBusiness&body=%7B%22skuId%22:%22100068388533%22%7D",
|
||||
reviewsTemplateUrl:
|
||||
"https://api.m.jd.com/?functionId=getLegoWareDetailComment&body=%7B%22sku%22:100068388533%7D"
|
||||
}
|
||||
});
|
||||
|
||||
expect(importResponse.statusCode).toBe(200);
|
||||
expect(importResponse.json().session).toMatchObject({
|
||||
configured: true,
|
||||
hasCookie: true,
|
||||
searchApiTemplate: {
|
||||
available: true
|
||||
},
|
||||
detailTemplate: {
|
||||
available: true
|
||||
},
|
||||
reviewsTemplate: {
|
||||
available: true
|
||||
}
|
||||
});
|
||||
|
||||
const summaryResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/jd/live-session"
|
||||
});
|
||||
expect(summaryResponse.statusCode).toBe(200);
|
||||
expect(summaryResponse.json().session).toMatchObject({
|
||||
configured: true,
|
||||
hasCookie: true
|
||||
});
|
||||
|
||||
const readinessResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/readiness"
|
||||
});
|
||||
expect(
|
||||
readinessResponse
|
||||
.json()
|
||||
.platforms.find((platform: { platform: string }) => platform.platform === "jd")
|
||||
).toMatchObject({
|
||||
platform: "jd",
|
||||
ready: true,
|
||||
status: "ready"
|
||||
});
|
||||
|
||||
const clearResponse = await app.inject({
|
||||
method: "DELETE",
|
||||
url: "/api/platforms/jd/live-session"
|
||||
});
|
||||
expect(clearResponse.statusCode).toBe(204);
|
||||
|
||||
const clearedSummaryResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/jd/live-session"
|
||||
});
|
||||
expect(clearedSummaryResponse.json().session).toMatchObject({
|
||||
configured: false,
|
||||
hasCookie: false
|
||||
});
|
||||
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("exposes JD live preview endpoints through the injected service", async () => {
|
||||
const jdLiveService = createJdLiveServiceStub();
|
||||
const app = createServer({ jdLiveService });
|
||||
await app.ready();
|
||||
|
||||
const searchResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/jd/live-search-preview?query=iPhone%2015"
|
||||
});
|
||||
expect(searchResponse.statusCode).toBe(200);
|
||||
expect(searchResponse.json().preview).toMatchObject({
|
||||
query: "iPhone 15",
|
||||
source: "api",
|
||||
candidateCount: 1
|
||||
});
|
||||
|
||||
const detailResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/jd/live-detail-preview?skuId=100068388533"
|
||||
});
|
||||
expect(detailResponse.statusCode).toBe(200);
|
||||
expect(detailResponse.json().preview.detail).toMatchObject({
|
||||
skuId: "100068388533",
|
||||
shopName: "JD Self Operated"
|
||||
});
|
||||
|
||||
const reviewsResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/jd/live-reviews-preview?skuId=100068388533&commentCount=3"
|
||||
});
|
||||
expect(reviewsResponse.statusCode).toBe(200);
|
||||
expect(reviewsResponse.json().preview.reviews).toMatchObject({
|
||||
skuId: "100068388533",
|
||||
goodRate: "95%"
|
||||
});
|
||||
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("surfaces JD live preview failures with service-provided status codes", async () => {
|
||||
const jdLiveService = createJdLiveServiceStub({
|
||||
async previewDetail() {
|
||||
const error = new Error("Imported detail template is bound to another sku.") as Error & {
|
||||
statusCode: number;
|
||||
};
|
||||
error.statusCode = 409;
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
const app = createServer({ jdLiveService });
|
||||
await app.ready();
|
||||
|
||||
const response = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/jd/live-detail-preview?skuId=100068388533"
|
||||
});
|
||||
|
||||
expect(response.statusCode).toBe(409);
|
||||
expect(response.json()).toMatchObject({
|
||||
message: "Imported detail template is bound to another sku."
|
||||
});
|
||||
|
||||
await app.close();
|
||||
});
|
||||
});
|
||||
@ -49,46 +49,6 @@ describe("API server", () => {
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("exposes session state with a 24-hour expiry window after preparation", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
|
||||
const prepareResponse = await app.inject({
|
||||
method: "POST",
|
||||
url: "/api/platforms/jd/prepare"
|
||||
});
|
||||
|
||||
expect(prepareResponse.statusCode).toBe(200);
|
||||
const preparedPayload = prepareResponse.json();
|
||||
expect(preparedPayload).toMatchObject({
|
||||
platform: "jd",
|
||||
session_ready: true,
|
||||
status: "ready",
|
||||
encrypted_snapshot_available: true
|
||||
});
|
||||
expect(
|
||||
Date.parse(preparedPayload.expires_at) - Date.parse(preparedPayload.last_prepared_at)
|
||||
).toBe(24 * 60 * 60 * 1000);
|
||||
|
||||
const sessionResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/sessions/jd"
|
||||
});
|
||||
|
||||
expect(sessionResponse.statusCode).toBe(200);
|
||||
expect(sessionResponse.json().session).toMatchObject({
|
||||
platform: "jd",
|
||||
ready: true,
|
||||
status: "ready",
|
||||
scope: "workspace",
|
||||
ttlHours: 24,
|
||||
encryptedSnapshotAvailable: true,
|
||||
cipherLabel: "mock-aes-gcm-v1"
|
||||
});
|
||||
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("creates a task and lands in AwaitingConfirmation with mock candidates", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
@ -122,102 +82,6 @@ describe("API server", () => {
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("records strategy attempts and observability metrics for platform search", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
|
||||
const createdTask = await createTask(app, "iPhone 15 Pro");
|
||||
|
||||
const attemptsResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: `/api/tasks/${createdTask.taskId}/strategy-attempts`
|
||||
});
|
||||
|
||||
expect(attemptsResponse.statusCode).toBe(200);
|
||||
expect(attemptsResponse.json().attempts).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
platform: "tmall",
|
||||
capability: "search",
|
||||
outcome: "succeeded",
|
||||
layer: "L1"
|
||||
}),
|
||||
expect.objectContaining({
|
||||
platform: "jd",
|
||||
capability: "search",
|
||||
outcome: "blocked",
|
||||
layer: "L0",
|
||||
errorType: "session_required"
|
||||
})
|
||||
])
|
||||
);
|
||||
|
||||
const overviewResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/observability/overview"
|
||||
});
|
||||
|
||||
expect(overviewResponse.statusCode).toBe(200);
|
||||
expect(overviewResponse.json().overview.strategyAttempts).toMatchObject({
|
||||
total: 2,
|
||||
searchSuccessRate: 50
|
||||
});
|
||||
expect(
|
||||
overviewResponse
|
||||
.json()
|
||||
.overview.platformRuns.find((metric: { platform: string }) => metric.platform === "jd")
|
||||
).toMatchObject({
|
||||
platform: "jd",
|
||||
searchDurationMs: expect.any(Number)
|
||||
});
|
||||
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("clears a prepared session and updates the readiness summary", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
|
||||
await preparePlatform(app, "jd");
|
||||
|
||||
const deleteResponse = await app.inject({
|
||||
method: "DELETE",
|
||||
url: "/api/sessions/jd"
|
||||
});
|
||||
|
||||
expect(deleteResponse.statusCode).toBe(204);
|
||||
|
||||
const sessionResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/sessions/jd"
|
||||
});
|
||||
|
||||
expect(sessionResponse.statusCode).toBe(200);
|
||||
expect(sessionResponse.json().session).toMatchObject({
|
||||
platform: "jd",
|
||||
ready: false,
|
||||
status: "missing",
|
||||
encryptedSnapshotAvailable: false
|
||||
});
|
||||
|
||||
const readinessResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/platforms/readiness"
|
||||
});
|
||||
|
||||
expect(
|
||||
readinessResponse
|
||||
.json()
|
||||
.platforms.find((platform: { platform: string }) => platform.platform === "jd")
|
||||
).toMatchObject({
|
||||
platform: "jd",
|
||||
ready: false,
|
||||
status: "missing"
|
||||
});
|
||||
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("supports NoSelection terminal state when user confirms nothing", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
@ -317,62 +181,6 @@ describe("API server", () => {
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("records recovery audit entries and retry metrics for recovered platforms", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
|
||||
const createdTask = await createTask(app, "iPhone 15 Pro");
|
||||
|
||||
await preparePlatform(app, "jd");
|
||||
const retryResponse = await app.inject({
|
||||
method: "POST",
|
||||
url: `/api/tasks/${createdTask.taskId}/platforms/jd/retry`
|
||||
});
|
||||
|
||||
expect(retryResponse.statusCode).toBe(200);
|
||||
|
||||
const auditResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: `/api/tasks/${createdTask.taskId}/audit`
|
||||
});
|
||||
|
||||
expect(auditResponse.statusCode).toBe(200);
|
||||
expect(auditResponse.json().audit).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
action: "platform.retry_started",
|
||||
platform: "jd"
|
||||
}),
|
||||
expect.objectContaining({
|
||||
action: "task.recovery_completed",
|
||||
platform: "jd"
|
||||
}),
|
||||
expect.objectContaining({
|
||||
action: "platform.retry_completed",
|
||||
platform: "jd"
|
||||
})
|
||||
])
|
||||
);
|
||||
|
||||
const overviewResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/observability/overview"
|
||||
});
|
||||
|
||||
expect(
|
||||
overviewResponse
|
||||
.json()
|
||||
.overview.platformRuns.find((metric: { platform: string }) => metric.platform === "jd")
|
||||
).toMatchObject({
|
||||
platform: "jd",
|
||||
retryCount: 1,
|
||||
recoveryCount: 1
|
||||
});
|
||||
expect(overviewResponse.json().overview.audits.recoveryActions).toBe(2);
|
||||
|
||||
await app.close();
|
||||
});
|
||||
|
||||
it("publishes a new report version when a blocked platform recovers successfully", async () => {
|
||||
const app = createServer();
|
||||
await app.ready();
|
||||
@ -532,16 +340,6 @@ describe("API server", () => {
|
||||
.tasks.some((task: { taskId: string }) => task.taskId === createdTask.taskId)
|
||||
).toBe(false);
|
||||
|
||||
const overviewResponse = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/observability/overview"
|
||||
});
|
||||
expect(overviewResponse.json().overview.retention).toMatchObject({
|
||||
taskDeletes: 1,
|
||||
deletedReports: 1
|
||||
});
|
||||
expect(overviewResponse.json().overview.audits.deleteActions).toBe(1);
|
||||
|
||||
await app.close();
|
||||
});
|
||||
});
|
||||
|
||||
@ -6,14 +6,11 @@ import type {
|
||||
import cors from "@fastify/cors";
|
||||
import Fastify from "fastify";
|
||||
|
||||
import { JdLiveSessionService, isJdLiveError } from "./platforms/jd/live-session";
|
||||
import type { JdLiveService, JdSearchMode } from "./platforms/jd/types";
|
||||
import { InMemoryTaskStore } from "./store";
|
||||
|
||||
export function createServer(options: { jdLiveService?: JdLiveService } = {}) {
|
||||
export function createServer() {
|
||||
const app = Fastify({ logger: false });
|
||||
const store = new InMemoryTaskStore();
|
||||
const jdLiveService = options.jdLiveService ?? new JdLiveSessionService();
|
||||
|
||||
app.register(cors, { origin: true });
|
||||
|
||||
@ -26,88 +23,18 @@ export function createServer(options: { jdLiveService?: JdLiveService } = {}) {
|
||||
platforms: store.getPlatformReadiness()
|
||||
}));
|
||||
|
||||
app.get("/api/sessions", async () => ({
|
||||
sessions: store.listSessions()
|
||||
}));
|
||||
|
||||
app.get<{
|
||||
Params: { platform: PlatformId };
|
||||
}>("/api/sessions/:platform", async (request, reply) => {
|
||||
try {
|
||||
const session = store.getSession(request.params.platform);
|
||||
return { session };
|
||||
} catch {
|
||||
reply.code(404);
|
||||
return { message: "Session not found." };
|
||||
}
|
||||
});
|
||||
|
||||
app.post<{
|
||||
Params: { platform: PlatformId };
|
||||
}>("/api/platforms/:platform/prepare", async (request, reply) => {
|
||||
const session = store.preparePlatform(request.params.platform);
|
||||
const readiness = store.preparePlatform(request.params.platform);
|
||||
reply.code(200);
|
||||
return {
|
||||
platform: session.platform,
|
||||
session_ready: session.ready,
|
||||
status: session.status,
|
||||
last_prepared_at: session.lastPreparedAt,
|
||||
expires_at: session.expiresAt,
|
||||
encrypted_snapshot_available: session.encryptedSnapshotAvailable
|
||||
platform: readiness.platform,
|
||||
session_ready: readiness.ready,
|
||||
last_prepared_at: readiness.lastPreparedAt
|
||||
};
|
||||
});
|
||||
|
||||
app.get("/api/platforms/jd/live-session", async () => ({
|
||||
session: jdLiveService.getSessionSummary()
|
||||
}));
|
||||
|
||||
app.post<{
|
||||
Body: {
|
||||
cookieHeader: string;
|
||||
userAgent?: string;
|
||||
searchApiTemplateUrl?: string;
|
||||
detailTemplateUrl?: string;
|
||||
reviewsTemplateUrl?: string;
|
||||
searchReferer?: string;
|
||||
detailReferer?: string;
|
||||
};
|
||||
}>("/api/platforms/jd/live-session", async (request, reply) => {
|
||||
try {
|
||||
const session = jdLiveService.importSession(request.body);
|
||||
store.preparePlatform("jd");
|
||||
reply.code(200);
|
||||
return { session };
|
||||
} catch (error) {
|
||||
reply.code(isJdLiveError(error) ? error.statusCode : 400);
|
||||
return {
|
||||
message: error instanceof Error ? error.message : "Invalid JD live session payload."
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
app.delete("/api/platforms/jd/live-session", async (_request, reply) => {
|
||||
jdLiveService.clearSession();
|
||||
store.clearPlatformSession("jd");
|
||||
reply.code(204);
|
||||
return null;
|
||||
});
|
||||
|
||||
app.delete<{
|
||||
Params: { platform: PlatformId };
|
||||
}>("/api/sessions/:platform", async (request, reply) => {
|
||||
try {
|
||||
store.clearPlatformSession(request.params.platform);
|
||||
if (request.params.platform === "jd") {
|
||||
jdLiveService.clearSession();
|
||||
}
|
||||
reply.code(204);
|
||||
return null;
|
||||
} catch {
|
||||
reply.code(404);
|
||||
return { message: "Session not found." };
|
||||
}
|
||||
});
|
||||
|
||||
app.post<{
|
||||
Body: CreateTaskInput;
|
||||
}>("/api/tasks", async (request, reply) => {
|
||||
@ -151,30 +78,6 @@ export function createServer(options: { jdLiveService?: JdLiveService } = {}) {
|
||||
return { candidates };
|
||||
});
|
||||
|
||||
app.get<{
|
||||
Params: { taskId: string };
|
||||
}>("/api/tasks/:taskId/strategy-attempts", async (request, reply) => {
|
||||
const attempts = store.getTaskStrategyAttempts(request.params.taskId);
|
||||
if (!attempts) {
|
||||
reply.code(404);
|
||||
return { message: "Task not found." };
|
||||
}
|
||||
|
||||
return { attempts };
|
||||
});
|
||||
|
||||
app.get<{
|
||||
Params: { taskId: string };
|
||||
}>("/api/tasks/:taskId/audit", async (request, reply) => {
|
||||
const audit = store.getTaskAuditLogs(request.params.taskId);
|
||||
if (!audit) {
|
||||
reply.code(404);
|
||||
return { message: "Task not found." };
|
||||
}
|
||||
|
||||
return { audit };
|
||||
});
|
||||
|
||||
app.post<{
|
||||
Params: { taskId: string };
|
||||
Body: ConfirmTaskPayload;
|
||||
@ -213,80 +116,10 @@ export function createServer(options: { jdLiveService?: JdLiveService } = {}) {
|
||||
return { report };
|
||||
});
|
||||
|
||||
app.get<{
|
||||
Querystring: { query?: string; mode?: JdSearchMode };
|
||||
}>("/api/platforms/jd/live-search-preview", async (request, reply) => {
|
||||
try {
|
||||
const query = request.query.query?.trim();
|
||||
if (!query) {
|
||||
reply.code(400);
|
||||
return { message: "query is required." };
|
||||
}
|
||||
|
||||
const preview = await jdLiveService.previewSearch(query, request.query.mode);
|
||||
return { preview };
|
||||
} catch (error) {
|
||||
reply.code(isJdLiveError(error) ? error.statusCode : 502);
|
||||
return {
|
||||
message:
|
||||
error instanceof Error ? error.message : "JD live search preview failed."
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
app.get<{
|
||||
Querystring: { skuId?: string };
|
||||
}>("/api/platforms/jd/live-detail-preview", async (request, reply) => {
|
||||
try {
|
||||
const skuId = request.query.skuId?.trim();
|
||||
if (!skuId) {
|
||||
reply.code(400);
|
||||
return { message: "skuId is required." };
|
||||
}
|
||||
|
||||
const preview = await jdLiveService.previewDetail(skuId);
|
||||
return { preview };
|
||||
} catch (error) {
|
||||
reply.code(isJdLiveError(error) ? error.statusCode : 502);
|
||||
return {
|
||||
message:
|
||||
error instanceof Error ? error.message : "JD live detail preview failed."
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
app.get<{
|
||||
Querystring: { skuId?: string; commentCount?: string };
|
||||
}>("/api/platforms/jd/live-reviews-preview", async (request, reply) => {
|
||||
try {
|
||||
const skuId = request.query.skuId?.trim();
|
||||
if (!skuId) {
|
||||
reply.code(400);
|
||||
return { message: "skuId is required." };
|
||||
}
|
||||
|
||||
const commentCount = request.query.commentCount
|
||||
? Number.parseInt(request.query.commentCount, 10)
|
||||
: undefined;
|
||||
const preview = await jdLiveService.previewReviews(skuId, commentCount);
|
||||
return { preview };
|
||||
} catch (error) {
|
||||
reply.code(isJdLiveError(error) ? error.statusCode : 502);
|
||||
return {
|
||||
message:
|
||||
error instanceof Error ? error.message : "JD live reviews preview failed."
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
app.get("/api/history", async () => ({
|
||||
tasks: store.listHistory()
|
||||
}));
|
||||
|
||||
app.get("/api/observability/overview", async () => ({
|
||||
overview: store.getObservabilityOverview()
|
||||
}));
|
||||
|
||||
app.get<{
|
||||
Params: { taskId: string };
|
||||
}>("/api/tasks/:taskId/events", async (request, reply) => {
|
||||
|
||||
@ -1,32 +1,17 @@
|
||||
import {
|
||||
auditActions,
|
||||
deriveTaskStatusFromConfirmedPlatforms,
|
||||
mapPlatformStatusToExecutionStatus,
|
||||
platformCatalog,
|
||||
platformCatalogMap,
|
||||
platforms,
|
||||
strategyAttemptOutcomes,
|
||||
strategyLayers,
|
||||
type AuditAction,
|
||||
type AuditLogRecord,
|
||||
type CandidateRecord,
|
||||
type ConfirmTaskPayload,
|
||||
type CreateTaskInput,
|
||||
type HistoryTaskRecord,
|
||||
type ObservabilityOverview,
|
||||
type PlatformId,
|
||||
type PlatformCapability,
|
||||
type PlatformRunMetricRecord,
|
||||
type PlatformRunRecord,
|
||||
type PlatformStatus,
|
||||
type ReportMetricRecord,
|
||||
type RetentionMetricRecord,
|
||||
type SessionReadinessRecord,
|
||||
type SessionStateRecord,
|
||||
type StrategyAttemptOutcome,
|
||||
type StrategyAttemptRecord,
|
||||
type StrategyAttemptTrigger,
|
||||
type StrategyLayer,
|
||||
type TaskEventRecord,
|
||||
type TaskRecord
|
||||
} from "@cross-ai/domain";
|
||||
@ -46,135 +31,6 @@ function createPlatformCandidatesRecord(): Record<PlatformId, CandidateRecord[]>
|
||||
};
|
||||
}
|
||||
|
||||
function createPlatformRunMetricsRecord(
|
||||
taskId: string,
|
||||
timestamp: string
|
||||
): Record<PlatformId, PlatformRunMetricRecord> {
|
||||
return {
|
||||
tmall: {
|
||||
taskId,
|
||||
platform: "tmall",
|
||||
searchDurationMs: 0,
|
||||
detailDurationMs: 0,
|
||||
reviewsDurationMs: 0,
|
||||
retryCount: 0,
|
||||
recoveryCount: 0,
|
||||
lastUpdatedAt: timestamp
|
||||
},
|
||||
jd: {
|
||||
taskId,
|
||||
platform: "jd",
|
||||
searchDurationMs: 0,
|
||||
detailDurationMs: 0,
|
||||
reviewsDurationMs: 0,
|
||||
retryCount: 0,
|
||||
recoveryCount: 0,
|
||||
lastUpdatedAt: timestamp
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const mockCapabilityDurations: Record<
|
||||
Exclude<PlatformCapability, "login">,
|
||||
Record<PlatformId, number>
|
||||
> = {
|
||||
search: {
|
||||
tmall: 420,
|
||||
jd: 760
|
||||
},
|
||||
detail: {
|
||||
tmall: 260,
|
||||
jd: 320
|
||||
},
|
||||
reviews: {
|
||||
tmall: 880,
|
||||
jd: 980
|
||||
}
|
||||
};
|
||||
|
||||
const mockLoginDurations: Record<PlatformId, number> = {
|
||||
tmall: 640,
|
||||
jd: 720
|
||||
};
|
||||
|
||||
function resolveDefaultLayer(
|
||||
platform: PlatformId,
|
||||
capability: PlatformCapability
|
||||
): StrategyLayer {
|
||||
if (capability === "login") {
|
||||
return "L3";
|
||||
}
|
||||
|
||||
if (capability === "search") {
|
||||
return platform === "jd" ? "L0" : "L1";
|
||||
}
|
||||
|
||||
return capability === "detail" && platform === "jd" ? "L0" : "L1";
|
||||
}
|
||||
|
||||
function getMockDurationMs(
|
||||
platform: PlatformId,
|
||||
capability: PlatformCapability,
|
||||
outcome: StrategyAttemptOutcome
|
||||
): number {
|
||||
const baseDuration =
|
||||
capability === "login"
|
||||
? mockLoginDurations[platform]
|
||||
: mockCapabilityDurations[capability][platform];
|
||||
|
||||
switch (outcome) {
|
||||
case "blocked":
|
||||
return baseDuration + 180;
|
||||
case "failed":
|
||||
return baseDuration + 220;
|
||||
case "no_result":
|
||||
return Math.max(120, baseDuration - 140);
|
||||
case "skipped":
|
||||
return Math.max(80, baseDuration - 200);
|
||||
default:
|
||||
return baseDuration;
|
||||
}
|
||||
}
|
||||
|
||||
const SESSION_TTL_HOURS = 24;
|
||||
const SESSION_TTL_MS = SESSION_TTL_HOURS * 60 * 60 * 1000;
|
||||
|
||||
type StoredSessionState = Omit<SessionStateRecord, "encryptedSnapshotAvailable"> & {
|
||||
encryptedSnapshot: string | null;
|
||||
};
|
||||
|
||||
function addDurationMs(timestamp: string, durationMs: number): string {
|
||||
return new Date(Date.parse(timestamp) + durationMs).toISOString();
|
||||
}
|
||||
|
||||
function createEncryptedSnapshot(platform: PlatformId, timestamp: string): string {
|
||||
return `mock-aes-gcm-v1:${platform}:${Buffer.from(timestamp).toString("base64url")}`;
|
||||
}
|
||||
|
||||
function createMissingSession(platform: PlatformId): StoredSessionState {
|
||||
return {
|
||||
platform,
|
||||
ready: false,
|
||||
status: "missing",
|
||||
searchRequirement: platformCatalogMap[platform].searchRequirement,
|
||||
scope: "workspace",
|
||||
ttlHours: SESSION_TTL_HOURS,
|
||||
encryptedSnapshot: null
|
||||
};
|
||||
}
|
||||
|
||||
function createPreparedSession(platform: PlatformId, timestamp: string): StoredSessionState {
|
||||
return {
|
||||
...createMissingSession(platform),
|
||||
ready: true,
|
||||
status: "ready",
|
||||
lastPreparedAt: timestamp,
|
||||
expiresAt: addDurationMs(timestamp, SESSION_TTL_MS),
|
||||
encryptedSnapshot: createEncryptedSnapshot(platform, timestamp),
|
||||
cipherLabel: "mock-aes-gcm-v1"
|
||||
};
|
||||
}
|
||||
|
||||
type MockExecutionScenario = {
|
||||
outcome: Extract<PlatformStatus, "Blocked" | "Failed">;
|
||||
mode: "once" | "always";
|
||||
@ -222,63 +78,39 @@ export class InMemoryTaskStore {
|
||||
private readonly tasks = new Map<string, TaskRecord>();
|
||||
private readonly reports = new Map<string, ReportSnapshot[]>();
|
||||
private readonly reportFingerprints = new Map<string, string[]>();
|
||||
private readonly readiness = new Map<PlatformId, StoredSessionState>();
|
||||
private readonly strategyAttempts = new Map<string, StrategyAttemptRecord[]>();
|
||||
private readonly platformRunMetrics = new Map<
|
||||
string,
|
||||
Record<PlatformId, PlatformRunMetricRecord>
|
||||
>();
|
||||
private readonly reportMetrics: ReportMetricRecord[] = [];
|
||||
private readonly retentionMetrics: RetentionMetricRecord[] = [];
|
||||
private readonly auditLogs: AuditLogRecord[] = [];
|
||||
private readonly readiness = new Map<PlatformId, SessionReadinessRecord>();
|
||||
private readonly executionScenarios = new Map<
|
||||
string,
|
||||
Partial<Record<PlatformId, MockExecutionScenario>>
|
||||
>();
|
||||
|
||||
constructor() {
|
||||
const timestamp = nowIso();
|
||||
this.readiness.set("tmall", createPreparedSession("tmall", timestamp));
|
||||
this.readiness.set("jd", createMissingSession("jd"));
|
||||
this.readiness.set("tmall", {
|
||||
platform: "tmall",
|
||||
ready: true,
|
||||
searchRequirement: "recommended",
|
||||
lastPreparedAt: nowIso()
|
||||
});
|
||||
this.readiness.set("jd", {
|
||||
platform: "jd",
|
||||
ready: false,
|
||||
searchRequirement: "required"
|
||||
});
|
||||
}
|
||||
|
||||
getPlatformReadiness(): SessionReadinessRecord[] {
|
||||
return platforms.map((platform) => this.requireReadiness(platform));
|
||||
}
|
||||
|
||||
listSessions(): SessionStateRecord[] {
|
||||
return platforms.map((platform) => this.getSession(platform));
|
||||
}
|
||||
|
||||
getSession(platform: PlatformId): SessionStateRecord {
|
||||
return this.toSessionRecord(this.requireSession(platform));
|
||||
}
|
||||
|
||||
preparePlatform(platform: PlatformId): SessionStateRecord {
|
||||
const timestamp = nowIso();
|
||||
const next = createPreparedSession(platform, timestamp);
|
||||
preparePlatform(platform: PlatformId): SessionReadinessRecord {
|
||||
const readiness = this.requireReadiness(platform);
|
||||
const next: SessionReadinessRecord = {
|
||||
...readiness,
|
||||
ready: true,
|
||||
lastPreparedAt: nowIso()
|
||||
};
|
||||
this.readiness.set(platform, next);
|
||||
this.pushAudit("session.prepared", {
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 会话已标记为预热完成。`,
|
||||
metadata: {
|
||||
search_requirement: next.searchRequirement,
|
||||
expires_at: next.expiresAt ?? null
|
||||
}
|
||||
});
|
||||
return this.toSessionRecord(next);
|
||||
}
|
||||
|
||||
clearPlatformSession(platform: PlatformId): void {
|
||||
const cleared = createMissingSession(platform);
|
||||
this.readiness.set(platform, cleared);
|
||||
this.pushAudit("session.cleared", {
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 会话已清理。`,
|
||||
metadata: {
|
||||
search_requirement: cleared.searchRequirement
|
||||
}
|
||||
});
|
||||
return next;
|
||||
}
|
||||
|
||||
createTask(input: CreateTaskInput): TaskRecord {
|
||||
@ -309,9 +141,6 @@ export class InMemoryTaskStore {
|
||||
reportVersions: []
|
||||
};
|
||||
|
||||
this.strategyAttempts.set(taskId, []);
|
||||
this.platformRunMetrics.set(taskId, createPlatformRunMetricsRecord(taskId, timestamp));
|
||||
|
||||
if (Object.keys(executionScenarios).length > 0) {
|
||||
this.executionScenarios.set(taskId, executionScenarios);
|
||||
}
|
||||
@ -355,93 +184,6 @@ export class InMemoryTaskStore {
|
||||
return this.tasks.get(taskId)?.platformCandidates;
|
||||
}
|
||||
|
||||
getTaskStrategyAttempts(taskId: string): StrategyAttemptRecord[] | undefined {
|
||||
if (!this.tasks.has(taskId)) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return [...(this.strategyAttempts.get(taskId) ?? [])];
|
||||
}
|
||||
|
||||
getTaskAuditLogs(taskId: string): AuditLogRecord[] | undefined {
|
||||
if (!this.tasks.has(taskId)) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return this.auditLogs.filter((entry) => entry.taskId === taskId);
|
||||
}
|
||||
|
||||
getObservabilityOverview(): ObservabilityOverview {
|
||||
const strategyAttempts = Array.from(this.strategyAttempts.values()).flat();
|
||||
const totalAttempts = strategyAttempts.length;
|
||||
const searchAttempts = strategyAttempts.filter((attempt) => attempt.capability === "search");
|
||||
const successfulSearchAttempts = searchAttempts.filter(
|
||||
(attempt) => attempt.outcome === "succeeded"
|
||||
);
|
||||
const platformRuns = Array.from(this.platformRunMetrics.values()).flatMap((record) =>
|
||||
platforms.map((platform) => record[platform])
|
||||
);
|
||||
|
||||
return {
|
||||
strategyAttempts: {
|
||||
total: totalAttempts,
|
||||
searchSuccessRate:
|
||||
searchAttempts.length > 0
|
||||
? Number(
|
||||
((successfulSearchAttempts.length / searchAttempts.length) * 100).toFixed(2)
|
||||
)
|
||||
: 0,
|
||||
browserFallbackShare:
|
||||
totalAttempts > 0
|
||||
? Number(
|
||||
(
|
||||
(strategyAttempts.filter((attempt) => attempt.layer === "L3").length /
|
||||
totalAttempts) *
|
||||
100
|
||||
).toFixed(2)
|
||||
)
|
||||
: 0,
|
||||
byLayer: strategyLayers.map((layer) => ({
|
||||
layer,
|
||||
count: strategyAttempts.filter((attempt) => attempt.layer === layer).length
|
||||
})),
|
||||
byOutcome: strategyAttemptOutcomes.map((outcome) => ({
|
||||
outcome,
|
||||
count: strategyAttempts.filter((attempt) => attempt.outcome === outcome).length
|
||||
}))
|
||||
},
|
||||
platformRuns,
|
||||
reports: {
|
||||
published: this.reportMetrics.length,
|
||||
unchanged: this.auditLogs.filter((entry) => entry.action === "report.unchanged").length,
|
||||
sampleInsufficient: this.reportMetrics.filter((metric) => metric.sampleInsufficient).length,
|
||||
partialFailures: this.reportMetrics.filter((metric) => metric.partialPlatformFailure).length
|
||||
},
|
||||
retention: {
|
||||
taskDeletes: this.retentionMetrics.filter((metric) => metric.action === "task_deleted")
|
||||
.length,
|
||||
cleanupRuns: this.retentionMetrics.filter((metric) => metric.action === "retention_cleanup")
|
||||
.length,
|
||||
deletedReports: this.retentionMetrics.reduce(
|
||||
(sum, metric) => sum + metric.deletedReportCount,
|
||||
0
|
||||
),
|
||||
residualArtifacts: this.retentionMetrics.reduce(
|
||||
(sum, metric) => sum + metric.residualArtifactCount,
|
||||
0
|
||||
)
|
||||
},
|
||||
audits: {
|
||||
total: this.auditLogs.length,
|
||||
recoveryActions: this.auditLogs.filter(
|
||||
(entry) =>
|
||||
entry.action === "session.prepared" || entry.action === "task.recovery_completed"
|
||||
).length,
|
||||
deleteActions: this.auditLogs.filter((entry) => entry.action === "task.deleted").length
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
confirmTask(taskId: string, payload: ConfirmTaskPayload): TaskRecord {
|
||||
const task = this.requireTask(taskId);
|
||||
const selectionMap = new Map(
|
||||
@ -511,27 +253,12 @@ export class InMemoryTaskStore {
|
||||
return task;
|
||||
}
|
||||
|
||||
this.incrementRetryCount(task.taskId, platform);
|
||||
this.incrementRecoveryCount(task.taskId, platform);
|
||||
this.pushEvent(
|
||||
task,
|
||||
`platform.${platform}.retry_started`,
|
||||
`${platformCatalogMap[platform].label} 正在重新获取候选结果。`
|
||||
);
|
||||
this.pushAudit("platform.retry_started", {
|
||||
taskId,
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 已发起候选重试。`,
|
||||
metadata: {
|
||||
previous_status: "SearchBlocked"
|
||||
}
|
||||
});
|
||||
this.pushAudit("task.recovery_completed", {
|
||||
taskId,
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 已完成恢复,准备重新搜索。`
|
||||
});
|
||||
this.runSearchForPlatform(task, run, "recovery");
|
||||
this.runSearchForPlatform(task, run);
|
||||
const recoveredCandidateCount = task.platformCandidates[platform].length;
|
||||
task.taskStage = "confirmation";
|
||||
task.taskStatus = "AwaitingConfirmation";
|
||||
@ -543,18 +270,6 @@ export class InMemoryTaskStore {
|
||||
? `${platformCatalogMap[platform].label} 已恢复候选确认。`
|
||||
: `${platformCatalogMap[platform].label} 重试后仍未返回候选结果。`
|
||||
);
|
||||
this.pushAudit("platform.retry_completed", {
|
||||
taskId,
|
||||
platform,
|
||||
message:
|
||||
recoveredCandidateCount > 0
|
||||
? `${platformCatalogMap[platform].label} 候选重试已恢复。`
|
||||
: `${platformCatalogMap[platform].label} 候选重试后仍无结果。`,
|
||||
metadata: {
|
||||
candidate_count: recoveredCandidateCount,
|
||||
next_status: run.status
|
||||
}
|
||||
});
|
||||
return task;
|
||||
}
|
||||
|
||||
@ -567,16 +282,6 @@ export class InMemoryTaskStore {
|
||||
return task;
|
||||
}
|
||||
|
||||
const previousStatus = run.status;
|
||||
this.incrementRetryCount(task.taskId, platform);
|
||||
if (previousStatus === "Blocked") {
|
||||
this.incrementRecoveryCount(task.taskId, platform);
|
||||
this.pushAudit("task.recovery_completed", {
|
||||
taskId,
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 已完成恢复,准备重新执行。`
|
||||
});
|
||||
}
|
||||
task.taskStatus = "Running";
|
||||
task.taskStage = "session_check";
|
||||
this.pushEvent(
|
||||
@ -584,28 +289,11 @@ export class InMemoryTaskStore {
|
||||
`platform.${platform}.retry_started`,
|
||||
`${platformCatalogMap[platform].label} 正在执行定向重试。`
|
||||
);
|
||||
this.pushAudit("platform.retry_started", {
|
||||
taskId,
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 已发起平台级重试。`,
|
||||
metadata: {
|
||||
previous_status: previousStatus
|
||||
}
|
||||
});
|
||||
|
||||
this.executeSelectedPlatforms(task, [run], previousStatus === "Blocked" ? "recovery" : "retry");
|
||||
this.executeSelectedPlatforms(task, [run]);
|
||||
task.taskStatus = deriveTaskStatusFromConfirmedPlatforms(task.platformRuns);
|
||||
task.updatedAt = nowIso();
|
||||
this.publishReportIfNeeded(task);
|
||||
this.pushAudit("platform.retry_completed", {
|
||||
taskId,
|
||||
platform,
|
||||
message: `${platformCatalogMap[platform].label} 平台级重试已结束。`,
|
||||
metadata: {
|
||||
next_status: run.status,
|
||||
task_status: task.taskStatus
|
||||
}
|
||||
});
|
||||
|
||||
return task;
|
||||
}
|
||||
@ -624,29 +312,10 @@ export class InMemoryTaskStore {
|
||||
}
|
||||
|
||||
deleteTask(taskId: string): void {
|
||||
const task = this.requireTask(taskId);
|
||||
const deletedReportCount = this.reports.get(taskId)?.length ?? 0;
|
||||
this.retentionMetrics.push({
|
||||
metricId: randomUUID(),
|
||||
action: "task_deleted",
|
||||
taskId,
|
||||
deletedReportCount,
|
||||
deletedArtifactCount: 0,
|
||||
residualArtifactCount: 0,
|
||||
recordedAt: nowIso()
|
||||
});
|
||||
this.pushAudit("task.deleted", {
|
||||
taskId,
|
||||
message: `任务“${task.query}”及其关联报告已删除。`,
|
||||
metadata: {
|
||||
deleted_report_count: deletedReportCount
|
||||
}
|
||||
});
|
||||
this.requireTask(taskId);
|
||||
this.tasks.delete(taskId);
|
||||
this.reports.delete(taskId);
|
||||
this.reportFingerprints.delete(taskId);
|
||||
this.strategyAttempts.delete(taskId);
|
||||
this.platformRunMetrics.delete(taskId);
|
||||
this.executionScenarios.delete(taskId);
|
||||
}
|
||||
|
||||
@ -835,28 +504,13 @@ export class InMemoryTaskStore {
|
||||
});
|
||||
}
|
||||
|
||||
private runSearchForPlatform(
|
||||
task: TaskRecord,
|
||||
run: PlatformRunRecord,
|
||||
trigger: StrategyAttemptTrigger = "system"
|
||||
): void {
|
||||
private runSearchForPlatform(task: TaskRecord, run: PlatformRunRecord): void {
|
||||
const readiness = this.requireReadiness(run.platform);
|
||||
if (readiness.searchRequirement === "required" && !readiness.ready) {
|
||||
run.status = "SearchBlocked";
|
||||
run.reason = platformCatalogMap[run.platform].recoveryHint;
|
||||
run.candidateCount = 0;
|
||||
run.lastUpdatedAt = nowIso();
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"search",
|
||||
"blocked",
|
||||
trigger,
|
||||
`${platformCatalogMap[run.platform].label} 搜索前检查缺少有效会话。`,
|
||||
{
|
||||
errorType: "session_required"
|
||||
}
|
||||
);
|
||||
this.pushEvent(
|
||||
task,
|
||||
`platform.${run.platform}.search_blocked`,
|
||||
@ -877,21 +531,6 @@ export class InMemoryTaskStore {
|
||||
? undefined
|
||||
: `${platformCatalogMap[run.platform].label} 当前未找到可供确认的候选。`;
|
||||
run.lastUpdatedAt = nowIso();
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"search",
|
||||
candidates.length > 0 ? "succeeded" : "no_result",
|
||||
trigger,
|
||||
candidates.length > 0
|
||||
? `${platformCatalogMap[run.platform].label} 已返回候选列表。`
|
||||
: `${platformCatalogMap[run.platform].label} 本轮搜索没有候选结果。`,
|
||||
candidates.length > 0
|
||||
? undefined
|
||||
: {
|
||||
errorType: "no_candidates"
|
||||
}
|
||||
);
|
||||
this.pushEvent(
|
||||
task,
|
||||
`platform.${run.platform}.searched`,
|
||||
@ -903,8 +542,7 @@ export class InMemoryTaskStore {
|
||||
|
||||
private executeSelectedPlatforms(
|
||||
task: TaskRecord,
|
||||
runs: PlatformRunRecord[],
|
||||
trigger: StrategyAttemptTrigger = "system"
|
||||
runs: PlatformRunRecord[]
|
||||
): void {
|
||||
task.taskStage = "crawl";
|
||||
|
||||
@ -914,18 +552,6 @@ export class InMemoryTaskStore {
|
||||
run.status = "Blocked";
|
||||
run.reason = platformCatalogMap[run.platform].recoveryHint;
|
||||
run.lastUpdatedAt = nowIso();
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"login",
|
||||
"blocked",
|
||||
trigger,
|
||||
`${platformCatalogMap[run.platform].label} 抓取前校验失败,需要先恢复会话。`,
|
||||
{
|
||||
layer: "L3",
|
||||
errorType: "session_required"
|
||||
}
|
||||
);
|
||||
this.pushEvent(
|
||||
task,
|
||||
`platform.${run.platform}.blocked`,
|
||||
@ -939,18 +565,6 @@ export class InMemoryTaskStore {
|
||||
run.status = "Blocked";
|
||||
run.reason = `${platformCatalogMap[run.platform].label} 命中模拟阻塞,需要先进入恢复流程。`;
|
||||
run.lastUpdatedAt = nowIso();
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"login",
|
||||
"blocked",
|
||||
trigger,
|
||||
`${platformCatalogMap[run.platform].label} 命中模拟阻塞,需要浏览器恢复。`,
|
||||
{
|
||||
layer: "L3",
|
||||
errorType: "platform_blocked"
|
||||
}
|
||||
);
|
||||
this.pushEvent(
|
||||
task,
|
||||
`platform.${run.platform}.blocked`,
|
||||
@ -967,27 +581,8 @@ export class InMemoryTaskStore {
|
||||
`platform.${run.platform}.running`,
|
||||
`${platformCatalogMap[run.platform].label} 开始抓取商品与评论。`
|
||||
);
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"detail",
|
||||
"succeeded",
|
||||
trigger,
|
||||
`${platformCatalogMap[run.platform].label} 已通过 ${resolveDefaultLayer(run.platform, "detail")} 抓取商品详情。`
|
||||
);
|
||||
|
||||
if (mockOutcome === "Failed") {
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"reviews",
|
||||
"failed",
|
||||
trigger,
|
||||
`${platformCatalogMap[run.platform].label} 评论抓取失败,可稍后定向重试。`,
|
||||
{
|
||||
errorType: "mock_failure"
|
||||
}
|
||||
);
|
||||
run.status = "Failed";
|
||||
run.reason = `${platformCatalogMap[run.platform].label} 命中模拟失败,可稍后发起定向重试。`;
|
||||
run.lastUpdatedAt = nowIso();
|
||||
@ -999,14 +594,6 @@ export class InMemoryTaskStore {
|
||||
continue;
|
||||
}
|
||||
|
||||
this.recordStrategyAttempt(
|
||||
task.taskId,
|
||||
run.platform,
|
||||
"reviews",
|
||||
"succeeded",
|
||||
trigger,
|
||||
`${platformCatalogMap[run.platform].label} 已完成评论抓取与抽样。`
|
||||
);
|
||||
run.status = "Completed";
|
||||
run.reason = undefined;
|
||||
run.lastUpdatedAt = nowIso();
|
||||
@ -1048,13 +635,6 @@ export class InMemoryTaskStore {
|
||||
|
||||
if (lastFingerprint === fingerprint) {
|
||||
this.pushEvent(task, "task.report_unchanged", "本轮重试未改变报告结果,沿用当前版本。");
|
||||
this.pushAudit("report.unchanged", {
|
||||
taskId: task.taskId,
|
||||
message: "报告结果未变化,继续沿用当前版本。",
|
||||
metadata: {
|
||||
task_status: task.taskStatus
|
||||
}
|
||||
});
|
||||
return false;
|
||||
}
|
||||
|
||||
@ -1065,28 +645,7 @@ export class InMemoryTaskStore {
|
||||
task.reportVersions = [...task.reportVersions, report.report_version];
|
||||
task.defaultReportVersion = report.report_version;
|
||||
task.latestSuccessfulReportVersion = report.report_version;
|
||||
this.reportMetrics.push({
|
||||
taskId: task.taskId,
|
||||
reportVersion: report.report_version,
|
||||
taskStatus: task.taskStatus,
|
||||
selectedLinkCount: report.product_snapshot.selected_link_count,
|
||||
reviewSampleCount: report.product_snapshot.review_sample_count,
|
||||
blockedPlatforms: report.quality_flags.blocked_platforms,
|
||||
failedPlatforms: report.quality_flags.failed_platforms,
|
||||
sampleInsufficient: report.quality_flags.sample_insufficient,
|
||||
partialPlatformFailure: report.quality_flags.partial_platform_failure,
|
||||
generatedAt: report.generated_at
|
||||
});
|
||||
this.pushEvent(task, "task.report_published", `报告 v${report.report_version} 已生成。`);
|
||||
this.pushAudit("report.published", {
|
||||
taskId: task.taskId,
|
||||
message: `报告 v${report.report_version} 已生成。`,
|
||||
metadata: {
|
||||
report_version: report.report_version,
|
||||
task_status: task.taskStatus,
|
||||
review_sample_count: report.product_snapshot.review_sample_count
|
||||
}
|
||||
});
|
||||
|
||||
return true;
|
||||
}
|
||||
@ -1132,98 +691,6 @@ export class InMemoryTaskStore {
|
||||
return scenario.outcome;
|
||||
}
|
||||
|
||||
private recordStrategyAttempt(
|
||||
taskId: string,
|
||||
platform: PlatformId,
|
||||
capability: PlatformCapability,
|
||||
outcome: StrategyAttemptOutcome,
|
||||
trigger: StrategyAttemptTrigger,
|
||||
detail: string,
|
||||
options?: {
|
||||
layer?: StrategyLayer;
|
||||
errorType?: string;
|
||||
}
|
||||
): StrategyAttemptRecord {
|
||||
const durationMs = getMockDurationMs(platform, capability, outcome);
|
||||
const finishedAt = nowIso();
|
||||
const startedAt = new Date(Date.parse(finishedAt) - durationMs).toISOString();
|
||||
const attempt: StrategyAttemptRecord = {
|
||||
attemptId: randomUUID(),
|
||||
taskId,
|
||||
platform,
|
||||
capability,
|
||||
layer: options?.layer ?? resolveDefaultLayer(platform, capability),
|
||||
outcome,
|
||||
trigger,
|
||||
startedAt,
|
||||
finishedAt,
|
||||
durationMs,
|
||||
errorType: options?.errorType,
|
||||
detail
|
||||
};
|
||||
const attempts = this.strategyAttempts.get(taskId) ?? [];
|
||||
attempts.push(attempt);
|
||||
this.strategyAttempts.set(taskId, attempts);
|
||||
this.addCapabilityDuration(taskId, platform, capability, durationMs);
|
||||
return attempt;
|
||||
}
|
||||
|
||||
private addCapabilityDuration(
|
||||
taskId: string,
|
||||
platform: PlatformId,
|
||||
capability: PlatformCapability,
|
||||
durationMs: number
|
||||
): void {
|
||||
const metrics = this.requirePlatformMetrics(taskId);
|
||||
const target = metrics[platform];
|
||||
|
||||
if (capability === "search") {
|
||||
target.searchDurationMs += durationMs;
|
||||
} else if (capability === "detail") {
|
||||
target.detailDurationMs += durationMs;
|
||||
} else if (capability === "reviews") {
|
||||
target.reviewsDurationMs += durationMs;
|
||||
}
|
||||
|
||||
target.lastUpdatedAt = nowIso();
|
||||
}
|
||||
|
||||
private incrementRetryCount(taskId: string, platform: PlatformId): void {
|
||||
const metrics = this.requirePlatformMetrics(taskId);
|
||||
metrics[platform].retryCount += 1;
|
||||
metrics[platform].lastUpdatedAt = nowIso();
|
||||
}
|
||||
|
||||
private incrementRecoveryCount(taskId: string, platform: PlatformId): void {
|
||||
const metrics = this.requirePlatformMetrics(taskId);
|
||||
metrics[platform].recoveryCount += 1;
|
||||
metrics[platform].lastUpdatedAt = nowIso();
|
||||
}
|
||||
|
||||
private pushAudit(
|
||||
action: AuditAction,
|
||||
input: {
|
||||
message: string;
|
||||
taskId?: string;
|
||||
platform?: PlatformId;
|
||||
metadata?: AuditLogRecord["metadata"];
|
||||
}
|
||||
): void {
|
||||
if (!auditActions.includes(action)) {
|
||||
throw new Error(`Unsupported audit action ${action}.`);
|
||||
}
|
||||
|
||||
this.auditLogs.push({
|
||||
auditId: randomUUID(),
|
||||
taskId: input.taskId,
|
||||
platform: input.platform,
|
||||
action,
|
||||
message: input.message,
|
||||
createdAt: nowIso(),
|
||||
metadata: input.metadata
|
||||
});
|
||||
}
|
||||
|
||||
private pushEvent(task: TaskRecord, type: string, message: string): void {
|
||||
const event: TaskEventRecord = {
|
||||
eventId: randomUUID(),
|
||||
@ -1243,82 +710,11 @@ export class InMemoryTaskStore {
|
||||
return task;
|
||||
}
|
||||
|
||||
private requirePlatformMetrics(
|
||||
taskId: string
|
||||
): Record<PlatformId, PlatformRunMetricRecord> {
|
||||
const metrics = this.platformRunMetrics.get(taskId);
|
||||
if (!metrics) {
|
||||
throw new Error(`Platform metrics for task ${taskId} not found.`);
|
||||
}
|
||||
return metrics;
|
||||
}
|
||||
|
||||
private toSessionRecord(session: StoredSessionState): SessionStateRecord {
|
||||
return {
|
||||
platform: session.platform,
|
||||
ready: session.ready,
|
||||
status: session.status,
|
||||
searchRequirement: session.searchRequirement,
|
||||
scope: session.scope,
|
||||
ttlHours: session.ttlHours,
|
||||
lastPreparedAt: session.lastPreparedAt,
|
||||
expiresAt: session.expiresAt,
|
||||
encryptedSnapshotAvailable: Boolean(session.encryptedSnapshot),
|
||||
cipherLabel: session.encryptedSnapshot ? session.cipherLabel : undefined
|
||||
};
|
||||
}
|
||||
|
||||
private toReadiness(session: StoredSessionState): SessionReadinessRecord {
|
||||
return {
|
||||
platform: session.platform,
|
||||
ready: session.ready,
|
||||
status: session.status,
|
||||
searchRequirement: session.searchRequirement,
|
||||
reason: this.getSessionReason(session),
|
||||
lastPreparedAt: session.lastPreparedAt,
|
||||
expiresAt: session.expiresAt
|
||||
};
|
||||
}
|
||||
|
||||
private getSessionReason(session: StoredSessionState): string {
|
||||
switch (session.status) {
|
||||
case "ready":
|
||||
return "当前工作区存在可复用会话,创建任务时会再次校验。";
|
||||
case "expired":
|
||||
return "最近一次会话已过期,请重新完成会话准备。";
|
||||
default:
|
||||
return session.searchRequirement === "required"
|
||||
? platformCatalogMap[session.platform].recoveryHint
|
||||
: "当前没有可复用会话,建议先预热以提升搜索稳定性。";
|
||||
}
|
||||
}
|
||||
|
||||
private requireSession(platform: PlatformId): StoredSessionState {
|
||||
const session = this.readiness.get(platform);
|
||||
if (!session) {
|
||||
private requireReadiness(platform: PlatformId): SessionReadinessRecord {
|
||||
const readiness = this.readiness.get(platform);
|
||||
if (!readiness) {
|
||||
throw new Error(`Platform ${platform} not found.`);
|
||||
}
|
||||
|
||||
if (
|
||||
session.status === "ready" &&
|
||||
session.expiresAt &&
|
||||
Date.parse(session.expiresAt) <= Date.now()
|
||||
) {
|
||||
const expired: StoredSessionState = {
|
||||
...session,
|
||||
ready: false,
|
||||
status: "expired",
|
||||
encryptedSnapshot: null,
|
||||
cipherLabel: undefined
|
||||
};
|
||||
this.readiness.set(platform, expired);
|
||||
return expired;
|
||||
}
|
||||
|
||||
return session;
|
||||
}
|
||||
|
||||
private requireReadiness(platform: PlatformId): SessionReadinessRecord {
|
||||
return this.toReadiness(this.requireSession(platform));
|
||||
return readiness;
|
||||
}
|
||||
}
|
||||
|
||||
@ -4,8 +4,6 @@ import {
|
||||
type CandidateRecord,
|
||||
type PlatformId,
|
||||
type PlatformStatus,
|
||||
type SessionReadinessRecord,
|
||||
type SessionStateRecord,
|
||||
type TaskRecord,
|
||||
type TaskStatus
|
||||
} from "@cross-ai/domain";
|
||||
@ -24,13 +22,11 @@ import { PlatformIdentity, PlatformStatusPill, TaskStatusPill } from "./componen
|
||||
import { TaskContextHeader } from "./components/TaskContextHeader";
|
||||
import { TaskSpine } from "./components/TaskSpine";
|
||||
import {
|
||||
clearPlatformSession,
|
||||
confirmTask,
|
||||
createTask,
|
||||
deleteTask,
|
||||
getHistoryTasks,
|
||||
getPlatformReadiness,
|
||||
getPlatformSession,
|
||||
getTask,
|
||||
getTaskCandidates,
|
||||
getTaskReport,
|
||||
@ -122,65 +118,12 @@ function getNoReportSummary(task: TaskRecord) {
|
||||
}
|
||||
}
|
||||
|
||||
function formatTimestamp(timestamp?: string) {
|
||||
if (!timestamp) {
|
||||
return "暂无";
|
||||
}
|
||||
|
||||
return new Date(timestamp).toLocaleString("zh-CN");
|
||||
}
|
||||
|
||||
function getSearchRequirementLabel(
|
||||
searchRequirement: SessionReadinessRecord["searchRequirement"]
|
||||
) {
|
||||
switch (searchRequirement) {
|
||||
case "required":
|
||||
return "需准备会话";
|
||||
case "recommended":
|
||||
return "建议预热";
|
||||
default:
|
||||
return "无需会话";
|
||||
}
|
||||
}
|
||||
|
||||
function getReadinessSummary(readiness: SessionReadinessRecord) {
|
||||
if (readiness.status === "ready") {
|
||||
return readiness.expiresAt
|
||||
? `当前工作区已有可复用会话,有效至 ${formatTimestamp(readiness.expiresAt)}。`
|
||||
: readiness.reason;
|
||||
}
|
||||
|
||||
if (readiness.status === "expired") {
|
||||
return `最近一次会话已过期,上次准备时间为 ${formatTimestamp(readiness.lastPreparedAt)}。`;
|
||||
}
|
||||
|
||||
return readiness.reason;
|
||||
}
|
||||
|
||||
function getSessionSnapshotSummary(session: SessionStateRecord) {
|
||||
if (session.status === "ready") {
|
||||
return session.expiresAt
|
||||
? `已保存加密快照,可复用至 ${formatTimestamp(session.expiresAt)}。`
|
||||
: "已保存加密快照,可用于当前工作区复用。";
|
||||
}
|
||||
|
||||
if (session.status === "expired") {
|
||||
return "最近一次会话已过期,需重新完成会话准备。";
|
||||
}
|
||||
|
||||
return "当前还没有可复用会话快照。";
|
||||
}
|
||||
|
||||
export function NewTaskPage() {
|
||||
function NewTaskPage() {
|
||||
const navigate = useNavigate();
|
||||
const readinessQuery = useQuery({
|
||||
queryKey: ["platform-readiness"],
|
||||
queryFn: getPlatformReadiness
|
||||
});
|
||||
const historyQuery = useQuery({
|
||||
queryKey: ["history"],
|
||||
queryFn: getHistoryTasks
|
||||
});
|
||||
const [query, setQuery] = useState("");
|
||||
const [perLinkLimit, setPerLinkLimit] = useState(100);
|
||||
const [taskTotalLimit, setTaskTotalLimit] = useState(500);
|
||||
@ -191,7 +134,6 @@ export function NewTaskPage() {
|
||||
navigate(`/tasks/${task.taskId}/confirm`);
|
||||
}
|
||||
});
|
||||
const recentTasks = (historyQuery.data?.tasks ?? []).slice(0, 4);
|
||||
|
||||
return (
|
||||
<Layout>
|
||||
@ -267,72 +209,26 @@ export function NewTaskPage() {
|
||||
status={platform.ready ? "Completed" : "SearchBlocked"}
|
||||
/>
|
||||
</div>
|
||||
<p className="readiness-card__caption">
|
||||
搜索要求:{getSearchRequirementLabel(platform.searchRequirement)}
|
||||
</p>
|
||||
<p>{getReadinessSummary(platform)}</p>
|
||||
<div className="readiness-card__meta">
|
||||
<span className="inline-note inline-note--subtle">
|
||||
最近准备:{formatTimestamp(platform.lastPreparedAt)}
|
||||
</span>
|
||||
<span className="inline-note inline-note--subtle">
|
||||
{platform.expiresAt
|
||||
? `有效至 ${formatTimestamp(platform.expiresAt)}`
|
||||
: "当前无有效期中的会话"}
|
||||
</span>
|
||||
</div>
|
||||
<div className="panel-actions">
|
||||
<a
|
||||
className="text-link"
|
||||
href={`/sessions/${platform.platform}/prepare?from=/tasks/new`}
|
||||
>
|
||||
进入会话准备
|
||||
</a>
|
||||
</div>
|
||||
<p>{platformCatalogMap[platform.platform].description}</p>
|
||||
<a
|
||||
className="text-link"
|
||||
href={`/sessions/${platform.platform}/prepare?from=/tasks/new`}
|
||||
>
|
||||
进入会话准备
|
||||
</a>
|
||||
</article>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="stack">
|
||||
<div className="page-panel">
|
||||
<p className="eyebrow">Recent Tasks</p>
|
||||
<h3>最近任务捷径</h3>
|
||||
<div className="stack stack--dense">
|
||||
{recentTasks.length > 0 ? (
|
||||
recentTasks.map((task) => (
|
||||
<a
|
||||
key={task.taskId}
|
||||
className="mini-task-link"
|
||||
href={getTaskDestination(
|
||||
task.taskId,
|
||||
task.taskStatus,
|
||||
task.hasReport,
|
||||
task.defaultReportVersion
|
||||
)}
|
||||
>
|
||||
<div className="mini-task-link__topline">
|
||||
<strong>{task.query}</strong>
|
||||
<TaskStatusPill status={task.taskStatus} />
|
||||
</div>
|
||||
<p>{new Date(task.updatedAt).toLocaleString("zh-CN")}</p>
|
||||
</a>
|
||||
))
|
||||
) : (
|
||||
<div className="empty-state">还没有历史任务,先创建第一条分析任务。</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="page-panel">
|
||||
<p className="eyebrow">Scope Reminder</p>
|
||||
<h3>P0 当前不做什么</h3>
|
||||
<ul className="list">
|
||||
<li>不做自动绕过风控。</li>
|
||||
<li>不做无人工确认的同款判断。</li>
|
||||
<li>当前工作台只覆盖天猫、京东。</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div className="page-panel">
|
||||
<p className="eyebrow">Scope Reminder</p>
|
||||
<h3>P0 当前不做什么</h3>
|
||||
<ul className="list">
|
||||
<li>不做自动绕过风控。</li>
|
||||
<li>不做无人工确认的同款判断。</li>
|
||||
<li>当前工作台只覆盖天猫、京东。</li>
|
||||
</ul>
|
||||
</div>
|
||||
</section>
|
||||
</Layout>
|
||||
@ -1087,87 +983,34 @@ export function HistoryPage() {
|
||||
);
|
||||
}
|
||||
|
||||
export function SessionPreparePage() {
|
||||
function SessionPreparePage() {
|
||||
const navigate = useNavigate();
|
||||
const queryClient = useQueryClient();
|
||||
const { platform = "tmall" } = useParams();
|
||||
const [searchParams] = useSearchParams();
|
||||
const from = searchParams.get("from") ?? "/tasks/new";
|
||||
const platformId = platform as PlatformId;
|
||||
const sessionQuery = useQuery({
|
||||
queryKey: ["session", platformId],
|
||||
queryFn: () => getPlatformSession(platformId)
|
||||
});
|
||||
const prepareMutation = useMutation({
|
||||
mutationFn: () => preparePlatform(platformId),
|
||||
mutationFn: () => preparePlatform(platform as PlatformId),
|
||||
onSuccess: async () => {
|
||||
await Promise.all([
|
||||
queryClient.invalidateQueries({ queryKey: ["platform-readiness"] }),
|
||||
queryClient.invalidateQueries({ queryKey: ["session", platformId] })
|
||||
]);
|
||||
await queryClient.invalidateQueries({ queryKey: ["platform-readiness"] });
|
||||
navigate(from);
|
||||
}
|
||||
});
|
||||
const clearMutation = useMutation({
|
||||
mutationFn: () => clearPlatformSession(platformId),
|
||||
onSuccess: async () => {
|
||||
await Promise.all([
|
||||
queryClient.invalidateQueries({ queryKey: ["platform-readiness"] }),
|
||||
queryClient.invalidateQueries({ queryKey: ["session", platformId] })
|
||||
]);
|
||||
}
|
||||
});
|
||||
const session = sessionQuery.data?.session;
|
||||
|
||||
return (
|
||||
<Layout>
|
||||
<section className="page-panel session-panel">
|
||||
<p className="eyebrow">Session Console</p>
|
||||
<h2>{platformCatalogMap[platformId].label} 会话准备</h2>
|
||||
<p>{platformCatalogMap[platformId].recoveryHint}</p>
|
||||
<h2>{platformCatalogMap[platform as PlatformId].label} 会话准备</h2>
|
||||
<p>{platformCatalogMap[platform as PlatformId].recoveryHint}</p>
|
||||
<div className="session-placeholder">
|
||||
<div className="session-placeholder__viewport">Remote Browser Viewport</div>
|
||||
<div className="session-placeholder__sidebar">
|
||||
<strong>当前模式:prepare</strong>
|
||||
<p>本轮实现先用按钮模拟会话预热,后续替换为真正的远程浏览器接管。</p>
|
||||
<div className="session-details">
|
||||
<div className="session-details__row">
|
||||
<span>当前状态</span>
|
||||
<PlatformStatusPill status={session?.ready ? "Completed" : "SearchBlocked"} />
|
||||
</div>
|
||||
<div className="session-details__row">
|
||||
<span>搜索要求</span>
|
||||
<strong>{getSearchRequirementLabel(platformCatalogMap[platformId].searchRequirement)}</strong>
|
||||
</div>
|
||||
<div className="session-details__row">
|
||||
<span>会话快照</span>
|
||||
<strong>{session?.encryptedSnapshotAvailable ? "已加密保存" : "尚未生成"}</strong>
|
||||
</div>
|
||||
<div className="session-details__row">
|
||||
<span>有效期</span>
|
||||
<strong>{session?.expiresAt ? formatTimestamp(session.expiresAt) : "暂无"}</strong>
|
||||
</div>
|
||||
<p>{session ? getSessionSnapshotSummary(session) : "正在读取当前会话状态..."}</p>
|
||||
<p className="session-return-target">完成后将返回:{from}</p>
|
||||
</div>
|
||||
<div className="panel-actions">
|
||||
<button
|
||||
className="primary-button"
|
||||
disabled={prepareMutation.isPending}
|
||||
onClick={() => prepareMutation.mutate()}
|
||||
type="button"
|
||||
>
|
||||
标记预热完成
|
||||
</button>
|
||||
<button
|
||||
className="ghost-button"
|
||||
disabled={clearMutation.isPending || !session?.encryptedSnapshotAvailable}
|
||||
onClick={() => clearMutation.mutate()}
|
||||
type="button"
|
||||
>
|
||||
清理当前会话
|
||||
</button>
|
||||
</div>
|
||||
<button className="primary-button" onClick={() => prepareMutation.mutate()} type="button">
|
||||
标记预热完成
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
@ -6,13 +6,11 @@ import { MemoryRouter } from "react-router-dom";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("./lib/api", () => ({
|
||||
clearPlatformSession: vi.fn(),
|
||||
confirmTask: vi.fn(),
|
||||
createTask: vi.fn(),
|
||||
deleteTask: vi.fn(),
|
||||
getHistoryTasks: vi.fn(),
|
||||
getPlatformReadiness: vi.fn(),
|
||||
getPlatformSession: vi.fn(),
|
||||
getTask: vi.fn(),
|
||||
getTaskCandidates: vi.fn(),
|
||||
getTaskReport: vi.fn(),
|
||||
|
||||
@ -1,158 +0,0 @@
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
import { cleanup, render, screen, waitFor } from "@testing-library/react";
|
||||
import userEvent from "@testing-library/user-event";
|
||||
import type { ReactNode } from "react";
|
||||
import { MemoryRouter, Route, Routes } from "react-router-dom";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("./lib/api", () => ({
|
||||
clearPlatformSession: vi.fn(),
|
||||
confirmTask: vi.fn(),
|
||||
createTask: vi.fn(),
|
||||
deleteTask: vi.fn(),
|
||||
getHistoryTasks: vi.fn(),
|
||||
getPlatformReadiness: vi.fn(),
|
||||
getPlatformSession: vi.fn(),
|
||||
getTask: vi.fn(),
|
||||
getTaskCandidates: vi.fn(),
|
||||
getTaskReport: vi.fn(),
|
||||
preparePlatform: vi.fn(),
|
||||
retryTaskPlatform: vi.fn()
|
||||
}));
|
||||
|
||||
import { NewTaskPage, SessionPreparePage } from "./App";
|
||||
import {
|
||||
clearPlatformSession,
|
||||
getHistoryTasks,
|
||||
getPlatformReadiness,
|
||||
getPlatformSession,
|
||||
preparePlatform
|
||||
} from "./lib/api";
|
||||
|
||||
function renderWithProviders(node: ReactNode, initialEntries?: string[]) {
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: {
|
||||
queries: {
|
||||
retry: false
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
return render(
|
||||
<QueryClientProvider client={queryClient}>
|
||||
{initialEntries ? (
|
||||
<MemoryRouter initialEntries={initialEntries}>{node}</MemoryRouter>
|
||||
) : (
|
||||
<MemoryRouter>{node}</MemoryRouter>
|
||||
)}
|
||||
</QueryClientProvider>
|
||||
);
|
||||
}
|
||||
|
||||
describe("task composer and session console", () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(getPlatformReadiness).mockResolvedValue({
|
||||
platforms: [
|
||||
{
|
||||
platform: "tmall",
|
||||
ready: true,
|
||||
status: "ready",
|
||||
searchRequirement: "recommended",
|
||||
reason: "当前工作区存在可复用会话,创建任务时会再次校验。",
|
||||
lastPreparedAt: "2026-04-02T12:00:00.000Z",
|
||||
expiresAt: "2026-04-03T12:00:00.000Z"
|
||||
},
|
||||
{
|
||||
platform: "jd",
|
||||
ready: false,
|
||||
status: "missing",
|
||||
searchRequirement: "required",
|
||||
reason: "需要先完成会话准备,否则系统会标记为 SearchBlocked。"
|
||||
}
|
||||
]
|
||||
} as any);
|
||||
vi.mocked(getHistoryTasks).mockResolvedValue({
|
||||
tasks: [
|
||||
{
|
||||
taskId: "task-1",
|
||||
query: "Nintendo Switch 2",
|
||||
taskStatus: "Completed",
|
||||
updatedAt: "2026-04-02T12:00:00.000Z",
|
||||
hasReport: true,
|
||||
defaultReportVersion: 2,
|
||||
failedPlatforms: [],
|
||||
blockedPlatforms: []
|
||||
},
|
||||
{
|
||||
taskId: "task-2",
|
||||
query: "DJI Pocket 3",
|
||||
taskStatus: "AwaitingConfirmation",
|
||||
updatedAt: "2026-04-02T11:30:00.000Z",
|
||||
hasReport: false,
|
||||
failedPlatforms: [],
|
||||
blockedPlatforms: ["jd"]
|
||||
}
|
||||
]
|
||||
} as any);
|
||||
vi.mocked(getPlatformSession).mockResolvedValue({
|
||||
session: {
|
||||
platform: "jd",
|
||||
ready: true,
|
||||
status: "ready",
|
||||
searchRequirement: "required",
|
||||
scope: "workspace",
|
||||
ttlHours: 24,
|
||||
lastPreparedAt: "2026-04-02T10:00:00.000Z",
|
||||
expiresAt: "2026-04-03T10:00:00.000Z",
|
||||
encryptedSnapshotAvailable: true,
|
||||
cipherLabel: "mock-aes-gcm-v1"
|
||||
}
|
||||
} as any);
|
||||
vi.mocked(clearPlatformSession).mockResolvedValue(undefined);
|
||||
vi.mocked(preparePlatform).mockResolvedValue({
|
||||
platform: "jd",
|
||||
session_ready: true,
|
||||
status: "ready",
|
||||
last_prepared_at: "2026-04-02T10:00:00.000Z",
|
||||
expires_at: "2026-04-03T10:00:00.000Z",
|
||||
encrypted_snapshot_available: true
|
||||
} as any);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
it("shows readiness details and recent task shortcuts on the new task page", async () => {
|
||||
renderWithProviders(<NewTaskPage />);
|
||||
|
||||
expect(await screen.findByText("最近任务捷径")).toBeInTheDocument();
|
||||
expect(await screen.findByText("Nintendo Switch 2")).toBeInTheDocument();
|
||||
expect(await screen.findByText("DJI Pocket 3")).toBeInTheDocument();
|
||||
expect(await screen.findByText("搜索要求:建议预热")).toBeInTheDocument();
|
||||
expect(
|
||||
await screen.findByText(/当前工作区已有可复用会话,有效至/)
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it("shows session details and allows clearing the current session", async () => {
|
||||
const user = userEvent.setup();
|
||||
|
||||
renderWithProviders(
|
||||
<Routes>
|
||||
<Route element={<SessionPreparePage />} path="/sessions/:platform/prepare" />
|
||||
</Routes>,
|
||||
["/sessions/jd/prepare?from=/tasks/new"]
|
||||
);
|
||||
|
||||
expect(await screen.findByText("已加密保存")).toBeInTheDocument();
|
||||
expect(screen.getByText(/完成后将返回:\/tasks\/new/)).toBeInTheDocument();
|
||||
|
||||
await user.click(screen.getByRole("button", { name: "清理当前会话" }));
|
||||
|
||||
await waitFor(() => {
|
||||
expect(clearPlatformSession).toHaveBeenCalledWith("jd");
|
||||
});
|
||||
});
|
||||
});
|
||||
@ -220,8 +220,7 @@ a {
|
||||
.candidate-card,
|
||||
.metric-card,
|
||||
.insight-card,
|
||||
.evidence-card,
|
||||
.mini-task-link {
|
||||
.evidence-card {
|
||||
border: 1px solid rgba(31, 42, 48, 0.08);
|
||||
border-radius: 18px;
|
||||
background: var(--bg-elevated);
|
||||
@ -232,8 +231,7 @@ a {
|
||||
.history-card,
|
||||
.metric-card,
|
||||
.insight-card,
|
||||
.evidence-card,
|
||||
.mini-task-link {
|
||||
.evidence-card {
|
||||
padding: 16px;
|
||||
}
|
||||
|
||||
@ -258,37 +256,6 @@ a {
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.readiness-card__caption {
|
||||
margin: 0;
|
||||
color: var(--text-secondary);
|
||||
font-size: 13px;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.readiness-card__meta {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.mini-task-link {
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
color: inherit;
|
||||
}
|
||||
|
||||
.mini-task-link__topline {
|
||||
display: flex;
|
||||
align-items: start;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.mini-task-link p {
|
||||
margin: 0;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.task-context-header {
|
||||
padding: 20px 0 0;
|
||||
}
|
||||
@ -457,12 +424,6 @@ a {
|
||||
background: rgba(20, 108, 110, 0.08);
|
||||
}
|
||||
|
||||
.inline-note--subtle {
|
||||
padding: 8px 10px;
|
||||
background: rgba(31, 42, 48, 0.06);
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.metric-card {
|
||||
display: grid;
|
||||
gap: 6px;
|
||||
@ -607,31 +568,6 @@ a {
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.session-details {
|
||||
display: grid;
|
||||
gap: 12px;
|
||||
padding: 14px;
|
||||
border-radius: 16px;
|
||||
background: rgba(31, 42, 48, 0.04);
|
||||
}
|
||||
|
||||
.session-details p {
|
||||
margin: 0;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.session-details__row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.session-return-target {
|
||||
font-family: "IBM Plex Mono", monospace;
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.list {
|
||||
margin: 0;
|
||||
padding-left: 18px;
|
||||
|
||||
@ -73,15 +73,12 @@
|
||||
4. 同时可观察到 `cactus.jd.com/request_algo`、`jra.jd.com/jsTk.do`、`sgm-w.jd.com/h5` 等风控/参数初始化请求。
|
||||
5. `https://item.jd.com/robots.txt` 的公开信息非常有限,不能据此推断搜索或详情路径可稳定匿名抓取。
|
||||
参考:<https://item.jd.com/robots.txt>
|
||||
6. 2026-04-02 登录后复测中,`https://search.jd.com/Search?keyword=iPhone%2015` 返回的是 Vite 壳页 HTML,响应内不再稳定包含商品卡片;同 query 的 `pc_search_searchWare` API 回放可稳定返回 30 条候选。
|
||||
7. 同次复测中,`pc_detailpage_wareBusiness` 与 `getLegoWareDetailComment` 在授权会话下可稳定返回价格、店铺、库存、主图、标签与评论正文。
|
||||
|
||||
判断:
|
||||
|
||||
1. 京东 PC 端存在明确的接口层,不是只能靠浏览器 DOM 抓;其中搜索页 HTML 已明显退化为前端壳页,不能再把稳定 DOM 解析当作默认主路径。
|
||||
1. 京东 PC 端存在明确的接口层,不是只能靠浏览器 DOM 抓。
|
||||
2. 但接口调用明显依赖会话与动态参数上下文,不能把它当作无状态公开接口。
|
||||
3. 当前更稳的默认路线应收敛为“授权会话下的搜索/详情/评论 API 回放”,浏览器负责登录、模板刷新与阻塞恢复。
|
||||
4. 因此京东的非浏览器路线是可行的,但必须建立在“先会话、后请求”的体系上。
|
||||
3. 京东的非浏览器路线是可行的,但必须建立在“先会话、后请求”的体系上。
|
||||
|
||||
### 3.3 淘宝
|
||||
|
||||
|
||||
@ -4,27 +4,6 @@ export type PlatformId = (typeof platforms)[number];
|
||||
export const searchRequirements = ["none", "recommended", "required"] as const;
|
||||
export type SearchRequirement = (typeof searchRequirements)[number];
|
||||
|
||||
export const sessionStatuses = ["missing", "ready", "expired"] as const;
|
||||
export type SessionStatus = (typeof sessionStatuses)[number];
|
||||
|
||||
export const strategyLayers = ["L0", "L1", "L2", "L3"] as const;
|
||||
export type StrategyLayer = (typeof strategyLayers)[number];
|
||||
|
||||
export const platformCapabilities = ["search", "detail", "reviews", "login"] as const;
|
||||
export type PlatformCapability = (typeof platformCapabilities)[number];
|
||||
|
||||
export const strategyAttemptOutcomes = [
|
||||
"succeeded",
|
||||
"blocked",
|
||||
"failed",
|
||||
"no_result",
|
||||
"skipped"
|
||||
] as const;
|
||||
export type StrategyAttemptOutcome = (typeof strategyAttemptOutcomes)[number];
|
||||
|
||||
export const strategyAttemptTriggers = ["system", "retry", "recovery"] as const;
|
||||
export type StrategyAttemptTrigger = (typeof strategyAttemptTriggers)[number];
|
||||
|
||||
export const taskStatuses = [
|
||||
"Draft",
|
||||
"Searching",
|
||||
@ -85,18 +64,3 @@ export type SampleFlag = (typeof sampleFlags)[number];
|
||||
|
||||
export const evidenceSourceTypes = ["product", "review"] as const;
|
||||
export type EvidenceSourceType = (typeof evidenceSourceTypes)[number];
|
||||
|
||||
export const retentionActions = ["task_deleted", "retention_cleanup"] as const;
|
||||
export type RetentionAction = (typeof retentionActions)[number];
|
||||
|
||||
export const auditActions = [
|
||||
"session.prepared",
|
||||
"session.cleared",
|
||||
"task.recovery_completed",
|
||||
"platform.retry_started",
|
||||
"platform.retry_completed",
|
||||
"task.deleted",
|
||||
"report.published",
|
||||
"report.unchanged"
|
||||
] as const;
|
||||
export type AuditAction = (typeof auditActions)[number];
|
||||
|
||||
@ -1,15 +1,7 @@
|
||||
import type {
|
||||
AuditAction,
|
||||
PlatformCapability,
|
||||
PlatformId,
|
||||
PlatformStatus,
|
||||
SearchRequirement,
|
||||
ReportableTaskStatus,
|
||||
RetentionAction,
|
||||
SessionStatus,
|
||||
StrategyAttemptOutcome,
|
||||
StrategyAttemptTrigger,
|
||||
StrategyLayer,
|
||||
TaskStage,
|
||||
TaskStatus
|
||||
} from "./enums";
|
||||
@ -54,120 +46,8 @@ export interface TaskEventRecord {
|
||||
export interface SessionReadinessRecord {
|
||||
platform: PlatformId;
|
||||
ready: boolean;
|
||||
status: SessionStatus;
|
||||
searchRequirement: SearchRequirement;
|
||||
reason: string;
|
||||
lastPreparedAt?: string | undefined;
|
||||
expiresAt?: string | undefined;
|
||||
}
|
||||
|
||||
export interface SessionStateRecord {
|
||||
platform: PlatformId;
|
||||
ready: boolean;
|
||||
status: SessionStatus;
|
||||
searchRequirement: SearchRequirement;
|
||||
scope: "workspace";
|
||||
ttlHours: number;
|
||||
lastPreparedAt?: string | undefined;
|
||||
expiresAt?: string | undefined;
|
||||
encryptedSnapshotAvailable: boolean;
|
||||
cipherLabel?: string | undefined;
|
||||
}
|
||||
|
||||
export interface StrategyAttemptRecord {
|
||||
attemptId: string;
|
||||
taskId: string;
|
||||
platform: PlatformId;
|
||||
capability: PlatformCapability;
|
||||
layer: StrategyLayer;
|
||||
outcome: StrategyAttemptOutcome;
|
||||
trigger: StrategyAttemptTrigger;
|
||||
startedAt: string;
|
||||
finishedAt: string;
|
||||
durationMs: number;
|
||||
errorType?: string | undefined;
|
||||
detail: string;
|
||||
}
|
||||
|
||||
export interface PlatformRunMetricRecord {
|
||||
taskId: string;
|
||||
platform: PlatformId;
|
||||
searchDurationMs: number;
|
||||
detailDurationMs: number;
|
||||
reviewsDurationMs: number;
|
||||
retryCount: number;
|
||||
recoveryCount: number;
|
||||
lastUpdatedAt: string;
|
||||
}
|
||||
|
||||
export interface ReportMetricRecord {
|
||||
taskId: string;
|
||||
reportVersion: number;
|
||||
taskStatus: ReportableTaskStatus;
|
||||
selectedLinkCount: number;
|
||||
reviewSampleCount: number;
|
||||
blockedPlatforms: PlatformId[];
|
||||
failedPlatforms: PlatformId[];
|
||||
sampleInsufficient: boolean;
|
||||
partialPlatformFailure: boolean;
|
||||
generatedAt: string;
|
||||
}
|
||||
|
||||
export interface RetentionMetricRecord {
|
||||
metricId: string;
|
||||
action: RetentionAction;
|
||||
taskId?: string | undefined;
|
||||
deletedReportCount: number;
|
||||
deletedArtifactCount: number;
|
||||
residualArtifactCount: number;
|
||||
recordedAt: string;
|
||||
}
|
||||
|
||||
export type AuditMetadataValue = string | number | boolean | null;
|
||||
export type AuditMetadata = Record<string, AuditMetadataValue>;
|
||||
|
||||
export interface AuditLogRecord {
|
||||
auditId: string;
|
||||
taskId?: string | undefined;
|
||||
platform?: PlatformId | undefined;
|
||||
action: AuditAction;
|
||||
message: string;
|
||||
createdAt: string;
|
||||
metadata?: AuditMetadata | undefined;
|
||||
}
|
||||
|
||||
export interface ObservabilityOverview {
|
||||
strategyAttempts: {
|
||||
total: number;
|
||||
searchSuccessRate: number;
|
||||
browserFallbackShare: number;
|
||||
byLayer: Array<{
|
||||
layer: StrategyLayer;
|
||||
count: number;
|
||||
}>;
|
||||
byOutcome: Array<{
|
||||
outcome: StrategyAttemptOutcome;
|
||||
count: number;
|
||||
}>;
|
||||
};
|
||||
platformRuns: PlatformRunMetricRecord[];
|
||||
reports: {
|
||||
published: number;
|
||||
unchanged: number;
|
||||
sampleInsufficient: number;
|
||||
partialFailures: number;
|
||||
};
|
||||
retention: {
|
||||
taskDeletes: number;
|
||||
cleanupRuns: number;
|
||||
deletedReports: number;
|
||||
residualArtifacts: number;
|
||||
};
|
||||
audits: {
|
||||
total: number;
|
||||
recoveryActions: number;
|
||||
deleteActions: number;
|
||||
};
|
||||
}
|
||||
|
||||
export interface TaskRecord {
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user