feat: export author spread metrics

This commit is contained in:
wxs
2026-06-29 16:06:14 +08:00
parent 7839380613
commit 121977fd0d
14 changed files with 1117 additions and 20 deletions
@@ -0,0 +1,105 @@
# 星图达人视频传播数据导出 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 导出 CSV 前按配置调用 `get_author_spread_info`,追加个人视频和星图视频传播指标列。
**Architecture:** 新增独立的 spread-info 模块负责参数配置、URL、响应映射和并发加载;列表解析保留 `authorId`,额外保存 `spreadAuthorId` 作为 `o_author_id`CSV exporter 只负责把已加载的 spread metrics 输出成列。导出入口在生成 CSV 前补齐 spread metrics。
**Tech Stack:** TypeScript, Chrome content script, Vitest, jsdom.
---
### Task 1: Spread Info Client And Mapping
**Files:**
- Create: `src/content/market/spread-info.ts`
- Modify: `src/content/market/types.ts`
- Test: `tests/spread-info.test.ts`
- [ ] **Step 1: Write failing tests**
Cover URL construction, label/header generation, response mapping, personal-video fixed params, and Xingtu-video multi-param configs.
- [ ] **Step 2: Run failing tests**
Run: `npx vitest run tests/spread-info.test.ts`
Expected: FAIL because `spread-info.ts` does not exist.
- [ ] **Step 3: Implement spread-info module and types**
Implement typed configs, formatter helpers, response mapper, client, and limited-concurrency loader.
- [ ] **Step 4: Run tests**
Run: `npx vitest run tests/spread-info.test.ts`
Expected: PASS.
### Task 2: Preserve Spread Author ID From Search Rows
**Files:**
- Modify: `src/content/market/types.ts`
- Modify: `src/content/market/market-list-row.ts`
- Modify: `src/content/market/page-bridge.ts`
- Test: `tests/market-page-bridge.test.ts`
- Test: `tests/silent-export-controller.test.ts`
- [ ] **Step 1: Write failing tests**
Verify `attribute_datas.id` is retained as `spreadAuthorId` and preferred over top-level `star_id` for spread-info requests.
- [ ] **Step 2: Run failing tests**
Run focused tests for row parsing and silent export.
- [ ] **Step 3: Implement parser changes**
Store `spreadAuthorId` on snapshots and merge it in result store.
- [ ] **Step 4: Run focused tests**
Expected: PASS.
### Task 3: CSV Columns
**Files:**
- Modify: `src/content/market/csv-exporter.ts`
- Test: `tests/csv-exporter.test.ts`
- [ ] **Step 1: Write failing tests**
Verify spread headers append after backend metrics and blank cells are exported when metrics are absent.
- [ ] **Step 2: Implement CSV spread columns**
Read `record.spreadMetrics` by generated header names.
- [ ] **Step 3: Run focused tests**
Run: `npx vitest run tests/csv-exporter.test.ts`
Expected: PASS.
### Task 4: Export Hydration
**Files:**
- Modify: `src/content/market/index.ts`
- Modify: `src/content/market/result-store.ts`
- Test: `tests/market-content-entry.test.ts`
- [ ] **Step 1: Write failing tests**
Verify export calls spread-info with `spreadAuthorId`, waits before CSV generation, preserves row order, and leaves blanks on failure.
- [ ] **Step 2: Implement hydration**
Inject `loadSpreadMetrics` for tests, default to spread-info loader, and hydrate records before `buildCsv`.
- [ ] **Step 3: Run focused tests**
Run focused content-entry tests.
### Task 5: Final Verification
- [ ] Run `npm test`.
- [ ] Run `npm run build`.
- [ ] Review `git diff`.
@@ -0,0 +1,242 @@
# 星图达人视频传播数据导出需求文档
## 目标
在现有星图达人 CSV 导出流程中,额外调用星图接口 `get_author_spread_info`,获取达人视频传播相关指标,并把这些指标追加到导出表格中。
因为同一个指标在不同参数组合下含义不同,所以导出字段名必须带上参数前缀。例如:
```text
只看指派_排除营销流量_星图视频_近30天_完播率
```
这个字段表示:它不是普通的“完播率”,而是在“只看指派 + 排除营销流量 + 星图视频 + 近30天”这组参数下获取到的完播率。
字段名前缀只体现会造成数据差异、且在当前导出中可变化的参数。固定不变的参数不用写进字段名前缀。
## 接口
调用接口:
```text
GET /gw/api/data_sp/get_author_spread_info
```
请求参数:
| 参数 | 含义 |
| --- | --- |
| `o_author_id` | 达人的星图 ID |
| `platform_source` | 固定传 `1` |
| `platform_channel` | 固定传 `1` |
| `type` | 视频类型 |
| `flow_type` | 是否排除营销流量 |
| `only_assign` | 是否只看指派 |
| `range` | 数据时间范围 |
请求需要带上当前星图网页登录态,所以实现时请求要使用浏览器当前 cookie,也就是 `credentials: "include"`
## 星图 ID 来源
`o_author_id` 需要从 `search_for_author_square` 接口返回值中获取:
```text
authors[i].attribute_datas.id
```
如果同一行数据里同时存在顶层 `star_id``attribute_datas.id`,这个接口优先使用 `attribute_datas.id` 作为 `o_author_id`
## 参数含义
### only_assign
| 值 | 含义 | 字段名前缀 |
| --- | --- | --- |
| `true` | 只看指派 | `只看指派` |
| `false` | 取消“只看指派”勾选 | `不限指派` |
### flow_type
| 值 | 含义 | 字段名前缀 |
| --- | --- | --- |
| `1` | 排除营销流量 | `排除营销流量` |
| `0` | 不排除营销流量 | `不排除营销流量` |
### range
| 值 | 含义 | 字段名前缀 |
| --- | --- | --- |
| `2` | 近 30 天 | `近30天` |
| `3` | 近 90 天 | `近90天` |
### type
| 值 | 含义 | 字段名前缀 |
| --- | --- | --- |
| `1` | 个人视频 | `个人视频` |
| `2` | 星图视频 | `星图视频` |
## 多组参数导出
第一版需要支持多组参数组合。
参数组合需要区分“个人视频”和“星图视频”两类处理:
- `type=1` 个人视频:`only_assign=false``flow_type=0` 固定,只允许调整 `range`
- `type=2` 星图视频:需要支持多组参数组合,因为 `only_assign``flow_type``range` 的不同设置会导致接口返回的数据不同。
个人视频固定参数:
```text
type=1
flow_type=0
only_assign=false
```
个人视频可变参数:
```text
range=2 或 range=3
```
因为个人视频里 `only_assign=false``flow_type=0` 是固定参数,所以它们不写入字段名前缀。个人视频字段只需要体现视频类型和时间范围,例如:
```text
个人视频_近30天_完播率
个人视频_近90天_完播率
```
星图视频可以配置多组参数。每一组参数都会调用一次 `get_author_spread_info`,并为这一组参数生成 7 个导出字段。
例如某一组参数是:
```text
only_assign=true
flow_type=1
type=2
range=2
```
那么这一组会生成:
- `只看指派_排除营销流量_星图视频_近30天_完播率`
- `只看指派_排除营销流量_星图视频_近30天_播放量中位数`
- `只看指派_排除营销流量_星图视频_近30天_互动率`
- `只看指派_排除营销流量_星图视频_近30天_作品平均时长`
- `只看指派_排除营销流量_星图视频_近30天_作品平均评论数`
- `只看指派_排除营销流量_星图视频_近30天_作品平均点赞数`
- `只看指派_排除营销流量_星图视频_近30天_作品平均转发数`
字段名规则固定为:
```text
<会变化的参数文案>_<视频类型文案>_<时间范围文案>_<指标名>
```
对星图视频来说,`only_assign``flow_type``range` 都可能变化,所以字段名要保留这些参数。对个人视频来说,只有 `range` 变化,所以字段名不需要写 `不限指派``不排除营销流量`
这里必须保留会变化参数的前缀,不能把不同参数组合下的同名指标合并。例如下面两个字段都叫“完播率”,但数据含义不同,必须作为两个独立字段导出:
```text
只看指派_排除营销流量_星图视频_近30天_完播率
不限指派_不排除营销流量_星图视频_近30天_完播率
```
## 需要导出的指标
每一组参数都要导出下面 7 个指标:
| 导出字段指标名 | 接口响应字段 | 示例值 | 说明 |
| --- | --- | --- | --- |
| 完播率 | `play_over_rate.value` | `2824` | 按万分比理解,导出时建议显示为 `28.24%` |
| 播放量中位数 | `play_mid`,兜底 `item_rate.play_mid.value` | `10913233` | 播放量中位数 |
| 互动率 | `interact_rate.value` | `402` | 按万分比理解,导出时建议显示为 `4.02%` |
| 作品平均时长 | `avg_duration` | `5600` | 按百分之一秒理解,导出时显示为秒,例如 `56` |
| 作品平均评论数 | `comment_avg` | `7502` | 平均评论数 |
| 作品平均点赞数 | `like_avg` | `494458` | 平均点赞数 |
| 作品平均转发数 | `share_avg` | `188267` | 平均转发数 |
示例响应:
```json
{
"avg_duration": "5600",
"comment_avg": "7502",
"interact_rate": {
"overtake": 5312,
"value": 402
},
"item_rate": {
"play_mid": {
"label": "",
"overtake": 10000,
"value": 10913233
}
},
"like_avg": "494458",
"play_mid": "10913233",
"play_over_rate": {
"overtake": 9584,
"value": 2824
},
"share_avg": "188267"
}
```
## 导出流程
1. 当前插件仍然先从星图达人搜索页收集达人列表。
2.`search_for_author_square``authors[i].attribute_datas.id` 取出每个达人的星图 ID。
3. 用户导出 CSV 时,先按现有逻辑确定导出范围,例如当前页、前 5 页、前 10 页、全部或自定义页数。
4. 对导出范围内的每个达人,先按个人视频参数调用 `get_author_spread_info``type=1``flow_type=0``only_assign=false` 固定,`range` 按配置取值。
5. 如果配置了星图视频参数组合,再按每一组星图视频参数分别调用 `get_author_spread_info`
6. 把每次接口返回值解析成 7 个指标。
7. CSV 保留原有字段顺序,在现有字段后追加这些带参数前缀的新字段。
## 失败处理
- 如果某个达人没有 `attribute_datas.id`,这一行的视频传播指标留空。
- 如果某个参数组合请求失败,这一组参数对应的 7 个字段留空。
- 如果接口响应结构异常,这一组参数对应的 7 个字段留空。
- 某个达人失败不能影响其他达人导出。
- 某组参数失败不能影响同一个达人的其他参数组导出。
## 性能要求
这个功能会产生比较多接口请求:
```text
请求数 = 导出的达人数量 * 参数组合数量
```
所以实现时需要:
- 做并发限制,避免一次性打太多请求。
- 保持最终 CSV 行顺序和原导出顺序一致。
- 给每个请求设置超时时间。
- 第一版不做激进重试,避免接口压力过大。
## 测试要求
需要补充测试覆盖:
- `get_author_spread_info` URL 参数构造是否正确。
- `type=1` 生成 `个人视频` 前缀。
- `type=2` 生成 `星图视频` 前缀。
- 个人视频是否固定使用:`type=1``flow_type=0``only_assign=false`
- 个人视频是否支持切换 `range=2``range=3`
- 个人视频字段名前缀是否不包含固定参数 `不限指派``不排除营销流量`
- 星图视频是否支持多组参数组合。
- `only_assign``flow_type``range` 前缀是否正确。
- 是否从 `attribute_datas.id` 读取 `o_author_id`
- 多组参数是否分别生成 7 个字段。
- 响应字段是否正确映射到 7 个导出指标。
- 接口失败时是否导出空字段。
- 多个达人并发请求完成顺序不一致时,最终 CSV 行顺序是否保持不变。
## 暂不做的事情
- 暂不新增页面上的参数配置 UI。
- 暂不改变星图搜索页原本的筛选条件。
- 暂不改变现有后端指标字段。
- 暂不改变批次提交 payload。
+12 -1
View File
@@ -1,5 +1,6 @@
import { normalizeRateDisplay } from "../../shared/rate-normalizer"; import { normalizeRateDisplay } from "../../shared/rate-normalizer";
import { escapeCsvCell } from "../../shared/csv"; import { escapeCsvCell } from "../../shared/csv";
import { buildSpreadInfoColumns } from "./spread-info";
import type { MarketRecord } from "./types"; import type { MarketRecord } from "./types";
export type CsvColumn = { export type CsvColumn = {
@@ -74,6 +75,11 @@ const BACKEND_METRIC_COLUMNS: CsvColumn[] = [
} }
]; ];
const SPREAD_INFO_COLUMNS: CsvColumn[] = buildSpreadInfoColumns().map((header) => ({
header,
readValue: (record: MarketRecord) => record.spreadMetrics?.[header] ?? ""
}));
export function listRateCsvHeaders(): string[] { export function listRateCsvHeaders(): string[] {
return RATE_COLUMNS.map((column) => column.header); return RATE_COLUMNS.map((column) => column.header);
} }
@@ -94,7 +100,12 @@ export function buildMarketCsv(records: MarketRecord[]): string {
export function buildMarketCsvColumns(records: MarketRecord[]): CsvColumn[] { export function buildMarketCsvColumns(records: MarketRecord[]): CsvColumn[] {
const baseColumns = buildBaseColumns(records); const baseColumns = buildBaseColumns(records);
return [...baseColumns, ...RATE_COLUMNS, ...BACKEND_METRIC_COLUMNS]; return [
...baseColumns,
...RATE_COLUMNS,
...BACKEND_METRIC_COLUMNS,
...SPREAD_INFO_COLUMNS
];
} }
export function buildBaseColumns(records: MarketRecord[]): CsvColumn[] { export function buildBaseColumns(records: MarketRecord[]): CsvColumn[] {
+10 -2
View File
@@ -76,6 +76,7 @@ type MarketDataRow = {
location?: string; location?: string;
price21To60s?: string; price21To60s?: string;
rates?: AfterSearchRates; rates?: AfterSearchRates;
spreadAuthorId?: string;
}; };
export interface MarketRowDom { export interface MarketRowDom {
@@ -88,6 +89,7 @@ export interface MarketRowDom {
personalCell: HTMLElement; personalCell: HTMLElement;
price21To60s?: string; price21To60s?: string;
rates?: AfterSearchRates; rates?: AfterSearchRates;
spreadAuthorId?: string;
row: HTMLElement; row: HTMLElement;
selectionCheckbox: HTMLInputElement; selectionCheckbox: HTMLInputElement;
singleCell: HTMLElement; singleCell: HTMLElement;
@@ -679,6 +681,7 @@ function syncDivGridRoot(root: HTMLElement): MarketTableDom | null {
row: authorCell, row: authorCell,
selectionCheckbox, selectionCheckbox,
singleCell, singleCell,
spreadAuthorId: fallbackMarketRow?.spreadAuthorId,
visibilityTargets: rowCells visibilityTargets: rowCells
} satisfies MarketRowDom } satisfies MarketRowDom
]; ];
@@ -1351,7 +1354,8 @@ function readSerializedMarketRows(
? { ? {
singleVideoAfterSearchRate singleVideoAfterSearchRate
} }
: undefined : undefined,
spreadAuthorId: readString(record.spreadAuthorId) ?? undefined
}; };
}) })
.filter((row) => Boolean(row.authorId || row.authorName)); .filter((row) => Boolean(row.authorId || row.authorName));
@@ -1673,7 +1677,11 @@ function mergeMarketDataRows(
baseRow.price21To60s, baseRow.price21To60s,
preferredRow.price21To60s preferredRow.price21To60s
), ),
rates: mergeRates(baseRow.rates, preferredRow.rates) rates: mergeRates(baseRow.rates, preferredRow.rates),
spreadAuthorId: mergeNonEmptyString(
baseRow.spreadAuthorId,
preferredRow.spreadAuthorId
)
}; };
} }
+54 -4
View File
@@ -31,6 +31,7 @@ import { createMarketApiClient } from "./api-client";
import { createExportRangeController } from "./export-range-controller"; import { createExportRangeController } from "./export-range-controller";
import { ensurePluginToolbar, isPluginToolbarMounted } from "./plugin-toolbar"; import { ensurePluginToolbar, isPluginToolbarMounted } from "./plugin-toolbar";
import { createSilentExportController } from "./silent-export-controller"; import { createSilentExportController } from "./silent-export-controller";
import { createSpreadInfoClient } from "./spread-info";
import { import {
readToolbarExportTarget, readToolbarExportTarget,
setToolbarBusyState, setToolbarBusyState,
@@ -79,6 +80,7 @@ export interface CreateMarketControllerOptions {
target: AudienceProfileRequestTarget target: AudienceProfileRequestTarget
) => Promise<AudienceProfileResult>; ) => Promise<AudienceProfileResult>;
loadAuthorMetrics?: (authorId: string) => Promise<MarketApiResult>; loadAuthorMetrics?: (authorId: string) => Promise<MarketApiResult>;
loadSpreadMetrics?: (spreadAuthorId: string) => Promise<Record<string, string>>;
searchBackendMetrics?: (starIds: string[]) => Promise< searchBackendMetrics?: (starIds: string[]) => Promise<
Array<BackendMetrics & { starId: string }> Array<BackendMetrics & { starId: string }>
>; >;
@@ -101,10 +103,13 @@ export function createMarketController(options: CreateMarketControllerOptions) {
const audienceProfileClient = createAudienceProfileClient(); const audienceProfileClient = createAudienceProfileClient();
const authorBaseClient = createAuthorBaseClient(); const authorBaseClient = createAuthorBaseClient();
const businessAbilityClient = createBusinessAbilityClient(); const businessAbilityClient = createBusinessAbilityClient();
const spreadInfoClient = createSpreadInfoClient();
const sendRuntimeMessage = createRuntimeMessageSender(); const sendRuntimeMessage = createRuntimeMessageSender();
const resultStore = options.resultStore ?? createMarketResultStore(); const resultStore = options.resultStore ?? createMarketResultStore();
const loadAuthorMetrics = const loadAuthorMetrics =
options.loadAuthorMetrics ?? marketApiClient.loadAuthorAseInfo; options.loadAuthorMetrics ?? marketApiClient.loadAuthorAseInfo;
const loadSpreadMetrics =
options.loadSpreadMetrics ?? spreadInfoClient.loadAuthorSpreadMetrics;
const searchBackendMetrics = const searchBackendMetrics =
options.searchBackendMetrics ?? options.searchBackendMetrics ??
(hasRuntimeMessageSender() ? (starIds: string[]) => readBackendMetrics(sendRuntimeMessage, starIds) : null); (hasRuntimeMessageSender() ? (starIds: string[]) => readBackendMetrics(sendRuntimeMessage, starIds) : null);
@@ -211,6 +216,7 @@ export function createMarketController(options: CreateMarketControllerOptions) {
try { try {
const records = filterRecordsBySelection( const records = filterRecordsBySelection(
await exportRecords(exportTarget.target, "导出中", { await exportRecords(exportTarget.target, "导出中", {
includeSpreadMetrics: true,
showDetailedProgress: selectedAuthorIds.size === 0 showDetailedProgress: selectedAuthorIds.size === 0
}) })
); );
@@ -707,6 +713,7 @@ export function createMarketController(options: CreateMarketControllerOptions) {
target: MarketExportTarget, target: MarketExportTarget,
inProgressLabel = "导出中", inProgressLabel = "导出中",
progressOptions: { progressOptions: {
includeSpreadMetrics?: boolean;
showDetailedProgress?: boolean; showDetailedProgress?: boolean;
} = {} } = {}
): Promise<MarketRecord[]> { ): Promise<MarketRecord[]> {
@@ -716,7 +723,9 @@ export function createMarketController(options: CreateMarketControllerOptions) {
if (target.mode === "count" && target.pageCount <= 1) { if (target.mode === "count" && target.pageCount <= 1) {
await prepareCurrentPageForExport(); await prepareCurrentPageForExport();
return getVisibleOrderedRecords(); return hydrateExportRecords(getVisibleOrderedRecords(), {
includeSpreadMetrics: progressOptions.includeSpreadMetrics ?? false
});
} }
const silentExportRecords = await silentExportController.exportRecords(target); const silentExportRecords = await silentExportController.exportRecords(target);
@@ -725,7 +734,10 @@ export function createMarketController(options: CreateMarketControllerOptions) {
silentExportRecords.map((record) => ({ silentExportRecords.map((record) => ({
...record, ...record,
status: record.status ?? "idle" status: record.status ?? "idle"
})) })),
{
includeSpreadMetrics: progressOptions.includeSpreadMetrics ?? false
}
); );
} }
@@ -1000,7 +1012,12 @@ export function createMarketController(options: CreateMarketControllerOptions) {
await runSyncCycle(); await runSyncCycle();
} }
async function hydrateExportRecords(records: MarketRecord[]): Promise<MarketRecord[]> { async function hydrateExportRecords(
records: MarketRecord[],
options: {
includeSpreadMetrics?: boolean;
} = {}
): Promise<MarketRecord[]> {
for (const record of records) { for (const record of records) {
resultStore.upsertMarketRow(record); resultStore.upsertMarketRow(record);
const existingRecord = resultStore.getRecord(record.authorId); const existingRecord = resultStore.getRecord(record.authorId);
@@ -1066,9 +1083,32 @@ export function createMarketController(options: CreateMarketControllerOptions) {
} }
} }
if (options.includeSpreadMetrics) {
await hydrateSpreadMetricsForRecords(records);
}
return records.map((record) => toMarketRecord(record)); return records.map((record) => toMarketRecord(record));
} }
async function hydrateSpreadMetricsForRecords(
records: MarketRecord[]
): Promise<void> {
await Promise.all(
records.map(async (record) => {
const storedRecord = resultStore.getRecord(record.authorId) ?? record;
const spreadAuthorId = storedRecord.spreadAuthorId ?? record.spreadAuthorId;
if (!spreadAuthorId || storedRecord.spreadMetrics) {
return;
}
const spreadMetrics = await loadSpreadMetrics(spreadAuthorId);
if (Object.keys(spreadMetrics).length > 0) {
resultStore.setSpreadMetricsSuccess(record.authorId, spreadMetrics);
}
})
);
}
async function harvestCurrentPageForExport(): Promise<void> { async function harvestCurrentPageForExport(): Promise<void> {
let hydrationSnapshot = await collectCurrentPageSnapshotsUntilSettled(); let hydrationSnapshot = await collectCurrentPageSnapshotsUntilSettled();
if ( if (
@@ -1148,6 +1188,10 @@ export function createMarketController(options: CreateMarketControllerOptions) {
existingRecord?.price21To60s, existingRecord?.price21To60s,
rowSnapshot.price21To60s rowSnapshot.price21To60s
); );
const spreadAuthorId = mergeStringValue(
existingRecord?.spreadAuthorId,
rowSnapshot.spreadAuthorId
);
return { return {
...existingRecord, ...existingRecord,
...rowSnapshot, ...rowSnapshot,
@@ -1169,6 +1213,11 @@ export function createMarketController(options: CreateMarketControllerOptions) {
location, location,
price21To60s, price21To60s,
rates: mergeFieldMap(existingRecord?.rates, rowSnapshot.rates), rates: mergeFieldMap(existingRecord?.rates, rowSnapshot.rates),
spreadAuthorId,
spreadMetrics: mergeFieldMap(
existingRecord?.spreadMetrics,
rowSnapshot.spreadMetrics
),
status: existingRecord?.status ?? "idle" status: existingRecord?.status ?? "idle"
} satisfies MarketRecord; } satisfies MarketRecord;
} }
@@ -1477,7 +1526,8 @@ function readRowSnapshot(rowDom: MarketRowDom): MarketRowSnapshot {
hasDirectRatesSource: rowDom.hasDirectRatesSource, hasDirectRatesSource: rowDom.hasDirectRatesSource,
location: rowDom.location, location: rowDom.location,
price21To60s: rowDom.price21To60s, price21To60s: rowDom.price21To60s,
rates: rowDom.rates rates: rowDom.rates,
spreadAuthorId: rowDom.spreadAuthorId
}; };
} }
+3 -1
View File
@@ -68,7 +68,9 @@ export function mapMarketListRow(
? { ? {
singleVideoAfterSearchRate singleVideoAfterSearchRate
} }
: undefined : undefined,
spreadAuthorId:
readString(readMarketFieldValue(row, attributeDatas, "id")) ?? undefined
}; };
} }
+2 -1
View File
@@ -206,7 +206,8 @@ function readSerializedMarketRows() {
authorName: authorName:
readString(attributeDatas.nickname) ?? readString(row.nick_name) ?? "", readString(attributeDatas.nickname) ?? readString(row.nick_name) ?? "",
coreUserId: readString(attributeDatas.core_user_id) ?? undefined, coreUserId: readString(attributeDatas.core_user_id) ?? undefined,
singleVideoAfterSearchRate singleVideoAfterSearchRate,
spreadAuthorId: readString(attributeDatas.id) ?? undefined
}; };
}) })
.filter((row) => Boolean(row.authorId || row.authorName)); .filter((row) => Boolean(row.authorId || row.authorName));
+17 -1
View File
@@ -2,7 +2,8 @@ import type {
BackendMetrics, BackendMetrics,
MarketApiFailureReason, MarketApiFailureReason,
MarketRecord, MarketRecord,
MarketRowSnapshot MarketRowSnapshot,
SpreadInfoMetrics
} from "./types"; } from "./types";
import type { AfterSearchRates } from "./types"; import type { AfterSearchRates } from "./types";
@@ -46,6 +47,13 @@ export function createMarketResultStore() {
...backendMetrics ...backendMetrics
}; };
}, },
setSpreadMetricsSuccess(authorId: string, spreadMetrics: SpreadInfoMetrics) {
const existingRecord = ensureRecord(authorId);
existingRecord.spreadMetrics = {
...existingRecord.spreadMetrics,
...spreadMetrics
};
},
setAuthorSuccess(authorId: string, rates: AfterSearchRates) { setAuthorSuccess(authorId: string, rates: AfterSearchRates) {
const existingRecord = ensureRecord(authorId); const existingRecord = ensureRecord(authorId);
existingRecord.status = "success"; existingRecord.status = "success";
@@ -73,6 +81,10 @@ export function createMarketResultStore() {
existingRecord.price21To60s, existingRecord.price21To60s,
row.price21To60s row.price21To60s
); );
existingRecord.spreadAuthorId = mergeStringValue(
existingRecord.spreadAuthorId,
row.spreadAuthorId
);
existingRecord.exportFields = mergeFieldMap( existingRecord.exportFields = mergeFieldMap(
existingRecord.exportFields, existingRecord.exportFields,
row.exportFields row.exportFields
@@ -81,6 +93,10 @@ export function createMarketResultStore() {
existingRecord.backendMetrics, existingRecord.backendMetrics,
row.backendMetrics row.backendMetrics
); );
existingRecord.spreadMetrics = mergeFieldMap(
existingRecord.spreadMetrics,
row.spreadMetrics
);
existingRecord.hasDirectRatesSource = existingRecord.hasDirectRatesSource =
existingRecord.hasDirectRatesSource || row.hasDirectRatesSource; existingRecord.hasDirectRatesSource || row.hasDirectRatesSource;
existingRecord.rates = mergeFieldMap(existingRecord.rates, row.rates); existingRecord.rates = mergeFieldMap(existingRecord.rates, row.rates);
+319
View File
@@ -0,0 +1,319 @@
import type { SpreadInfoMetrics } from "./types";
interface FetchResponseLike {
json(): Promise<unknown>;
ok: boolean;
}
type FetchLike = (
input: string,
init?: RequestInit
) => Promise<FetchResponseLike>;
export interface SpreadInfoConfig {
flowType: 0 | 1;
onlyAssign: boolean;
range: 2 | 3;
type: 1 | 2;
}
interface SpreadInfoClientOptions {
baseUrl?: string;
configs?: SpreadInfoConfig[];
fetchImpl?: FetchLike;
timeoutMs?: number;
}
interface SpreadInfoMetricDefinition {
key: keyof MappedSpreadInfoResponse;
label: string;
}
interface MappedSpreadInfoResponse {
averageCommentCount?: string;
averageDuration?: string;
averageLikeCount?: string;
averageShareCount?: string;
finishRate?: string;
interactionRate?: string;
playMedian?: string;
}
const SPREAD_INFO_METRICS: SpreadInfoMetricDefinition[] = [
{
key: "finishRate",
label: "完播率"
},
{
key: "playMedian",
label: "播放量中位数"
},
{
key: "interactionRate",
label: "互动率"
},
{
key: "averageDuration",
label: "作品平均时长"
},
{
key: "averageCommentCount",
label: "作品平均评论数"
},
{
key: "averageLikeCount",
label: "作品平均点赞数"
},
{
key: "averageShareCount",
label: "作品平均转发数"
}
];
export const DEFAULT_SPREAD_INFO_CONFIGS: SpreadInfoConfig[] = [
{
flowType: 0,
onlyAssign: false,
range: 2,
type: 1
},
{
flowType: 0,
onlyAssign: false,
range: 3,
type: 1
},
...buildXingtuVideoConfigs()
];
export function createSpreadInfoClient(options: SpreadInfoClientOptions = {}) {
const baseUrl = options.baseUrl ?? resolveBaseUrl();
const configs = options.configs ?? DEFAULT_SPREAD_INFO_CONFIGS;
const fetchImpl = options.fetchImpl ?? defaultFetch;
const timeoutMs = options.timeoutMs ?? 8000;
return {
async loadAuthorSpreadMetrics(authorId: string): Promise<SpreadInfoMetrics> {
const metrics: SpreadInfoMetrics = {};
for (const config of configs) {
const mappedResponse = await loadSpreadInfoFromUrl(
buildSpreadInfoUrl(authorId, config, baseUrl)
);
Object.entries(buildSpreadInfoMetricMap(config, mappedResponse)).forEach(
([header, value]) => {
metrics[header] = value;
}
);
}
return metrics;
}
};
async function loadSpreadInfoFromUrl(
url: string
): Promise<MappedSpreadInfoResponse> {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
try {
const response = await fetchImpl(url, {
credentials: "include",
method: "GET",
signal: controller.signal
});
if (!response.ok) {
return {};
}
return mapSpreadInfoResponse(await response.json());
} catch {
return {};
} finally {
clearTimeout(timeoutId);
}
}
}
export function buildSpreadInfoUrl(
authorId: string,
config: SpreadInfoConfig,
baseUrl: string
): string {
const url = new URL("/gw/api/data_sp/get_author_spread_info", baseUrl);
url.searchParams.set("o_author_id", authorId);
url.searchParams.set("platform_source", "1");
url.searchParams.set("platform_channel", "1");
url.searchParams.set("type", String(config.type));
url.searchParams.set("flow_type", String(config.flowType));
url.searchParams.set("only_assign", String(config.onlyAssign));
url.searchParams.set("range", String(config.range));
return url.toString();
}
export function buildSpreadInfoColumns(
configs: SpreadInfoConfig[] = DEFAULT_SPREAD_INFO_CONFIGS
): string[] {
return configs.flatMap((config) =>
SPREAD_INFO_METRICS.map((metric) => buildSpreadInfoColumnHeader(config, metric))
);
}
export function buildSpreadInfoMetricMap(
config: SpreadInfoConfig,
metrics: MappedSpreadInfoResponse
): SpreadInfoMetrics {
const values: SpreadInfoMetrics = {};
SPREAD_INFO_METRICS.forEach((metric) => {
const value = metrics[metric.key];
if (hasTextValue(value)) {
values[buildSpreadInfoColumnHeader(config, metric)] = value;
}
});
return values;
}
export function mapSpreadInfoResponse(
payload: unknown
): MappedSpreadInfoResponse {
const data = getPayloadData(payload);
if (!data) {
return {};
}
return {
averageCommentCount: readStringLike(data.comment_avg),
averageDuration: formatMillisecondsAsSeconds(readNumberLike(data.avg_duration)),
averageLikeCount: readStringLike(data.like_avg),
averageShareCount: readStringLike(data.share_avg),
finishRate: formatBasisPointPercent(
readNumberLike(readNestedValue(data.play_over_rate, "value"))
),
interactionRate: formatBasisPointPercent(
readNumberLike(readNestedValue(data.interact_rate, "value"))
),
playMedian:
readStringLike(data.play_mid) ??
readStringLike(readNestedValue(readNestedValue(data.item_rate, "play_mid"), "value"))
};
}
function buildSpreadInfoColumnHeader(
config: SpreadInfoConfig,
metric: SpreadInfoMetricDefinition
): string {
return [...buildConfigPrefixParts(config), metric.label].join("_");
}
function buildConfigPrefixParts(config: SpreadInfoConfig): string[] {
const typeLabel = config.type === 1 ? "个人视频" : "星图视频";
const rangeLabel = config.range === 2 ? "近30天" : "近90天";
if (config.type === 1) {
return [typeLabel, rangeLabel];
}
return [
config.onlyAssign ? "只看指派" : "不限指派",
config.flowType === 1 ? "排除营销流量" : "不排除营销流量",
typeLabel,
rangeLabel
];
}
function buildXingtuVideoConfigs(): SpreadInfoConfig[] {
const configs: SpreadInfoConfig[] = [];
[false, true].forEach((onlyAssign) => {
([0, 1] as const).forEach((flowType) => {
([2, 3] as const).forEach((range) => {
configs.push({
flowType,
onlyAssign,
range,
type: 2
});
});
});
});
return configs;
}
function getPayloadData(payload: unknown): Record<string, unknown> | null {
if (!isRecord(payload)) {
return null;
}
return isRecord(payload.data) ? payload.data : payload;
}
function readNestedValue(value: unknown, key: string): unknown {
return isRecord(value) ? value[key] : undefined;
}
function readStringLike(value: unknown): string | undefined {
if (typeof value === "string") {
return value;
}
if (typeof value === "number") {
return String(value);
}
return undefined;
}
function readNumberLike(value: unknown): number | null {
if (typeof value === "number" && Number.isFinite(value)) {
return value;
}
if (typeof value === "string" && value.trim().length > 0) {
const parsedValue = Number(value);
return Number.isFinite(parsedValue) ? parsedValue : null;
}
return null;
}
function formatBasisPointPercent(value: number | null): string | undefined {
if (value === null) {
return undefined;
}
return `${formatDecimal(value / 100)}%`;
}
function formatMillisecondsAsSeconds(value: number | null): string | undefined {
if (value === null) {
return undefined;
}
return formatDecimal(value / 100);
}
function formatDecimal(value: number): string {
return value.toFixed(2).replace(/\.?0+$/, "");
}
function resolveBaseUrl(): string {
if (typeof location !== "undefined" && location.origin) {
return location.origin;
}
return "https://www.xingtu.cn";
}
async function defaultFetch(input: string, init?: RequestInit) {
return fetch(input, init);
}
function hasTextValue(value: string | undefined): value is string {
return typeof value === "string" && value.trim().length > 0;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null;
}
+4
View File
@@ -12,6 +12,8 @@ export interface BackendMetrics {
newA3Rate?: string; newA3Rate?: string;
} }
export type SpreadInfoMetrics = Record<string, string>;
export type MarketSortField = export type MarketSortField =
| keyof Required<AfterSearchRates> | keyof Required<AfterSearchRates>
| keyof Required<BackendMetrics>; | keyof Required<BackendMetrics>;
@@ -28,6 +30,8 @@ export interface MarketRowSnapshot {
location?: string; location?: string;
price21To60s?: string; price21To60s?: string;
rates?: AfterSearchRates; rates?: AfterSearchRates;
spreadAuthorId?: string;
spreadMetrics?: SpreadInfoMetrics;
} }
export interface MarketRecord extends MarketRowSnapshot { export interface MarketRecord extends MarketRowSnapshot {
+69 -9
View File
@@ -1,6 +1,7 @@
import { describe, expect, test } from "vitest"; import { describe, expect, test } from "vitest";
import { buildMarketCsv } from "../src/content/market/csv-exporter"; import { buildMarketCsv } from "../src/content/market/csv-exporter";
import { buildSpreadInfoColumns } from "../src/content/market/spread-info";
import type { MarketRecord } from "../src/content/market/types"; import type { MarketRecord } from "../src/content/market/types";
describe("csv-exporter", () => { describe("csv-exporter", () => {
@@ -21,12 +22,13 @@ describe("csv-exporter", () => {
"秒思api-新增A3数", "秒思api-新增A3数",
"秒思api-新增A3率", "秒思api-新增A3率",
"秒思api-CPA3", "秒思api-CPA3",
"秒思api-cp_search" "秒思api-cp_search",
...buildSpreadInfoColumns()
].join(",") ].join(",")
); );
}); });
test("uses page export field order and appends the two plugin columns", () => { test("uses page export field order and appends the plugin columns", () => {
const csv = buildMarketCsv([ const csv = buildMarketCsv([
{ {
authorId: "123", authorId: "123",
@@ -66,11 +68,16 @@ describe("csv-exporter", () => {
"秒思api-新增A3数", "秒思api-新增A3数",
"秒思api-新增A3率", "秒思api-新增A3率",
"秒思api-CPA3", "秒思api-CPA3",
"秒思api-cp_search" "秒思api-cp_search",
...buildSpreadInfoColumns()
].join(",") ].join(",")
); );
expect(rowLine).toBe( expect(rowLine).toMatch(
'Alice,100w,"¥450,000",0.5% - 1%,1% - 3%,0.36%,"9,689.96","78,366.22",3.44%,1.79,14.46' new RegExp(
`^${escapeRegExp(
'Alice,100w,"¥450,000",0.5% - 1%,1% - 3%,0.36%,"9,689.96","78,366.22",3.44%,1.79,14.46'
)}`
)
); );
}); });
@@ -101,10 +108,13 @@ describe("csv-exporter", () => {
"秒思api-新增A3数", "秒思api-新增A3数",
"秒思api-新增A3率", "秒思api-新增A3率",
"秒思api-CPA3", "秒思api-CPA3",
"秒思api-cp_search" "秒思api-cp_search",
...buildSpreadInfoColumns()
].join(",") ].join(",")
); );
expect(rowLine).toBe("Alice,100w,,,,,,,,"); expect(rowLine.split(",").slice(0, 10).join(",")).toBe(
"Alice,100w,,,,,,,,"
);
}); });
test("escapes commas and quotes", () => { test("escapes commas and quotes", () => {
@@ -137,7 +147,10 @@ describe("csv-exporter", () => {
]); ]);
const [, rowLine] = csv.split("\n"); const [, rowLine] = csv.split("\n");
expect(rowLine).toBe("123,Alice,,,,,,,,,,"); expect(rowLine.split(",").slice(0, 12).join(",")).toBe(
"123,Alice,,,,,,,,,,"
);
expect(rowLine.split(",").slice(12).every((cell) => cell === "")).toBe(true);
}); });
test("uses normalized display values in export rows", () => { test("uses normalized display values in export rows", () => {
@@ -172,6 +185,53 @@ describe("csv-exporter", () => {
]); ]);
const [, rowLine] = csv.split("\n"); const [, rowLine] = csv.split("\n");
expect(rowLine).toBe("123,Alice,,,0.5% - 1%,0.02% - 0.1%,,,,,,"); expect(rowLine.split(",").slice(0, 12).join(",")).toBe(
"123,Alice,,,0.5% - 1%,0.02% - 0.1%,,,,,,"
);
});
test("appends spread info metric columns after backend metrics", () => {
const csv = buildMarketCsv([
{
authorId: "123",
authorName: "Alice",
spreadMetrics: {
"个人视频_近30天_完播率": "28.24%",
"只看指派_排除营销流量_星图视频_近30天_互动率": "4.02%"
},
status: "success"
} satisfies MarketRecord
]);
const [headerLine, rowLine] = csv.split("\n");
expect(headerLine).toContain(
[
"秒思api-cp_search",
"个人视频_近30天_完播率",
"个人视频_近30天_播放量中位数"
].join(",")
);
expect(headerLine).toContain(
"只看指派_排除营销流量_星图视频_近30天_互动率"
);
expect(rowLine).toContain("28.24%");
expect(rowLine).toContain("4.02%");
});
test("emits empty spread info cells when spread metrics are absent", () => {
const csv = buildMarketCsv([
{
authorId: "123",
authorName: "Alice",
status: "success"
} satisfies MarketRecord
]);
const [, rowLine] = csv.split("\n");
expect(rowLine.split(",").slice(-70).every((cell) => cell === "")).toBe(true);
}); });
}); });
function escapeRegExp(value: string): string {
return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}
+69 -1
View File
@@ -293,7 +293,10 @@ describe("market-content-entry", () => {
}); });
test("renders the plugin action bar inside the native market action row", async () => { test("renders the plugin action bar inside the native market action row", async () => {
document.body.innerHTML = buildMarketFixture(); document.body.innerHTML = buildRealMarketFixture([
{ authorId: "a", authorName: "Alpha", price21To60s: "450000" },
{ authorId: "b", authorName: "Beta", price21To60s: "70000" }
]);
const { createMarketController } = await import("../src/content/market/index"); const { createMarketController } = await import("../src/content/market/index");
const controller = trackController(createMarketController({ const controller = trackController(createMarketController({
@@ -1217,6 +1220,71 @@ describe("market-content-entry", () => {
expect(onCsvReady).toHaveBeenCalledWith("csv-output"); expect(onCsvReady).toHaveBeenCalledWith("csv-output");
}); });
test("export hydrates spread info with attribute_datas.id before building csv", async () => {
document.body.innerHTML = buildRealMarketFixture([
{ authorId: "a", authorName: "Alpha", price21To60s: "450000" },
{ authorId: "b", authorName: "Beta", price21To60s: "70000" }
]);
attachMarketListState([
{
attribute_datas: {
id: "spread-a",
nickname: "Alpha"
},
star_id: "a"
},
{
attribute_datas: {
id: "spread-b",
nickname: "Beta"
},
star_id: "b"
}
]);
const buildCsv = vi.fn(() => "csv-output");
const loadSpreadMetrics = vi.fn(async (spreadAuthorId: string) => ({
"个人视频_近30天_完播率": spreadAuthorId === "spread-a" ? "28.24%" : "18.24%"
}));
const { createMarketController } = await import("../src/content/market/index");
const controller = trackController(createMarketController({
buildCsv,
document,
loadAuthorMetrics: async () => ({
success: false,
reason: "request-failed"
}),
loadSpreadMetrics,
onCsvReady: vi.fn(),
window
}));
await controller.ready;
setSelectValue('[data-plugin-export-range="select"]', "current");
dispatchChange('[data-plugin-export-range="select"]');
click('[data-plugin-export="button"]');
await waitForMockCall(buildCsv, 80, 50);
expect(loadSpreadMetrics).toHaveBeenCalledWith("spread-a");
expect(loadSpreadMetrics).toHaveBeenCalledWith("spread-b");
expect(buildCsv.mock.calls[0][0]).toEqual([
expect.objectContaining({
authorId: "a",
spreadAuthorId: "spread-a",
spreadMetrics: {
"个人视频_近30天_完播率": "28.24%"
}
}),
expect.objectContaining({
authorId: "b",
spreadAuthorId: "spread-b",
spreadMetrics: {
"个人视频_近30天_完播率": "18.24%"
}
})
]);
});
test( test(
"default export captures the first 5 pages and keeps non-empty fields when merging duplicates", "default export captures the first 5 pages and keeps non-empty fields when merging duplicates",
async () => { async () => {
+43
View File
@@ -155,6 +155,49 @@ describe("silent-export-controller", () => {
expect(records?.map((record) => record.authorId)).toEqual(["2", "3"]); expect(records?.map((record) => record.authorId)).toEqual(["2", "3"]);
}); });
test("keeps attribute_datas.id as the spread author id while preserving star_id as row id", async () => {
document.documentElement.setAttribute(
"data-sces-market-request-snapshot",
JSON.stringify({
body: JSON.stringify({
page_param: {
page: 1
}
}),
method: "POST",
url: "https://xingtu.cn/api/mock-market-search"
})
);
const controller = createSilentExportController({
document,
fetchImpl: async () => ({
json: async () => ({
authors: [
{
attribute_datas: {
id: "spread-1",
nickname: "达人1"
},
star_id: "row-1"
}
]
}),
ok: true
})
});
const records = await controller.exportRecords({
mode: "count",
pageCount: 1
});
expect(records?.[0]).toMatchObject({
authorId: "row-1",
spreadAuthorId: "spread-1"
});
});
test("starts from page 1 when the captured request omits an explicit page number", async () => { test("starts from page 1 when the captured request omits an explicit page number", async () => {
document.documentElement.setAttribute( document.documentElement.setAttribute(
"data-sces-market-request-snapshot", "data-sces-market-request-snapshot",
+168
View File
@@ -0,0 +1,168 @@
import { describe, expect, test, vi } from "vitest";
import {
buildSpreadInfoColumns,
buildSpreadInfoUrl,
createSpreadInfoClient,
DEFAULT_SPREAD_INFO_CONFIGS,
mapSpreadInfoResponse
} from "../src/content/market/spread-info";
describe("spread-info", () => {
test("builds the spread info url with all request parameters", () => {
expect(
buildSpreadInfoUrl(
"7361012802036695050",
{
flowType: 1,
onlyAssign: true,
range: 2,
type: 2
},
"https://www.xingtu.cn"
)
).toBe(
"https://www.xingtu.cn/gw/api/data_sp/get_author_spread_info?o_author_id=7361012802036695050&platform_source=1&platform_channel=1&type=2&flow_type=1&only_assign=true&range=2"
);
});
test("defines personal video ranges with fixed non-prefix parameters", () => {
const personalConfigs = DEFAULT_SPREAD_INFO_CONFIGS.filter(
(config) => config.type === 1
);
expect(personalConfigs).toEqual([
{
flowType: 0,
onlyAssign: false,
range: 2,
type: 1
},
{
flowType: 0,
onlyAssign: false,
range: 3,
type: 1
}
]);
expect(buildSpreadInfoColumns(personalConfigs).slice(0, 2)).toEqual([
"个人视频_近30天_完播率",
"个人视频_近30天_播放量中位数"
]);
});
test("defines all xingtu video assign flow and range combinations", () => {
const xingtuConfigs = DEFAULT_SPREAD_INFO_CONFIGS.filter(
(config) => config.type === 2
);
expect(xingtuConfigs).toHaveLength(8);
expect(xingtuConfigs).toContainEqual({
flowType: 1,
onlyAssign: true,
range: 2,
type: 2
});
expect(xingtuConfigs).toContainEqual({
flowType: 0,
onlyAssign: false,
range: 3,
type: 2
});
expect(buildSpreadInfoColumns([
{
flowType: 1,
onlyAssign: true,
range: 2,
type: 2
}
])).toEqual([
"只看指派_排除营销流量_星图视频_近30天_完播率",
"只看指派_排除营销流量_星图视频_近30天_播放量中位数",
"只看指派_排除营销流量_星图视频_近30天_互动率",
"只看指派_排除营销流量_星图视频_近30天_作品平均时长",
"只看指派_排除营销流量_星图视频_近30天_作品平均评论数",
"只看指派_排除营销流量_星图视频_近30天_作品平均点赞数",
"只看指派_排除营销流量_星图视频_近30天_作品平均转发数"
]);
});
test("maps spread info response values into display values", () => {
expect(
mapSpreadInfoResponse({
avg_duration: "5600",
comment_avg: "7502",
interact_rate: {
value: 402
},
item_rate: {
play_mid: {
value: 10913233
}
},
like_avg: "494458",
play_over_rate: {
value: 2824
},
share_avg: "188267"
})
).toEqual({
averageCommentCount: "7502",
averageDuration: "56",
averageLikeCount: "494458",
averageShareCount: "188267",
finishRate: "28.24%",
interactionRate: "4.02%",
playMedian: "10913233"
});
});
test("loads each configured spread metric column for one author", async () => {
const fetchImpl = vi.fn(async () => ({
json: async () => ({
avg_duration: "5600",
comment_avg: "7502",
interact_rate: {
value: 402
},
like_avg: "494458",
play_mid: "10913233",
play_over_rate: {
value: 2824
},
share_avg: "188267"
}),
ok: true
}));
const client = createSpreadInfoClient({
configs: [
{
flowType: 0,
onlyAssign: false,
range: 2,
type: 1
}
],
fetchImpl
});
await expect(
client.loadAuthorSpreadMetrics("7361012802036695050")
).resolves.toEqual({
"个人视频_近30天_互动率": "4.02%",
"个人视频_近30天_作品平均点赞数": "494458",
"个人视频_近30天_作品平均评论数": "7502",
"个人视频_近30天_作品平均时长": "56",
"个人视频_近30天_作品平均转发数": "188267",
"个人视频_近30天_完播率": "28.24%",
"个人视频_近30天_播放量中位数": "10913233"
});
expect(fetchImpl).toHaveBeenCalledWith(
"https://www.xingtu.cn/gw/api/data_sp/get_author_spread_info?o_author_id=7361012802036695050&platform_source=1&platform_channel=1&type=1&flow_type=0&only_assign=false&range=2",
expect.objectContaining({
credentials: "include",
method: "GET"
})
);
});
});