feat: add selected audience profile csv export
This commit is contained in:
@@ -0,0 +1,332 @@
|
||||
# Market Audience Profile Export Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a selected-creators-only `导出画像CSV` flow that exports current market columns plus detail-page "连接用户" audience profile distributions.
|
||||
|
||||
**Architecture:** Keep the existing CSV export untouched and add a separate profile export path. The content controller reuses current selection and market row hydration, loads one selected creator profile at a time through a focused detail-page profile client, then writes a separate CSV with structured audience columns.
|
||||
|
||||
**Tech Stack:** TypeScript, Chrome MV3 content scripts, Xingtu authenticated pages, Vitest, jsdom, tsup
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
- Modify: `src/background/auth/controller.ts`
|
||||
- Keep token-readable auth state behavior from the previous fix.
|
||||
- Modify: `src/background/auth/state.ts`
|
||||
- Keep logged-out `lastError` support from the previous fix.
|
||||
- Modify: `tests/background-auth-controller.test.ts`
|
||||
- Keep token-expired regression coverage.
|
||||
- Modify: `src/content/market/auth-gate.ts`
|
||||
- Render expired-login text when auth state carries a token-expired error.
|
||||
- Modify: `src/content/index.ts`
|
||||
- Pass auth failure text into the market auth gate if needed.
|
||||
- Modify: `src/content/market/plugin-toolbar.ts`
|
||||
- Add a `导出画像CSV` button and handler.
|
||||
- Create: `src/content/market/audience-profile-types.ts`
|
||||
- Define normalized distribution and export-row types.
|
||||
- Create: `src/content/market/audience-profile-client.ts`
|
||||
- Load one creator detail page and extract normalized audience profile data.
|
||||
- Create: `src/content/market/audience-profile-csv.ts`
|
||||
- Build CSV columns from market records plus profile distributions.
|
||||
- Modify: `src/content/market/index.ts`
|
||||
- Add selected-only profile export flow and serial profile loading.
|
||||
- Test: `tests/market-auth-gating.test.ts`
|
||||
- Verify expired-login text.
|
||||
- Test: `tests/plugin-toolbar.test.ts`
|
||||
- Verify new toolbar button wiring.
|
||||
- Test: `tests/audience-profile-csv.test.ts`
|
||||
- Verify structured CSV column expansion.
|
||||
- Test: `tests/audience-profile-client.test.ts`
|
||||
- Verify parser behavior against representative detail-page payload/state shapes.
|
||||
- Modify: `tests/market-content-entry.test.ts`
|
||||
- Verify selected-only export behavior and failure handling.
|
||||
|
||||
## Task 1: Expired Login Message
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/content/market/auth-gate.ts`
|
||||
- Modify: `src/content/index.ts`
|
||||
- Test: `tests/market-auth-gating.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing auth gate test**
|
||||
|
||||
Add a test where `sendAuthMessage` returns:
|
||||
|
||||
```ts
|
||||
{
|
||||
ok: true,
|
||||
type: "auth:state",
|
||||
value: {
|
||||
isAuthenticated: false,
|
||||
lastError: "Token 已过期"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Assert the page shows `登录已过期，请重新登录`.
|
||||
|
||||
- [ ] **Step 2: Run the failing test**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/market-auth-gating.test.ts
|
||||
```
|
||||
|
||||
Expected: FAIL because the gate only renders `请先登录插件`.
|
||||
|
||||
- [ ] **Step 3: Implement minimal auth gate text support**
|
||||
|
||||
Update `renderMarketAuthGate` to accept an optional message string and render it instead of the default title. Update `bootContentScript` to pass `登录已过期，请重新登录` when `lastError` contains `token` or `过期`.
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/market-auth-gating.test.ts tests/popup-entry.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
## Task 2: Toolbar Button
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/content/market/plugin-toolbar.ts`
|
||||
- Test: `tests/plugin-toolbar.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write failing toolbar tests**
|
||||
|
||||
Create tests that:
|
||||
|
||||
- render the toolbar
|
||||
- assert a button with text `导出画像CSV` exists
|
||||
- click it and assert `onExportAudienceProfile` was called
|
||||
- assert `setToolbarBusyState` disables the new button
|
||||
|
||||
- [ ] **Step 2: Run the failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/plugin-toolbar.test.ts
|
||||
```
|
||||
|
||||
Expected: FAIL because the button and handler do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement toolbar support**
|
||||
|
||||
Add `onExportAudienceProfile` to `PluginToolbarHandlers`, add `audienceProfileExportButton` to `PluginToolbarDom`, render the new button, wire click handling, and include it in busy-state disabling.
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/plugin-toolbar.test.ts tests/market-content-entry.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
## Task 3: Profile CSV Builder
|
||||
|
||||
**Files:**
|
||||
- Create: `src/content/market/audience-profile-types.ts`
|
||||
- Create: `src/content/market/audience-profile-csv.ts`
|
||||
- Test: `tests/audience-profile-csv.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write failing CSV tests**
|
||||
|
||||
Define a sample market record and sample profile:
|
||||
|
||||
```ts
|
||||
const profile = {
|
||||
status: "success",
|
||||
gender: [
|
||||
{ label: "男性", value: "40.6%" },
|
||||
{ label: "女性", value: "59.4%" }
|
||||
],
|
||||
age: [{ label: "18-23", value: "28.6%" }],
|
||||
province: [{ label: "广东", value: "15%" }],
|
||||
regionTop: [{ label: "北京", value: "15%" }],
|
||||
cityTier: [{ label: "一线", value: "20%" }],
|
||||
interestTop: [{ label: "亲子", value: "18%" }],
|
||||
crowd: [{ label: "精致妈妈", value: "12%" }]
|
||||
};
|
||||
```
|
||||
|
||||
Assert the CSV contains separate headers like `连接用户-男性占比`, `省份-广东占比`, `地域TOP1名称`, `地域TOP1占比`.
|
||||
|
||||
- [ ] **Step 2: Run the failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/audience-profile-csv.test.ts
|
||||
```
|
||||
|
||||
Expected: FAIL because the files do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement CSV builder**
|
||||
|
||||
Create:
|
||||
|
||||
- `AudienceProfileDistributionItem`
|
||||
- `AudienceProfileResult`
|
||||
- `buildAudienceProfileCsv(records, profilesByAuthorId)`
|
||||
|
||||
Reuse `escapeCsvCell` and existing base/rate/backend metric column conventions. Add `画像抓取状态`.
|
||||
|
||||
- [ ] **Step 4: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/audience-profile-csv.test.ts tests/csv-exporter.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
## Task 4: Detail Profile Client and Parser
|
||||
|
||||
**Files:**
|
||||
- Create: `src/content/market/audience-profile-client.ts`
|
||||
- Test: `tests/audience-profile-client.test.ts`
|
||||
|
||||
- [ ] **Step 1: Use a logged-in browser to identify the real data source**
|
||||
|
||||
Run a Playwright probe against an authenticated `https://xingtu.cn/ad/creator/author-homepage/douyin-video/<authorId>` page. Capture only `/gw/api/...` JSON responses and page Vue/ECharts state. Record representative payload/state samples in the test file as small fixtures.
|
||||
|
||||
- [ ] **Step 2: Write failing parser tests**
|
||||
|
||||
Use the captured fixture to assert the parser returns normalized arrays for gender, age, province, region top 10, city tier, interest top 10, and crowd.
|
||||
|
||||
- [ ] **Step 3: Run the failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/audience-profile-client.test.ts
|
||||
```
|
||||
|
||||
Expected: FAIL because the client/parser does not exist.
|
||||
|
||||
- [ ] **Step 4: Implement parser and client**
|
||||
|
||||
Implement a small parser first. Then implement the client with injectable dependencies:
|
||||
|
||||
```ts
|
||||
createAudienceProfileClient({
|
||||
fetchDetailPage?: (authorId: string) => Promise<unknown>;
|
||||
readProfileFromPage?: (authorId: string) => Promise<unknown>;
|
||||
})
|
||||
```
|
||||
|
||||
Prefer parsed API JSON. Fall back to page state when API JSON is unavailable.
|
||||
|
||||
- [ ] **Step 5: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/audience-profile-client.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
## Task 5: Controller Export Flow
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/content/market/index.ts`
|
||||
- Test: `tests/market-content-entry.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write failing selected-only export tests**
|
||||
|
||||
Add tests that:
|
||||
|
||||
- select one of two visible rows
|
||||
- click `导出画像CSV`
|
||||
- assert only the selected author profile is requested
|
||||
- assert `onCsvReady` receives a CSV containing profile columns
|
||||
|
||||
- [ ] **Step 2: Write failing no-selection test**
|
||||
|
||||
Assert clicking `导出画像CSV` with no selected rows sets status to `请先勾选需要导出画像的达人` and makes no profile requests.
|
||||
|
||||
- [ ] **Step 3: Write failing partial-failure test**
|
||||
|
||||
Mock two selected profiles where one succeeds and one fails. Assert CSV is still generated with one success row and one `画像抓取状态=失败` row.
|
||||
|
||||
- [ ] **Step 4: Run failing tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/market-content-entry.test.ts
|
||||
```
|
||||
|
||||
Expected: FAIL because the controller has no profile export flow.
|
||||
|
||||
- [ ] **Step 5: Implement controller flow**
|
||||
|
||||
Add an injected option `loadAudienceProfile?: (record: MarketRecord) => Promise<AudienceProfileResult>`. Add handler:
|
||||
|
||||
- sync selected state from DOM
|
||||
- reject empty selection
|
||||
- hydrate current-page selected records
|
||||
- load profiles serially
|
||||
- cache successful results by author ID
|
||||
- build profile CSV
|
||||
- call `onCsvReady`
|
||||
- update toolbar progress/status
|
||||
|
||||
- [ ] **Step 6: Verify**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/market-content-entry.test.ts tests/audience-profile-csv.test.ts tests/plugin-toolbar.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
## Task 6: Full Verification
|
||||
|
||||
**Files:**
|
||||
- Modify only if failures identify necessary scoped fixes.
|
||||
|
||||
- [ ] **Step 1: Run focused suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test -- tests/market-auth-gating.test.ts tests/plugin-toolbar.test.ts tests/audience-profile-csv.test.ts tests/audience-profile-client.test.ts tests/market-content-entry.test.ts
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run all tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Run build**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 4: Manual logged-in browser verification**
|
||||
|
||||
Load the built extension in Chrome, select one or two market creators, click `导出画像CSV`, and verify the downloaded CSV contains structured profile columns with detail-page data.
|
||||
@@ -0,0 +1,105 @@
|
||||
# Market Audience Profile Export Design
|
||||
|
||||
## Goal
|
||||
|
||||
Add a separate CSV export for selected creators that includes the current market export fields plus audience profile data from each creator detail page's "连接用户" tab.
|
||||
|
||||
## User-Approved Decisions
|
||||
|
||||
- Add a new toolbar button named `导出画像CSV`.
|
||||
- Keep the existing `导出CSV` behavior unchanged.
|
||||
- Only allow the new export when at least one creator row is selected.
|
||||
- Do not support "export all" for profile data in this change because detail-page data costs extra API/page loads.
|
||||
- Suggested downloaded filename: `达人连接用户画像_YYYYMMDD_HHmm.csv`.
|
||||
- Export profile distributions as separate structured CSV columns, not as JSON blobs.
|
||||
|
||||
## Data Scope
|
||||
|
||||
Each exported row represents one selected creator. Start with the current market CSV columns, then append audience profile columns for:
|
||||
|
||||
- 性别分布
|
||||
- 年龄分布
|
||||
- 全国省份分布
|
||||
- 地域占比 TOP10
|
||||
- 城市等级分布
|
||||
- 兴趣分布
|
||||
- 八大人群占比
|
||||
|
||||
Fixed distributions should become fixed columns, for example:
|
||||
|
||||
- `连接用户-男性占比`
|
||||
- `连接用户-女性占比`
|
||||
- `连接用户-18-23占比`
|
||||
- `连接用户-24-30占比`
|
||||
- `省份-广东占比`
|
||||
- `城市等级-一线占比`
|
||||
- `八大人群-精致妈妈占比`
|
||||
|
||||
Ranked distributions should become name/value column pairs:
|
||||
|
||||
- `地域TOP1名称`
|
||||
- `地域TOP1占比`
|
||||
- ...
|
||||
- `地域TOP10名称`
|
||||
- `地域TOP10占比`
|
||||
- `兴趣TOP1名称`
|
||||
- `兴趣TOP1占比`
|
||||
- ...
|
||||
- `兴趣TOP10名称`
|
||||
- `兴趣TOP10占比`
|
||||
|
||||
Add a `画像抓取状态` column so partial failures are visible in CSV output.
|
||||
|
||||
## Data Acquisition
|
||||
|
||||
Use an on-demand detail-page probe. The implementation must first confirm the real data source from an authenticated creator detail page:
|
||||
|
||||
1. Prefer Xingtu `/gw/api/...` JSON responses if they expose the required profile data.
|
||||
2. If the API payload is difficult to locate or unstable, read the detail page's Vue/ECharts state from the page context.
|
||||
3. Avoid screen/OCR parsing and avoid relying on rendered chart pixels.
|
||||
|
||||
The export should process selected creators one at a time by default to respect API limits and reduce anti-abuse risk. Cache successful profile results in memory for the current page session.
|
||||
|
||||
## UX
|
||||
|
||||
Toolbar behavior:
|
||||
|
||||
- Add `导出画像CSV` next to the existing export actions.
|
||||
- Disable the button while any export/submission action is running.
|
||||
- If no creators are selected, show `请先勾选需要导出画像的达人`.
|
||||
- While exporting, show progress such as `画像导出中 3/12...`.
|
||||
- If plugin auth is expired, show `登录已过期，请重新登录`.
|
||||
- If one creator fails, keep the row in the CSV with `画像抓取状态=失败` and leave profile columns empty.
|
||||
- If all creators fail, do not download a CSV and show a failure message.
|
||||
|
||||
## Architecture
|
||||
|
||||
Add the feature as a separate export path instead of extending the existing `导出CSV` action. Reuse current selection state, market record hydration, CSV escaping, and runtime download path.
|
||||
|
||||
Proposed units:
|
||||
|
||||
- `audience-profile-client`: load and parse audience profile data for one creator detail page.
|
||||
- `audience-profile-csv`: combine existing market CSV columns with profile-specific columns.
|
||||
- Toolbar additions: add a button and handler for profile export.
|
||||
- Controller additions: filter to selected creators, call the profile client serially, build the CSV, then reuse `onCsvReady`.
|
||||
|
||||
## Testing
|
||||
|
||||
Use TDD. Add focused tests for:
|
||||
|
||||
- Auth state expired detection and user-facing expired-login text.
|
||||
- Toolbar renders and wires the new `导出画像CSV` button.
|
||||
- New export refuses to run without selected creators.
|
||||
- Profile CSV expands fixed and ranked distributions into separate columns.
|
||||
- Controller exports only selected creators and fetches profiles serially.
|
||||
- Failed creator profile fetches produce a failed row while successful rows still export.
|
||||
|
||||
Run focused tests first, then full `npm test`, then `npm run build`.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Unselected/all-page profile export.
|
||||
- Persistent cross-session profile cache.
|
||||
- Visual dashboard UI for profile data.
|
||||
- Changing batch submission payloads.
|
||||
- OCR or screenshot-based chart extraction.
|
||||
Reference in New Issue
Block a user