Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ handles today, roughly in the order a campaign comes together:

- "How many VPs of demand gen at US SaaS companies can we reach?" (resolves targeting
facets, estimates reach before any money is involved)
- "Upload this CSV as a matched audience." (auto-cleans columns, hashes emails, converts
company domains to URLs)
- "Upload this CSV as a matched audience." (auto-cleans columns, hashes emails, keeps names,
companies and titles for LinkedIn's people match, converts company domains to URLs)
- "Build an audience from Salesforce: every contact on an account flagged as a target."
(SOQL straight to a matched audience)
- "Turn our HubSpot target accounts list into a LinkedIn audience." (a HubSpot list or search
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Adwarp is unofficial and not affiliated with or endorsed by LinkedIn or Google.
Once Adwarp is connected, you talk to your assistant in plain language. These are requests it handles today, roughly in the order a campaign comes together:

- "How many VPs of demand gen at US SaaS companies can we reach?" Adwarp resolves the targeting facets and estimates reach before any money is involved.
- "Upload this CSV as a matched audience." Adwarp cleans the columns, hashes emails, and converts company domains to URLs.
- "Upload this CSV as a matched audience." Adwarp cleans the columns, hashes emails, keeps names, companies and titles for LinkedIn's people match, and converts company domains to URLs.
- "Build an audience from Salesforce: every contact on an account flagged as a target." A SOQL query goes straight to a matched audience.
- "Turn our HubSpot target accounts list into a LinkedIn audience." A HubSpot list or search becomes a matched audience of contacts or companies.
- "Launch a draft campaign for that audience, $100/day, tracking the demo-booked conversion." The campaign group, campaign, and draft ads are created in a single call.
Expand Down
2 changes: 1 addition & 1 deletion docs/tools.json
Original file line number Diff line number Diff line change
Expand Up @@ -2050,7 +2050,7 @@
"hosted": false,
"hostedSupport": "none",
"hostedNote": "Reads a CSV path on the server, not on your machine.",
"description": "Clean a CSV at csvPath and upload it as a matched-audience DMP segment. Auto-detects contact (email) vs company (account) lists, normalizes column names, drops non-matcher columns, hashes emails, and converts company domains to website URLs. Requires the Audiences product; matching takes up to 48h.",
"description": "Clean a CSV at csvPath and upload it as a matched-audience DMP segment. Auto-detects contact (email, or first and last name) vs company (account) lists, normalizes column names, drops non-matcher columns, hashes emails, and converts company domains to website URLs. Contact lists keep first name, last name, company, title and country for LinkedIn's people match, which raises the match rate on work emails; a row with no valid email but a first and last name matches on name. Requires the Audiences product; matching takes up to 48h.",
"inputSchema": {
"type": "object",
"properties": {
Expand Down
2 changes: 1 addition & 1 deletion docs/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Talk in plain language ("VPs of demand gen at SaaS companies in the US") and Adw

## Audiences

- `upload_audience_csv`: cleans a CSV and uploads it as a matched audience. It normalizes column names, drops non-matcher columns, hashes emails, and converts company domains to website URLs. Contact and company lists are both supported.
- `upload_audience_csv`: cleans a CSV and uploads it as a matched audience. It normalizes column names, drops non-matcher columns, hashes emails, and converts company domains to website URLs. Contact and company lists are both supported. Contact lists keep first name, last name, company, title and country for LinkedIn's people match, and a row with a first and last name but no valid email still uploads as a name match.
- `audience_from_salesforce`: a SOQL query to a matched audience. See [Salesforce](./salesforce.md).
- `audience_from_hubspot`: a HubSpot list, or a CRM search by property filters, to a matched audience of contacts or companies. Pass `dryRun` first to see the counts and a masked sample. Local only. See [HubSpot](./hubspot.md).
- `get_audience_status`: matching status and resolved size.
Expand Down
71 changes: 61 additions & 10 deletions packages/core/src/audience.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,35 @@ async function uploadHashedAudience(
});
}

/**
* Render a cleaned contact list with people-match columns as LinkedIn's upload
* CSV: the email SHA256-hashed, the name, company, title and country as plain
* text. A row whose email is invalid stays in as a name match when it has a
* first and last name. Deduped on the email hash, or on name + company.
*/
export function contactListCsv(clean: CleanedCsv): { csvText: string; uploaded: number; skipped: number; nameOnly: number } {
const esc = (v: string) => (/[",\n]/.test(v) ? `"${v.replace(/"/g, '""')}"` : v);
const seen = new Set<string>();
const lines: string[] = [];
let skipped = 0;
let nameOnly = 0;
for (const r of clean.rows) {
const row = { ...r };
if (isValidEmail(row.email ?? "")) row.email = hashEmail(row.email!);
else delete row.email;
if (!row.email && !(row.firstname && row.lastname)) {
skipped += 1;
continue;
}
const key = row.email ?? `name:${[row.firstname, row.lastname, row.employeecompany ?? ""].join("|").toLowerCase()}`;
if (seen.has(key)) continue;
seen.add(key);
if (!row.email) nameOnly += 1;
lines.push(clean.columns.map((c) => esc(row[c] ?? "")).join(","));
}
return { csvText: `${clean.columns.join(",")}\n${lines.join("\n")}\n`, uploaded: lines.length, skipped, nameOnly };
}

/** Min recommended rows for a company list (contacts use RECOMMENDED_MIN_ROWS). */
const RECOMMENDED_MIN_COMPANIES = 1000;

Expand Down Expand Up @@ -186,7 +215,8 @@ export function audienceSizeWarnings(type: AudienceType, rows: number): string[]
/**
* Clean a CSV (fix column names, drop non-matcher columns, turn domains into
* website URLs) and upload it as the right list type (auto-detected, or forced
* via `type`). Contacts are hashed; companies upload name + website.
* via `type`). Contact emails are hashed, with any name, company, title and
* country columns kept for LinkedIn's people match; companies upload name + website.
*/
export async function uploadAudienceFromFile(
client: LinkedInClient,
Expand All @@ -201,9 +231,11 @@ export async function uploadAudienceFromFile(

/**
* Upload a list that has already been cleaned (by cleanAudienceCsv or
* cleanAudienceRows) as the matching segment type. Contacts are SHA256-hashed
* and deduped, so only hashes leave the machine; companies upload name +
* website. Shared by CSV files and CRM sources, so they upload the same way.
* cleanAudienceRows) as the matching segment type. Contact emails are
* SHA256-hashed and deduped, so an email-only list sends only hashes; name,
* company, title and country columns go as plain text, as LinkedIn requires.
* Companies upload name + website. Shared by CSV files and CRM sources, so
* they upload the same way.
*/
export async function uploadCleanedAudience(
client: LinkedInClient,
Expand All @@ -212,14 +244,33 @@ export async function uploadCleanedAudience(
): Promise<AudienceUploadResult & { audienceType: AudienceType }> {
const { clean } = input;
if (clean.type === "contact") {
const { hashes, total, skipped } = hashAndDedupeEmails(clean.rows.map((r) => r.email ?? ""));
const res = await uploadHashedAudience(client, getToken, {
if (clean.columns.length === 1 && clean.columns[0] === "email") {
const { hashes, total, skipped } = hashAndDedupeEmails(clean.rows.map((r) => r.email ?? ""));
const res = await uploadHashedAudience(client, getToken, {
accountId: input.accountId,
name: input.name,
hashes,
total,
skipped,
preWarnings: clean.warnings,
});
return { ...res, audienceType: "contact" };
}

const list = contactListCsv(clean);
if (list.uploaded === 0) throw new Error("No contacts with a valid email or a first and last name to upload.");
const warnings = [...clean.warnings, ...audienceSizeWarnings("contact", list.uploaded)];
if (list.skipped > 0) warnings.push(`Skipped ${list.skipped} rows with no valid email and no first and last name.`);
if (list.nameOnly > 0) warnings.push(`${list.nameOnly} contacts have no valid email and will match on name only.`);
const res = await uploadCsvToSegment(client, getToken, {
accountId: input.accountId,
name: input.name,
hashes,
total,
skipped,
preWarnings: clean.warnings,
csvText: list.csvText,
type: "USER_LIST_UPLOAD",
uploaded: list.uploaded,
totalRows: clean.rowCount,
skipped: list.skipped,
warnings,
});
return { ...res, audienceType: "contact" };
}
Expand Down
65 changes: 60 additions & 5 deletions packages/core/src/audienceCsv.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ import { parse } from "csv-parse/sync";
* isn't a matcher, and turns company domains into website URLs.
*
* Two list types:
* - contact (USER_LIST_UPLOAD): matched by email. Kept column: `email`.
* - contact (USER_LIST_UPLOAD): matched by email, and by LinkedIn's people match
* on name, company, title and country. Kept columns: `email`, `firstname`,
* `lastname`, `employeecompany`, `title`, `country` when present. A row needs
* an email or a first and last name.
* - company (COMPANY_LIST_UPLOAD): matched by website. Kept columns:
* `companyname`, `companywebsite` (full https:// URL, since LinkedIn matches
* accounts on the website URL), and `linkedincompanypageurl` (the LinkedIn
Expand Down Expand Up @@ -34,6 +37,20 @@ const LINKEDIN_PAGE_KEYS = new Set(
"linkedincompanypage", "linkedincompany"],
);

// Contact-list people-match columns. `name` is left out of the company keys here,
// since on a contact list it is usually the person's name.
const FIRST_NAME_KEYS = new Set(["firstname", "first", "fn", "givenname", "fname"]);
const LAST_NAME_KEYS = new Set(["lastname", "last", "ln", "surname", "familyname", "lname"]);
const EMPLOYEE_COMPANY_KEYS = new Set(
["employeecompany", "company", "companyname", "accountname", "account", "organization", "organisation",
"employer", "currentcompany"],
);
const TITLE_KEYS = new Set(["title", "jobtitle", "position", "currenttitle"]);
const COUNTRY_KEYS = new Set(["country", "countrycode"]);

/** LinkedIn's length caps for the plain-text contact columns. */
const CONTACT_MAX_LENGTH: Record<string, number> = { firstname: 35, lastname: 35, employeecompany: 50, title: 50 };

/** Turn a domain or messy URL into a clean https:// website URL (or "" if not a domain). */
export function toWebsiteUrl(raw: string): string {
let s = (raw ?? "").trim().toLowerCase();
Expand Down Expand Up @@ -75,6 +92,7 @@ function findHeader(headers: string[], keys: Set<string>): string | undefined {

function detectType(headers: string[]): AudienceType {
if (findHeader(headers, EMAIL_KEYS)) return "contact";
if (findHeader(headers, FIRST_NAME_KEYS) && findHeader(headers, LAST_NAME_KEYS)) return "contact";
if (findHeader(headers, COMPANY_NAME_KEYS) || findHeader(headers, WEBSITE_KEYS)) return "company";
throw new Error("Could not detect list type from columns. Pass an explicit type (contact|company).");
}
Expand All @@ -99,11 +117,48 @@ export function cleanAudienceRows(parsed: Record<string, string>[], opts: { type

if (type === "contact") {
const emailCol = findHeader(headers, EMAIL_KEYS);
if (!emailCol) throw new Error("No email column found for a contact list.");
const rows = parsed.map((r) => ({ email: (r[emailCol] ?? "").trim().toLowerCase() })).filter((r) => r.email);
const dropped = headers.filter((h) => h !== emailCol);
const firstCol = findHeader(headers, FIRST_NAME_KEYS);
const lastCol = findHeader(headers, LAST_NAME_KEYS);
if (!emailCol && !(firstCol && lastCol)) {
throw new Error("No email column, or first and last name columns, found for a contact list.");
}
const sources: [string, string | undefined][] = [
["email", emailCol],
["firstname", firstCol],
["lastname", lastCol],
["employeecompany", findHeader(headers, EMPLOYEE_COMPANY_KEYS)],
["title", findHeader(headers, TITLE_KEYS)],
["country", findHeader(headers, COUNTRY_KEYS)],
];
const kept = sources.filter((s): s is [string, string] => !!s[1]);
const columns = kept.map(([c]) => c);

let badCountry = 0;
const rows: Record<string, string>[] = [];
for (const r of parsed) {
const out: Record<string, string> = {};
for (const [col, header] of kept) {
let v = (r[header] ?? "").trim();
if (col === "email") v = v.toLowerCase();
if (col === "country" && v) {
// LinkedIn takes an ISO two-letter code only.
if (/^[a-z]{2}$/i.test(v)) v = v.toUpperCase();
else {
badCountry += 1;
v = "";
}
}
const max = CONTACT_MAX_LENGTH[col];
if (max && v.length > max) v = v.slice(0, max).trim();
if (v) out[col] = v;
}
if (out.email || (out.firstname && out.lastname)) rows.push(out);
}
const used = new Set(kept.map(([, h]) => h));
const dropped = headers.filter((h) => !used.has(h));
if (dropped.length) warnings.push(`Dropped ${dropped.length} non-matcher column(s): ${dropped.join(", ")}.`);
return { type, columns: ["email"], rows, dropped, rowCount: rows.length, warnings };
if (badCountry) warnings.push(`${badCountry} country value(s) were not two-letter ISO codes and were left blank.`);
return { type, columns, rows, dropped, rowCount: rows.length, warnings };
}

// company
Expand Down
131 changes: 131 additions & 0 deletions packages/core/test/contactList.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
/**
* Contact lists with LinkedIn's people-match columns: the cleaner keeps first and last
* name, company, title and country next to the email, and the upload hashes only the
* email, as LinkedIn's list-upload spec requires.
*
* Runs against dist (`pnpm --filter @adwarp/core build` first). LinkedIn answers from a
* fake client and the signed CSV upload from a stubbed fetch, so nothing leaves the process.
*/
import { test } from "node:test";
import assert from "node:assert/strict";
import { createHash } from "node:crypto";

const { cleanAudienceRows, cleanAudienceCsv, contactListCsv, uploadCleanedAudience } = await import("../dist/index.js");

const UPLOAD_URL = "https://upload.linkedin.test/list";
const sha256 = (s) => createHash("sha256").update(s).digest("hex");

/** A LinkedIn client that records calls and answers the matched-audience upload flow. */
function fakeLinkedIn() {
const calls = [];
const client = {
calls,
request: async (req) => {
calls.push(req);
if (req.query?.action === "generateUploadUrl") return { data: { value: UPLOAD_URL }, status: 200 };
if (req.path === "/dmpSegments" && req.method === "POST") return { data: undefined, status: 201, restliId: "9001" };
if (req.path === "/dmpSegments/9001/listUploads") return { data: undefined, status: 201 };
throw new Error(`Unexpected LinkedIn call: ${req.method} ${req.path}`);
},
};
return { client, getToken: async () => "li-token" };
}

/** Captures the CSV bodies posted to the signed upload URL for a single test. */
function stubUpload(t) {
const bodies = [];
const saved = globalThis.fetch;
globalThis.fetch = async (input, init = {}) => {
if (String(input) !== UPLOAD_URL || init.method !== "POST") throw new Error(`Unexpected request: ${input}`);
bodies.push(init.body);
return new Response(null, { status: 201, headers: { location: "D4E10AQ" } });
};
t.after(() => {
globalThis.fetch = saved;
});
return bodies;
}

test("contact lists keep the people-match columns under LinkedIn's names", () => {
const clean = cleanAudienceRows([
{ "Work Email": " Ada@Example.com ", "First Name": "Ada", "Last Name": "Lovelace", Company: "Analytical", "Job Title": "VP RevOps", Country: "gb", Notes: "x" },
{ "Work Email": "", "First Name": "Grace", "Last Name": "Hopper", Company: "Navy", "Job Title": "", Country: "United States", Notes: "" },
{ "Work Email": "", "First Name": "Solo", "Last Name": "", Company: "Nowhere", "Job Title": "", Country: "", Notes: "" },
]);

assert.equal(clean.type, "contact");
assert.deepEqual(clean.columns, ["email", "firstname", "lastname", "employeecompany", "title", "country"]);
assert.deepEqual(clean.dropped, ["Notes"]);
assert.deepEqual(clean.rows, [
{ email: "ada@example.com", firstname: "Ada", lastname: "Lovelace", employeecompany: "Analytical", title: "VP RevOps", country: "GB" },
{ firstname: "Grace", lastname: "Hopper", employeecompany: "Navy" },
]);
assert.ok(clean.warnings.some((w) => /1 country value\(s\) were not two-letter ISO codes/.test(w)));
});

test("a list with names but no email column is a contact list, and `name` is not read as the company", () => {
const clean = cleanAudienceCsv("firstname,lastname,name,title\nAda,Lovelace,Ada Lovelace,An extremely long job title that runs well past fifty characters\n");
assert.equal(clean.type, "contact");
assert.deepEqual(clean.columns, ["firstname", "lastname", "title"]);
assert.equal(clean.rows[0].title.length, 50);
});

test("a company column on a contact list becomes employeecompany, and other extras are still dropped", () => {
const clean = cleanAudienceCsv("email,company\nAda@Example.com,Analytical\n", { type: "contact" });
assert.deepEqual(clean.columns, ["email", "employeecompany"]);
const emailOnly = cleanAudienceCsv("email,notes\nAda@Example.com,hi\n");
assert.deepEqual(emailOnly.columns, ["email"]);
assert.deepEqual(emailOnly.rows, [{ email: "ada@example.com" }]);
});

test("the upload CSV hashes the email, keeps names as text, and dedupes", () => {
const clean = cleanAudienceRows([
{ email: "ada@example.com", firstname: "Ada", lastname: "Lovelace", company: "Analytical, Ltd" },
{ email: "ADA@example.com", firstname: "Ada", lastname: "Lovelace", company: "Analytical, Ltd" },
{ email: "not-an-email", firstname: "Grace", lastname: "Hopper", company: "Navy" },
{ email: "", firstname: "Grace", lastname: "Hopper", company: "Navy" },
{ email: "broken@", firstname: "", lastname: "", company: "Nobody" },
]);
const list = contactListCsv(clean);

assert.equal(list.uploaded, 2);
assert.equal(list.skipped, 1);
assert.equal(list.nameOnly, 1);
assert.equal(
list.csvText,
`email,firstname,lastname,employeecompany\n${sha256("ada@example.com")},Ada,Lovelace,"Analytical, Ltd"\n,Grace,Hopper,Navy\n`,
);
});

test("uploading a contact list with names sends one USER_LIST_UPLOAD CSV with no raw email", async (t) => {
const bodies = stubUpload(t);
const li = fakeLinkedIn();
const clean = cleanAudienceRows([
{ email: "ada@example.com", firstname: "Ada", lastname: "Lovelace", title: "VP RevOps" },
{ email: "", firstname: "Grace", lastname: "Hopper", title: "Admiral" },
]);

const res = await uploadCleanedAudience(li.client, li.getToken, { accountId: "123", name: "Emailed", clean });

assert.equal(res.segmentId, "9001");
assert.equal(res.audienceType, "contact");
assert.equal(res.uploaded, 2);
assert.ok(res.warnings.some((w) => /1 contacts have no valid email and will match on name only/.test(w)));
assert.equal(bodies.length, 1);
assert.equal(bodies[0], `email,firstname,lastname,title\n${sha256("ada@example.com")},Ada,Lovelace,VP RevOps\n,Grace,Hopper,Admiral\n`);
assert.ok(!bodies[0].includes("@"));
const segment = li.client.calls.find((c) => c.path === "/dmpSegments" && c.body?.name);
assert.equal(segment.body.type, "USER_LIST_UPLOAD");
});

test("an email-only upload still sends just the hash column", async (t) => {
const bodies = stubUpload(t);
const li = fakeLinkedIn();
const clean = cleanAudienceCsv("email\nAda@Example.com\nada@example.com\nnope\n");

const res = await uploadCleanedAudience(li.client, li.getToken, { accountId: "123", name: "Emails", clean });

assert.equal(res.uploaded, 1);
assert.equal(res.skipped, 1);
assert.equal(bodies[0], `email\n${sha256("ada@example.com")}\n`);
});
Loading
Loading