Intent Data Provenance: Source Records, Lineage and Review Tests

Ten proposed provenance record categories

1. Source and responsible party

Record the provider or collector, source system, contract or order reference, business owner, vendor contact, and the parties’ documented roles. If several sources are combined, keep each source record separate before creating the derived dataset.

Control: A reviewer should be able to identify who can answer a question or correct a problem. “Marketing vendor” is not a responsible-party field.

2. Collection method and context

Describe what generated the data: a first-party site event, topic-level research aggregation, form, partner feed, public record, CRM event, or another documented method. Record the page, topic, event, or category definition and the relevant notice or policy reference where applicable.

Control: Do not infer collection context from the field name. A value called “intent” might represent a page visit, an account-level surge, a model score, or an imported label.

3. Permitted-use and legal-review record

Capture the business purpose, jurisdiction and role review, contractual restrictions, approved destinations, reuse limits, and the accountable privacy or legal decision. Keep the evidence or policy reference linked to the record.

Control: A record of a decision is not proof that the decision is legally correct. ICO’s records-of-processing and lawful-basis guidance is an official UK reference, currently under review following the Data (Use and Access) Act. Check the current guidance and obtain review for the actual use.

4. Signal definition and analysis unit

Define what the signal represents, its inclusion and exclusion logic, the account, person, browser, or device unit, its sensitivity, and known false-positive modes. State whether the value is observed, inferred, modeled, or aggregated.

Control: Write a sentence that can stand alone: “This field indicates X, derived from Y during Z type of window.” If the team cannot do that, the field is not ready for activation.

5. Observation time and freshness

Keep event time, ingestion time, processing time, time zone, expiry or decay rule, and late-arriving-data treatment separate. A single “created” timestamp is rarely enough to explain whether the signal was fresh when it influenced an action.

Control: Record the rule used to expire or downgrade stale evidence and the owner who can change it.

6. Transformation and model lineage

Document normalization, filtering, aggregation, joins, scoring, thresholds, rule or model version, and the source entities from which the output was derived. Preserve reproducible queries, code versions, or transformation references where feasible.

Control: A score without a version and derivation path is a label, not an auditable result. Keep a change record when definitions or thresholds move.

7. Identity match and confidence

Record the identifiers supplied, the resolved account or person, deterministic and probabilistic methods, match confidence, conflicts, manual corrections, and the challenge or correction path. Keep the original and resolved identifiers logically distinct.

Control: Do not turn an account or device candidate into a statement that a named person performed the research. Record the actual matching method; deterministic equality and a probabilistic candidate have different semantics, and neither alone validates every associated field or purchase plan.

8. Recipients and destinations

List every CRM, warehouse, ad platform, report, client, integration, or subprocessor that receives the data. Include fields sent, purpose, transfer or activation time, owner, and onward-use restrictions.

Control: Reconcile the documented destination list with actual exports, connectors, audiences, and client deliverables. A contract appendix is not proof that no other destination exists.

9. Retention, deletion, and suppression lineage

Record the retention class, expiry, deletion propagation, backup treatment, suppression rule, and reingestion control for raw and derived data. Link the source signal to scores, identity relationships, audiences, and exports that may need correction or deletion.

Control: A suppression flag and a deletion action are not interchangeable. State which action applies to which system and why.

10. Approvals, incidents, and audit evidence

Name the person or role accountable for activation, changes, exceptions, incidents, and periodic review. Preserve approval records, control-test results, unresolved gaps, and remediation status.

Control: Agent-generated documentation is a draft until a responsible human reviews it. Automation cannot be the accountable owner.

A minimum viable provenance record in practice

Consider a fictional, account-level signal. A provider supplies a topic observation for “warehouse automation.” An ingestion job normalizes the company domain. A ruleset checks geography and size, then produces a fit-qualified account cohort. An identity process attaches known business contacts with confidence and suppression status. An operator approves a campaign audience. Outcome events later return from the CRM.

The minimum record should keep these as linked but separate objects:

• Raw entity: provider, topic definition, observed window, account identifier, source reference, and restrictions.
• Ingestion activity: job ID, processing time, validation results, and failed-record treatment.
• Qualification activity: rule version, input fields, exclusions, and resulting cohort.
• Identity activity: inputs, method, confidence, conflicts, and unresolved records.
• Activation entity: exact fields, destination, audience version, owner, and approval.
• Outcome entity: CRM definition, join method, attribution limitation, and time received.

In this fictional workflow, retain each derivation and its responsible owner. Prefer scoped internal identifiers and references rather than copying unnecessary personal data into the audit report. The example is not a formal PROV serialization or production BrandWell schema.

Operate provenance as a workflow, not a static spreadsheet

Assign clear responsibilities:

• The data owner defines the source, signal, transformation, and quality controls.
• The privacy or legal reviewer decides applicable purpose, role, restriction, and rights requirements.
• The security owner reviews access, transfer, vendor, incident, and evidence protection.
• The marketing or RevOps operator owns destination configuration and outcome feedback.
• The agency and client approver define handoffs, permitted scope, and who authorizes consequential activation.
• The vendor contact handles source questions, corrections, notices, and contract evidence.

Update the record when a source, definition, model, identity method, destination, purpose, contract, retention rule, or client changes. Trigger a review after an incident, rights request, unexpected audience expansion, or material quality failure. A quarterly checkbox is not enough if the workflow changed yesterday.

NIST’s Privacy Framework is a voluntary privacy-risk reference. Use it alongside the actual review process; it does not certify a provider or establish compliance.

Four ways to implement provenance documentation

1. Controlled spreadsheet or register

This is the fastest starting point for a small, stable workflow. Require controlled access, version history, unique IDs, evidence links, owners, and a review process.

Limitation: Manual registers drift when integrations or models change. They are weak at automated field-level lineage unless the team deliberately reconciles them.

2. Warehouse tables with version control

Store source, transformation, identity, destination, and approval metadata next to reproducible queries or code. This supports technical lineage and sampling at scale.

Limitation: A technically complete warehouse can still be unusable to a client, legal reviewer, or procurement team. Add a plain-language dictionary and responsibility layer.

3. Data catalog or lineage platform

Catalogs can discover systems, fields, owners, and transformations across a broad stack. Evaluate connectors, versioning, access controls, API and export, custom business metadata, and how well the tool represents marketing destinations.

Limitation: Automated discovery finds movement, not business purpose or lawful use. Human owners must supply and maintain that context.

4. Governance or privacy workflow platform

These systems can coordinate records, requests, approvals, assessments, and evidence. Test whether they cover the actual intent-data sources, identity graph, CRM, ad platforms, client reports, and deletion paths.

Limitation: A platform cannot compensate for missing source terms, weak identity logic, or an organization that will not resolve exceptions.

Legal and jurisdiction boundaries

Before relying on a dataset, assign reviewers for the actual collection, people, jurisdictions, organizational roles, purpose, channel, recipients and restrictions. Record their decisions and any unresolved questions. This article does not determine which legal duties or exceptions apply.

Do not turn this section into a universal checklist. The organization must assess where people are located, what data is involved, how it was obtained, the parties’ roles, the intended channel, contractual terms, and any sector-specific or sensitive-data restrictions.

Most importantly, do not claim that provenance proves consent. It shows what the organization documented and did. Legal permission, accuracy, identity, and intent each require their own evidence and review.

What does provenance documentation cost?

Cost depends on the number of sources, systems, fields, transformations, destinations, clients, jurisdictions, contracts, and changes. Model platform or data-catalog fees, integrations, process design, metadata stewardship, privacy and legal review, security review, audit sampling, vendor coordination, training, and remediation.

A spreadsheet can have a low license cost and a high ongoing labor cost. A platform can have a higher license cost while still failing if connectors and business definitions are incomplete. Compare total cost per governed workflow and the time required to answer a real traceability question.

Do not use a generic compliance budget. Write the scope, select a representative workflow, test retrieval and reconciliation, and obtain current written quotes for tooling and professional review.

Five tests for provenance control effectiveness

1. Traceability sample

Select activated records and trace each one back through identity and transformation activities to its source, owner, and permitted-use decision.

2. Destination reconciliation

Compare the documented recipients with actual integrations, exports, CRM fields, ad audiences, reports, and client deliveries.

3. Change-control test

Change a source definition, rule, or model version in a safe test environment and confirm that the provenance record, approvals, and affected outputs update.

4. Correction and deletion test

Confirm that lineage can locate raw and derived copies and route an approved correction, suppression, or deletion action without relying on tribal knowledge.

5. Exception-aging test

Review records with an unknown source, missing owner, expired evidence, unclear destination, or unresolved legal/security question. Track remediation rather than hiding them in a completion percentage.

Report coverage, retrieval time, reconciliation failures, unresolved exceptions, and remediation status. These are control indicators, not a guarantee of compliance.

Where BrandWell fits for agencies

An agency can design a research brief and provenance packet around a documented workflow. MarketPulse supplies topic research; it does not establish a native field-level lineage platform, complete client dossier or automatic legal review. Keep the agency-created record separate from what the product actually exposes.

Review the available topic fields and note any source, timestamp or transformation detail that is absent. Ask for current evidence rather than inventing the missing lineage or assuming an export includes the entire provenance history.

Use AIMEE through configured tools for a bounded draft of the field dictionary or brief. Retain references and unknowns, and have a named owner verify the output and actual permitted actions.

Before operational use, complete product, pricing, privacy, security, compliance, legal, and platform-policy review.

Five provenance failures to stop before activation

1. Static inventory drift

The register describes last month’s stack while new exports and destinations operate outside it.

2. Missing derivation

A score exists, but nobody can reproduce the rules, model version, or source entities that produced it.

3. Identity certainty inflation

An account, device, or probabilistic match is written as a known person’s behavior.

4. Provenance is treated as permission

A traceable record is misrepresented as proof of consent, lawful basis, or purchase intent.

5. Automation has no accountable owner

An agent generates polished documentation, but nobody approves, tests, corrects, or remediates it.

If the source, purpose, identity method, destination, or responsible party cannot be explained, pause activation until the gap is resolved.

Provenance packet handoff checklist

Before a company or agency activates intent data, confirm that the packet includes:

• Source, collector or provider, contract reference, and responsible owner.
• Collection method, context, signal definition, and analysis unit.
• Purpose, permitted destinations, restrictions, and legal/privacy review state.
• Observation, ingestion, transformation, and expiry timestamps or rules.
• Transformation, model, threshold, and version lineage.
• Identity inputs, method, confidence, conflicts, and correction path.
• Recipient and destination inventory with actual field-level reconciliation.
• Retention, deletion, backup, suppression, and reingestion rules.
• Approvals, exceptions, incidents, control tests, and remediation owners.
• Client acceptance and an update trigger for every material change.

The test is simple: a reviewer should be able to reconstruct what happened and who decided it without exposing more data than the review requires.