Forage AI Custom Data vs a Pre-Built Dataset: Who Owns the Part That Distinguishes You

Search a data catalogue for labelled support tickets while you build a ticket classifier, and one of the off-the-shelf datasets will look close. It has the obvious fields, a fair price, and it is available today.

Then the trial runs. The product categories are not yours. The tickets come from other companies' help desks, the file refreshes quarterly while you retrain monthly, and "resolved" means something else in its schema. A B2B team buying a company file hits the same wall at headcount and industry code.

A pre-built dataset is built once and licensed to every buyer who wants it. Custom data is collected to your fields, sources, definitions and refresh cadence.

A catalogue is built for the median buyer's schema. The fields every buyer shares are in stock; the fields that make your product different are the least likely to be. One large cloud marketplace's standard data subscription agreement is candid about it: the provider does not warrant "that the Data will meet Subscriber's requirements" (the July 2022 version, still the standard as of October 2026).

The short answer: buy the commodity layer, have the part that distinguishes you built to your spec, and plan to run both. Forage AI builds datasets to each client's spec and does not sell off-the-shelf datasets, so the pre-built wins below are ones we would lose.

By the end, you will have a rating and a committed winner for 11 factors, a field-by-field coverage audit for your own spec, five licence clauses to read before you buy, and a rule for when to buy, build or combine.

Factor	Forage AI custom (0-5)	Pre-built dataset (0-5)	One-line rationale
1. Cost	2	4	Collection and upkeep are shared across buyers on commodity fields
2. Speed	2	4	Available today and sampleable; a build starts with 1-2 weeks of onboarding
3. History	2	4	A catalogue ships backfill; a new collection starts at go-live
4. Fit	4	2	Your fields and definitions are spec items, where a public signal exists
5. Coverage	3	3	Pre-built wins breadth; neither fills entities that emit no signal
6. Freshness	4	2	Cadence per field is yours to set, not the provider's
7. Breakage	4	2	One accountable team owns the fix; not a correctness guarantee
8. Change	4	2	A field can be added by request, at the cost of a ticket and a wait
9. Rights	3	3	Decided by the clause you read and the record of collection
10. Ownership	4	2	You keep the delivered dataset; a licence ends in deletion
11. Benchmarks	1	4	Shared data is what makes results comparable

Scores anchor to the evidence cited in each factor. No review-platform ratings are used; every survey cited states its sample size, or says the sample size is unpublished. As of October 2026. Drafted by Forage AI's content team.

Off-the-shelf datasets vs custom data, factor by factor

Each factor below asks one question a buyer asks before signing. Every factor opens with a training-data example, then a B2B, alternative-data or ecommerce one. And every winner names a side and a layer: the commodity layer every buyer shares, or the part of your spec that distinguishes you.

1. Cost: which is cheaper to start and to run?

A general-language corpus licensed to several labs is cheap per buyer because the provider paid for collection, cleaning and upkeep once and spreads it across every licence. A build does the opposite: you fund the collection and the upkeep of a feed only you use.

In alternative data, Neudata reported in February 2026 that the average dataset on its platform is sold to about 20 clients, down from 25; it does not publish the sample size. We read that as cost sharing, not a price.

Cheap is not sufficient, though. A Hacker News commenter at a Tier-1 automotive supplier, describing a bought survey dataset in June 2026: "A quick regression showed that satisfaction was strongly correlated with buyer age, and there was very little signal otherwise." A file without your signal gets supplemented anyway, by a build.

Winner: Pre-built dataset, for commodity fields.

2. Speed: which gets you usable data first?

A training set in a catalogue exists today, and you can test a sample of it before you sign. A build cannot be tested before it exists. It starts with onboarding (ours typically takes 1-2 weeks), and the first delivery follows from the spec.

Speed to a file is not speed to an answer. In Neudata's September 2026 survey of 69 alternative-data buyers (self-reported), the top reason a trial did not convert was "no discernible signal", at 39%, against 25% for price. Our guide to alternative data for hedge funds walks through that evaluation loop.

Even so, the catalogue gets you to a first test sooner.

Winner: Pre-built dataset.

3. History: do you need years of data on day one?

Training on several years of text or prices needs those years to exist. A catalogue ships whatever backfill it holds; a new collection starts at go-live, and a web page's past states cannot be collected after they change.

Certain sources cannot be re-collected on the same terms either. An audit of robots.txt and terms of service across 14,000 domains (Longpre et al., "Consent in Crisis", arXiv, July 2024) found 2023-2024 restrictions made "28%+ of the most actively maintained, critical sources in C4" fully restricted from use, a limit that bites "if respected or enforced", in the authors' words.

For a new collection, web archives allow partial backfill: a 2012 study found no more than 31.3% of sampled URIs were archived more than once a month.

Backtests on alternative data need history too, though Deloitte Insights (November 2023) notes that "many alternative data sets are relatively new and therefore offer a very limited historical view of data", so check the catalogue's start date.

Winner: Pre-built dataset.

4. Fit: does it carry your fields, defined your way?

Fit starts with whether the text in a training corpus is usable at all.

A catalogue is assembled from signals that are easy to harvest and shaped to the median buyer's schema. In B2B files, headcount can be estimated: one company-data vendor's published data dictionary defines employees_count_inferred as "Estimated number of employees, calculated based on inferred employee data", and its plain employee count is the number of professional-network profiles linked to the company. Neither is payroll headcount.

Definitions also move. Etsy, a company we have no relationship with, reported 2,375 employees at the end of 2025, "including approximately 475 Depop employees"; after selling Depop to eBay, it said in August 2026 that headcount "is expected to be approximately 1,600 people". Three defensible numbers in about seven months, and none of SEC EDGAR, GLEIF or the New York corporations register carries Etsy's website domain.

Built to spec, your fields and definitions are line items, as long as the field has a public signal. Our roundup of B2B data enrichment tools covers the bought side.

Winner: Forage AI custom, for fields that have a public signal.

5. Coverage: which reaches more of your entities?

Breadth is where catalogues are strongest; the tail is where everyone runs out. The Data Provenance Initiative's December 2024 audit of about 4,000 training datasets in 608 languages (arXiv; ICLR 2025 per the authors' arXiv record) found multilingual representation has "failed to significantly improve their coverage since 2013". See our guide to multilingual datasets.

Firm data has the same shape. A 2017 Federal Reserve working paper, an older source, found a widely used commercial U.S. establishment database covers "about three quarters of U.S. private sector employment" under its strictest restrictions (a share of employment, not of firms). Its gaps sat among small establishments, "where imputation is prevalent": in 2014, 71.88% of establishments with 1-4 employees carried imputed employment. In the largest cross-country commercial firm database, financial variables "when downloaded, turn out to be missing, especially going back in time" (AEJ: Macroeconomics, April 2024).

A build cannot reach an entity that emits no public signal either, and Forage AI shares that limit.

Winner: Tie: pre-built wins breadth; neither fills the no-signal tail.

6. Freshness: who sets the refresh cadence?

A model retrained monthly on a quarterly file sees new rows once every three runs, and on a catalogue the provider sets that clock. One cloud marketplace's standard licence (current as of October 2026) says update frequency "shall be as set forth in the applicable DSA Offer", meaning the provider's listing. An ecommerce team pricing daily from a monthly file meets the same gap.

A dataset's refresh date is not a field's last-verified date. The New York corporations dataset was refreshed on October 13, 2026, yet its CEO-name field for Etsy still read Josh Silverman, though Kruti Patel Goyal has been CEO since January 1, 2026. GLEIF recorded eBay as Depop's parent about six weeks after the sale. Registers run on their own clocks, and a file built from them adds another.

Built to spec, cadence per field is a spec item; if the provider's cadence already matches yours, there is no gap. Decay rates live in our B2B data decay statistics.

Winner: Forage AI custom.

7. Breakage: who notices a fault, and who fixes it?

When a bought training feed starts returning wrong values, standard terms settle who fixes it. One data-cloud marketplace's terms (updated March 31, 2026) make the provider "solely responsible for addressing any … support or maintenance matters, including those related to quality, content, errors, or refunds".

A cloud marketplace's standard licence does not warrant that the data "will be accurate, complete, or up-to-date". Its 30-day cure covers only its named warranties, such as personal data and malware: a cure exists, but not for accuracy. The provider indemnifies you against rights claims for the use the licence allows; it does not warrant the data is accurate or complete. On one cloud data exchange, a revoked revision cannot be queried.

In practice, the buyer can be the first to notice, and the cost is the wait. One buyer of a query-delivered feed wrote in 2024: "periodically that query stops working because 'woops we changed our schema', and we have to wait for a new query for us to run."

Built to spec, Forage AI gives you one accountable team and source-change monitoring with alerts; it detects a source change, not a wrong value. See why enterprise extraction pipelines break for the failure modes.

Winner: Forage AI custom, for owning the fix. Not a correctness guarantee: test both on your own sample.

8. Change: can you add or change a field later?

Suppose your classifier needs a new label next quarter, or your company file needs a funding-stage field. On a catalogue, you get the provider's schema. One standard licence lets you "use, and modify the Data and to create Derived Data" internally, so you can compute new columns in your own copy. You cannot add a field to the product.

Built to spec, a field can be added or changed by request. That request joins the partner's queue, which is the honest con of a managed build: you wait on someone else's turnaround. Even so, a change request still beats a new purchase.

Winner: Forage AI custom, though a change is a ticket and a wait.

9. Rights: what can you do with the data?

A licence, ownership and permission to train are three separate rights. On widely used dataset hosting sites, the Data Provenance Initiative found "license omission of 70%+ and error rates of 50%+" across 1,800+ text datasets (Nature Machine Intelligence, August 2024). Its December 2024 follow-up found "while less than 33% of datasets are restrictively licensed, over 80% of the source content in widely-used text, speech, and video datasets, carry non-commercial restrictions."

Paid licences differ by clause. Of three standard licences we read in October 2026, one forbids training, one never mentions it, and one allows it for models but not the data (clause 1 below).

A build is not rights-clear by default either: collected web content carries its source's terms, and provenance exists only if source and date are recorded at collection. Ask any partner, us included, what they record. Pre-labelled sets add label opacity, which a data labelling company addresses and we do not.

Winner: Tie, decided by the clause: pre-built wins where a licence names training in words; built wins where you need the record of collection.

10. Ownership: what survives when you stop paying?

A training set you license is not one you own. One cloud marketplace's standard licence (as of October 2026) states: "Provider will retain all right, title and interest … in and to the Data". Derived data you create, minus the unmodified data, is yours; the data itself must be removed within 90 days of expiry and is the provider's confidential information, "including the selection, arrangement, and compilation".

An AI training-data marketplace lets either party terminate on 60 days' notice, after which you "cease using and delete, destroy, or return all copies of the Data and certify in writing". UK wholesale market-data users told the Financial Conduct Authority that contracts require them to purge history on exit (users' testimony, not an FCA finding).

Built to spec, you own the delivered dataset, and Forage AI does not resell it; rights in the source content are a separate question. The edge is not exclusivity but a dataset fitted to your spec that stays with you after the contract ends.

Winner: Forage AI custom, on the delivered dataset; source-content rights stay separate.

11. Benchmarks: which lets you compare results with others?

Benchmarks are the one place where shared data is the point. Stanford's HELM project reported that before it, language models "were evaluated on just 17.9% of the core HELM scenarios" on average, "with some prominent models not sharing a single scenario in common". Running 30 models on the same scenarios raised that to 96.0% (arXiv, November 2022).

Comparability comes from everyone using the same data, and a bespoke build defeats it. Versioned catalogue revisions are reproducible, and established language-resource distributors document commercial training rights corpus by corpus.

Winner: Pre-built dataset.

Factor	Forage AI custom	Pre-built dataset	Winner
Cost	You fund collection and upkeep of a feed only you use	Collection and upkeep are spread across every buyer	Pre-built (commodity fields)
Speed	Onboarding first; cannot be tested before it exists	Available today; sample before signing	Pre-built
History	Starts at go-live; past web states cannot be re-collected	Ships the provider's backfill; check the start date	Pre-built
Fit	Your fields and definitions are spec items, where a public signal exists	Built for the median buyer's schema; headcount may be estimated	Forage AI custom (fields with a public signal)
Coverage	Cannot reach entities with no public signal	Broader, but thin in the small-firm and language tail	Tie
Freshness	Cadence set per field in the spec	Cadence set by the provider's listing	Forage AI custom
Breakage	One accountable team; source-change monitoring with alerts	Fixes on the provider's calendar; accuracy not warranted	Forage AI custom (owning the fix, not correctness)
Change	A field is added by request, after a ticket and a wait	The provider's schema; derived columns in your copy only	Forage AI custom
Rights	Collected content carries its source's terms	Training forbidden, silent or allowed, by licence	Tie (decided by the clause)
Ownership	You own the delivered dataset; no reselling; source rights separate	Provider keeps title; delete or remove at exit	Forage AI custom (delivered dataset)
Benchmarks	A bespoke build cannot be compared with others' results	Shared, versioned data makes results comparable	Pre-built

How do you find what a catalogue dataset won't cover?

The factors point one way: a catalogue serves the median buyer's schema, so what matters is how far your spec sits from it. No study measures that gap for your requirement. The audit does, with your spec as the input.

Suppose the catalogue covers most of your fields, say 60%. The other 40% is where your product differs, and it is the part to audit. That split is an illustration, not a measured industry figure; your own split is what the audit produces.

1. Write your spec field by field: name each field, define it your way and set the refresh you need.
2. Get the data dictionary and a known-good sample: test on records where you already know the truth; a sample drawn only from your own CRM imports its biases.
3. Mark each field: covered, partly (a different definition), modelled or estimated, or missing.
4. Check the public signal for every gap: a missing field can be built only if it shows up in a website, a filing or a document. No signal means nobody can build it.
5. Count your own split: re-run the audit at renewal or whenever your spec changes.

Field	Your definition	Catalogue status	Public signal exists?	Refresh needed vs offered	Who fixes it	Rights note
Headcount (example)	Payroll employees, continuing operations	Modelled or estimated	Yes, in annual filings for listed companies	Quarterly vs annual	The provider, on its calendar	The licence decides reuse
Seller fee (example)	Current marketplace transaction fee	Missing	Yes, on the company's fee pages	Monthly vs not offered	You, or a build partner	Check the source website's terms

We ran the audit on one real company on October 14, 2026: Etsy, Inc., traced through SEC EDGAR, GLEIF, the New York Department of State's corporations register and Etsy's own filings. The table scores what a catalogue-style record built from those registers would carry against a 12-field spec.

Field	What the public record shows	Status
Legal name	"ETSY INC" (EDGAR) vs "ETSY, INC." (GLEIF, New York register)	Covered; punctuation breaks exact joins
Website domain	No field in GLEIF or the New York register; empty in EDGAR; in filing prose only	Missing in registers; public signal exists
HQ address	EDGAR: 117 Adams Street, Brooklyn. GLEIF: legal address a registered agent in Delaware. New York register: an agent in Harrison, NY	Partly: three definitions of "address"
Industry code	SIC 7389, "Services-Business Services, NEC" (EDGAR only); no NAICS anywhere	Partly: a catch-all code
Headcount	In no register. Filings: 2,375 at end-2025 incl. about 475 at Depop; "approximately 1,600" expected (August 2026)	Missing in registers; dated, moving filing signal
Revenue	10-K: $2.88bn total revenue, FY2025	Covered (listed company; annual, lagged)
Ultimate parent	GLEIF: no parent (reporting exception)	Covered
Subsidiaries	GLEIF links 2 children; the 10-K exhibit lists 3 "significant" ones	Partly: sources disagree
CEO	New York register: Josh Silverman. Filing: Kruti Patel Goyal, CEO since January 1, 2026	Stale at source
Status	Active in all three	Covered
Last-verified date	GLEIF only: last update July 2, 2026, "fully corroborated"	Partly
Seller transaction fee	10-K: "a 6.5 % transaction fee"; also on the company's fee pages	Missing in registers; public signal exists

Our count for this one record: 4 covered, 6 partly covered, differently defined or stale, and 2 missing from the registers but collectable. One company, our scoring: an illustration, not a rate.

Identity fields are clean and authoritative, which is why the commodity layer is worth buying. The fields that move (CEO, subsidiaries, headcount) changed inside ten months, each register catching up on its own clock. The fee sits in no register, yet it is public, so it can be collected. And this is a best case: a private company files no headcount or revenue, so a catalogue's value there is likely estimated, and a build cannot do better without a public signal.

Run the audit before the trial: a trial that ends in "no discernible signal" cannot tell you whether the data failed or the test did. Measure coverage against a known register, not the provider's record count. Our enterprise evaluation checklist covers the partner side of the same test.

Five licence clauses to read before you buy

This section is for informational purposes only and does not constitute legal advice. Consult a qualified attorney for legal guidance specific to your situation.

The audit tells you which fields you need; the licence tells you what you may do with them. Take these five clauses, from standard terms as of October 2026, into the procurement meeting, and read each clause rather than trusting a "fully licensed" badge.

1. Training: named in a few licences and absent from others. A large content library's standard licence (April 2026) forbids machine-learning and AI use of its images, media and metadata unless separately authorised, a right it sells separately. An AI training-data marketplace allows commercial use of models trained on its data, provided the data itself is not made available. A cloud marketplace's licence puts models under derived data and never says "train".
2. Modification: one standard licence lets you "use, and modify the Data and to create Derived Data"; a data-cloud marketplace bars derivative works by default unless the listing terms say otherwise.
3. Redistribution: barred in all of them. One licence forbids you to "publish, disseminate, distribute or provide access of any kind to the Data" to any third party.
4. Exit: remove the data within 90 days of expiry but keep your derived data (one licence); delete and certify in writing after 60 days' notice (another); lose query access to a revoked revision (a third).
5. Accuracy: not warranted. The cloud marketplace's standard licence disclaims it, and the AI training-data marketplace supplies its data "AS IS".

GDPR, CCPA and the EU AI Act remain your obligations as the user of the data, whichever route you take. For the provider side, see our guide to AI training data providers.

Buy, build to spec, or combine?

With the audit run and the clauses read, the decision sorts field by field. The commodity layer is the set of fields every buyer in your category needs and a catalogue already sells; the differentiating layer is the fields, sources, definitions and cadence only your product needs.

• Buy when the audit marks fields as covered, when you need years of history or a shared benchmark, or when the data is licensed or exhaust data only its owner can sell, such as a card-transaction panel. In the Etsy example: legal name, status, parent and revenue.
• Build to spec when the missing fields have a public signal, your definitions or cadence differ from the provider's, or you must keep the result after the contract ends. In the Etsy example: the website domain and the fee, plus CEO and headcount if you need them current and defined your way.
• Combine as the default: buy the commodity layer, have the differentiating layer built, and own the join between them. Entity matching across the layers is buildable as a service, though no join yields one perfect version of a company; see our guide to automated entity matching.

Sort each dataset by where its rights originate. Data whose rights start with a provider stays bought; data that starts at a public source can be built. It is the kept + 1 logic of our comparison of Forage AI and a multi-vendor data stack: the licensed layer stays. If the open question is tooling rather than data, read custom web data extraction vs pre-built tools.

The part missing from the catalogue is not bad luck. A catalogue is built for the median buyer, so the fields it lacks are, by construction, the ones closest to what makes your product different. That is the part worth owning, and the audit you ran shows which part it is.

Why Forage AI for the part a catalogue won't cover

Once the audit has shown which fields sit outside the median buyer's schema, the remaining question is who builds them. Forage AI runs an end-to-end managed data pipeline, built to your own fields and systems. One accountable team owns it, source-change monitoring sends alerts when a source changes, and a new field is a request rather than a new purchase. Onboarding typically takes 1-2 weeks, and you own the delivered dataset; Forage AI does not resell it.

Choose Forage AI if your audit shows fields with a public signal that the catalogue lacks, definitions or cadence of your own, a need to own the fix, or a need to keep the dataset at exit. Our managed extraction comparison covers choosing a managed partner.

Choose an off-the-shelf dataset from a catalogue or data marketplace if the fields are commodity, you need history today, you need a shared benchmark, or the data is licensed and only its owner sells it.

Frequently asked questions

Where can you buy datasets for machine learning?

From dataset catalogues and data marketplaces, which list pre-built datasets under standard licences, and from owners who license their own data directly. Our guide to AI dataset marketplaces holds the roster. Before buying, run the field-by-field audit on a sample and read the training, exit and accuracy clauses.

Are off-the-shelf datasets good enough for production AI?

For commodity fields, they can be, once tested against your own spec: check fit and refresh cadence field by field on a sample where you know the truth. Standard catalogue terms do not warrant accuracy, so "production-ready" is something you verify, not something a listing proves.

What does "fully licensed" mean for AI training data?

It means a licence exists, not that training is allowed. In the three standard licences we read, training was forbidden in one, never mentioned in another and allowed for models but not the data in the third. Modification, redistribution and exit sit in separate clauses, so read each one.

What is a dataset vendor responsible for when the data is wrong?

Under the standard terms we read, support and maintenance sit with the provider, on its schedule. Accuracy, completeness and currency are not warranted. One cloud licence has a 30-day cure, but only for its named warranties, such as personal data and malware. The provider indemnifies you against rights claims, not against wrong values.

Can you combine an off-the-shelf dataset with custom data acquisition?

Yes, and it is the default we recommend. Buy the commodity layer, such as identifiers, legal names and status, and have the fields that distinguish you collected to your spec. Then own the join, and test the entity matching on records whose correct values you already hold.

How long does a custom dataset take?

With Forage AI, onboarding typically takes 1-2 weeks. The first delivery then depends on the spec: the sources, the fields, any documents involved and the refresh cadence you need. A catalogue dataset is available the day you sign, which is why speed goes to the pre-built side.

Related articles

• Top AI Training Data Providers (2026): Buyer's Guide: how to evaluate a training-data provider on sourcing, licensing and quality before you sign.
• Forage AI vs a Multi-Vendor Data Stack: What Happens at Twelve Vendors: the kept + 1 rule for which data layers to consolidate and which licensed suppliers to keep.
• Data Extraction Company: Enterprise Evaluation Checklist: what to ask a managed partner before you move a feed to it.