LLM Training Data Services: 20 Compared by Pipeline Stage

Search "LLM training data services" and nearly every page sells people who write fine-tuning answers, rank outputs and red-team models. On Google's first page on 16 November 2026, 7 of 9 rankable results were human-data vendors or their content, and none built corpora.
That suits a human-judgement gap, not a gap in LLM training data the model has never seen.
Three moves fix it: pick the band by pipeline stage, judge each service on what its page commits to in writing, and test the first delivery. By the end, you can name the two or three services that fit your gap, and what to ask each before signing.
- LLM training data services are two different purchases.
- Corpus-building services source and build the text a model learns from, for pre-training, continued training, fine-tuning and retrieval-augmented generation (RAG).
- Human-data services add human judgement afterwards: supervised fine-tuning (SFT) demonstrations, preference and RLHF data, red-teaming and evaluation.
- Most AI training data services ranking for the term sell the second.
- Name your gap, then shortlist in that band.
| Your gap | Pipeline stage | Kind of service | Go to |
|---|---|---|---|
| Domain or language text the model has not seen | Pre-training / continued training | Corpus building | Band 1 |
| Fine-tuning source text built from your documents and the web | Fine-tuning (source text) | Corpus building | Band 1 |
| Grounding records with source and refresh metadata | RAG / grounding | Corpus building | Band 1 |
| A corpus you already hold that needs dedup, filtering or decontamination | Any | Curation of your own corpus | Band 1 aside |
| People writing ideal answers | SFT demonstrations | Human data | Band 2 |
| People ranking or comparing outputs | Preference / RLHF / DPO | Human data | Band 2 |
| People attacking or grading the model | Red-teaming / evaluation | Human data | Band 2 |
| A ready-made corpus today | Any | Not a service: dataset or marketplace | What does this list leave out? |
Every statement about a service was read on its own page on 16 November 2026 and is quoted as its claim; nobody paid for placement. Forage AI publishes this page and builds corpora to spec, so it is first in Band 1 by house convention, not as a ranking, with the same rows as its peers.
How we picked these services
A service is listed if its own website offers that band's work for LLMs. Band 1's floor: it builds a corpus to the client's spec and names where the text comes from.
Band 1 tests are graded P (parameter or field list stated), S (step stated without one) or "Not stated by vendor" (the page is silent, not proof the service cannot). Samples, demo panels, testimonials and blog posts never count, on our page too.
| Test | What we checked |
|---|---|
| R1 Sources named | Kinds of sources |
| R2 Robots, terms, opt-outs | How each is handled |
| R3 Extraction | HTML, PDFs and scans to clean text |
| R4 Deduplication | Exact and near-duplicate: level, threshold, scope |
| R5 Quality filtering | Removing off-target text |
| R6 Personal data | Removal or masking step |
| R7 Benchmark decontamination | Overlap with evaluation sets |
| R8 Per-document provenance | Fields per record |
| R9 Refresh for RAG | How changes and deletions reach you |
| R10 Delivery format | Named formats and routes |
| Who writes the data | Experts or crowd |
| Quality control and agreement | How measured |
| Ownership default | Who owns the output |
| Pricing posture | Published unit or quote |
| Turnaround and cadence | Pilot, first batch, rhythm |

After Forage AI, Band 1 runs by tests stated, then tests with a parameter, then alphabetically; Band 2 is alphabetical.
No ratings column: buyer reviews are thin. Most services have none on the major platforms, the largest samples sit with one vendor (Appen: 35 on G2, 40 on Capterra, read 22 October 2026), and vendor-page scores can come from workers, not buyers. No quality ranking either: no neutral comparison of output quality across these services exists. We compare what each page commits to.
Band 1 disclosure: Forage AI is listed first in this band by house convention, not by the count; on the 10 tests its page states 8 and would place 4th, behind Actowiz, Import.io and Zyte.
LLM training data services at a glance
Numbers are list positions, not ranks; the two bands are different purchases, not ranked against each other. Stages as each page states them, read 16 November 2026.
| # | Service | Band | Stages stated |
|---|---|---|---|
| 1 | Forage AI | Corpus | Pre-training, fine-tuning, continued training; RAG and grounding; evaluation sets |
| 2 | Actowiz Solutions | Corpus | Pre-training, fine-tuning, evaluation sets, RAG |
| 3 | Import.io | Corpus | Training corpora, retrieval freshness, evaluation sets |
| 4 | Zyte | Corpus | Training, fine-tuning, evaluation sets, RAG, agent grounding |
| 5 | Octoparse | Corpus | Fine-tuning, continual pre-training, RAG, evaluation |
| 6 | Hitech BPO | Corpus (also human data) | Pre-training, fine-tuning, RAG |
| 7 | LXT | Corpus (also human data) | Pre-training, fine-tuning, evaluation |
| 8 | Abaka AI | Human data | SFT, preference / RLHF, red-teaming, evaluation, agentic |
| 9 | Appen | Human data | SFT, preference / RLHF, red-teaming, evaluation, agentic |
| 10 | Argos Data | Human data | SFT, preference / RLHF / DPO, evaluation |
| 11 | Digital Divide Data | Human data | SFT, preference, evaluation, agentic |
| 12 | iMerit | Human data | SFT, RLHF, red-teaming, evaluation, agentic |
| 13 | Innodata | Human data | SFT, RLHF, DPO, red-teaming, evaluation, agentic |
| 14 | Neurvix | Human data | SFT, preference / DPO, evaluation |
| 15 | Sama | Human data | Preference ranking, evaluation |
| 16 | Scale AI | Human data | SFT, RLHF, red-teaming, evaluation |
| 17 | Shaip | Human data | SFT, RLHF / DPO, evaluation, agentic |
| 18 | Surge AI | Human data | SFT, preference, evaluation, RL environments |
| 19 | Toloka | Human data | SFT, preference, red-teaming, evaluation, RL gyms |
| 20 | Turing | Human data | RL environments, datasets |
The services, grouped by pipeline stage
The router sets the band: corpus building when the model lacks text, human data when it lacks judgement.
Band 1: corpus-building services
Teams hand corpus collection off because sources block crawlers and documents resist extraction. The risk sits in steps a page can claim in three words.
"Deduplicated" can mean exact hashes inside one delivery or near-duplicate matching across every delivery, and scope changes the result. FineWeb (Penedo et al., Hugging Face, 2024) deduplicated 96 Common Crawl snapshots two ways. Each was tested with 1.71B-parameter models on equal token budgets. Global deduplication left 4 trillion tokens, and its models "showed little improvement over a model trained on the non-deduplicated data". Per-snapshot deduplication left 20 trillion tokens and improved scores.
The threshold matters too: FineWeb's MinHash setting, "at least 75% similar", catches a 75%-similar pair about 77% of the time.
"Deduplicated" is not an answer: ask the level (exact, near-duplicate, repeated paragraphs), the threshold, the scope (this delivery, all deliveries, against your corpus) and the share removed.
1. Forage AI
Forage AI is listed first in this band by house convention, not by the count; on the 10 tests its page states 8 and would place 4th, behind Actowiz, Import.io and Zyte.
| Attribute | Detail |
|---|---|
| Stages stated | Pre-training, fine-tuning, continued training; RAG and grounding; evaluation sets |
| Sources and rights | Maps "the source universe by region, language, publisher, and document type", coverage "confirmed during discovery"; access, licensing, permitted use, retention and privacy reviewed in scoping; approved sources only |
| Extraction | S: dynamic pages, scanned files, long documents, complex tables |
| Processing stated | Dedup S · filtering S · PII: Not stated by vendor |
| Provenance per record | P: source URL, collection timestamp, extraction method, validation and review status, lineage |
| RAG refresh | S: "refresh rules"; outputs revalidated as sources change |
| Delivery | P: JSONL, JSON, CSV, Parquet or custom, via S3, APIs, secure files or warehouse |
| Ownership | Built to order; clients own the data; never resold |
| Pricing posture | A scoped managed programme, quoted per project, not priced per record |
A fully managed service that builds "domain-specific web and document corpora, filtered, deduplicated, and structured for pretraining, fine-tuning, or continued training", plus RAG records "built around your retrieval and chunking design". Rights review is a scoping process, not a guarantee; quality runs on "automated validation, confidence scoring, and targeted human review". See how Forage AI builds data for AI.
Read on Forage AI's Data for AI page, 16 November 2026; ownership from its web data extraction page, pricing posture from its data extraction pricing guide. Not stated: a personal-data step; benchmark decontamination.
Not for: people-written SFT, output ranking or red-teaming (Band 2); a ready-made corpus this week.

2. Actowiz Solutions
| Attribute | Detail |
|---|---|
| Stages stated | Pre-training, fine-tuning, evaluation sets, RAG |
| Sources and rights | P: regulatory, government, patent and forum text "where terms permit"; robots snapshot; opt-outs respected |
| Extraction | S: boilerplate removed, share recorded |
| Processing stated | Dedup P · filtering S · PII P (flags, optional redaction) |
| Provenance per record | P: domain, URL hash, fetch timestamp, robots snapshot, declared licence |
| RAG refresh | S: incremental re-index; per-chunk freshness timestamp |
| Delivery | P: JSONL or Parquet with manifest; JSON, CSV, XLSX |
| Ownership | Not stated |
| Pricing posture | Quote, then fixed monthly |
Actowiz's page names a contamination check, with "contamination rates" reported, and states its limit: "we can check against corpora we can see". Not for: human-written SFT or preference data; self-serve buyers.
3. Import.io
| Attribute | Detail |
|---|---|
| Stages stated | Training corpora, RAG freshness, evaluation sets |
| Sources and rights | P: seed lists, sitemaps, link graph; robots and opt-outs applied pre-fetch; no paywalls |
| Extraction | S: main content, boilerplate removal, PDF parsing |
| Processing stated | Dedup P · filtering S · PII S ("PII detection and removal") |
| Provenance per record | P: URL, fetch timestamp, content hash, language, licence signal, robots status |
| RAG refresh | S: changes and removals delivered as events |
| Delivery | P: Parquet or JSONL with manifest and quality report |
| Ownership | Not stated |
| Pricing posture | "priced per corpus" |
The only page read that states a client-set threshold: "exact and MinHash near-duplicate removal at the threshold you set, with rates reported". Not for: human-written SFT or preference data.
4. Zyte
| Attribute | Detail |
|---|---|
| Stages stated | Training, fine-tuning, evaluation sets, RAG, agent grounding |
| Sources and rights | S: news, forums, docs, e-commerce, jobs; public only; opt-out detection, copyright flags |
| Extraction | S: JavaScript rendering; one schema per content type |
| Processing stated | Dedup S · filtering S · PII S |
| Provenance per record | P: "origin URLs, collection method, extraction date, and schema version" |
| RAG refresh | S: news "daily or intraday"; broader corpora "weekly or monthly" |
| Delivery | P: JSONL, Parquet, CSV; S3, SFTP, API, warehouse |
| Ownership | Not stated |
| Pricing posture | Not stated on the AI page |
Zyte states a scoped dataset "typically takes two to four weeks from kickoff to first delivery". Not for: human-written SFT or preference data.
5. Octoparse
| Attribute | Detail |
|---|---|
| Stages stated | Fine-tuning, "continual pre-training", RAG, evaluation |
| Sources and rights | Reviews, forums, news, docs, jobs; public only; robots and opt-outs: Not stated by vendor |
| Extraction | S: "cleaned body text, normalized encoding" |
| Processing stated | Dedup P ("against prior batches") · filtering: Not stated by vendor · PII S (scoped option) |
| Provenance per record | P: source, URL, publish timestamp, language, domain label |
| RAG refresh | S: batches carry new or changed records only |
| Delivery | P: JSONL, Parquet, CSV; S3, GCS, Snowflake, BigQuery |
| Ownership | Not stated; legal basis put on the customer |
| Pricing posture | Published entry prices; larger scopes quoted |
The only Band 1 service read that publishes a corpus price: "$699/project" one-time, "$599/month" recurring. Not for: corpora that must document opt-outs per document.
6. Hitech BPO
| Attribute | Detail |
|---|---|
| Stages stated | Pre-training, fine-tuning, RAG |
| Sources and rights | "public and proprietary datasets"; rights: Not stated by vendor |
| Extraction | S: boilerplate, noise, malformed tokens cleaned |
| Processing stated | Dedup S · filtering S · PII: Not stated by vendor |
| Provenance per record | S: "Metadata tagging" (no fields named) |
| RAG refresh | Not stated by vendor |
| Delivery | P: "JSONL, TFRecord, CSV" |
| Ownership | Not stated |
| Pricing posture | Quote |
Also sells human data (Band 2). Not for: buyers who need provenance fields named upfront.
7. LXT
| Attribute | Detail |
|---|---|
| Stages stated | Pre-training, fine-tuning, evaluation |
| Sources and rights | Contributor-collected manuals, publications, regulatory and conversational text; web sourcing: Not stated by vendor |
| Extraction | S: scans, handwriting "converted into machine-readable form" |
| Processing stated | Dedup: Not stated by vendor · filtering S · PII: Not stated by vendor |
| Provenance per record; RAG refresh | Not stated by vendor |
| Delivery | "your preferred format" |
| Ownership | Not stated |
| Pricing posture | Quote after scoping |
Contributor-led: corpora "sourced, curated, or transcribed" by its network. LXT states projects run "from small pilots to billion-token corpora". Not for: web-sourced corpora with per-document provenance.
What the corpus pages state: in our read of eight corpus-building services' own pages on 16 November 2026, one stated a benchmark-contamination check, limited to corpora it holds; three named a personal-data step as standard, a fourth offers one as a separately scoped option, and none named its detection method; one stated a client-set deduplication threshold.
If you already hold the corpus
Curating a corpus you hold is a different job, not a third band. DatologyAI states: "Datology runs on your data. We don't sell data. We don't source tokens." Its first stage removes "eval-contaminated data". NVIDIA NeMo Curator is open tooling for deduplication and filtering, not a service.
Band 2: human-data and post-training services
Vendor choice changes model outcomes. In its Llama 2 technical report (2023), Meta wrote that "different annotation platforms and vendors can result in markedly different downstream model performance, highlighting the importance of data checks even when using vendors to source annotations." The pilot decides, not the brochure, and the work is bought as a scoped contract delivered in batches.
Nathan Lambert, co-founder and executive director of Trillium Labs, a nonprofit for open science in frontier AI, and author of Reinforcement Learning from Human Feedback (Manning, 2026), writes that "a typical preference data contract might span a 6-week delivery period", with data "delivered in weekly batches".
| Service | SFT demonstrations | Preference / RLHF / DPO | Red-teaming | Evaluation | Agentic / RL environments |
|---|---|---|---|---|---|
| Abaka AI | ✓ | ✓ | ✓ | ✓ | ✓ |
| Appen | ✓ | ✓ | ✓ | ✓ | ✓ |
| Argos Data | ✓ | ✓ | - | ✓ | - |
| Digital Divide Data | ✓ | ✓ | - | ✓ | ✓ |
| iMerit | ✓ | ✓ | ✓ | ✓ | ✓ |
| Innodata | ✓ | ✓ | ✓ | ✓ | ✓ |
| Neurvix | ✓ | ✓ | - | ✓ | - |
| Sama | - | ✓ | - | ✓ | - |
| Scale AI | ✓ | ✓ | ✓ | ✓ | - |
| Shaip | ✓ | ✓ | - | ✓ | ✓ |
| Surge AI | ✓ | ✓ | - | ✓ | ✓ |
| Toloka | ✓ | ✓ | ✓ | ✓ | ✓ |
| Turing | - | - | - | - | ✓ |
"-" means the page does not state that stage. The five Band 2 tests:
| Service | Who writes the data | Quality control and agreement | Ownership default | Pricing posture | Turnaround and cadence |
|---|---|---|---|---|---|
| Abaka AI | Vetted domain experts | "multi-layer QAs", reviewer consensus | Client data "exclusively yours" | Published hourly, per evaluation | About 3 days to staff; 1-2 week pilot; weekly batches |
| Appen | Verified experts | Not stated | Not stated | Quote | Not stated |
| Argos Data | Vetted experts | "QA controls and expert review" | Not stated | Quote | Not stated |
| Digital Divide Data | Not stated | Not stated | Not stated | Quote | Not stated |
| iMerit | Scholars programme experts | "Quality Guaranteed" (no method) | Not stated | Quote | Not stated |
| Innodata | Linguists, taxonomists, experts | Not stated | Not stated | Quote | Not stated |
| Neurvix | Domain experts | Senior and bias review | Not stated | Quote | Pilot scoped "within 48 hours" |
| Sama | Associates | Not stated | Not stated | Scoped on contact | Not stated |
| Scale AI | Domain experts | Not stated | Not stated | Demo request | Not stated |
| Shaip | Domain experts | QA reviewers audit rankings | Not stated | Quote | Not stated |
| Surge AI | Doctors, lawyers, engineers, researchers | Not stated | Not stated | Not stated | Pilot "results within a week" |
| Toloka | Contributors and experts | Automated quality control | Not stated | Not stated | Not stated |
| Turing | Expert network | Not stated | Turing retains; client licensed | Not stated | Not stated |
Pages read 16 November 2026; figures are the vendor's own claims.
8. Abaka AI
The only managed human-data service read that publishes unit rates. Abaka AI states "over 1 million vetted domain experts". Not for: corpus building.
9. Appen
Appen claims "500+ global locales covered" on an LLM page structured by stage. Not for: buyers who need the QC method before a call.
10. Argos Data
Multilingual SFT and preference data. Its pre-training page names no source, so it is not in Band 1. Not for: sourcing a new corpus.
11. Digital Divide Data
Digital Divide Data also sells off-the-shelf datasets. Not for: buyers who want the method on the page.
12. iMerit
Expert pools for SFT, RLHF, red-teaming and agent evaluation; iMerit states "25,000+ Domain Experts". EXL announced its completed acquisition of iMerit on 3 August 2026. Not for: buyers who need published ownership terms.
13. Innodata
One page gives two language counts, so get coverage in writing. It also sells off-the-shelf datasets. Not for: buyers who need stated QC.
14. Neurvix
Expert-written instruction and DPO preference pairs, delivered as JSONL "with data cards, documentation and quality metrics". Not for: red-teaming or RL environments.
15. Sama
Sama states a "99% first-batch acceptance rate". Not for: SFT written from scratch; red-teaming.
16. Scale AI
Prompt-response pairs written "from scratch", RLHF, red-teaming and evaluation. Meta holds a non-voting minority stake in Scale AI, per Meta's July 2025 10-Q, and our Scale AI alternatives guide covers other routes. Not for: buyers who need published prices.
17. Shaip
Question-and-answer pairs created "by thoroughly reading large documents". Not for: red-teaming.
18. Surge AI
Surge AI's enterprise page names programme types but not how quality or agreement is measured; it also lists off-the-shelf data. Not for: buyers who need the QC method first.
19. Toloka
RL gyms, agent trajectories and "Reinforcement learning tasks with built-in verification". Toloka states "50+ methods of automated Quality control". Not for: corpus building.
20. Turing
The one published ownership default we read. Its Standard Terms (updated 24 February 2026, section 8) license deliverables "for its internal testing purposes and for any other purpose specified in the Order Form"; otherwise "Turing retains all right and ownership". Not for: owning and open-sourcing output without negotiation.
Hitech BPO and LXT also sell SFT, RLHF and red-teaming data. Also selling post-training data on their own pages: Invisible, Mercor and micro1 (expert human data, evaluations, training environments) and Prolific (self-serve participants for preference tuning and safety evaluations).
All 20 services side by side
"Stated" is not "verified"; the sample check does that. Pages read 16 November 2026.
| Service | Dedup · PII step · decontamination | Ownership | Pricing | Not for |
|---|---|---|---|---|
| Forage AI | Stated · Not stated · Not stated | Client owns (built to order) | Quote per project | People-written SFT, ranking |
| Actowiz Solutions | Stated · Stated · Stated | Not stated | Quote | People-written SFT |
| Import.io | Stated · Stated · Not stated | Not stated | Per corpus (quote) | People-written SFT |
| Zyte | Stated · Stated · Not stated | Not stated | Not stated | People-written SFT |
| Octoparse | Stated · Stated (scoped option) · Not stated | Not stated | From $599/month | Documented opt-outs |
| Hitech BPO | Stated · Not stated · Not stated | Not stated | Quote | Named provenance fields |
| LXT | Not stated · Not stated · Not stated | Not stated | Quote | Web-sourced provenance |
| Abaka AI | n/a | Client data "exclusively yours" | Published hourly, per evaluation | Corpus building |
| Appen | n/a | Not stated | Quote | QC method up front |
| Argos Data | n/a | Not stated | Quote | New corpus sourcing |
| Digital Divide Data | n/a | Not stated | Quote | Method on the page |
| iMerit | n/a | Not stated | Quote | Published ownership |
| Innodata | n/a | Not stated | Quote | Stated QC |
| Neurvix | n/a | Not stated | Quote | Red-teaming |
| Sama | n/a | Not stated | Quote | Written SFT, red-teaming |
| Scale AI | n/a | Not stated | Demo request | Published prices |
| Shaip | n/a | Not stated | Quote | Red-teaming |
| Surge AI | n/a | Not stated | Not stated | QC method up front |
| Toloka | n/a | Not stated | Not stated | Corpus building |
| Turing | n/a | Turing retains; client licence | Not stated | Default open-sourcing |
What does this list leave out?
A ready-made dataset is not a service, so these buys are routed, not ranked:
- Dataset marketplaces and catalogues (Datarade, Defined.ai and Shaip catalogues, SyndiGate): see AI dataset marketplaces.
- Open corpora such as Common Crawl, which you process yourself.
- Fine-tuning shops: on Google, "LLM fine-tuning data services" mostly means firms that fine-tune models.
- Labelling software and non-LLM annotation: see data labeling companies.
- Web-scraping companies: see web scraping companies for AI data.
- Supplier types across modalities: see AI training data providers.
- Read, not listed: GTS.AI (sources named; no processing step, provenance or format); Handshake AI and Alignerr (pages recruit workers); TELUS Digital was not read.
What should you ask a service before signing?
An audit of ten corpora that popular models trained on found duplicates, personal data and benchmark contamination: about 50% of RedPajama's documents were duplicates (Elazar et al., "What's In My Big Data?", ICLR 2024). Each question below makes a service answer one failure point in writing.
Questions for a corpus-building service
- Where does each record come from, and is the right to train on it written down?
- How are exact and near-duplicates removed: level, threshold, scope (this delivery, all deliveries, against my corpus), and share removed?
- What is your personal-data method, not your compliance label: categories, detection, action, and residual rate on my sample?
- How was the quality filter built for my domain and languages, and can I see rejected documents?
- Do you check for overlap with evaluation sets, by what method, and will you run it against my hold-out sets?
- What does each record carry for my stage: source URL, collection timestamp, lineage, and for RAG the refresh rule and deletion handling?
Why 1: a licence, data ownership and permission to train are three different rights; robots.txt is not a licence, and its absence is not consent. In the EU, the AI Act requires general-purpose model providers to comply with rights reservations under Article 4(3) of the copyright directive (Art. 53(1)(c)).
Why 2: duplicated text gets memorised. Lee et al. (ACL 2022) found "over 1% of the unprompted output" of models trained on common datasets was copied verbatim, and deduplicated training emitted memorised text "ten times less frequently".
Why 3: FineWeb removes only emails and public IP addresses by regex, which is a floor. Elazar et al. counted 7.6 million email addresses and 19.7 million phone numbers in C4 by regex, hand-checking precision on 100 random examples; recall was not measured. Leftover personal data comes back out: Pythia 6.9B reproduced 13.6% of 483 hand-checked personal-information strings verbatim when prompted with the preceding text (Subramani, Ghate and Diab, EACL Findings 2026). Ask for measured recall per category on a pilot sample.
Why 4: DataComp-LM (Li et al., 2024; 53 evaluations) found "model-based filtering is key", training a 7B model to 64% 5-shot MMLU on 2.6T tokens. Filters cost something too: blocklist filtering on C4 "disproportionately removes text from and about minority individuals" (Dodge et al., EMNLP 2021).
Why 5: Yang et al. (2023) showed a 13B model trained on paraphrased test items scored "on par with GPT-4", so string matching is not enough. Tülu 3 (Lambert et al., 2024) used 8-gram matching and removed 11.3% of one maths training set that overlapped MATH. Hand over your hold-out list and still run your own check.
Why 6: per-record provenance is what a training-content summary or deletion request is built from. EU AI Act Art. 53(1)(d), in force since 2 August 2025, requires a general-purpose model provider to publish "a sufficiently detailed summary about the content used for training": your duty, not the supplier's. For RAG, get the refresh rule in writing. Records to ask for (illustrative, not a Forage AI specification):
{"stage": "pretraining", "text": "...", "lang": "de", "source_url": "https://...", "collected_at": "2026-11-02T10:14:00Z", "extraction_method": "html", "dedup_cluster": null, "pii_redacted": ["email"]}
{"stage": "rag", "chunk_id": "doc-118#s3", "text": "...", "doc_title": "...", "section_path": ["Part II", "3.1"], "source_url": "https://...", "collected_at": "2026-11-02T10:14:00Z", "refresh_rule": "weekly"}
Questions for a human-data service
- Who are the raters, and how is their expertise verified?
- How is agreement measured, and will I see it on the pilot? (See pilot and agreement method.)
- Is there a paid pilot with acceptance criteria written before it ships? Lambert reports vendors that "have not delivered data as contracted without the customer threatening legal or financial action".
- Who owns the output, and may I open-source it? Lambert notes "the default terms on the contracts often prohibit the open sourcing of artifacts after acquisition"; check the contract clauses before signing.
- Is it built for me alone, or also sold off the shelf?
How to check a sample corpus delivery
Run machine checks on every document, human reads on a random sample, and write pass thresholds into the contract before delivery; we found no published acceptance threshold for LLM corpora.
| Check | Run on | Pass looks like |
|---|---|---|
| Manifest complete; counts reconcile | All 10,000 | Every contracted field present |
| Exact and near-duplicates, within and across deliveries | All | At or under the contracted rate |
| Boilerplate (paragraphs in over 0.1% of documents) | All | Each repeated line explained or removed |
| Personal data: detector, then hand-check flagged and unflagged | All + 200 | Residual misses as a range per category |
| 13-gram overlap with your evaluation sets | All | Zero overlapping documents (paraphrase not covered) |
| Stratified human read of HTML / PDF / scan extraction | 371 | Per-stratum error rate as a range |
| Provenance spot check (URL, hash, robots status) | 50 rows, about 20 domains | Fields true where testable |
With the error rate unknown, 385 documents give ±5 points at 95% confidence. On a 10,000-document delivery the finite-population correction brings that to 371 (the exact figure is 370; 371 is the conservative round), and 200 documents give about ±7 points. Report a range: 6 failed extractions in 371 means a true rate between about 0.7% and 3.5%. A sample tests this delivery, not the vendor's next one.
The first script checks every document (tested 16 November 2026, Python 3.9, standard library):
"""Acceptance checks a buyer runs on every document of a delivery (JSONL, one record per line):
manifest completeness, exact duplicates, and paragraphs repeated across documents (leftover boilerplate)."""
import json, hashlib, sys, collections
REQUIRED = ["text", "source_url", "collected_at", "content_hash", "license_signal", "robots_status"] # your contract's list
rows = [json.loads(l) for l in open(sys.argv[1], encoding="utf8")]
missing = collections.Counter(f for r in rows for f in REQUIRED if not r.get(f))
norm = lambda t: " ".join(t.lower().split())
hashes = collections.Counter(hashlib.sha256(norm(r["text"]).encode()).hexdigest() for r in rows)
exact_dups = sum(c - 1 for c in hashes.values() if c > 1)
para_docs = collections.Counter()
for r in rows:
for p in {norm(p) for p in r["text"].split("\n") if len(p.split()) >= 5}:
para_docs[p] += 1
threshold = max(3, len(rows) // 1000) # a paragraph in more than 0.1% of documents
repeated = [(p[:60], n) for p, n in para_docs.most_common(10) if n >= threshold]
print(f"docs={len(rows)} missing_fields={dict(missing)} exact_duplicates={exact_dups}")
for p, n in repeated:
print(f" repeated in {n} docs: {p!r}")
The second spot-checks a sample before signing (tested 16 November 2026, Python 3.9, datasketch 2.0.0):
"""Spot-check a vendor's sample corpus before signing: near-duplicates, obvious PII,
and 13-gram overlap with a benchmark you care about. Input: JSONL, one {"text": ...} per line."""
import json, re, sys
from datasketch import MinHash, MinHashLSH
def shingles(text, n=5):
words = text.lower().split()
return {" ".join(words[i:i + n]) for i in range(max(1, len(words) - n + 1))}
def minhash(text):
m = MinHash(num_perm=112) # FineWeb setting: 112 hashes...
for s in shingles(text):
m.update(s.encode("utf8"))
return m
docs = [json.loads(l)["text"] for l in open(sys.argv[1], encoding="utf8")]
lsh = MinHashLSH(num_perm=112, params=(14, 8)) # ...in 14 buckets of 8
near_dups = 0
for i, d in enumerate(docs):
m = minhash(d)
if lsh.query(m):
near_dups += 1
lsh.insert(str(i), m)
email = re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+")
ipv4 = re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b")
pii_docs = sum(1 for d in docs if email.search(d) or ipv4.search(d))
def ngrams(text, n=13):
w = text.lower().split()
return {" ".join(w[i:i + n]) for i in range(len(w) - n + 1)}
bench = set()
if len(sys.argv) > 2: # optional: benchmark test set, one item per line
for line in open(sys.argv[2], encoding="utf8"):
bench |= ngrams(line)
contaminated = sum(1 for d in docs if ngrams(d) & bench)
print(f"docs={len(docs)} near_duplicates={near_dups} "
f"docs_with_email_or_ip={pii_docs} docs_with_13gram_benchmark_overlap={contaminated}")
On a 42-document test file with planted defects, the first script found the missing licence field, the exact duplicate and a boilerplate line in five documents. The second found the email, the IP address and the benchmark sentence, but missed a planted near-duplicate at 0.767 similarity, which FineWeb's settings catch about 83% of the time. It is a sample check, not proof: the threshold in your contract decides what counts as a duplicate.
Limits: the email and IP regex is a floor, not a PII check (regex precision on IP addresses was 43-56% in two large corpora, WIMBD); a clean 13-gram result is necessary, not sufficient; the field names are a buyer's contract list, not a Forage AI specification.
Provenance with the delivery: if you would rather receive per-record source URL, collection timestamp, extraction method and lineage than reconstruct them, Forage AI builds corpora with those fields. See Forage AI's data for AI service.

How are LLM training data services priced?
Human data is priced per hour, per evaluation or per contract. On pages read 16 November 2026, Abaka AI's FAQ states "STEM Generalist tasks are priced at $12/hr", "LLM Math and Coding experts are available at $18/hr" and red-teaming at "$8 per evaluation". Prolific, a self-serve platform, adds a fee "usually 42.8% for corporate customers" to participant pay. Lambert calls a preference-data contract of about $500K "a smaller data contract", and first batches are often discounted, so they are not the run rate.
Corpus building is mostly quoted after scoping. Octoparse publishes "$699/project for one-time datasets and $599/month for recurring feeds"; Import.io prices "per corpus"; Actowiz quotes each engagement; LXT quotes after scoping; and Forage AI runs each corpus as a scoped managed programme, quoted per project, not priced per record. Zyte and Hitech BPO publish no price on the pages read.
An entry price is a floor for a defined scope, and no neutral price data exists for corpus building, so compare quotes against one written spec.
Which services should you shortlist for your stage?
Names are alphabetical, chosen by what each page states. Send each the questions above and keep the two or three that answer in writing.
- A domain or language corpus for pre-training or continued training (stage and record fields stated): Actowiz Solutions, Forage AI, Import.io, Octoparse, Zyte.
- RAG or grounding records that stay current (change handling stated): Actowiz Solutions, Import.io, Octoparse, Zyte.
- SFT and preference data (both stages and a timeline stated): Abaka AI, Neurvix, Surge AI.
- Red-teaming, evaluation and agentic or RL data (all three stated): Abaka AI, Appen, iMerit, Innodata, Toloka.
Re-run the sample check on every delivery, because a vendor's page and its pipeline both change. Every entry carries its read date, so when a service publishes a method, tell us and the census line changes.
Why Forage AI for LLM training data services
Forage AI builds domain-specific web and document corpora and RAG records to your spec, with per-record provenance and refresh rules, delivered into your systems. Forage AI runs each corpus as a scoped managed programme, quoted per project, not priced per record.
Choose Forage AI if your gap is a domain or language corpus, or RAG records, built from web and document sources to your schema, with provenance and a refresh cadence, and you want the build and maintenance handed off.
Choose a human-data service if you need people to write SFT examples or rank and red-team outputs (Band 2). For a ready-made corpus this week, use a dataset marketplace; for labelling-only work, a specialist labelling provider.
Scope a corpus with Forage AI.

FAQ
Which companies provide LLM training data services?
Corpus building: Actowiz Solutions, Forage AI and Hitech BPO, among seven in Band 1. Human data: Abaka AI, Appen and Argos Data, among thirteen in Band 2. Pick the band from your pipeline stage first.
What is the difference between corpus-building and RLHF data services?
Corpus builders source the text a model learns from, for pre-training, fine-tuning and RAG. Human-data services have people write, rank or attack outputs: SFT, preference and RLHF data, red-teaming and evaluation. Fine-tuning splits by who writes the examples.
Where can I get training data for an LLM?
Open corpora and dataset marketplaces are ready today. Corpus-building and human-data services build to spec, including fine-tuning or RAG data from your own documents.
How are LLM training data services priced?
Human data is priced per hour, per evaluation or per contract. Corpus building is mostly quoted after scoping; one Band 1 service publishes entry prices.
Who owns data a service builds for me?
Few pages say. One vendor's terms keep ownership and license deliverables for internal testing and the order form's purposes; Lambert notes default terms often bar open-sourcing. Get the clause in writing before the pilot.
Do LLM training data services check for benchmark contamination?
One of eight corpus-building pages we read states a check, limited to corpora that service holds. String matching also misses paraphrased test items (Yang et al., 2023), so run the 13-gram check yourself.
How much training data does an LLM need?
It depends on the stage. Meta stopped at 27,540 human-written SFT annotations for Llama 2 (2023); DCLM trained a 7B model on 2.6T tokens (2024); Tülu 3 (2024) used 939,344 SFT prompts, including synthetic ones. There is no general number.
Related articles
- AI training data providers: which kind of supplier fits, across data types and modalities.
- Data labeling companies: how to pilot a labelling partner and measure agreement.
- Forage AI vs pre-built datasets: when a ready-made dataset is enough and when to build to spec.
- Data vendor due diligence: the contract clauses to check before signing a data vendor.
- Multilingual datasets: where multilingual training text comes from and what to check in it.
Sai is a data infrastructure enthusiast who has spent the past two to three years following the AI space closely, from the infrastructure layer to the fast-growing world of data for AI. He is genuinely curious about how modern data pipelines get built and where the data industry is heading, and he writes insightful pieces on the core topics that shape this niche.