# Arcadium Data public tenders feed: schema v1

Every file holds the same columns. Sources: EU Tenders Electronic Daily (TED, official Search API, all EU/EEA notices above the EU thresholds and many below) and UK Find a Tender (official OCDS API, every UK public notice since the Procurement Act 2023).

## Tables

| Table | Layout | Cadence |
|---|---|---|
| `daily` | `daily/dt=YYYY-MM-DD/notices.{parquet,csv}` | daily; notices first stored since the previous run (new publications of the day and late ones) |
| `open` | `open/dt=YYYY-MM-DD/open_competitions.{parquet,csv}` | daily; every call for tenders whose deadline is today or later (the forward pipeline) |
| `snapshot` | `snapshot/week=YYYY-Www/notices.{parquet,csv.gz}` | weekly (Monday); every notice we hold, all history |
| `by_country` | `snapshot/week=YYYY-Www/by_country/country=XX.{parquet,csv.gz}` | weekly; the snapshot cut by buyer country |
| `by_cpv` | `snapshot/week=YYYY-Www/by_cpv/division=NN.{parquet,csv.gz}` | weekly; the snapshot cut by CPV division |

## Columns

| Column | Type | Nullable | Description |
|---|---|---|---|
| `notice_id` | STRING | no | Stable id `<source>:<native id>`, e.g. `ted:730004-2026`. |
| `source` | STRING | no | `ted` (EU Tenders Electronic Daily) or `fts` (UK Find a Tender). |
| `native_id` | STRING | no | Notice id at the source (TED publication number, FTS notice id). |
| `procedure_id` | STRING | yes | Procedure the notice belongs to (TED procedure id, OCDS ocid): joins a call to its award. |
| `kind` | STRING | no | planning, competition (open call), award, modification, cancellation or other. |
| `notice_type` | STRING | yes | Notice type as published (eForms `cn-standard`, OCDS tags `award,contract`). |
| `published_on` | STRING | no | Publication day (YYYY-MM-DD). |
| `deadline_on` | STRING | yes | Earliest deadline for tenders or requests to participate (competitions only). |
| `title` | STRING | no | English title (TED: official machine translation "Country – CPV – title"), else original. |
| `title_original` | STRING | yes | Title in the language of the notice. |
| `language` | STRING | yes | ISO 639-1 of the notice. |
| `buyer_name` | STRING | yes | Contracting authority (an organisation). |
| `buyer_country` | STRING | yes | ISO 3166-1 alpha-2 of the buyer. |
| `buyer_nuts` | STRING | yes | NUTS 3 (EU) / ITL 3 (UK) region of the buyer. |
| `buyer_city` | STRING | yes | City of the buyer. |
| `buyer_type` | STRING | yes | Buyer legal type as published (e.g. `cga` central government, `ra` regional authority). |
| `cpv_main` | STRING | yes | Main CPV 2008 code (8 digits, no check digit). |
| `cpv_division` | STRING | yes | CPV division: first two digits of `cpv_main`. |
| `cpv_division_label` | STRING | yes | English label of the CPV division. |
| `cpv_codes` | STRING | yes | Every CPV code of the notice, `;`-separated, main first. |
| `contract_nature` | STRING | yes | works, supplies, services (`,`-separated when lots differ). |
| `procedure_type` | STRING | yes | Procedure as published (open, restricted, neg-w-call, ...). |
| `estimated_value` | DOUBLE | yes | Estimated value as published, in `estimated_currency` (net of VAT). |
| `estimated_currency` | STRING | yes | ISO 4217. |
| `estimated_value_eur` | DOUBLE | yes | Estimated value in EUR at the fixed monthly ECB reference rate. |
| `award_value` | DOUBLE | yes | Awarded value (total of the notice), in `award_currency`. |
| `award_currency` | STRING | yes | ISO 4217. |
| `award_value_eur` | DOUBLE | yes | Awarded value in EUR at the fixed monthly ECB reference rate. |
| `winners` | STRING | yes | Winning organisations, `;`-separated (lot suffixes removed). |
| `winner_countries` | STRING | yes | ISO 3166-1 alpha-2 of the winners, `;`-separated. |
| `winners_withheld` | INT64 | no | Winner names left out because they could name a private person (sole traders). |
| `place_nuts` | STRING | yes | NUTS / ITL codes of the place of performance, `;`-separated. |
| `url` | STRING | no | Official notice page. |
| `flags` | STRING | yes | `;`-separated data-quality flags: no_fx, mixed_currency, value_outlier. |
| `first_seen_at` | TIMESTAMP | no | When we first stored the notice (UTC). |

## Method

- **One row per published notice.** A procedure usually has a call (`competition`) and later a result (`award`); `procedure_id` joins them.
- **CPV**: codes normalised to 8 digits without the check digit; `cpv_division` = the first two digits.
- **Values**: as published (net of VAT where the source says so). EUR uses a fixed monthly table of ECB reference rates, so a rebuilt file gives the same numbers. Values above EUR 50 bn are placeholders and are dropped (`value_outlier`).
- **Award value**: the notice total, else the result value, else the sum of the winning tenders (TED); the awards, else the contracts (UK).
- **Deadline**: the earliest deadline over the lots.
- **Privacy**: buyers and winners are organisations. Contact persons, emails and phone numbers are never read from the source. A winner name that could be a private person (a sole trader trading under their own name) is left out and counted in `winners_withheld`. Free text goes through a filter that removes emails, phone numbers and personal profile links.