Dataset delivery specification

A delivery specification should identify the exact collection licensed, not just the destination where files will be uploaded. Define the snapshot date, source partitions, inclusions and exclusions. State what counts as one record and how records relate to attachments, labels or sensor streams. A manifest should list the delivered objects and the identifiers used to connect them. Include a version identifier that both parties can refer to in review findings. This gives acceptance a stable object and prevents later uncertainty about whether a file belonged to the original delivery, a correction or a separate refresh.

Describe formats in enough detail to test them. For structured records, define field names, types, required values and treatment of missing data. For media, specify encoding and the associated metadata needed to interpret it. For text, explain document boundaries, character encoding and whether formatting is preserved or normalized. Use examples that illustrate the agreed schema without exposing confidential records. A format name alone rarely describes every convention that a recipient needs. Establish how the specification handles unsupported values or additional fields instead of leaving each exception to an improvised interpretation.

Agree which checks establish that the delivery arrived intact. A file manifest can support count comparisons and integrity checks without asserting that the content is semantically correct. Keep those technical checks separate from label review, provenance review and task suitability. If a usable item requires several linked files, specify how the recipient identifies missing components and whether the whole item is considered incomplete. The specification should also describe naming conventions, stable identifiers and any mapping from source IDs to delivered IDs. These details help a recipient investigate a defect without needing access to your original business systems.

Train, validation and held-out partitions need explicit treatment when they are part of the agreement. State who creates each split, how related records are kept together and which uses are allowed for each partition. A held-out set intended for evaluation should not be silently included in the training export. If no split is supplied, say so rather than implying that the collection arrives ready for every workflow. The owner and recipient can negotiate responsibilities for later partitioning. This keeps technical preparation consistent with the licence and with the documentation describing the collection.

Transfer arrangements should identify the approved recipient, access method and expiry of temporary credentials or links. Agree how receipt is acknowledged and who can raise technical questions. Do not include unrelated system access merely because it is convenient for export. If a recipient uses contractors to process the delivery, the agreement should address their access consistently with the permitted downstream rights. Keep transfer logs appropriate to the task, without using a public inquiry to exchange credentials or raw confidential material. A delivery plan can describe security requirements before any particular transfer service is selected.

Corrections and refreshes should use predictable versioning. Explain whether a corrected batch replaces an earlier version, adds a patch or supplies a new snapshot. Provide a change record identifying removed, changed and added items. State the support window, review procedure and any separately agreed duties for further extraction. At the end of a licence, obligations for stored copies, derived material and trained models belong in the agreement; the specification should implement those decisions rather than invent them. Where retention or deletion depends on a contract clause, reference the agreed requirement in the private delivery record.

Test the specification with a controlled sample before full delivery. The test should exercise the same export procedure, manifest and documentation planned for the final collection. Ask the recipient to report concrete interpretation problems, such as inconsistent timestamps, unclear units or broken record links. Resolve those questions in the written specification rather than relying on explanations that only one person remembers. A sample that can be understood and checked under the proposed conventions creates a clearer starting point for acceptance. It is still not a guarantee of model performance or approval of every record in the complete archive.