Dataset APIs

Import LedgeKitTesting for these APIs. Credentials come from LedgeKit.

LedgeDatasetClient

public init(credentials: LedgeCredentials, session: URLSession = .shared)
public func load(_ id: UUID) async throws -> LedgeDataset

Use an app read key. A successful load validates the response ID, unique item IDs, item count, source shape, and limits of 1,000 items and 4 MiB. A downloaded value is a snapshot; later edits in the console do not mutate it.

LedgeDataset

Immutable fields: id, appId, name, sourceKind, definitionName, createdAt, itemCount, and items. SourceKind is .trace or .span; SourceMode is .live or .test.

Each Item has id, sourceMode, sourceTraceId, optional sourceSpanId, input, and createdAt.

public func decodeInputs<Value: Decodable & Sendable>(
    _ type: Value.Type,
    sourceKind expectedKind: SourceKind,
    definitionName expectedDefinition: String,
    decoder: JSONDecoder = JSONDecoder(),
    validate: (Value) throws -> Void = { _ in }
) throws -> [LedgeDatasetInput<Value>]

public func evaluationInfo() throws -> [String: String]
public func validateResults(in report: LedgeEvaluationFile) throws

decodeInputs checks the expected source definition, decodes every input, and runs your validator before returning any values. It rejects empty datasets. Validation errors identify the item without including its input; cancellation propagates.

evaluationInfo() returns a ledgeDataset JSON string for Apple's native report metadata. validateResults checks that report cases contain exactly the dataset item IDs with no duplicates or omissions. It does not judge model quality or replace checks for native execution failures.

Input and provenance types

The source identifies the original captured input, independently of a new trace created when evaluating it. Preserve the dataset item's id as the native evaluation sample ID.

LedgeDatasetError

Cases: downloadRejected(Int), invalidResponse, tooLarge, incompatibleDefinition, emptyDataset, invalidInput(UUID), and incompleteResults. The enum conforms to Error, LocalizedError, Equatable, and Sendable.

See loading datasets and dataset evaluations.