Skip to main content

DARPA Common Data Model (CDM)

The Common Data Model (CDM) is a standardized format developed for the DARPA Transparent Computing (TC) program. It is designed to represent causal system events in a consistent way across different operating systems and tracking tools, enabling the construction of end-to-end provenance graphs.

📊 Format Specifications​

  • Serialization: Apache Avro (Binary).
  • Goal: To provide a platform-independent schema for serializing system-level interactions.
  • Versions: Primarily CDM 18 (Engagement 3) and CDM 20 (Engagement 5).

🏗️ Core Entities​

The CDM schema organizes system information into six primary categories:

1. Hosts​

Represents the physical or virtual machines where the data collection occurs.

2. Principals​

Represents local users or accounts that own objects or initiate process subjects.

3. Subjects​

Represents execution contexts, primarily processes and threads. This is the "active" part of the system that initiates actions.

4. Objects​

Represents system resources that subjects interact with. Common objects include:

  • Files: Regular files, directories, pipes.
  • Network Sockets: IP addresses, ports, and protocols.
  • Memory Regions: Shared memory or mapped regions.
  • Registry Keys: Specific to Windows environments.

5. Events​

The "edges" in the provenance graph. Events represent actions taken by a Subject on an Object or another Subject.

  • Examples: read, write, execve, fork, connect, open.
  • Metadata: Includes timestamps, sequence numbers, and success/failure status.

6. ProvenanceTags​

Security or integrity metadata associated with entities to track information flow (e.g., tagging data as Benign, Untrusted, Secret, or Public).

🛠️ Data Consumption​

Because the data is stored in Avro binary format, it must be parsed using the schema (.avsc or .avdl). DARPA provides a Java Consumer tool in each engagement release to:

  1. Parse the binary stream.
  2. Filter or transform records.
  3. Convert records into human-readable JSON format.
info

The CDM ensures that regardless of whether the data comes from a FreeBSD host (CADETS) or a Linux host (THEIA), the representation of a "process writing to a file" remains identical in the resulting graph.