Troubleshooting: Salesforce “Data Cloud” ingestion failures (Metadata Mismatch)

A Metadata Mismatch error in Salesforce Data Cloud occurs when the structure, formatting, or data types of an incoming data payload do not align with the target Data Lake Object (DLO) schema. When ingesting records through the Ingestion API, Amazon S3, Google Cloud Storage, or external webhooks, Data Cloud validates every column header, delimiter, and field property before processing. If a single column contains an unexpected string format, an incompatible timestamp, or an unmapped header, the platform halts the stream or routes the rejected batch to the error partition.

Fast Fix

Open Data Cloud > Data Streams, select the failing stream, and inspect the Data Stream History error details. Check whether your source payload contains unescaped header spaces, missing primary key fields, or invalid DateTime strings. Format all dates to strict ISO 8601 standards (YYYY-MM-DD'T'HH:mm:ss.SSS'Z') and verify that column headers match the DLO schema casing exactly.

---

Diagnostic Snapshot

• Error Signatures: Metadata mismatch detected, Schema validation failed for Data Lake Object, Failed to parse date/time value, Record does not contain defined primary key.
• Affected Ingestion Channels: Ingestion API (REST/Bulk), Amazon S3 / Snowflake connectors, Webhooks, CRM Data Streams.
• Core Mechanism: Data Lake Object (DLO) schema validation during the transformation and staging phase.
• Permissions Needed: Data Cloud Admin or Data Cloud Data Aware Specialist permissions to modify Data Streams, schemas, and field mappings.

---

How Schema Validation Works in Data Cloud

When external data enters Data Cloud, it does not write directly into standard database tables. Instead, it follows a strict two-tier modeling pipeline:

1. Source Data -> Data Lake Object (DLO): The ingestion engine receives the raw data and stages it in the DLO according to a fixed schema defined when the stream was created.
2. Data Lake Object (DLO) -> Data Model Object (DMO): Data Cloud maps fields from the DLO to canonical CIM (Customer Information Model) entities like Individual, Contact Point Email, or custom DMOs.

A Metadata Mismatch happens at Step 1. The ingestion layer applies rigid type enforcement:

• Column names are case-sensitive. A source column named customer_id will fail if the DLO expects Customer_Id.
• Data Cloud does not perform automatic type coercion. If a DLO schema expects a Number or Date and the incoming JSON or CSV sends an empty string ("") rather than a valid numeric string or null, the parser fails the record.
• Once created, a DLO’s primary key and core field types cannot be modified in place.

---

Diagnostic Path

Data Cloud Ingestion Failure
 ├── Check Data Stream History (Data Cloud Setup)
 │    ├── "Missing primary key" ────────► Payload lacks the defined record identifier
 │    ├── "Timestamp / Date format" ────► Convert dates to ISO 8601 (UTC)
 │    └── "Column count mismatch" ──────► Unescaped commas/newlines in source CSV
 └── Was the source schema changed externally?
      ├── New columns added to source ──► Refresh Data Stream schema
      └── Field type changed at source ─► Create a new Data Stream and remap DLO

---

Step-by-Step Resolution

1. Audit Header Names and Case Sensitivity

If ingesting data via the Ingestion API or cloud storage buckets:

1. Compare the first row of your CSV file or the keys in your JSON object directly against the DLO field list.
2. Check for hidden whitespace. A header like "Email " (with a trailing space) will not match "Email".
3. Verify letter casing. Data Cloud treats AccountId, accountid, and ACCOUNTID as three distinct properties.
4. Remove special characters from column headers. Only alphanumeric characters and underscores are supported for field API names.

2. Standardize Date and DateTime Formats

Date formatting is the most frequent cause of parsing failures in Data Cloud.

• Data Cloud requires strict ISO 8601 date-time representations in UTC.
• Correct Format: 2026-03-31T14:30:00.000Z or 2026-03-31 (for Date-only fields).
• Common Failing Formats: 03/31/2026, 31-03-2026 02:30 PM, or Unix epoch timestamps in milliseconds (unless explicitly mapped as a numeric field).
• If sending data from external data warehouses (e.g., Snowflake, BigQuery), cast timestamps using TO_VARCHAR(my_timestamp, 'YYYY-MM-DD"T"HH24:MI:SS.FF3"Z"') before pushing to the ingestion bucket.

3. Handle Null Values vs. Empty Strings

When delivering CSV files to S3 or Google Cloud Storage for Data Cloud ingestion:

• A blank string between delimiters (,,) is often parsed as an empty string (""). If the target field type is Date or Number, this generates a metadata parse error.
• Ensure your extraction tools output genuine null values or use explicit NULL representations configured in your Data Stream’s source settings.
• For boolean fields, ensure the values evaluate strictly to true or false (case-insensitive); integers like 1 or 0 will fail if the target DLO specifies a Boolean data type.

4. Resolve Schema Drift (Handling Modified Source Data)

If the upstream team added, removed, or renamed columns in the source data source, the ingestion engine detects schema drift and halts.

• Adding New Columns: Go to Data Cloud > Data Streams > open your stream > click Refresh Schema. Once refreshed, add the newly discovered fields to the DLO and map them to your target DMO.
• Changed Data Types: If an upstream field changed from a Text field to a Number, Data Cloud will not allow you to change the existing DLO data type. You must deactivate the stream, create a new stream with the updated schema, and retire the legacy stream.

5. Verify Ingestion API Schema Manifests

When sending data through the Ingestion API using OpenAPI schema definitions:

• Ensure the request header Content-Type is set to application/json.
• Confirm that the object name referenced in the URL endpoint (/api/v1/ingest/sources/<source_name>/<object_name>) matches the exact name provided in the uploaded OpenAPI YAML/JSON schema file. If the connector returns authentication faults alongside ingestion issues, check How to Resolve: Salesforce “Connected App” failing OIDC Handshakes.

---

Common Misdiagnoses

• DLO Ingestion vs. DMO Mapping Failures: Ingestion errors occur when data fails to land in the Data Lake Object. If data lands in the DLO successfully but does not appear in calculated insights or identity resolution rules, the problem is an unmapped or misconfigured Data Model Object (DMO) relationship, not an ingestion metadata mismatch.
• Salesforce CRM Connector Permissions: If using the native Salesforce CRM connector and records fail to ingest, the issue is often missing Object or Field-Level Security on the Data Cloud Salesforce Connector Integration User. Confirm permissions using Fix: Salesforce “You do not have the required permissions” (Identity Loop).

---

Hard Stops

• Do not delete an active Data Stream without evaluating downstream dependencies. Deleting a stream breaks all downstream data transformations, calculated insights, and segments built on the associated DLO. Always pause the stream and remove references in data model mappings before attempting to rebuild a schema from scratch.