Solutions
Silent Upstream Failure Detection
Detect when upstream connectors return 0 rows and dbt runs successfully on empty data.
What is a silent upstream failure?
A silent upstream failure occurs when an upstream data connector — Fivetran, Airbyte, a custom ETL script, or any data source — successfully completes its sync but delivers 0 rows to the staging table. dbt then runs its transformation pipeline on empty input and produces empty downstream models. The Airflow DAG shows green. The dbt run shows success. But every downstream table, dashboard, and metric is empty or wrong.
Why silent upstream failures are the hardest to diagnose
Unlike a model compilation error or a schema drift, a silent upstream failure produces no error logs. The pipeline ran correctly — on bad input. Data engineers typically discover the issue hours later when a business stakeholder reports that dashboard metrics are all zero, or when a data quality check fires on an empty table. By then, tracing the issue back to an upstream 0-row sync requires manually checking connector logs, comparing row counts, and correlating timestamps — a 1–3 hour investigation.
Common causes of silent upstream failures
- Source API rate limiting that caused the connector to return empty results instead of retrying
- Source schema change that caused a connector to extract 0 matching rows
- Upstream service outage where the API returned HTTP 200 with an empty payload
- Connector authentication expiry causing a silent empty sync
- Incremental sync window misconfiguration fetching a time range with no new data
How Ordo detects silent upstream failures
Ordo collects upstream row counts for source tables and compares them against the 7-day rolling average baseline. When row counts drop to zero or fall more than 95% below baseline, Ordo flags the upstream connector as the root cause before the dbt run completes — or immediately after if the empty run already succeeded.
Silent upstream failure detected (97% confidence)
Source table: raw.salesforce_opportunities
Row count at 02:14 UTC: 0 rows (7-day avg: 1,240,000 rows)
dbt run: succeeded on empty input
Root cause: Upstream Fivetran connector returned empty payload
Fix: Check Fivetran connector logs for sync errors at 02:10 UTC
Affected models: fct_pipeline, rpt_revenue, dim_accounts (+11 downstream)
Why Ordo catches what dbt tests miss
dbt source freshness and not_null tests only fire if you've written them — and they only check data after the run. Ordo checks upstream row counts before and during the run, regardless of whether you have tests configured. It acts as a pre-run validator that catches empty-input conditions before they propagate downstream.