Incremental Processing Using Netflix Maestro and Apache Iceberg
Incremental processing reduces cost and execution time of workflows by only processing new or changed data. Netflix Maestro & Apache Iceberg enable data accuracy, freshness & backfill. ICDC table allows MERGE incremental change data into target table, reducing transformation jobs with range parameters in the business logic to optimize ETL workflows. Highlighted efficiency simplifies user workflows when dealing with late arriving data.
Read More