Inline Synthetic CDC

An inline Synthetic CDC operation can be added to a Data Pipeline directly, as part of the data transformation handling logic. However, unlike the DataZen-Managed CDC, this operation uses a customer-supplied database table. This allows administrators to monitor which records are changed, and optionally force changes by modifying the CDC tracking table directly. Because this operation is performed inline, as part of the Data Pipeline, it can be performed at any time against the source and/or target system. When applied against the source Data Pipeline, the Change Log only contains the records that were modified. When applied at the target Data Pipeline, you can choose to have different Change Capture logic based on the target system.

To add the Inline CDC operation, select your Data Pipeline, and choose Add -> Change Capture -> Apply Synthetic Change Capture.

To configure this inline operation, select a database connection, a table name (it will be created automatically), and the CDC Key Columns. Check the Add CDC Tracking Column to include a new column called __cdc__ containing an 'I' for inserted or a 'U' for updated records. This can be useful for downstream processing when target systems require different operations based on Inserted or Updated records.

You can apply a inline CDC operation at the source and/or the target. To apply to the target, select the the Target Pipeline tab under your Target System.

Performing an inline CDC operation may change the state of your CDC table unless you check the "PREVIEW CHANGES" option.

The inline CDC operation does not support the identification of deleted records.

At this time, only a SQL Server database is supported to store the CDC table.