¶ Description
Compute the checksum of the content of several columns.
¶ Parameters
¶ Standard tab
Parameters:
- Unique mode
- Mode
- Column name with result
- Compute checksum on
- Select columns
¶ Advanced tab
Parameters:
- Ignore error if a selected column is missing
- UTF16 Code point of Separator char for mode
¶ About
The ActionCRC action computes a row-level checksum (CRC) across selected columns in a dataset using hashing or concatenation methods. It appends a new column containing the computed value for each row, which is useful for identifying duplicates, verifying data integrity, or creating unique keys in ETL processes.
This action is especially helpful when tracking changes, generating fingerprints of records, or detecting modified rows in incremental data pipelines.
¶ Parameters
-
Unique mode
Enables row-level CRC calculation. Each row gets its own unique checksum.
Default: Enabled -
Mode
Algorithm used to compute the checksum. Available options:MD5 (v2)SHA256concatenationconcatenation (compressed)
-
Column name with result
Specifies the name of the output column where the checksum will be stored.
Default:crc -
Compute checksum on
Determines which columns to use in checksum calculation:these selected columns: only selected columns will be includedall columns minus these selected columns: all columns except the ones selected
-
Select columns
Selects columns to include or exclude from the CRC calculation, depending on the mode above.
¶ Example 1: MD5 on selected columns
¶ Input
| ID | Name | Age | Country |
|---|---|---|---|
| 1 | Alice | 30 | USA |
| 2 | Bob | 25 | Canada |
| 3 | Charlie | 35 | Australia |
¶ Configuration
- Unique mode: Enabled
- Mode:
MD5 (v2) - Column name with result:
crc - Compute checksum on:
these selected columns - Selected columns:
Name,Age
¶ Output
¶ Example 2: SHA256 on all columns minus ID
¶ Input
| ID | Product | Price | Quantity |
|---|---|---|---|
| 101 | Laptop | 1200 | 5 |
| 102 | Keyboard | 50 | 10 |
| 103 | Mouse | 25 | 15 |
¶ Configuration
- Unique mode: Enabled
- Mode:
SHA256 - Column name with result:
crc - Compute checksum on:
all columns minus these selected columns - Selected columns:
ID
¶ Output
¶ When to Use
- Change detection: Identify modified rows between two datasets by comparing CRC values.
- Deduplication: Generate hashes to detect and eliminate duplicate records.
- Auditing: Track row-level changes without storing full record history.
- Data integrity: Confirm that row content has not changed between stages of processing.
Notes
- When using
"these selected columns"mode, you must select at least one column — otherwise, the action will throw an error:
No columns to compute the CRC.- Output CRC values vary depending on the hash algorithm selected.
