¶ Description
Sort the input table, partition-by-partition.
¶ Parameters
¶ Standard tab
Parameters:
- Partitioning variable
¶ Advanced tab
Parameters:
- Data block size
- Abort when encountering a character in a numeric sort
¶ Explanation
A partition is a set of contiguous rows within the input table. The end of a partition (i.e., the last row of a partition) is marked by a change in the value of the partitioning variable. In other words, the partitioning variable remains constant within a single partition.
For example:
The algorithm is the following:
- ETL loads into an in-memory RAM buffer a whole partition.
- ETL sorts the in-memory RAM buffer.
- ETL outputs each row of the (sorted) buffer.
- ETL clears the buffer and, if there are some more input rows (i.e. some more partitions), it goes back to step 1.
The size of the in-memory RAM buffer automatically increases to be able to store a whole partition. The size increment is present in parameters editor.
The memory consumption of the PartitionedSort action is equal to the amount of RAM memory required to store the largest partition.
In the general case, the output table does not contain any sort-meta-data (to indicate that the output table is, in the general case, not sorted), unless we are in the following "special" case: If the most significant sort-variable of the input table is equal to the partitioning-variable, then the sort-meta-data of the output table is set to the following:
- The most significant sort-variable of the output table is equal to the partitioning-variable.
- The other less-significant sort-variables are set accordingly to the sorting parameters of the Partitioned Sort Action.
The PartitionedSort action can run inside a N-Way Multithread Section only if the partitioning variable of the N-Way Multithread Section is equal to the partitioning variable of the [PartitionedSort] action. When executing a PartitionedSort action using N CPUs, the RAM memory consumption is multiplied by N.
