Merge Data with ORC Data
Merge data with your Apache ORC data quickly
Merge Data
Merging stacks your Apache ORC file together with one or more datasets, appending their records underneath so everything lands in one table. The result includes every column found across all datasets, and you choose which column from each dataset fills each one. Perfect for combining exports from different periods, regions, or systems into a single dataset.
Apache ORC
Apache ORC (Optimized Row Columnar) is a self-describing, columnar file format that supports high compression ratios and fast data retrieval. ORC supports complex types, including structs, lists, maps, and unions. ORC files are divided into blocks of data (stripes) containing statistics (such as min, max, sum, and count) and lightweight indexing which can be used to skip over irrelevant data during queries. ORC also supports predicate pushdown, meaning that filters can be applied as the data is read from disk, reducing the amount of data loaded into memory and processed. Due to its high performance in terms of compression and speed of access, ORC is particularly well-suited for heavy read operations and is commonly used in data warehousing and analytics applications.