7,305 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:07,349 INFO hadoop.InternalParquetRecordReader: block read in memory in 44 ms. row count = 715
2025-02-13 06:34:07,351 INFO parquet.MapredParquetOutputFormat: creating new record writer...org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat@39f55d48
2025-02-13 06:34:07,352 INFO write.ParquetRecordWriterWrapper: initialize serde with table properties.
2025-02-13 06:34:07,352 INFO write.ParquetRecordWriterWrapper: creating real writer to write at hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/_temporary/attempt_202502130626211125544307711486521_0000_m_000314_314/part-00314-787063eb-4883-4300-ba10-614b8cda885e-c000
2025-02-13 06:34:07,352 INFO codec.CodecConfig: Compression: SNAPPY
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Parquet block size to 134217728
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Parquet page size to 1048576
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Parquet dictionary page size to 1048576
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Dictionary is on
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Validation is off
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Writer version is: PARQUET_1_0
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Page size checking is: estimated
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Min row count for page size check is: 100
2025-02-13 06:34:07,352 INFO hadoop.ParquetOutputFormat: Max row count for page size check is: 10000
2025-02-13 06:34:07,634 INFO write.ParquetRecordWriterWrapper: real writer: org.apache.parquet.hadoop.ParquetRecordWriter@2b9303bd
2025-02-13 06:34:07,670 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45717, range: 0-215391, partition values: [2025-01-18,07]
2025-02-13 06:34:07,909 INFO hadoop.InternalParquetRecordReader: block read in memory in 4410 ms. row count = 2467
2025-02-13 06:34:07,931 INFO hadoop.InternalParquetRecordReader: block read in memory in 4085 ms. row count = 5733
2025-02-13 06:34:08,082 INFO hadoop.InternalParquetRecordReader: block read in memory in 4594 ms. row count = 1781
2025-02-13 06:34:08,108 INFO hadoop.InternalParquetRecordReader: block read in memory in 4596 ms. row count = 1911
2025-02-13 06:34:08,141 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47613, range: 0-605319, partition values: [2025-01-18,12]
2025-02-13 06:34:08,214 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44980, range: 0-429442, partition values: [2025-01-18,05]
2025-02-13 06:34:08,258 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47281, range: 0-464202, partition values: [2025-01-18,12]
2025-02-13 06:34:08,382 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 32020417
2025-02-13 06:34:10,093 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1861 records.
2025-02-13 06:34:10,096 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:10,242 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2445 records.
2025-02-13 06:34:10,245 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:10,276 INFO hadoop.InternalParquetRecordReader: block read in memory in 180 ms. row count = 1861
2025-02-13 06:34:10,319 INFO hadoop.InternalParquetRecordReader: block read in memory in 73 ms. row count = 2445
2025-02-13 06:34:10,465 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47309, range: 0-464161, partition values: [2025-01-18,12]
2025-02-13 06:34:10,534 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=04/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44731, range: 0-604896, partition values: [2025-01-18,04]
2025-02-13 06:34:10,847 INFO hadoop.InternalParquetRecordReader: block read in memory in 6721 ms. row count = 2613
2025-02-13 06:34:10,939 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2445 records.
2025-02-13 06:34:10,941 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:11,068 INFO hadoop.InternalParquetRecordReader: block read in memory in 127 ms. row count = 2445
2025-02-13 06:34:11,132 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47474, range: 0-648746, partition values: [2025-01-18,12]
2025-02-13 06:34:11,301 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=17/part-f9905227-f900-4842-8731-2d79a4a9626d-0-49441, range: 0-604855, partition values: [2025-01-18,17]
2025-02-13 06:34:12,314 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2688 records.
2025-02-13 06:34:12,317 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:12,699 INFO hadoop.InternalParquetRecordReader: block read in memory in 382 ms. row count = 2688
2025-02-13 06:34:12,897 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44755, range: 0-604521, partition values: [2025-01-18,05]
2025-02-13 06:34:14,814 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2148 records.
2025-02-13 06:34:14,816 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:17,809 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2155 records.
2025-02-13 06:34:17,811 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:20,840 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626216263086832560879297_0000_m_000173_173' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626216263086832560879297_0000_m_000173
2025-02-13 06:34:20,840 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626216263086832560879297_0000_m_000173_173: Committed
2025-02-13 06:34:20,843 INFO executor.Executor: Finished task 173.0 in stage 0.0 (TID 173). 1909 bytes result sent to driver
2025-02-13 06:34:20,882 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 320
2025-02-13 06:34:20,883 INFO executor.Executor: Running task 320.0 in stage 0.0 (TID 320)
2025-02-13 06:34:20,933 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1
2025-02-13 06:34:20,933 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2025-02-13 06:34:20,933 INFO datasources.SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2025-02-13 06:34:20,933 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46332, range: 0-199311, partition values: [2025-01-18,09]
2025-02-13 06:34:21,371 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 790 records.
2025-02-13 06:34:21,373 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:21,439 INFO hadoop.InternalParquetRecordReader: block read in memory in 66 ms. row count = 790
2025-02-13 06:34:21,441 INFO parquet.MapredParquetOutputFormat: creating new record writer...org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat@16324e45
2025-02-13 06:34:21,442 INFO write.ParquetRecordWriterWrapper: initialize serde with table properties.
2025-02-13 06:34:21,442 INFO write.ParquetRecordWriterWrapper: creating real writer to write at hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/_temporary/attempt_202502130626215321616387315054969_0000_m_000320_320/part-00320-787063eb-4883-4300-ba10-614b8cda885e-c000
2025-02-13 06:34:21,442 INFO codec.CodecConfig: Compression: SNAPPY
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Parquet block size to 134217728
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Parquet page size to 1048576
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Parquet dictionary page size to 1048576
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Dictionary is on
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Validation is off
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Writer version is: PARQUET_1_0
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Page size checking is: estimated
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Min row count for page size check is: 100
2025-02-13 06:34:21,442 INFO hadoop.ParquetOutputFormat: Max row count for page size check is: 10000
2025-02-13 06:34:21,508 INFO write.ParquetRecordWriterWrapper: real writer: org.apache.parquet.hadoop.ParquetRecordWriter@47edaf54
2025-02-13 06:34:21,549 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45716, range: 0-199234, partition values: [2025-01-18,07]
2025-02-13 06:34:37,113 INFO hadoop.InternalParquetRecordReader: block read in memory in 22297 ms. row count = 2148
2025-02-13 06:34:37,274 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47594, range: 0-582045, partition values: [2025-01-18,12]
2025-02-13 06:34:38,408 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2398 records.
2025-02-13 06:34:38,411 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:38,461 INFO hadoop.InternalParquetRecordReader: block read in memory in 50 ms. row count = 2398
2025-02-13 06:34:38,629 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47443, range: 0-581982, partition values: [2025-01-18,12]
2025-02-13 06:34:40,369 INFO hadoop.InternalParquetRecordReader: block read in memory in 22558 ms. row count = 2155
2025-02-13 06:34:40,558 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47416, range: 0-571630, partition values: [2025-01-18,12]
2025-02-13 06:34:40,694 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2461 records.
2025-02-13 06:34:40,697 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:40,803 INFO hadoop.InternalParquetRecordReader: block read in memory in 105 ms. row count = 2461
2025-02-13 06:34:40,997 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47585, range: 0-581936, partition values: [2025-01-18,12]
2025-02-13 06:34:41,193 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2445 records.
2025-02-13 06:34:41,198 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:41,210 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2374 records.
2025-02-13 06:34:41,213 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:41,256 INFO hadoop.InternalParquetRecordReader: block read in memory in 43 ms. row count = 2374
2025-02-13 06:34:41,446 INFO hadoop.InternalParquetRecordReader: block read in memory in 248 ms. row count = 2445
2025-02-13 06:34:41,451 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44844, range: 0-581748, partition values: [2025-01-18,05]
2025-02-13 06:34:41,601 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=04/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44634, range: 0-571527, partition values: [2025-01-18,04]
2025-02-13 06:34:55,576 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 786 records.
2025-02-13 06:34:55,579 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:56,235 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2852 records.
2025-02-13 06:34:56,240 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:56,256 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1678 records.
2025-02-13 06:34:56,260 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:56,268 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1825 records.
2025-02-13 06:34:56,270 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:56,307 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2447 records.
2025-02-13 06:34:56,309 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:58,328 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 666 records.
2025-02-13 06:34:58,332 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:58,876 INFO hadoop.InternalParquetRecordReader: block read in memory in 3297 ms. row count = 786
2025-02-13 06:34:58,951 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46958, range: 0-215294, partition values: [2025-01-18,11]
2025-02-13 06:34:59,561 INFO hadoop.InternalParquetRecordReader: block read in memory in 3321 ms. row count = 2852
2025-02-13 06:34:59,583 INFO hadoop.InternalParquetRecordReader: block read in memory in 3323 ms. row count = 1678
2025-02-13 06:34:59,596 INFO hadoop.InternalParquetRecordReader: block read in memory in 3326 ms. row count = 1825
2025-02-13 06:34:59,609 INFO hadoop.InternalParquetRecordReader: block read in memory in 3300 ms. row count = 2447
2025-02-13 06:34:59,636 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 935 records.
2025-02-13 06:34:59,638 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:34:59,712 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44973, range: 0-429394, partition values: [2025-01-18,05]
2025-02-13 06:34:59,726 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=23/part-f9905227-f900-4842-8731-2d79a4a9626d-0-51317, range: 0-464113, partition values: [2025-01-18,23]
2025-02-13 06:34:59,784 INFO hadoop.InternalParquetRecordReader: block read in memory in 146 ms. row count = 935
2025-02-13 06:34:59,799 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=04/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44711, range: 0-648561, partition values: [2025-01-18,04]
2025-02-13 06:34:59,836 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=04/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44422, range: 0-604396, partition values: [2025-01-18,04]
2025-02-13 06:34:59,848 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45395, range: 0-215208, partition values: [2025-01-18,06]
2025-02-13 06:35:00,036 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2569 records.
2025-02-13 06:35:00,038 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:00,128 INFO hadoop.InternalParquetRecordReader: block read in memory in 90 ms. row count = 2569
2025-02-13 06:35:00,246 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 697 records.
2025-02-13 06:35:00,249 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:00,289 INFO hadoop.InternalParquetRecordReader: block read in memory in 40 ms. row count = 697
2025-02-13 06:35:00,335 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47809, range: 0-648191, partition values: [2025-01-18,13]
2025-02-13 06:35:00,347 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45685, range: 0-215179, partition values: [2025-01-18,07]
2025-02-13 06:35:00,499 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2329 records.
2025-02-13 06:35:00,501 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:00,528 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2215 records.
2025-02-13 06:35:00,530 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:00,566 INFO hadoop.InternalParquetRecordReader: block read in memory in 2234 ms. row count = 666
2025-02-13 06:35:00,610 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46452, range: 0-199047, partition values: [2025-01-18,09]
2025-02-13 06:35:00,782 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1665 records.
2025-02-13 06:35:00,785 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:01,067 INFO hadoop.InternalParquetRecordReader: block read in memory in 282 ms. row count = 1665
2025-02-13 06:35:01,197 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47279, range: 0-429257, partition values: [2025-01-18,12]
2025-02-13 06:35:01,208 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2611 records.
2025-02-13 06:35:01,210 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:01,278 INFO hadoop.InternalParquetRecordReader: block read in memory in 68 ms. row count = 2611
2025-02-13 06:35:01,490 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44799, range: 0-647352, partition values: [2025-01-18,05]
2025-02-13 06:35:01,982 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 758 records.
2025-02-13 06:35:01,985 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:02,369 INFO hadoop.InternalParquetRecordReader: block read in memory in 384 ms. row count = 758
2025-02-13 06:35:02,435 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46374, range: 0-198948, partition values: [2025-01-18,09]
2025-02-13 06:35:03,455 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 736 records.
2025-02-13 06:35:03,458 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:03,690 INFO hadoop.InternalParquetRecordReader: block read in memory in 3189 ms. row count = 2329
2025-02-13 06:35:03,715 INFO hadoop.InternalParquetRecordReader: block read in memory in 256 ms. row count = 736
2025-02-13 06:35:03,735 INFO hadoop.InternalParquetRecordReader: block read in memory in 3205 ms. row count = 2215
2025-02-13 06:35:03,774 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46447, range: 0-198869, partition values: [2025-01-18,09]
2025-02-13 06:35:03,958 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47334, range: 0-581417, partition values: [2025-01-18,12]
2025-02-13 06:35:04,006 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47537, range: 0-571496, partition values: [2025-01-18,12]
2025-02-13 06:35:04,498 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 751 records.
2025-02-13 06:35:04,501 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:04,840 INFO hadoop.InternalParquetRecordReader: block read in memory in 339 ms. row count = 751
2025-02-13 06:35:04,879 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46493, range: 0-198863, partition values: [2025-01-18,09]
2025-02-13 06:35:05,896 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2386 records.
2025-02-13 06:35:05,898 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:06,148 INFO hadoop.InternalParquetRecordReader: block read in memory in 249 ms. row count = 2386
2025-02-13 06:35:06,284 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47733, range: 0-570943, partition values: [2025-01-18,13]
2025-02-13 06:35:06,614 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2283 records.
2025-02-13 06:35:06,616 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:06,794 INFO hadoop.InternalParquetRecordReader: block read in memory in 178 ms. row count = 2283
2025-02-13 06:35:06,960 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44798, range: 0-570494, partition values: [2025-01-18,05]
2025-02-13 06:35:08,582 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2270 records.
2025-02-13 06:35:08,586 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:08,947 INFO hadoop.InternalParquetRecordReader: block read in memory in 361 ms. row count = 2270
2025-02-13 06:35:09,111 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47560, range: 0-570089, partition values: [2025-01-18,12]
2025-02-13 06:35:09,814 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2344 records.
2025-02-13 06:35:09,816 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:09,948 INFO hadoop.InternalParquetRecordReader: block read in memory in 132 ms. row count = 2344
2025-02-13 06:35:10,163 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44778, range: 0-569564, partition values: [2025-01-18,05]
2025-02-13 06:35:10,685 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2224 records.
2025-02-13 06:35:10,688 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:10,805 INFO hadoop.InternalParquetRecordReader: block read in memory in 117 ms. row count = 2224
2025-02-13 06:35:10,951 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44927, range: 0-569162, partition values: [2025-01-18,05]
2025-02-13 06:35:14,407 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2445 records.
2025-02-13 06:35:14,413 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:14,590 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1777 records.
2025-02-13 06:35:14,592 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:15,417 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 771 records.
2025-02-13 06:35:15,426 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:16,111 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1656 records.
2025-02-13 06:35:16,118 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:16,400 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2716 records.
2025-02-13 06:35:16,404 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:17,109 INFO hadoop.InternalParquetRecordReader: block read in memory in 2696 ms. row count = 2445
2025-02-13 06:35:17,341 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=04/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44649, range: 0-604040, partition values: [2025-01-18,04]
2025-02-13 06:35:17,369 INFO hadoop.InternalParquetRecordReader: block read in memory in 2777 ms. row count = 1777
2025-02-13 06:35:17,535 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47267, range: 0-463309, partition values: [2025-01-18,11]
2025-02-13 06:35:18,051 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2442 records.
2025-02-13 06:35:18,054 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:18,122 INFO hadoop.InternalParquetRecordReader: block read in memory in 68 ms. row count = 2442
2025-02-13 06:35:18,259 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1860 records.
2025-02-13 06:35:18,261 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:18,316 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47542, range: 0-604032, partition values: [2025-01-18,12]
2025-02-13 06:35:18,336 INFO hadoop.InternalParquetRecordReader: block read in memory in 75 ms. row count = 1860
2025-02-13 06:35:18,526 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47264, range: 0-462876, partition values: [2025-01-18,11]
2025-02-13 06:35:18,607 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2475 records.
2025-02-13 06:35:18,610 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:18,639 INFO hadoop.InternalParquetRecordReader: block read in memory in 3213 ms. row count = 771
2025-02-13 06:35:18,678 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 757 records.
2025-02-13 06:35:18,682 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:18,707 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45493, range: 0-215022, partition values: [2025-01-18,07]
2025-02-13 06:35:18,959 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1944 records.
2025-02-13 06:35:18,962 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:18,995 INFO hadoop.InternalParquetRecordReader: block read in memory in 33 ms. row count = 1944
2025-02-13 06:35:19,092 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 774 records.
2025-02-13 06:35:19,095 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:19,174 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47284, range: 0-462084, partition values: [2025-01-18,12]
2025-02-13 06:35:19,226 INFO hadoop.InternalParquetRecordReader: block read in memory in 131 ms. row count = 774
2025-02-13 06:35:19,287 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46520, range: 0-215022, partition values: [2025-01-18,09]
2025-02-13 06:35:19,707 INFO hadoop.InternalParquetRecordReader: block read in memory in 3589 ms. row count = 1656
2025-02-13 06:35:19,791 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1853 records.
2025-02-13 06:35:19,794 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:19,801 INFO hadoop.InternalParquetRecordReader: block read in memory in 3397 ms. row count = 2716
2025-02-13 06:35:19,835 INFO hadoop.InternalParquetRecordReader: block read in memory in 41 ms. row count = 1853
2025-02-13 06:35:19,854 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47314, range: 0-429172, partition values: [2025-01-18,12]
2025-02-13 06:35:20,015 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47395, range: 0-461743, partition values: [2025-01-18,12]
2025-02-13 06:35:20,100 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47671, range: 0-646892, partition values: [2025-01-18,13]
2025-02-13 06:35:21,427 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1874 records.
2025-02-13 06:35:21,430 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:21,723 INFO hadoop.InternalParquetRecordReader: block read in memory in 3113 ms. row count = 2475
2025-02-13 06:35:21,769 INFO hadoop.InternalParquetRecordReader: block read in memory in 3087 ms. row count = 757
2025-02-13 06:35:21,824 INFO hadoop.InternalParquetRecordReader: block read in memory in 394 ms. row count = 1874
2025-02-13 06:35:21,826 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46519, range: 0-198836, partition values: [2025-01-18,09]
2025-02-13 06:35:21,905 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47566, range: 0-581268, partition values: [2025-01-18,12]
2025-02-13 06:35:21,977 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47240, range: 0-461735, partition values: [2025-01-18,11]
2025-02-13 06:35:22,472 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2437 records.
2025-02-13 06:35:22,475 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:22,767 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2371 records.
2025-02-13 06:35:22,770 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:22,814 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 753 records.
2025-02-13 06:35:22,816 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:22,865 INFO hadoop.InternalParquetRecordReader: block read in memory in 49 ms. row count = 753
2025-02-13 06:35:22,911 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45459, range: 0-198796, partition values: [2025-01-18,06]
2025-02-13 06:35:22,958 INFO hadoop.InternalParquetRecordReader: block read in memory in 188 ms. row count = 2371
2025-02-13 06:35:23,121 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47690, range: 0-580926, partition values: [2025-01-18,13]
2025-02-13 06:35:23,250 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 699 records.
2025-02-13 06:35:23,252 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:23,297 INFO hadoop.InternalParquetRecordReader: block read in memory in 45 ms. row count = 699
2025-02-13 06:35:23,342 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45668, range: 0-198678, partition values: [2025-01-18,07]
2025-02-13 06:35:24,639 INFO hadoop.InternalParquetRecordReader: block read in memory in 2164 ms. row count = 2437
2025-02-13 06:35:24,864 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 19940993
2025-02-13 06:35:28,480 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626217464476036957821274_0000_m_000248_248' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626217464476036957821274_0000_m_000248
2025-02-13 06:35:28,480 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626217464476036957821274_0000_m_000248_248: Committed
2025-02-13 06:35:28,482 INFO executor.Executor: Finished task 248.0 in stage 0.0 (TID 248). 1909 bytes result sent to driver
2025-02-13 06:35:29,229 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2562 records.
2025-02-13 06:35:29,232 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:29,240 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 777 records.
2025-02-13 06:35:29,242 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:30,217 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2655 records.
2025-02-13 06:35:30,219 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:30,508 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1651 records.
2025-02-13 06:35:30,510 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:31,588 INFO hadoop.InternalParquetRecordReader: block read in memory in 2346 ms. row count = 777
2025-02-13 06:35:31,605 INFO hadoop.InternalParquetRecordReader: block read in memory in 2373 ms. row count = 2562
2025-02-13 06:35:31,640 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46464, range: 0-215011, partition values: [2025-01-18,09]
2025-02-13 06:35:31,756 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47667, range: 0-603310, partition values: [2025-01-18,13]
2025-02-13 06:35:31,915 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 817 records.
2025-02-13 06:35:31,959 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:31,999 INFO hadoop.InternalParquetRecordReader: block read in memory in 40 ms. row count = 817
2025-02-13 06:35:32,046 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=10/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46665, range: 0-214983, partition values: [2025-01-18,10]
2025-02-13 06:35:32,322 INFO hadoop.InternalParquetRecordReader: block read in memory in 2103 ms. row count = 2655
2025-02-13 06:35:32,497 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47825, range: 0-646740, partition values: [2025-01-18,13]
2025-02-13 06:35:32,654 INFO hadoop.InternalParquetRecordReader: block read in memory in 2144 ms. row count = 1651
2025-02-13 06:35:32,760 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47162, range: 0-428686, partition values: [2025-01-18,11]
2025-02-13 06:35:32,835 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1945 records.
2025-02-13 06:35:32,837 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:33,137 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1732 records.
2025-02-13 06:35:33,139 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:33,201 INFO hadoop.InternalParquetRecordReader: block read in memory in 62 ms. row count = 1732
2025-02-13 06:35:33,309 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47330, range: 0-428672, partition values: [2025-01-18,12]
2025-02-13 06:35:33,642 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2366 records.
2025-02-13 06:35:33,644 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:33,676 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1658 records.
2025-02-13 06:35:33,678 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:33,703 INFO hadoop.InternalParquetRecordReader: block read in memory in 25 ms. row count = 1658
2025-02-13 06:35:33,782 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 707 records.
2025-02-13 06:35:33,785 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:33,808 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47198, range: 0-428352, partition values: [2025-01-18,11]
2025-02-13 06:35:34,778 INFO hadoop.InternalParquetRecordReader: block read in memory in 1941 ms. row count = 1945
2025-02-13 06:35:34,944 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44943, range: 0-461560, partition values: [2025-01-18,05]
2025-02-13 06:35:35,254 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1774 records.
2025-02-13 06:35:35,256 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:35,342 INFO hadoop.InternalParquetRecordReader: block read in memory in 86 ms. row count = 1774
2025-02-13 06:35:35,458 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44975, range: 0-460816, partition values: [2025-01-18,05]
2025-02-13 06:35:35,732 INFO hadoop.InternalParquetRecordReader: block read in memory in 2088 ms. row count = 2366
2025-02-13 06:35:35,801 INFO hadoop.InternalParquetRecordReader: block read in memory in 2016 ms. row count = 707
2025-02-13 06:35:35,801 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1824 records.
2025-02-13 06:35:35,803 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:35,851 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46285, range: 0-198499, partition values: [2025-01-18,09]
2025-02-13 06:35:35,939 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47502, range: 0-580888, partition values: [2025-01-18,12]
2025-02-13 06:35:35,973 INFO hadoop.InternalParquetRecordReader: block read in memory in 170 ms. row count = 1824
2025-02-13 06:35:36,118 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47133, range: 0-460224, partition values: [2025-01-18,11]
2025-02-13 06:35:39,187 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2496 records.
2025-02-13 06:35:39,191 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:39,277 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 849 records.
2025-02-13 06:35:39,323 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:39,958 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2732 records.
2025-02-13 06:35:39,961 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:40,982 INFO hadoop.InternalParquetRecordReader: block read in memory in 1659 ms. row count = 849
2025-02-13 06:35:40,987 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1726 records.
2025-02-13 06:35:40,989 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:41,042 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45527, range: 0-214936, partition values: [2025-01-18,07]
2025-02-13 06:35:41,045 INFO hadoop.InternalParquetRecordReader: block read in memory in 1854 ms. row count = 2496
2025-02-13 06:35:41,252 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47568, range: 0-602973, partition values: [2025-01-18,12]
2025-02-13 06:35:41,277 INFO hadoop.InternalParquetRecordReader: block read in memory in 1316 ms. row count = 2732
2025-02-13 06:35:41,520 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47650, range: 0-646340, partition values: [2025-01-18,13]
2025-02-13 06:35:41,660 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2559 records.
2025-02-13 06:35:41,662 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:41,973 INFO hadoop.InternalParquetRecordReader: block read in memory in 311 ms. row count = 2559
2025-02-13 06:35:42,156 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2411 records.
2025-02-13 06:35:42,159 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:42,169 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47750, range: 0-602673, partition values: [2025-01-18,13]
2025-02-13 06:35:42,193 INFO hadoop.InternalParquetRecordReader: block read in memory in 1204 ms. row count = 1726
2025-02-13 06:35:42,313 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47142, range: 0-427791, partition values: [2025-01-18,11]
2025-02-13 06:35:42,316 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 765 records.
2025-02-13 06:35:42,319 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:42,888 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1973 records.
2025-02-13 06:35:42,892 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:43,440 INFO hadoop.InternalParquetRecordReader: block read in memory in 1281 ms. row count = 2411
2025-02-13 06:35:43,710 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44848, range: 0-580554, partition values: [2025-01-18,05]
2025-02-13 06:35:43,913 INFO hadoop.InternalParquetRecordReader: block read in memory in 1594 ms. row count = 765
2025-02-13 06:35:43,973 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46443, range: 0-198458, partition values: [2025-01-18,09]
2025-02-13 06:35:44,589 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2374 records.
2025-02-13 06:35:44,592 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:44,662 INFO hadoop.InternalParquetRecordReader: block read in memory in 70 ms. row count = 2374
2025-02-13 06:35:44,822 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 19816478
2025-02-13 06:35:45,363 INFO hadoop.InternalParquetRecordReader: block read in memory in 2471 ms. row count = 1973
2025-02-13 06:35:45,527 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47202, range: 0-460130, partition values: [2025-01-18,11]
2025-02-13 06:35:47,913 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626216298576230617110178_0000_m_000247_247' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626216298576230617110178_0000_m_000247
2025-02-13 06:35:47,913 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626216298576230617110178_0000_m_000247_247: Committed
2025-02-13 06:35:47,917 INFO executor.Executor: Finished task 247.0 in stage 0.0 (TID 247). 1909 bytes result sent to driver
2025-02-13 06:35:48,081 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 741 records.
2025-02-13 06:35:48,083 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:48,445 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2732 records.
2025-02-13 06:35:48,449 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:49,096 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2396 records.
2025-02-13 06:35:49,100 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:49,503 INFO hadoop.InternalParquetRecordReader: block read in memory in 1420 ms. row count = 741
2025-02-13 06:35:49,543 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1831 records.
2025-02-13 06:35:49,545 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:49,556 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46484, range: 0-214768, partition values: [2025-01-18,09]
2025-02-13 06:35:49,824 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 822 records.
2025-02-13 06:35:49,826 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:49,887 INFO hadoop.InternalParquetRecordReader: block read in memory in 61 ms. row count = 822
2025-02-13 06:35:49,933 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46257, range: 0-214732, partition values: [2025-01-18,09]
2025-02-13 06:35:50,404 INFO hadoop.InternalParquetRecordReader: block read in memory in 1955 ms. row count = 2732
2025-02-13 06:35:50,445 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 833 records.
2025-02-13 06:35:50,447 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:50,588 INFO hadoop.InternalParquetRecordReader: block read in memory in 141 ms. row count = 833
2025-02-13 06:35:50,600 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=02/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44001, range: 0-646261, partition values: [2025-01-18,02]
2025-02-13 06:35:50,636 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45112, range: 0-214729, partition values: [2025-01-18,06]
2025-02-13 06:35:51,020 INFO hadoop.InternalParquetRecordReader: block read in memory in 1920 ms. row count = 2396
2025-02-13 06:35:51,266 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=04/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44740, range: 0-602542, partition values: [2025-01-18,04]
2025-02-13 06:35:51,295 INFO hadoop.InternalParquetRecordReader: block read in memory in 1750 ms. row count = 1831
2025-02-13 06:35:51,420 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47088, range: 0-427537, partition values: [2025-01-18,11]
2025-02-13 06:35:51,475 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 729 records.
2025-02-13 06:35:51,478 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:51,518 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2322 records.
2025-02-13 06:35:51,521 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:51,653 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 772 records.
2025-02-13 06:35:51,656 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:51,748 INFO hadoop.InternalParquetRecordReader: block read in memory in 227 ms. row count = 2322
2025-02-13 06:35:51,789 INFO hadoop.InternalParquetRecordReader: block read in memory in 133 ms. row count = 772
2025-02-13 06:35:51,854 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45454, range: 0-214491, partition values: [2025-01-18,06]
2025-02-13 06:35:51,911 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2489 records.
2025-02-13 06:35:51,914 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:51,942 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47583, range: 0-646224, partition values: [2025-01-18,12]
2025-02-13 06:35:51,988 INFO hadoop.InternalParquetRecordReader: block read in memory in 74 ms. row count = 2489
2025-02-13 06:35:52,107 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1994 records.
2025-02-13 06:35:52,110 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:52,195 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47741, range: 0-602527, partition values: [2025-01-18,13]
2025-02-13 06:35:52,696 INFO hadoop.InternalParquetRecordReader: block read in memory in 1218 ms. row count = 729
2025-02-13 06:35:52,743 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2442 records.
2025-02-13 06:35:52,746 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:52,770 INFO hadoop.InternalParquetRecordReader: block read in memory in 24 ms. row count = 2442
2025-02-13 06:35:52,770 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45696, range: 0-198354, partition values: [2025-01-18,07]
2025-02-13 06:35:52,990 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47533, range: 0-601809, partition values: [2025-01-18,12]
2025-02-13 06:35:53,032 INFO hadoop.InternalParquetRecordReader: block read in memory in 922 ms. row count = 1994
2025-02-13 06:35:53,211 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=00/part-f9905227-f900-4842-8731-2d79a4a9626d-0-43176, range: 0-459989, partition values: [2025-01-18,00]
2025-02-13 06:35:53,296 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2476 records.
2025-02-13 06:35:53,299 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:53,366 INFO hadoop.InternalParquetRecordReader: block read in memory in 67 ms. row count = 2476
2025-02-13 06:35:53,536 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47685, range: 0-601345, partition values: [2025-01-18,13]
2025-02-13 06:35:53,901 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2394 records.
2025-02-13 06:35:53,904 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:54,135 INFO hadoop.InternalParquetRecordReader: block read in memory in 231 ms. row count = 2394
2025-02-13 06:35:54,336 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47749, range: 0-601058, partition values: [2025-01-18,13]
2025-02-13 06:35:54,677 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2347 records.
2025-02-13 06:35:54,679 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:54,745 INFO hadoop.InternalParquetRecordReader: block read in memory in 66 ms. row count = 2347
2025-02-13 06:35:54,899 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44768, range: 0-601009, partition values: [2025-01-18,05]
2025-02-13 06:35:55,085 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2019 records.
2025-02-13 06:35:55,087 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:55,145 INFO hadoop.InternalParquetRecordReader: block read in memory in 58 ms. row count = 2019
2025-02-13 06:35:55,272 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47388, range: 0-459925, partition values: [2025-01-18,12]
2025-02-13 06:35:55,541 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1801 records.
2025-02-13 06:35:55,545 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:55,596 INFO hadoop.InternalParquetRecordReader: block read in memory in 51 ms. row count = 1801
2025-02-13 06:35:55,706 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47018, range: 0-459771, partition values: [2025-01-18,11]
2025-02-13 06:35:55,864 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1760 records.
2025-02-13 06:35:55,866 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:56,371 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2795 records.
2025-02-13 06:35:56,373 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:56,464 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 814 records.
2025-02-13 06:35:56,466 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:58,563 INFO hadoop.InternalParquetRecordReader: block read in memory in 2697 ms. row count = 1760
2025-02-13 06:35:58,712 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=20/part-f9905227-f900-4842-8731-2d79a4a9626d-0-50320, range: 0-427418, partition values: [2025-01-18,20]
2025-02-13 06:35:58,866 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 659 records.
2025-02-13 06:35:58,870 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:35:59,411 INFO hadoop.InternalParquetRecordReader: block read in memory in 2945 ms. row count = 814
2025-02-13 06:35:59,433 INFO hadoop.InternalParquetRecordReader: block read in memory in 3059 ms. row count = 2795
2025-02-13 06:35:59,526 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45444, range: 0-214478, partition values: [2025-01-18,06]
2025-02-13 06:35:59,664 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47666, range: 0-646220, partition values: [2025-01-18,13]
2025-02-13 06:36:01,174 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2746 records.
2025-02-13 06:36:01,176 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:01,243 INFO hadoop.InternalParquetRecordReader: block read in memory in 67 ms. row count = 2746
2025-02-13 06:36:01,453 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47649, range: 0-646215, partition values: [2025-01-18,13]
2025-02-13 06:36:01,681 INFO hadoop.InternalParquetRecordReader: block read in memory in 2811 ms. row count = 659
2025-02-13 06:36:01,748 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46204, range: 0-198317, partition values: [2025-01-18,09]
2025-02-13 06:36:01,976 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 751 records.
2025-02-13 06:36:01,983 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:02,022 INFO hadoop.InternalParquetRecordReader: block read in memory in 39 ms. row count = 751
2025-02-13 06:36:02,066 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45669, range: 0-198292, partition values: [2025-01-18,07]
2025-02-13 06:36:02,427 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 725 records.
2025-02-13 06:36:02,429 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:02,453 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2700 records.
2025-02-13 06:36:02,455 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:02,506 INFO hadoop.InternalParquetRecordReader: block read in memory in 51 ms. row count = 2700
2025-02-13 06:36:02,568 INFO hadoop.InternalParquetRecordReader: block read in memory in 139 ms. row count = 725
2025-02-13 06:36:02,610 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46460, range: 0-198288, partition values: [2025-01-18,09]
2025-02-13 06:36:02,678 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47620, range: 0-645278, partition values: [2025-01-18,12]
2025-02-13 06:36:03,618 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2366 records.
2025-02-13 06:36:03,622 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:03,632 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2007 records.
2025-02-13 06:36:03,635 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:03,724 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2678 records.
2025-02-13 06:36:03,726 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:04,198 INFO hadoop.InternalParquetRecordReader: block read in memory in 472 ms. row count = 2678
2025-02-13 06:36:04,400 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44843, range: 0-645043, partition values: [2025-01-18,05]
2025-02-13 06:36:04,930 INFO hadoop.InternalParquetRecordReader: block read in memory in 1308 ms. row count = 2366
2025-02-13 06:36:05,107 INFO hadoop.InternalParquetRecordReader: block read in memory in 1472 ms. row count = 2007
2025-02-13 06:36:05,176 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2760 records.
2025-02-13 06:36:05,177 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47669, range: 0-600850, partition values: [2025-01-18,13]
2025-02-13 06:36:05,179 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:05,333 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45083, range: 0-459305, partition values: [2025-01-18,05]
2025-02-13 06:36:05,357 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 756 records.
2025-02-13 06:36:05,360 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:05,371 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1754 records.
2025-02-13 06:36:05,375 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:05,382 INFO hadoop.InternalParquetRecordReader: block read in memory in 203 ms. row count = 2760
2025-02-13 06:36:05,585 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47830, range: 0-644637, partition values: [2025-01-18,13]
2025-02-13 06:36:06,174 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2622 records.
2025-02-13 06:36:06,176 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:06,397 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2434 records.
2025-02-13 06:36:06,402 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:06,428 INFO hadoop.InternalParquetRecordReader: block read in memory in 252 ms. row count = 2622
2025-02-13 06:36:06,507 INFO hadoop.InternalParquetRecordReader: block read in memory in 105 ms. row count = 2434
2025-02-13 06:36:06,548 INFO hadoop.InternalParquetRecordReader: block read in memory in 1188 ms. row count = 756
2025-02-13 06:36:06,596 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47934, range: 0-644593, partition values: [2025-01-18,13]
2025-02-13 06:36:06,602 INFO hadoop.InternalParquetRecordReader: block read in memory in 1227 ms. row count = 1754
2025-02-13 06:36:06,612 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45525, range: 0-214129, partition values: [2025-01-18,07]
2025-02-13 06:36:06,721 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47139, range: 0-427149, partition values: [2025-01-18,11]
2025-02-13 06:36:06,726 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-44860, range: 0-600481, partition values: [2025-01-18,05]
2025-02-13 06:36:06,908 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 754 records.
2025-02-13 06:36:06,910 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:06,996 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 740 records.
2025-02-13 06:36:06,999 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:07,050 INFO hadoop.InternalParquetRecordReader: block read in memory in 140 ms. row count = 754
2025-02-13 06:36:07,153 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45683, range: 0-213857, partition values: [2025-01-18,07]
2025-02-13 06:36:07,181 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1777 records.
2025-02-13 06:36:07,184 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:07,422 INFO hadoop.InternalParquetRecordReader: block read in memory in 238 ms. row count = 1777
2025-02-13 06:36:07,615 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47125, range: 0-427095, partition values: [2025-01-18,11]
2025-02-13 06:36:07,812 INFO hadoop.InternalParquetRecordReader: block read in memory in 813 ms. row count = 740
2025-02-13 06:36:07,869 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46345, range: 0-198286, partition values: [2025-01-18,09]
2025-02-13 06:36:08,014 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 770 records.
2025-02-13 06:36:08,019 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:08,235 INFO hadoop.InternalParquetRecordReader: block read in memory in 216 ms. row count = 770
2025-02-13 06:36:08,355 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46477, range: 0-213725, partition values: [2025-01-18,09]
2025-02-13 06:36:08,365 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 752 records.
2025-02-13 06:36:08,369 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:08,499 INFO hadoop.InternalParquetRecordReader: block read in memory in 130 ms. row count = 752
2025-02-13 06:36:08,565 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45538, range: 0-198234, partition values: [2025-01-18,07]
2025-02-13 06:36:08,900 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 695 records.
2025-02-13 06:36:08,903 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:09,089 INFO hadoop.InternalParquetRecordReader: block read in memory in 186 ms. row count = 695
2025-02-13 06:36:09,136 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=10/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46638, range: 0-198069, partition values: [2025-01-18,10]
2025-02-13 06:36:09,316 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1925 records.
2025-02-13 06:36:09,320 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:10,116 INFO hadoop.InternalParquetRecordReader: block read in memory in 796 ms. row count = 1925
2025-02-13 06:36:10,299 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47008, range: 0-459263, partition values: [2025-01-18,11]
2025-02-13 06:36:10,875 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1870 records.
2025-02-13 06:36:10,877 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:10,896 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2460 records.
2025-02-13 06:36:10,900 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:11,019 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 2540 records.
2025-02-13 06:36:11,021 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:11,124 INFO hadoop.InternalParquetRecordReader: block read in memory in 247 ms. row count = 1870
2025-02-13 06:36:11,246 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47116, range: 0-459247, partition values: [2025-01-18,11]
2025-02-13 06:36:11,980 INFO hadoop.InternalParquetRecordReader: block read in memory in 1080 ms. row count = 2460
2025-02-13 06:36:12,162 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 21953243
2025-02-13 06:36:12,218 INFO hadoop.InternalParquetRecordReader: block read in memory in 1197 ms. row count = 2540
2025-02-13 06:36:12,363 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 20203603
2025-02-13 06:36:12,577 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1807 records.
2025-02-13 06:36:12,581 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:14,120 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 812 records.
2025-02-13 06:36:14,122 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:14,134 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 677 records.
2025-02-13 06:36:14,138 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:14,170 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626216348931178202641401_0000_m_000244_244' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626216348931178202641401_0000_m_000244
2025-02-13 06:36:14,170 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626216348931178202641401_0000_m_000244_244: Committed
2025-02-13 06:36:14,173 INFO executor.Executor: Finished task 244.0 in stage 0.0 (TID 244). 1909 bytes result sent to driver
2025-02-13 06:36:14,369 INFO hadoop.InternalParquetRecordReader: block read in memory in 1788 ms. row count = 1807
2025-02-13 06:36:14,532 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47195, range: 0-427077, partition values: [2025-01-18,11]
2025-02-13 06:36:14,612 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626218038776247761369969_0000_m_000237_237' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626218038776247761369969_0000_m_000237
2025-02-13 06:36:14,612 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626218038776247761369969_0000_m_000237_237: Committed
2025-02-13 06:36:14,614 INFO executor.Executor: Finished task 237.0 in stage 0.0 (TID 237). 1909 bytes result sent to driver
2025-02-13 06:36:15,023 INFO hadoop.InternalParquetRecordReader: block read in memory in 901 ms. row count = 812
2025-02-13 06:36:15,029 INFO hadoop.InternalParquetRecordReader: block read in memory in 891 ms. row count = 677
2025-02-13 06:36:15,095 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46227, range: 0-198046, partition values: [2025-01-18,09]
2025-02-13 06:36:15,105 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=13/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47637, range: 0-213690, partition values: [2025-01-18,13]
2025-02-13 06:36:15,109 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1976 records.
2025-02-13 06:36:15,111 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:15,462 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1702 records.
2025-02-13 06:36:15,464 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:15,628 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 863 records.
2025-02-13 06:36:15,630 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:15,696 INFO hadoop.InternalParquetRecordReader: block read in memory in 66 ms. row count = 863
2025-02-13 06:36:15,748 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45458, range: 0-213617, partition values: [2025-01-18,06]
2025-02-13 06:36:15,980 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 717 records.
2025-02-13 06:36:15,982 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:16,111 INFO hadoop.InternalParquetRecordReader: block read in memory in 1000 ms. row count = 1976
2025-02-13 06:36:16,121 INFO hadoop.InternalParquetRecordReader: block read in memory in 139 ms. row count = 717
2025-02-13 06:36:16,167 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46371, range: 0-213488, partition values: [2025-01-18,09]
2025-02-13 06:36:16,223 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 18347840
2025-02-13 06:36:16,562 INFO hadoop.InternalParquetRecordReader: block read in memory in 1098 ms. row count = 1702
2025-02-13 06:36:16,665 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47223, range: 0-426730, partition values: [2025-01-18,11]
2025-02-13 06:36:16,709 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 885 records.
2025-02-13 06:36:16,711 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:16,778 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 755 records.
2025-02-13 06:36:16,780 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:16,786 INFO hadoop.InternalParquetRecordReader: block read in memory in 75 ms. row count = 885
2025-02-13 06:36:16,835 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46262, range: 0-213465, partition values: [2025-01-18,09]
2025-02-13 06:36:16,854 INFO hadoop.InternalParquetRecordReader: block read in memory in 74 ms. row count = 755
2025-02-13 06:36:16,895 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=14/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47999, range: 0-197988, partition values: [2025-01-18,14]
2025-02-13 06:36:17,478 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 729 records.
2025-02-13 06:36:17,480 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:17,508 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1695 records.
2025-02-13 06:36:17,510 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:17,561 INFO hadoop.InternalParquetRecordReader: block read in memory in 51 ms. row count = 1695
2025-02-13 06:36:17,612 INFO hadoop.InternalParquetRecordReader: block read in memory in 132 ms. row count = 729
2025-02-13 06:36:17,653 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46524, range: 0-197904, partition values: [2025-01-18,09]
2025-02-13 06:36:17,679 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=12/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47287, range: 0-426482, partition values: [2025-01-18,12]
2025-02-13 06:36:17,870 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626218709479335253863463_0000_m_000263_263' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626218709479335253863463_0000_m_000263
2025-02-13 06:36:17,870 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626218709479335253863463_0000_m_000263_263: Committed
2025-02-13 06:36:17,872 INFO executor.Executor: Finished task 263.0 in stage 0.0 (TID 263). 1909 bytes result sent to driver
2025-02-13 06:36:18,331 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 842 records.
2025-02-13 06:36:18,332 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:18,697 INFO hadoop.InternalParquetRecordReader: block read in memory in 365 ms. row count = 842
2025-02-13 06:36:18,747 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46364, range: 0-213464, partition values: [2025-01-18,09]
2025-02-13 06:36:19,249 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 888 records.
2025-02-13 06:36:19,251 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:19,283 INFO hadoop.InternalParquetRecordReader: block read in memory in 32 ms. row count = 888
2025-02-13 06:36:19,291 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 721 records.
2025-02-13 06:36:19,293 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:19,343 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1680 records.
2025-02-13 06:36:19,345 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:19,377 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46381, range: 0-213324, partition values: [2025-01-18,09]
2025-02-13 06:36:19,639 INFO hadoop.InternalParquetRecordReader: block read in memory in 346 ms. row count = 721
2025-02-13 06:36:19,679 INFO hadoop.InternalParquetRecordReader: block read in memory in 334 ms. row count = 1680
2025-02-13 06:36:19,683 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46463, range: 0-197822, partition values: [2025-01-18,09]
2025-02-13 06:36:19,781 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47269, range: 0-425924, partition values: [2025-01-18,11]
2025-02-13 06:36:20,283 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1641 records.
2025-02-13 06:36:20,285 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:20,427 INFO hadoop.InternalParquetRecordReader: block read in memory in 142 ms. row count = 1641
2025-02-13 06:36:20,561 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46990, range: 0-425552, partition values: [2025-01-18,11]
2025-02-13 06:36:21,134 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 869 records.
2025-02-13 06:36:21,138 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:21,449 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 752 records.
2025-02-13 06:36:21,454 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:21,824 INFO hadoop.InternalParquetRecordReader: block read in memory in 686 ms. row count = 869
2025-02-13 06:36:21,929 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45442, range: 0-213160, partition values: [2025-01-18,06]
2025-02-13 06:36:22,412 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1715 records.
2025-02-13 06:36:22,415 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:22,770 INFO hadoop.InternalParquetRecordReader: block read in memory in 1316 ms. row count = 752
2025-02-13 06:36:22,834 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=10/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46678, range: 0-197743, partition values: [2025-01-18,10]
2025-02-13 06:36:22,856 INFO hadoop.InternalParquetRecordReader: block read in memory in 441 ms. row count = 1715
2025-02-13 06:36:22,993 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47121, range: 0-425549, partition values: [2025-01-18,11]
2025-02-13 06:36:23,218 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1652 records.
2025-02-13 06:36:23,221 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:23,340 INFO hadoop.InternalParquetRecordReader: block read in memory in 119 ms. row count = 1652
2025-02-13 06:36:23,366 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 742 records.
2025-02-13 06:36:23,369 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:23,465 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45003, range: 0-425470, partition values: [2025-01-18,05]
2025-02-13 06:36:23,554 INFO hadoop.InternalParquetRecordReader: block read in memory in 185 ms. row count = 742
2025-02-13 06:36:23,616 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46284, range: 0-213112, partition values: [2025-01-18,09]
2025-02-13 06:36:23,805 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 856 records.
2025-02-13 06:36:23,808 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:23,828 INFO hadoop.InternalParquetRecordReader: block read in memory in 20 ms. row count = 856
2025-02-13 06:36:23,890 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46266, range: 0-213040, partition values: [2025-01-18,09]
2025-02-13 06:36:25,674 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 730 records.
2025-02-13 06:36:25,679 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:26,337 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1647 records.
2025-02-13 06:36:26,340 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:26,343 INFO hadoop.InternalParquetRecordReader: block read in memory in 664 ms. row count = 730
2025-02-13 06:36:26,349 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 834 records.
2025-02-13 06:36:26,351 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:26,396 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45585, range: 0-197687, partition values: [2025-01-18,07]
2025-02-13 06:36:26,801 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 682 records.
2025-02-13 06:36:26,804 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:26,843 INFO hadoop.InternalParquetRecordReader: block read in memory in 39 ms. row count = 682
2025-02-13 06:36:26,895 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46473, range: 0-197493, partition values: [2025-01-18,09]
2025-02-13 06:36:26,954 INFO hadoop.InternalParquetRecordReader: block read in memory in 614 ms. row count = 1647
2025-02-13 06:36:26,968 INFO hadoop.InternalParquetRecordReader: block read in memory in 617 ms. row count = 834
2025-02-13 06:36:27,043 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=08/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46184, range: 0-212962, partition values: [2025-01-18,08]
2025-02-13 06:36:27,132 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=20/part-f9905227-f900-4842-8731-2d79a4a9626d-0-50436, range: 0-425461, partition values: [2025-01-18,20]
2025-02-13 06:36:27,279 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 733 records.
2025-02-13 06:36:27,281 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:27,337 INFO hadoop.InternalParquetRecordReader: block read in memory in 56 ms. row count = 733
2025-02-13 06:36:27,401 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46421, range: 0-197489, partition values: [2025-01-18,09]
2025-02-13 06:36:28,126 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 766 records.
2025-02-13 06:36:28,130 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:28,305 INFO hadoop.InternalParquetRecordReader: block read in memory in 175 ms. row count = 766
2025-02-13 06:36:28,374 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46254, range: 0-197425, partition values: [2025-01-18,09]
2025-02-13 06:36:28,911 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 786 records.
2025-02-13 06:36:28,914 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:29,044 INFO hadoop.InternalParquetRecordReader: block read in memory in 130 ms. row count = 786
2025-02-13 06:36:29,102 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45536, range: 0-197266, partition values: [2025-01-18,07]
2025-02-13 06:36:29,931 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 882 records.
2025-02-13 06:36:29,936 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:30,237 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1608 records.
2025-02-13 06:36:30,241 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:30,597 INFO hadoop.InternalParquetRecordReader: block read in memory in 661 ms. row count = 882
2025-02-13 06:36:30,690 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=06/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45456, range: 0-212540, partition values: [2025-01-18,06]
2025-02-13 06:36:31,055 INFO hadoop.InternalParquetRecordReader: block read in memory in 814 ms. row count = 1608
2025-02-13 06:36:31,205 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47236, range: 0-425458, partition values: [2025-01-18,11]
2025-02-13 06:36:31,567 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1700 records.
2025-02-13 06:36:31,570 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:31,669 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 690 records.
2025-02-13 06:36:31,672 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:31,737 INFO hadoop.InternalParquetRecordReader: block read in memory in 167 ms. row count = 1700
2025-02-13 06:36:31,848 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=05/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45087, range: 0-425358, partition values: [2025-01-18,05]
2025-02-13 06:36:32,103 INFO hadoop.InternalParquetRecordReader: block read in memory in 431 ms. row count = 690
2025-02-13 06:36:32,151 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=07/part-f9905227-f900-4842-8731-2d79a4a9626d-0-45559, range: 0-197147, partition values: [2025-01-18,07]
2025-02-13 06:36:32,404 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 768 records.
2025-02-13 06:36:32,406 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:32,902 INFO hadoop.InternalParquetRecordReader: block read in memory in 496 ms. row count = 768
2025-02-13 06:36:32,969 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=08/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46185, range: 0-212459, partition values: [2025-01-18,08]
2025-02-13 06:36:33,720 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1712 records.
2025-02-13 06:36:33,723 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:34,076 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 728 records.
2025-02-13 06:36:34,078 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:34,195 INFO hadoop.InternalParquetRecordReader: block read in memory in 472 ms. row count = 1712
2025-02-13 06:36:34,308 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=11/part-f9905227-f900-4842-8731-2d79a4a9626d-0-47187, range: 0-425352, partition values: [2025-01-18,11]
2025-02-13 06:36:34,495 INFO hadoop.InternalParquetRecordReader: block read in memory in 417 ms. row count = 728
2025-02-13 06:36:34,540 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46486, range: 0-197101, partition values: [2025-01-18,09]
2025-02-13 06:36:34,797 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 725 records.
2025-02-13 06:36:34,799 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:34,825 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 833 records.
2025-02-13 06:36:34,828 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:34,843 INFO hadoop.InternalParquetRecordReader: block read in memory in 44 ms. row count = 725
2025-02-13 06:36:34,896 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46363, range: 0-197051, partition values: [2025-01-18,09]
2025-02-13 06:36:35,215 INFO hadoop.InternalParquetRecordReader: block read in memory in 387 ms. row count = 833
2025-02-13 06:36:35,294 INFO datasources.FileScanRDD: Reading File path: hdfs://namenodes-cluster/user/hive/warehouse/record.db/app_play/dt=2025-01-18/hr=09/part-f9905227-f900-4842-8731-2d79a4a9626d-0-46339, range: 0-212380, partition values: [2025-01-18,09]
2025-02-13 06:36:35,402 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 813 records.
2025-02-13 06:36:35,404 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:35,447 INFO hadoop.InternalParquetRecordReader: block read in memory in 43 ms. row count = 813
2025-02-13 06:36:35,499 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 8434416
2025-02-13 06:36:35,635 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 795 records.
2025-02-13 06:36:35,638 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:35,769 INFO hadoop.InternalParquetRecordReader: block read in memory in 131 ms. row count = 795
2025-02-13 06:36:35,866 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 9180108
2025-02-13 06:36:36,281 INFO hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 1675 records.
2025-02-13 06:36:36,285 INFO hadoop.InternalParquetRecordReader: at row 0. reading next block
2025-02-13 06:36:36,664 INFO hadoop.InternalParquetRecordReader: block read in memory in 379 ms. row count = 1675
2025-02-13 06:36:36,811 INFO hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 16837622
2025-02-13 06:36:37,045 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626215321616387315054969_0000_m_000320_320' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626215321616387315054969_0000_m_000320
2025-02-13 06:36:37,045 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626215321616387315054969_0000_m_000320_320: Committed
2025-02-13 06:36:37,047 INFO executor.Executor: Finished task 320.0 in stage 0.0 (TID 320). 1909 bytes result sent to driver
2025-02-13 06:36:37,187 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626211125544307711486521_0000_m_000314_314' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626211125544307711486521_0000_m_000314
2025-02-13 06:36:37,187 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626211125544307711486521_0000_m_000314_314: Committed
2025-02-13 06:36:37,188 INFO executor.Executor: Finished task 314.0 in stage 0.0 (TID 314). 1909 bytes result sent to driver
2025-02-13 06:36:38,011 INFO output.FileOutputCommitter: Saved output of task 'attempt_202502130626211414847864263294059_0000_m_000268_268' to hdfs://namenodes-cluster/home/hadoop/hive_backup/record/app_play/app_play_2025-01-18/.hive-staging_hive_2025-02-13_06-26-19_441_4357489605114561573-1/-ext-10000/_temporary/0/task_202502130626211414847864263294059_0000_m_000268
2025-02-13 06:36:38,012 INFO mapred.SparkHadoopMapRedUtil: attempt_202502130626211414847864263294059_0000_m_000268_268: Committed
2025-02-13 06:36:38,013 INFO executor.Executor: Finished task 268.0 in stage 0.0 (TID 268). 1909 bytes result sent to driver
2025-02-13 06:42:35,344 INFO executor.CoarseGrainedExecutorBackend: Driver commanded a shutdown
2025-02-13 06:42:35,387 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
tdown