09/24 21:18:20 INFO TorrentBroadcast: Started reading broadcast variable 33 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_33_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO TorrentBroadcast: Reading broadcast variable 33 took 5 ms
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_33 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:20 INFO Executor: Finished task 2.0 in stage 31.0 (TID 54). 2989 bytes result sent to driver
23/09/24 21:18:20 INFO CoarseGrainedExecutorBackend: Got assigned task 59
23/09/24 21:18:20 INFO Executor: Running task 2.0 in stage 34.0 (TID 59)
23/09/24 21:18:20 INFO MapOutputTrackerWorker: Updating epoch to 12 and clearing cache
23/09/24 21:18:20 INFO TorrentBroadcast: Started reading broadcast variable 35 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_35_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO TorrentBroadcast: Reading broadcast variable 35 took 5 ms
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_35 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:20 INFO Executor: Finished task 2.0 in stage 34.0 (TID 59). 2989 bytes result sent to driver
23/09/24 21:18:20 INFO CoarseGrainedExecutorBackend: Got assigned task 63
23/09/24 21:18:20 INFO Executor: Running task 2.0 in stage 37.0 (TID 63)
23/09/24 21:18:20 INFO MapOutputTrackerWorker: Updating epoch to 13 and clearing cache
23/09/24 21:18:20 INFO TorrentBroadcast: Started reading broadcast variable 37 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_37_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO TorrentBroadcast: Reading broadcast variable 37 took 6 ms
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_37 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:20 INFO Executor: Finished task 2.0 in stage 37.0 (TID 63). 2989 bytes result sent to driver
23/09/24 21:18:20 INFO CoarseGrainedExecutorBackend: Got assigned task 66
23/09/24 21:18:20 INFO Executor: Running task 0.0 in stage 38.0 (TID 66)
23/09/24 21:18:20 INFO MapOutputTrackerWorker: Updating epoch to 14 and clearing cache
23/09/24 21:18:20 INFO TorrentBroadcast: Started reading broadcast variable 38 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_38_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO TorrentBroadcast: Reading broadcast variable 38 took 5 ms
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_38 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 13, fetching them
23/09/24 21:18:20 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:20 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:20 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:20 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:20 INFO Executor: Finished task 0.0 in stage 38.0 (TID 66). 2450 bytes result sent to driver
23/09/24 21:18:20 INFO CoarseGrainedExecutorBackend: Got assigned task 69
23/09/24 21:18:20 INFO Executor: Running task 2.0 in stage 40.0 (TID 69)
23/09/24 21:18:20 INFO TorrentBroadcast: Started reading broadcast variable 39 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_39_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO TorrentBroadcast: Reading broadcast variable 39 took 5 ms
23/09/24 21:18:20 INFO MemoryStore: Block broadcast_39 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:20 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:20 INFO Executor: Finished task 2.0 in stage 40.0 (TID 69). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 73
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 43.0 (TID 73)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 15 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 41 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_41_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 41 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_41 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 43.0 (TID 73). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 79
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 46.0 (TID 79)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 16 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 43 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_43_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 43 took 7 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_43 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 46.0 (TID 79). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 85
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 51.0 (TID 85)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 17 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 46 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_46_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 46 took 6 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_46 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 51.0 (TID 85). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 87
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 52.0 (TID 87)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 18 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 47 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_47_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 47 took 5 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_47 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 17, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 52.0 (TID 87). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 89
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 54.0 (TID 89)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 48 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_48_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 48 took 6 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_48 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 54.0 (TID 89). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 95
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 57.0 (TID 95)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 19 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 50 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_50_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 50 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_50 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 57.0 (TID 95). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 99
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 60.0 (TID 99)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 20 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 52 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_52_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 52 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_52 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 60.0 (TID 99). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 102
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 61.0 (TID 102)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 21 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 53 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_53_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 53 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_53 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 20, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 61.0 (TID 102). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 105
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 63.0 (TID 105)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 54 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_54_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 54 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_54 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 63.0 (TID 105). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 107
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 64.0 (TID 107)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 22 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 55 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_55_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 55 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_55 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 21, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 64.0 (TID 107). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 108
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 66.0 (TID 108)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 56 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_56_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 56 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_56 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 66.0 (TID 108). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 115
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 69.0 (TID 115)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 23 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 58 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_58_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 58 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_58 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 69.0 (TID 115). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 117
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 70.0 (TID 117)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 24 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 59 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_59_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 59 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_59 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 23, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 70.0 (TID 117). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 118
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 72.0 (TID 118)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 60 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_60_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 60 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_60 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 72.0 (TID 118). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 122
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 73.0 (TID 122)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 25 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 61 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_61_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 61 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_61 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 24, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 73.0 (TID 122). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 123
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 75.0 (TID 123)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 62 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_62_piece0 stored as bytes in memory (estimated size 9.7 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 62 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_62 stored as values in memory (estimated size 24.5 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 75.0 (TID 123). 3715 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 125
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 77.0 (TID 125)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 63 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_63_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 63 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_63 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 77.0 (TID 125). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 128
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 78.0 (TID 128)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 26 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 64 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_64_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 64 took 2 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_64 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 25, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 78.0 (TID 128). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 130
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 80.0 (TID 130)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 65 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_65_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 65 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_65 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 80.0 (TID 130). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 133
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 81.0 (TID 133)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 27 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 66 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_66_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 66 took 3 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_66 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 26, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 81.0 (TID 133). 2450 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 134
23/09/24 21:18:21 INFO Executor: Running task 2.0 in stage 83.0 (TID 134)
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 67 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_67_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 67 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_67 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:21 INFO Executor: Finished task 2.0 in stage 83.0 (TID 134). 2990 bytes result sent to driver
23/09/24 21:18:21 INFO CoarseGrainedExecutorBackend: Got assigned task 138
23/09/24 21:18:21 INFO Executor: Running task 0.0 in stage 84.0 (TID 138)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Updating epoch to 28 and clearing cache
23/09/24 21:18:21 INFO TorrentBroadcast: Started reading broadcast variable 68 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_68_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO TorrentBroadcast: Reading broadcast variable 68 took 4 ms
23/09/24 21:18:21 INFO MemoryStore: Block broadcast_68 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 27, fetching them
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:21 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:21 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:21 INFO Executor: Finished task 0.0 in stage 84.0 (TID 138). 2450 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 140
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 86.0 (TID 140)
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 69 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_69_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 69 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_69 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 86.0 (TID 140). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 144
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 89.0 (TID 144)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 29 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 71 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_71_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 71 took 4 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_71 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 89.0 (TID 144). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 151
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 92.0 (TID 151)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 30 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 73 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_73_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 73 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_73 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 92.0 (TID 151). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 153
23/09/24 21:18:22 INFO Executor: Running task 0.0 in stage 93.0 (TID 153)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 31 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 74 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_74_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 74 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_74 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 30, fetching them
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:22 INFO Executor: Finished task 0.0 in stage 93.0 (TID 153). 2450 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 154
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 95.0 (TID 154)
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 75 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_75_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 75 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_75 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 95.0 (TID 154). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 159
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 98.0 (TID 159)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 32 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 77 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_77_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 77 took 4 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_77 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 98.0 (TID 159). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 163
23/09/24 21:18:22 INFO Executor: Running task 0.0 in stage 99.0 (TID 163)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 33 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 78 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_78_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 78 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_78 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 32, fetching them
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:22 INFO Executor: Finished task 0.0 in stage 99.0 (TID 163). 2450 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 166
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 103.0 (TID 166)
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 80 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_80_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 80 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_80 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 103.0 (TID 166). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 170
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 106.0 (TID 170)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 34 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 82 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_82_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 82 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_82 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 106.0 (TID 170). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 174
23/09/24 21:18:22 INFO Executor: Running task 0.0 in stage 107.0 (TID 174)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 35 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 83 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_83_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 83 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_83 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 34, fetching them
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:22 INFO Executor: Finished task 0.0 in stage 107.0 (TID 174). 2450 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 177
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 109.0 (TID 177)
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 84 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_84_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 84 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_84 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 109.0 (TID 177). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 179
23/09/24 21:18:22 INFO Executor: Running task 0.0 in stage 110.0 (TID 179)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 36 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 85 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_85_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 85 took 5 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_85 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 35, fetching them
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:22 INFO Executor: Finished task 0.0 in stage 110.0 (TID 179). 2450 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 181
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 112.0 (TID 181)
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 86 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_86_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 86 took 4 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_86 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 112.0 (TID 181). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 185
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 115.0 (TID 185)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 37 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 88 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_88_piece0 stored as bytes in memory (estimated size 10.2 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 88 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_88 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 115.0 (TID 185). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 190
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 118.0 (TID 190)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 38 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 90 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_90_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 90 took 5 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_90 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 118.0 (TID 190). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 197
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 121.0 (TID 197)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 39 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 92 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_92_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 92 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_92 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 121.0 (TID 197). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 199
23/09/24 21:18:22 INFO Executor: Running task 0.0 in stage 122.0 (TID 199)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 40 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 93 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_93_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 93 took 2 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_93 stored as values in memory (estimated size 10.1 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 39, fetching them
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Getting 4 (240.0 B) non-empty blocks including 1 (60.0 B) local and 3 (180.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:22 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:22 INFO Executor: Finished task 0.0 in stage 122.0 (TID 199). 2450 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 202
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 124.0 (TID 202)
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 94 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_94_piece0 stored as bytes in memory (estimated size 10.2 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 94 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_94 stored as values in memory (estimated size 26.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_94_2 locally
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 124.0 (TID 202). 2990 bytes result sent to driver
23/09/24 21:18:22 INFO CoarseGrainedExecutorBackend: Got assigned task 205
23/09/24 21:18:22 INFO Executor: Running task 2.0 in stage 126.0 (TID 205)
23/09/24 21:18:22 INFO MapOutputTrackerWorker: Updating epoch to 41 and clearing cache
23/09/24 21:18:22 INFO TorrentBroadcast: Started reading broadcast variable 96 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_96_piece0 stored as bytes in memory (estimated size 11.6 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO TorrentBroadcast: Reading broadcast variable 96 took 3 ms
23/09/24 21:18:22 INFO MemoryStore: Block broadcast_96 stored as values in memory (estimated size 34.2 KiB, free 5.1 GiB)
23/09/24 21:18:22 INFO BlockManager: Found block rdd_40_2 locally
23/09/24 21:18:22 INFO CodeGenerator: Code generated in 14.054948 ms
23/09/24 21:18:22 INFO CodeGenerator: Code generated in 9.352304 ms
23/09/24 21:18:22 INFO Executor: Finished task 2.0 in stage 126.0 (TID 205). 3786 bytes result sent to driver
23/09/24 21:18:23 INFO CoarseGrainedExecutorBackend: Got assigned task 208
23/09/24 21:18:23 INFO Executor: Running task 0.0 in stage 127.0 (TID 208)
23/09/24 21:18:23 INFO TorrentBroadcast: Started reading broadcast variable 98 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_98_piece0 stored as bytes in memory (estimated size 16.4 KiB, free 5.1 GiB)
23/09/24 21:18:23 INFO TorrentBroadcast: Reading broadcast variable 98 took 3 ms
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_98 stored as values in memory (estimated size 47.7 KiB, free 5.1 GiB)
23/09/24 21:18:23 INFO CodeGenerator: Code generated in 8.46537 ms
23/09/24 21:18:23 INFO BlockManager: Found block rdd_40_0 remotely
23/09/24 21:18:23 INFO CodeGenerator: Code generated in 11.198466 ms
23/09/24 21:18:23 INFO CodeGenerator: Code generated in 14.217573 ms
23/09/24 21:18:23 INFO TorrentBroadcast: Started reading broadcast variable 97 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_97_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 5.1 GiB)
23/09/24 21:18:23 INFO TorrentBroadcast: Reading broadcast variable 97 took 4 ms
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_97 stored as values in memory (estimated size 64.0 MiB, free 5.0 GiB)
23/09/24 21:18:23 INFO MemoryStore: Block rdd_382_0 stored as values in memory (estimated size 7.0 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO BlockManager: Found block rdd_40_1 remotely
23/09/24 21:18:23 INFO MemoryStore: Block rdd_382_1 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO BlockManager: Found block rdd_40_2 locally
23/09/24 21:18:23 INFO MemoryStore: Block rdd_382_2 stored as values in memory (estimated size 16.0 B, free 5.0 GiB)
23/09/24 21:18:23 INFO Executor: Finished task 0.0 in stage 127.0 (TID 208). 2363 bytes result sent to driver
23/09/24 21:18:23 INFO CoarseGrainedExecutorBackend: Got assigned task 209
23/09/24 21:18:23 INFO Executor: Running task 0.0 in stage 128.0 (TID 209)
23/09/24 21:18:23 INFO TorrentBroadcast: Started reading broadcast variable 99 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_99_piece0 stored as bytes in memory (estimated size 15.3 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO TorrentBroadcast: Reading broadcast variable 99 took 3 ms
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_99 stored as values in memory (estimated size 44.4 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_0 locally
23/09/24 21:18:23 INFO CodeGenerator: Code generated in 12.07985 ms
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_1 locally
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_2 locally
23/09/24 21:18:23 INFO Executor: Finished task 0.0 in stage 128.0 (TID 209). 4253 bytes result sent to driver
23/09/24 21:18:23 INFO CoarseGrainedExecutorBackend: Got assigned task 217
23/09/24 21:18:23 INFO Executor: Running task 0.0 in stage 136.0 (TID 217)
23/09/24 21:18:23 INFO TorrentBroadcast: Started reading broadcast variable 107 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_107_piece0 stored as bytes in memory (estimated size 16.4 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO TorrentBroadcast: Reading broadcast variable 107 took 3 ms
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_107 stored as values in memory (estimated size 47.7 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_0 locally
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_1 locally
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_2 locally
23/09/24 21:18:23 INFO Executor: Finished task 0.0 in stage 136.0 (TID 217). 2320 bytes result sent to driver
23/09/24 21:18:23 INFO CoarseGrainedExecutorBackend: Got assigned task 220
23/09/24 21:18:23 INFO Executor: Running task 0.0 in stage 139.0 (TID 220)
23/09/24 21:18:23 INFO TorrentBroadcast: Started reading broadcast variable 110 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_110_piece0 stored as bytes in memory (estimated size 16.4 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO TorrentBroadcast: Reading broadcast variable 110 took 3 ms
23/09/24 21:18:23 INFO MemoryStore: Block broadcast_110 stored as values in memory (estimated size 47.7 KiB, free 5.0 GiB)
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_0 locally
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_1 locally
23/09/24 21:18:23 INFO BlockManager: Found block rdd_382_2 locally
23/09/24 21:18:23 INFO Executor: Finished task 0.0 in stage 139.0 (TID 220). 2320 bytes result sent to driver
23/09/24 21:18:24 INFO CoarseGrainedExecutorBackend: Got assigned task 225
23/09/24 21:18:24 INFO CoarseGrainedExecutorBackend: Got assigned task 226
23/09/24 21:18:24 INFO Executor: Running task 0.0 in stage 143.0 (TID 225)
23/09/24 21:18:24 INFO Executor: Running task 1.0 in stage 143.0 (TID 226)
23/09/24 21:18:24 INFO TorrentBroadcast: Started reading broadcast variable 114 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_114_piece0 stored as bytes in memory (estimated size 16.9 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO TorrentBroadcast: Reading broadcast variable 114 took 2 ms
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_114 stored as values in memory (estimated size 47.9 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO BlockManager: Found block rdd_382_1 locally
23/09/24 21:18:24 INFO BlockManager: Found block rdd_382_0 locally
23/09/24 21:18:24 INFO Executor: Finished task 1.0 in stage 143.0 (TID 226). 2602 bytes result sent to driver
23/09/24 21:18:24 INFO Executor: Finished task 0.0 in stage 143.0 (TID 225). 2602 bytes result sent to driver
23/09/24 21:18:24 INFO CoarseGrainedExecutorBackend: Got assigned task 230
23/09/24 21:18:24 INFO Executor: Running task 2.0 in stage 145.0 (TID 230)
23/09/24 21:18:24 INFO MapOutputTrackerWorker: Updating epoch to 42 and clearing cache
23/09/24 21:18:24 INFO TorrentBroadcast: Started reading broadcast variable 116 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_116_piece0 stored as bytes in memory (estimated size 11.6 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO TorrentBroadcast: Reading broadcast variable 116 took 4 ms
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_116 stored as values in memory (estimated size 34.2 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO BlockManager: Found block rdd_40_2 locally
23/09/24 21:18:24 INFO Executor: Finished task 2.0 in stage 145.0 (TID 230). 3786 bytes result sent to driver
23/09/24 21:18:24 INFO CoarseGrainedExecutorBackend: Got assigned task 233
23/09/24 21:18:24 INFO Executor: Running task 2.0 in stage 146.0 (TID 233)
23/09/24 21:18:24 INFO TorrentBroadcast: Started reading broadcast variable 118 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_118_piece0 stored as bytes in memory (estimated size 16.8 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO TorrentBroadcast: Reading broadcast variable 118 took 3 ms
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_118 stored as values in memory (estimated size 47.9 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO BlockManager: Found block rdd_40_2 locally
23/09/24 21:18:24 INFO CodeGenerator: Code generated in 15.729216 ms
23/09/24 21:18:24 INFO TorrentBroadcast: Started reading broadcast variable 117 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_117_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO TorrentBroadcast: Reading broadcast variable 117 took 13 ms
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_117 stored as values in memory (estimated size 64.0 MiB, free 5.0 GiB)
23/09/24 21:18:24 INFO MemoryStore: Block rdd_497_2 stored as values in memory (estimated size 6.6 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO Executor: Finished task 2.0 in stage 146.0 (TID 233). 2645 bytes result sent to driver
23/09/24 21:18:24 INFO CoarseGrainedExecutorBackend: Got assigned task 234
23/09/24 21:18:24 INFO Executor: Running task 0.0 in stage 147.0 (TID 234)
23/09/24 21:18:24 INFO MapOutputTrackerWorker: Updating epoch to 43 and clearing cache
23/09/24 21:18:24 INFO TorrentBroadcast: Started reading broadcast variable 119 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_119_piece0 stored as bytes in memory (estimated size 5.0 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO TorrentBroadcast: Reading broadcast variable 119 took 2 ms
23/09/24 21:18:24 INFO MemoryStore: Block broadcast_119 stored as values in memory (estimated size 10.1 KiB, free 5.0 GiB)
23/09/24 21:18:24 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 42, fetching them
23/09/24 21:18:24 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:24 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:24 INFO ShuffleBlockFetcherIterator: Getting 3 (180.0 B) non-empty blocks including 1 (60.0 B) local and 2 (120.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:24 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:24 INFO Executor: Finished task 0.0 in stage 147.0 (TID 234). 2450 bytes result sent to driver
23/09/24 21:18:27 INFO BlockManager: Removing RDD 547
23/09/24 21:18:28 INFO BlockManager: Removing RDD 547
23/09/24 21:18:28 INFO CoarseGrainedExecutorBackend: Got assigned task 276
23/09/24 21:18:28 INFO Executor: Running task 2.0 in stage 188.0 (TID 276)
23/09/24 21:18:28 INFO TorrentBroadcast: Started reading broadcast variable 198 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:28 INFO MemoryStore: Block broadcast_198_piece0 stored as bytes in memory (estimated size 17.8 KiB, free 5.0 GiB)
23/09/24 21:18:28 INFO TorrentBroadcast: Reading broadcast variable 198 took 3 ms
23/09/24 21:18:28 INFO MemoryStore: Block broadcast_198 stored as values in memory (estimated size 63.4 KiB, free 5.0 GiB)
23/09/24 21:18:28 INFO BlockManager: Found block rdd_88_2 locally
23/09/24 21:18:28 INFO CodeGenerator: Code generated in 41.897105 ms
23/09/24 21:18:28 INFO CodeGenerator: Code generated in 7.851083 ms
23/09/24 21:18:28 INFO Executor: Finished task 2.0 in stage 188.0 (TID 276). 2574 bytes result sent to driver
23/09/24 21:18:28 INFO CoarseGrainedExecutorBackend: Got assigned task 280
23/09/24 21:18:28 INFO Executor: Running task 2.0 in stage 189.0 (TID 280)
23/09/24 21:18:28 INFO MapOutputTrackerWorker: Updating epoch to 44 and clearing cache
23/09/24 21:18:28 INFO TorrentBroadcast: Started reading broadcast variable 199 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:28 INFO MemoryStore: Block broadcast_199_piece0 stored as bytes in memory (estimated size 7.9 KiB, free 5.0 GiB)
23/09/24 21:18:28 INFO TorrentBroadcast: Reading broadcast variable 199 took 4 ms
23/09/24 21:18:28 INFO MemoryStore: Block broadcast_199 stored as values in memory (estimated size 19.4 KiB, free 5.0 GiB)
23/09/24 21:18:28 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 43, fetching them
23/09/24 21:18:28 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:28 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:28 INFO ShuffleBlockFetcherIterator: Getting 2 (844.0 B) non-empty blocks including 0 (0.0 B) local and 2 (844.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:28 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
23/09/24 21:18:28 INFO MemoryStore: Block rdd_610_2 stored as values in memory (estimated size 2.9 KiB, free 5.0 GiB)
23/09/24 21:18:28 INFO CodeGenerator: Code generated in 8.98417 ms
23/09/24 21:18:28 INFO CodeGenerator: Code generated in 16.764098 ms
23/09/24 21:18:28 INFO Executor: Finished task 2.0 in stage 189.0 (TID 280). 2981 bytes result sent to driver
23/09/24 21:18:28 INFO CoarseGrainedExecutorBackend: Got assigned task 283
23/09/24 21:18:28 INFO Executor: Running task 2.0 in stage 192.0 (TID 283)
23/09/24 21:18:28 INFO TorrentBroadcast: Started reading broadcast variable 201 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:28 INFO MemoryStore: Block broadcast_201_piece0 stored as bytes in memory (estimated size 51.3 KiB, free 5.0 GiB)
23/09/24 21:18:28 INFO TorrentBroadcast: Reading broadcast variable 201 took 3 ms
23/09/24 21:18:28 INFO MemoryStore: Block broadcast_201 stored as values in memory (estimated size 139.3 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO BlockManager: Found block rdd_610_2 locally
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 13.00147 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 15.322977 ms
23/09/24 21:18:30 INFO TorrentBroadcast: Started reading broadcast variable 200 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_200_piece0 stored as bytes in memory (estimated size 3.9 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO TorrentBroadcast: Reading broadcast variable 200 took 4 ms
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_200 stored as values in memory (estimated size 7.9 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO MemoryStore: Block rdd_621_2 stored as values in memory (estimated size 4.5 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 12.990549 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 44.276392 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 5.958722 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 4.10759 ms
23/09/24 21:18:30 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
23/09/24 21:18:30 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
23/09/24 21:18:30 INFO MemoryStore: Block rdd_627_2 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 7.767684 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 10.121445 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 9.634858 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 4.435178 ms
23/09/24 21:18:30 INFO Executor: Finished task 2.0 in stage 192.0 (TID 283). 2876 bytes result sent to driver
23/09/24 21:18:30 INFO CoarseGrainedExecutorBackend: Got assigned task 287
23/09/24 21:18:30 INFO Executor: Running task 2.0 in stage 195.0 (TID 287)
23/09/24 21:18:30 INFO TorrentBroadcast: Started reading broadcast variable 202 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_202_piece0 stored as bytes in memory (estimated size 51.3 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO TorrentBroadcast: Reading broadcast variable 202 took 3 ms
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_202 stored as values in memory (estimated size 139.3 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO BlockManager: Found block rdd_627_2 locally
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 6.974086 ms
23/09/24 21:18:30 INFO CodeGenerator: Code generated in 8.374369 ms
23/09/24 21:18:30 INFO Executor: Finished task 2.0 in stage 195.0 (TID 287). 2763 bytes result sent to driver
23/09/24 21:18:30 INFO CoarseGrainedExecutorBackend: Got assigned task 292
23/09/24 21:18:30 INFO Executor: Running task 2.0 in stage 197.0 (TID 292)
23/09/24 21:18:30 INFO TorrentBroadcast: Started reading broadcast variable 203 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_203_piece0 stored as bytes in memory (estimated size 7.1 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO TorrentBroadcast: Reading broadcast variable 203 took 3 ms
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_203 stored as values in memory (estimated size 15.1 KiB, free 5.0 GiB)
23/09/24 21:18:30 INFO BlockManager: Found block rdd_88_2 locally
23/09/24 21:18:30 INFO Executor: Finished task 2.0 in stage 197.0 (TID 292). 3439 bytes result sent to driver
23/09/24 21:18:30 INFO CoarseGrainedExecutorBackend: Got assigned task 296
23/09/24 21:18:30 INFO Executor: Running task 2.0 in stage 200.0 (TID 296)
23/09/24 21:18:30 INFO TorrentBroadcast: Started reading broadcast variable 205 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:30 INFO MemoryStore: Block broadcast_205_piece0 stored as bytes in memory (estimated size 52.0 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 300
23/09/24 21:18:31 INFO Executor: Running task 2.0 in stage 202.0 (TID 300)
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 206 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 205 took 8 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_205 stored as values in memory (estimated size 140.3 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_206_piece0 stored as bytes in memory (estimated size 51.9 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 206 took 4 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_206 stored as values in memory (estimated size 140.2 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 17.660669 ms
23/09/24 21:18:31 INFO BlockManager: Found block rdd_627_2 locally
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 9.490806 ms
23/09/24 21:18:31 INFO BlockManager: Found block rdd_627_2 locally
23/09/24 21:18:31 INFO Executor: Finished task 2.0 in stage 200.0 (TID 296). 3001 bytes result sent to driver
23/09/24 21:18:31 INFO Executor: Finished task 2.0 in stage 202.0 (TID 300). 3125 bytes result sent to driver
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 302
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 305
23/09/24 21:18:31 INFO Executor: Running task 0.0 in stage 203.0 (TID 302)
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 308
23/09/24 21:18:31 INFO Executor: Running task 3.0 in stage 203.0 (TID 305)
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 311
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Updating epoch to 45 and clearing cache
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 207 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO Executor: Running task 6.0 in stage 203.0 (TID 308)
23/09/24 21:18:31 INFO Executor: Running task 9.0 in stage 203.0 (TID 311)
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 314
23/09/24 21:18:31 INFO Executor: Running task 1.0 in stage 201.0 (TID 314)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Updating epoch to 46 and clearing cache
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 208 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_207_piece0 stored as bytes in memory (estimated size 53.6 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 207 took 6 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_208_piece0 stored as bytes in memory (estimated size 53.6 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_207 stored as values in memory (estimated size 147.5 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 208 took 3 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_208 stored as values in memory (estimated size 147.6 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 44, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 44, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 44, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 44, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (368.0 B) non-empty blocks including 0 (0.0 B) local and 1 (368.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (334.0 B) non-empty blocks including 0 (0.0 B) local and 1 (334.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (405.0 B) non-empty blocks including 0 (0.0 B) local and 1 (405.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (368.0 B) non-empty blocks including 0 (0.0 B) local and 1 (368.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 46, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (334.0 B) non-empty blocks including 0 (0.0 B) local and 1 (334.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 12.718445 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 10.296677 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 9.872817 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 9.189207 ms
23/09/24 21:18:31 INFO MemoryStore: Block rdd_639_9 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_1 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MemoryStore: Block rdd_639_0 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MemoryStore: Block rdd_639_6 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MemoryStore: Block rdd_639_3 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 11.949815 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 13.935001 ms
23/09/24 21:18:31 INFO Executor: Finished task 1.0 in stage 201.0 (TID 314). 4580 bytes result sent to driver
23/09/24 21:18:31 INFO Executor: Finished task 0.0 in stage 203.0 (TID 302). 4580 bytes result sent to driver
23/09/24 21:18:31 INFO Executor: Finished task 6.0 in stage 203.0 (TID 308). 4580 bytes result sent to driver
23/09/24 21:18:31 INFO Executor: Finished task 3.0 in stage 203.0 (TID 305). 4580 bytes result sent to driver
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 317
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 318
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 319
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 320
23/09/24 21:18:31 INFO Executor: Running task 4.0 in stage 201.0 (TID 317)
23/09/24 21:18:31 INFO Executor: Running task 6.0 in stage 201.0 (TID 319)
23/09/24 21:18:31 INFO Executor: Running task 7.0 in stage 201.0 (TID 320)
23/09/24 21:18:31 INFO Executor: Finished task 9.0 in stage 203.0 (TID 311). 4580 bytes result sent to driver
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (445.0 B) non-empty blocks including 0 (0.0 B) local and 1 (445.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 321
23/09/24 21:18:31 INFO Executor: Running task 8.0 in stage 201.0 (TID 321)
23/09/24 21:18:31 INFO Executor: Running task 5.0 in stage 201.0 (TID 318)
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_6 stored as values in memory (estimated size 4.9 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (445.0 B) non-empty blocks including 0 (0.0 B) local and 1 (445.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (445.0 B) non-empty blocks including 0 (0.0 B) local and 1 (445.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_7 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (445.0 B) non-empty blocks including 0 (0.0 B) local and 1 (445.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO Executor: Finished task 7.0 in stage 201.0 (TID 320). 4537 bytes result sent to driver
23/09/24 21:18:31 INFO Executor: Finished task 6.0 in stage 201.0 (TID 319). 4580 bytes result sent to driver
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 328
23/09/24 21:18:31 INFO Executor: Running task 15.0 in stage 201.0 (TID 328)
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_8 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (445.0 B) non-empty blocks including 0 (0.0 B) local and 1 (445.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO Executor: Finished task 8.0 in stage 201.0 (TID 321). 4537 bytes result sent to driver
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_5 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 1 (368.0 B) non-empty blocks including 0 (0.0 B) local and 1 (368.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_4 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO Executor: Finished task 5.0 in stage 201.0 (TID 318). 4537 bytes result sent to driver
23/09/24 21:18:31 INFO MemoryStore: Block rdd_658_15 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO Executor: Finished task 4.0 in stage 201.0 (TID 317). 4537 bytes result sent to driver
23/09/24 21:18:31 INFO Executor: Finished task 15.0 in stage 201.0 (TID 328). 4537 bytes result sent to driver
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 331
23/09/24 21:18:31 INFO Executor: Running task 2.0 in stage 204.0 (TID 331)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Updating epoch to 48 and clearing cache
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 209 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_209_piece0 stored as bytes in memory (estimated size 75.1 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 209 took 3 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_209 stored as values in memory (estimated size 233.4 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 45, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 11 (4.0 KiB) non-empty blocks including 4 (1475.0 B) local and 7 (2.5 KiB) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 7.697392 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 12.722824 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 10.437121 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 34.836954 ms
23/09/24 21:18:31 INFO MemoryStore: Block rdd_679_0 stored as values in memory (estimated size 7.6 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 8.423432 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 4.23494 ms
23/09/24 21:18:31 INFO MemoryStore: Block rdd_689_0 stored as values in memory (estimated size 4.2 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 7.208118 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 4.366698 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 3.972194 ms
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 204 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_204_piece0 stored as bytes in memory (estimated size 3.7 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 204 took 3 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_204 stored as values in memory (estimated size 7.6 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 2.957569 ms
23/09/24 21:18:31 INFO Executor: Finished task 2.0 in stage 204.0 (TID 331). 7966 bytes result sent to driver
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 334
23/09/24 21:18:31 INFO Executor: Running task 2.0 in stage 211.0 (TID 334)
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 210 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_210_piece0 stored as bytes in memory (estimated size 75.3 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 210 took 2 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_210 stored as values in memory (estimated size 234.1 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 5.498374 ms
23/09/24 21:18:31 INFO BlockManager: Found block rdd_689_0 locally
23/09/24 21:18:31 INFO Executor: Finished task 2.0 in stage 211.0 (TID 334). 7948 bytes result sent to driver
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 337
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 340
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 343
23/09/24 21:18:31 INFO Executor: Running task 3.0 in stage 212.0 (TID 340)
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 346
23/09/24 21:18:31 INFO CoarseGrainedExecutorBackend: Got assigned task 349
23/09/24 21:18:31 INFO Executor: Running task 6.0 in stage 212.0 (TID 343)
23/09/24 21:18:31 INFO Executor: Running task 12.0 in stage 212.0 (TID 349)
23/09/24 21:18:31 INFO Executor: Running task 0.0 in stage 212.0 (TID 337)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Updating epoch to 49 and clearing cache
23/09/24 21:18:31 INFO Executor: Running task 9.0 in stage 212.0 (TID 346)
23/09/24 21:18:31 INFO TorrentBroadcast: Started reading broadcast variable 211 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_211_piece0 stored as bytes in memory (estimated size 129.5 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO TorrentBroadcast: Reading broadcast variable 211 took 5 ms
23/09/24 21:18:31 INFO MemoryStore: Block broadcast_211 stored as values in memory (estimated size 379.7 KiB, free 5.0 GiB)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 48, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 48, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:41787)
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 48, fetching them
23/09/24 21:18:31 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 2 (608.0 B) non-empty blocks including 1 (304.0 B) local and 1 (304.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 2 (552.0 B) non-empty blocks including 0 (0.0 B) local and 2 (552.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 2 (608.0 B) non-empty blocks including 1 (304.0 B) local and 1 (304.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 2 (608.0 B) non-empty blocks including 1 (304.0 B) local and 1 (304.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Getting 2 (552.0 B) non-empty blocks including 0 (0.0 B) local and 2 (552.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 21:18:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 24.477149 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 31.422633 ms
23/09/24 21:18:31 INFO CodeGenerator: Code generated in 9.761387 ms
23/09/24 21:18:31 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:31 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:31 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:31 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:31 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:32 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:31 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:32 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:32 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:32 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:32 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:32 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:32 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 21:18:32 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO CodecConfig: Compression: SNAPPY
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Dictionary is on
23/09/24 21:18:32 INFO ParquetOutputFormat: Dictionary is on
23/09/24 21:18:32 INFO ParquetOutputFormat: Validation is off
23/09/24 21:18:32 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 21:18:32 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 21:18:32 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 21:18:32 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 21:18:32 INFO ParquetOutputFormat: Dictionary is on
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Validation is off
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 21:18:32 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 21:18:32 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 21:18:32 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 21:18:32 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 21:18:32 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 21:18:32 INFO ParquetOutputFormat: Validation is off
23/09/24 21:18:32 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 21:18:32 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 21:18:32 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 21:18:32 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 21:18:32 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 21:18:32 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 21:18:32 INFO ParquetOutputFormat: Dictionary is on
23/09/24 21:18:32 INFO ParquetOutputFormat: Validation is off
23/09/24 21:18:32 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 21:18:32 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 21:18:32 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 21:18:32 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 21:18:32 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 21:18:32 INFO ParquetOutputFormat: Dictionary is on
23/09/24 21:18:32 INFO ParquetOutputFormat: Validation is off
23/09/24 21:18:32 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 21:18:32 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 21:18:32 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 21:18:32 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 21:18:32 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 21:18:32 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 21:18:32 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 21:18:32 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 21:18:32 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 21:18:32 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 21:18:32 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 21:18:32 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 21:18:32 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 21:18:32 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 21:18:32 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 21:18:32 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 345
23/09/24 21:18:32 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 375
23/09/24 21:18:32 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 438
23/09/24 21:18:32 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 439
23/09/24 21:18:32 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 415
23/09/24 21:18:32 INFO FileOutputCommitter: Saved output of task 'attempt_202309242118316893023308221393206_0212_m_000000_337' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242118316893023308221393206_0212_m_000000
23/09/24 21:18:32 INFO FileOutputCommitter: Saved output of task 'attempt_202309242118314575907445849601313_0212_m_000009_346' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242118314575907445849601313_0212_m_000009
23/09/24 21:18:32 INFO FileOutputCommitter: Saved output of task 'attempt_202309242118313994480841377018069_0212_m_000006_343' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242118313994480841377018069_0212_m_000006
23/09/24 21:18:32 INFO FileOutputCommitter: Saved output of task 'attempt_202309242118317452238099110113441_0212_m_000012_349' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242118317452238099110113441_0212_m_000012
23/09/24 21:18:32 INFO SparkHadoopMapRedUtil: attempt_202309242118314575907445849601313_0212_m_000009_346: Committed
23/09/24 21:18:32 INFO FileOutputCommitter: Saved output of task 'attempt_202309242118318097414691544540641_0212_m_000003_340' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242118318097414691544540641_0212_m_000003
23/09/24 21:18:32 INFO SparkHadoopMapRedUtil: attempt_202309242118316893023308221393206_0212_m_000000_337: Committed
23/09/24 21:18:32 INFO SparkHadoopMapRedUtil: attempt_202309242118318097414691544540641_0212_m_000003_340: Committed
23/09/24 21:18:32 INFO SparkHadoopMapRedUtil: attempt_202309242118313994480841377018069_0212_m_000006_343: Committed
23/09/24 21:18:32 INFO SparkHadoopMapRedUtil: attempt_202309242118317452238099110113441_0212_m_000012_349: Committed
23/09/24 21:18:32 INFO Executor: Finished task 6.0 in stage 212.0 (TID 343). 10191 bytes result sent to driver
23/09/24 21:18:32 INFO Executor: Finished task 9.0 in stage 212.0 (TID 346). 10191 bytes result sent to driver
23/09/24 21:18:32 INFO Executor: Finished task 0.0 in stage 212.0 (TID 337). 10191 bytes result sent to driver
23/09/24 21:18:32 INFO Executor: Finished task 12.0 in stage 212.0 (TID 349). 10148 bytes result sent to driver
23/09/24 21:18:32 INFO Executor: Finished task 3.0 in stage 212.0 (TID 340). 10191 bytes result sent to driver
23/09/24 21:18:32 INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
23/09/24 21:18:32 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
tdown