ytes in memory (estimated size 16.9 KiB, free 5.1 GiB)
23/09/24 20:12:28 INFO TorrentBroadcast: Reading broadcast variable 34 took 6 ms
23/09/24 20:12:28 INFO MemoryStore: Block broadcast_34 stored as values in memory (estimated size 47.9 KiB, free 5.1 GiB)
23/09/24 20:12:28 INFO BlockManager: Found block rdd_40_1 locally
23/09/24 20:12:28 INFO CodeGenerator: Code generated in 18.266679 ms
23/09/24 20:12:29 INFO CodeGenerator: Code generated in 35.676301 ms
23/09/24 20:12:29 INFO TorrentBroadcast: Started reading broadcast variable 33 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:29 INFO MemoryStore: Block broadcast_33_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 5.1 GiB)
23/09/24 20:12:29 INFO TorrentBroadcast: Reading broadcast variable 33 took 6 ms
23/09/24 20:12:29 INFO MemoryStore: Block broadcast_33 stored as values in memory (estimated size 64.0 MiB, free 5.0 GiB)
23/09/24 20:12:29 INFO MemoryStore: Block rdd_141_1 stored as values in memory (estimated size 6.8 KiB, free 5.0 GiB)
23/09/24 20:12:29 INFO Executor: Finished task 1.0 in stage 26.0 (TID 53). 2644 bytes result sent to driver
23/09/24 20:12:30 INFO CoarseGrainedExecutorBackend: Got assigned task 55
23/09/24 20:12:30 INFO Executor: Running task 0.0 in stage 28.0 (TID 55)
23/09/24 20:12:30 INFO MapOutputTrackerWorker: Updating epoch to 10 and clearing cache
23/09/24 20:12:30 INFO TorrentBroadcast: Started reading broadcast variable 36 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:30 INFO MemoryStore: Block broadcast_36_piece0 stored as bytes in memory (estimated size 52.5 KiB, free 5.1 GiB)
23/09/24 20:12:30 INFO TorrentBroadcast: Reading broadcast variable 36 took 5 ms
23/09/24 20:12:30 INFO MemoryStore: Block broadcast_36 stored as values in memory (estimated size 194.2 KiB, free 5.1 GiB)
23/09/24 20:12:31 INFO BlockManager: Found block rdd_117_0 remotely
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 12.298364 ms
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 9.820346 ms
23/09/24 20:12:31 INFO MemoryStore: Block rdd_155_0 stored as values in memory (estimated size 7.1 KiB, free 5.1 GiB)
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 12.745809 ms
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 28.288886 ms
23/09/24 20:12:31 INFO BlockManager: Found block rdd_117_1 remotely
23/09/24 20:12:31 INFO MemoryStore: Block rdd_155_1 stored as values in memory (estimated size 4.8 KiB, free 5.1 GiB)
23/09/24 20:12:31 WARN BlockManager: Failed to fetch block after 1 fetch failures. Most recent failure cause:
java.lang.IllegalStateException: Empty buffer received for non empty block
	at org.apache.spark.storage.BlockManager.fetchRemoteManagedBuffer(BlockManager.scala:1063)
	at org.apache.spark.storage.BlockManager.$anonfun$getRemoteBlock$8(BlockManager.scala:1005)
	at scala.Option.orElse(Option.scala:447)
	at org.apache.spark.storage.BlockManager.getRemoteBlock(BlockManager.scala:1005)
	at org.apache.spark.storage.BlockManager.getRemoteValues(BlockManager.scala:951)
	at org.apache.spark.storage.BlockManager.get(BlockManager.scala:1168)
	at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1230)
	at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:386)
	at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1423)
	at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1350)
	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1414)
	at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1237)
	at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.UnionRDD.compute(UnionRDD.scala:106)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.CoalescedRDD.$anonfun$compute$1(CoalescedRDD.scala:99)
	at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:484)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:490)
	at org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer$$anon$1.next(InMemoryRelation.scala:87)
	at org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer$$anon$1.next(InMemoryRelation.scala:79)
	at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
	at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:222)
	at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)
	at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1423)
	at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1350)
	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1414)
	at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1237)
	at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:384)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:335)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.sql.execution.SQLExecutionRDD.$anonfun$compute$1(SQLExecutionRDD.scala:52)
	at org.apache.spark.sql.internal.SQLConf$.withExistingConf(SQLConf.scala:124)
	at org.apache.spark.sql.execution.SQLExecutionRDD.compute(SQLExecutionRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:498)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:501)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
23/09/24 20:12:31 INFO BlockManager: Found block rdd_40_2 remotely
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 10.511341 ms
23/09/24 20:12:31 INFO TorrentBroadcast: Started reading broadcast variable 28 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:31 INFO MemoryStore: Block broadcast_28_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 5.1 GiB)
23/09/24 20:12:31 INFO TorrentBroadcast: Reading broadcast variable 28 took 5 ms
23/09/24 20:12:31 INFO MemoryStore: Block broadcast_28 stored as values in memory (estimated size 64.0 MiB, free 5.0 GiB)
23/09/24 20:12:31 INFO MemoryStore: Block rdd_117_2 stored as values in memory (estimated size 16.0 B, free 5.0 GiB)
23/09/24 20:12:31 INFO MemoryStore: Block rdd_155_2 stored as values in memory (estimated size 16.0 B, free 5.0 GiB)
23/09/24 20:12:31 INFO BlockManager: Found block rdd_141_0 remotely
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 10.310802 ms
23/09/24 20:12:31 INFO MemoryStore: Block rdd_165_0 stored as values in memory (estimated size 4.3 KiB, free 5.0 GiB)
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 19.761481 ms
23/09/24 20:12:31 INFO BlockManager: Found block rdd_141_1 locally
23/09/24 20:12:31 INFO MemoryStore: Block rdd_165_1 stored as values in memory (estimated size 6.8 KiB, free 5.0 GiB)
23/09/24 20:12:31 INFO BlockManager: Found block rdd_141_2 remotely
23/09/24 20:12:31 INFO MemoryStore: Block rdd_165_2 stored as values in memory (estimated size 6.7 KiB, free 5.0 GiB)
23/09/24 20:12:31 INFO MemoryStore: Block rdd_173_0 stored as values in memory (estimated size 17.7 KiB, free 5.0 GiB)
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 17.608287 ms
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 15.367456 ms
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 7.152941 ms
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 8.362925 ms
23/09/24 20:12:31 INFO CodeGenerator: Code generated in 4.282273 ms
23/09/24 20:12:31 INFO Executor: Finished task 0.0 in stage 28.0 (TID 55). 35766 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 56
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 29.0 (TID 56)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 39 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_39_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 39 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_39 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_173_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 37 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_37_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 37 took 6 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_37 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO MemoryStore: Block rdd_191_0 stored as values in memory (estimated size 395.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 38 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_38_piece0 stored as bytes in memory (estimated size 103.0 B, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 38 took 5 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_38 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
23/09/24 20:12:32 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 29.0 (TID 56). 14043 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 57
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 30.0 (TID 57)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 41 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_41_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 41 took 5 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_41 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 40 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_40_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 40 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_40 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 30.0 (TID 57). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 58
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 31.0 (TID 58)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 43 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_43_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 43 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_43 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 42 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_42_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 42 took 3 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_42 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 31.0 (TID 58). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 59
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 32.0 (TID 59)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 45 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_45_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 45 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_45 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 44 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_44_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 44 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_44 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 32.0 (TID 59). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 60
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 33.0 (TID 60)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 47 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_47_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 47 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_47 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 46 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_46_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 46 took 3 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_46 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 33.0 (TID 60). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 61
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 34.0 (TID 61)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 49 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_49_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 49 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_49 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 48 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_48_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 48 took 3 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_48 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 34.0 (TID 61). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 62
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 35.0 (TID 62)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 51 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_51_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 51 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_51 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 50 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_50_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 50 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_50 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 35.0 (TID 62). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 63
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 36.0 (TID 63)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 53 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_53_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 53 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_53 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 52 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_52_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 52 took 3 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_52 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 36.0 (TID 63). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 64
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 37.0 (TID 64)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 55 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_55_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 55 took 3 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_55 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 54 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_54_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 54 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_54 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 37.0 (TID 64). 14000 bytes result sent to driver
23/09/24 20:12:32 INFO CoarseGrainedExecutorBackend: Got assigned task 65
23/09/24 20:12:32 INFO Executor: Running task 0.0 in stage 38.0 (TID 65)
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 57 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_57_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 57 took 4 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_57 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:32 INFO TorrentBroadcast: Started reading broadcast variable 56 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_56_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO TorrentBroadcast: Reading broadcast variable 56 took 3 ms
23/09/24 20:12:32 INFO MemoryStore: Block broadcast_56 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:32 INFO Executor: Finished task 0.0 in stage 38.0 (TID 65). 14000 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 66
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 39.0 (TID 66)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 59 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_59_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 59 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_59 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 58 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_58_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 58 took 5 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_58 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 39.0 (TID 66). 14000 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 67
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 40.0 (TID 67)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 61 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_61_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 61 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_61 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 60 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_60_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 60 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_60 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 40.0 (TID 67). 14000 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 68
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 41.0 (TID 68)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 63 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_63_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 63 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_63 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 62 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_62_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 62 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_62 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 41.0 (TID 68). 14000 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 69
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 42.0 (TID 69)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 65 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_65_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 65 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_65 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 64 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_64_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 64 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_64 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 42.0 (TID 69). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 70
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 43.0 (TID 70)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 67 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_67_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 67 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_67 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 66 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_66_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 66 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_66 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 43.0 (TID 70). 14001 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 71
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 44.0 (TID 71)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 69 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_69_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 69 took 5 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_69 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 68 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_68_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 68 took 5 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_68 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 44.0 (TID 71). 14045 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 72
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 45.0 (TID 72)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 71 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_71_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 71 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_71 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 70 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_70_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 70 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_70 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 45.0 (TID 72). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 73
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 46.0 (TID 73)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 73 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_73_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 73 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_73 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 72 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_72_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 72 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_72 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 46.0 (TID 73). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 74
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 47.0 (TID 74)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 75 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_75_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 75 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_75 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 74 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_74_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 74 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_74 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 47.0 (TID 74). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 75
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 48.0 (TID 75)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 77 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_77_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 77 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_77 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 76 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_76_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 76 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_76 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 48.0 (TID 75). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 76
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 49.0 (TID 76)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 79 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_79_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 79 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_79 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 78 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_78_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 78 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_78 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 49.0 (TID 76). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 77
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 50.0 (TID 77)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 81 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_81_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 81 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_81 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 80 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_80_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 80 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_80 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 50.0 (TID 77). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 78
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 51.0 (TID 78)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 83 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_83_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 83 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_83 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 82 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_82_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 82 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_82 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 51.0 (TID 78). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 79
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 52.0 (TID 79)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 85 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_85_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 85 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_85 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 84 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_84_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 84 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_84 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 52.0 (TID 79). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 80
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 53.0 (TID 80)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 87 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_87_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 87 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_87 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 86 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_86_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 86 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_86 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 53.0 (TID 80). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 81
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 54.0 (TID 81)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 89 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_89_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 89 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_89 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 88 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_88_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 88 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_88 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 54.0 (TID 81). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 82
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 55.0 (TID 82)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 91 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_91_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 91 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_91 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 90 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_90_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 90 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_90 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 55.0 (TID 82). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 83
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 56.0 (TID 83)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 93 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_93_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 93 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_93 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 92 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_92_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 92 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_92 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 56.0 (TID 83). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 84
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 57.0 (TID 84)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 95 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_95_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 95 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_95 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 94 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_94_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 94 took 2 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_94 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 57.0 (TID 84). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 85
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 58.0 (TID 85)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 97 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_97_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 97 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_97 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 96 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_96_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 96 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_96 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 58.0 (TID 85). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 86
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 59.0 (TID 86)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 99 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_99_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 99 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_99 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 98 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_98_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 98 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_98 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 59.0 (TID 86). 14002 bytes result sent to driver
23/09/24 20:12:33 INFO CoarseGrainedExecutorBackend: Got assigned task 87
23/09/24 20:12:33 INFO Executor: Running task 0.0 in stage 60.0 (TID 87)
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 101 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_101_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 101 took 4 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_101 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:33 INFO TorrentBroadcast: Started reading broadcast variable 100 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_100_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO TorrentBroadcast: Reading broadcast variable 100 took 3 ms
23/09/24 20:12:33 INFO MemoryStore: Block broadcast_100 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:33 INFO Executor: Finished task 0.0 in stage 60.0 (TID 87). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO CoarseGrainedExecutorBackend: Got assigned task 88
23/09/24 20:12:34 INFO Executor: Running task 0.0 in stage 61.0 (TID 88)
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 103 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_103_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 103 took 3 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_103 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 102 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_102_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 102 took 3 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_102 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO Executor: Finished task 0.0 in stage 61.0 (TID 88). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO CoarseGrainedExecutorBackend: Got assigned task 89
23/09/24 20:12:34 INFO Executor: Running task 0.0 in stage 62.0 (TID 89)
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 105 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_105_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 105 took 4 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_105 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 104 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_104_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 104 took 3 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_104 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO Executor: Finished task 0.0 in stage 62.0 (TID 89). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO CoarseGrainedExecutorBackend: Got assigned task 90
23/09/24 20:12:34 INFO Executor: Running task 0.0 in stage 63.0 (TID 90)
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 107 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_107_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 107 took 3 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_107 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 106 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_106_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 106 took 2 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_106 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO Executor: Finished task 0.0 in stage 63.0 (TID 90). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO CoarseGrainedExecutorBackend: Got assigned task 91
23/09/24 20:12:34 INFO Executor: Running task 0.0 in stage 64.0 (TID 91)
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 109 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_109_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 109 took 4 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_109 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 108 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_108_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 108 took 4 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_108 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO Executor: Finished task 0.0 in stage 64.0 (TID 91). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO CoarseGrainedExecutorBackend: Got assigned task 92
23/09/24 20:12:34 INFO Executor: Running task 0.0 in stage 65.0 (TID 92)
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 111 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_111_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 111 took 4 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_111 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 110 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_110_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 110 took 4 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_110 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO Executor: Finished task 0.0 in stage 65.0 (TID 92). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO CoarseGrainedExecutorBackend: Got assigned task 93
23/09/24 20:12:34 INFO Executor: Running task 0.0 in stage 66.0 (TID 93)
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 113 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_113_piece0 stored as bytes in memory (estimated size 53.1 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 113 took 4 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_113 stored as values in memory (estimated size 195.7 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO BlockManager: Found block rdd_191_0 locally
23/09/24 20:12:34 INFO TorrentBroadcast: Started reading broadcast variable 112 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_112_piece0 stored as bytes in memory (estimated size 10.0 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO TorrentBroadcast: Reading broadcast variable 112 took 2 ms
23/09/24 20:12:34 INFO MemoryStore: Block broadcast_112 stored as values in memory (estimated size 10.4 KiB, free 5.0 GiB)
23/09/24 20:12:34 INFO Executor: Finished task 0.0 in stage 66.0 (TID 93). 14002 bytes result sent to driver
23/09/24 20:12:34 INFO BlockManager: Removing RDD 191
23/09/24 20:12:34 INFO BlockManager: Removing RDD 191
23/09/24 20:12:35 INFO CoarseGrainedExecutorBackend: Got assigned task 95
23/09/24 20:12:35 INFO CoarseGrainedExecutorBackend: Got assigned task 97
23/09/24 20:12:35 INFO Executor: Running task 0.0 in stage 68.0 (TID 95)
23/09/24 20:12:35 INFO Executor: Running task 3.0 in stage 68.0 (TID 97)
23/09/24 20:12:35 INFO TorrentBroadcast: Started reading broadcast variable 114 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_114_piece0 stored as bytes in memory (estimated size 17.8 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO TorrentBroadcast: Reading broadcast variable 114 took 3 ms
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_114 stored as values in memory (estimated size 63.4 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO BlockManager: Found block rdd_88_0 locally
23/09/24 20:12:35 INFO BlockManager: Found block rdd_88_3 locally
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 40.121601 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 15.918386 ms
23/09/24 20:12:35 INFO Executor: Finished task 0.0 in stage 68.0 (TID 95). 2574 bytes result sent to driver
23/09/24 20:12:35 INFO Executor: Finished task 3.0 in stage 68.0 (TID 97). 2574 bytes result sent to driver
23/09/24 20:12:35 INFO CoarseGrainedExecutorBackend: Got assigned task 100
23/09/24 20:12:35 INFO Executor: Running task 2.0 in stage 69.0 (TID 100)
23/09/24 20:12:35 INFO MapOutputTrackerWorker: Updating epoch to 11 and clearing cache
23/09/24 20:12:35 INFO TorrentBroadcast: Started reading broadcast variable 115 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_115_piece0 stored as bytes in memory (estimated size 7.8 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO TorrentBroadcast: Reading broadcast variable 115 took 4 ms
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_115 stored as values in memory (estimated size 19.4 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 10, fetching them
23/09/24 20:12:35 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:39929)
23/09/24 20:12:35 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 20:12:35 INFO ShuffleBlockFetcherIterator: Getting 4 (1223.0 B) non-empty blocks including 2 (502.0 B) local and 2 (721.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:35 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:35 INFO MemoryStore: Block rdd_254_2 stored as values in memory (estimated size 3.0 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 11.30026 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 10.593366 ms
23/09/24 20:12:35 INFO Executor: Finished task 2.0 in stage 69.0 (TID 100). 3220 bytes result sent to driver
23/09/24 20:12:35 INFO CoarseGrainedExecutorBackend: Got assigned task 103
23/09/24 20:12:35 INFO Executor: Running task 2.0 in stage 72.0 (TID 103)
23/09/24 20:12:35 INFO TorrentBroadcast: Started reading broadcast variable 117 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_117_piece0 stored as bytes in memory (estimated size 51.3 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO TorrentBroadcast: Reading broadcast variable 117 took 4 ms
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_117 stored as values in memory (estimated size 139.3 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO BlockManager: Found block rdd_254_2 locally
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 11.385571 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 14.689917 ms
23/09/24 20:12:35 INFO TorrentBroadcast: Started reading broadcast variable 116 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_116_piece0 stored as bytes in memory (estimated size 3.8 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO TorrentBroadcast: Reading broadcast variable 116 took 4 ms
23/09/24 20:12:35 INFO MemoryStore: Block broadcast_116 stored as values in memory (estimated size 7.6 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO MemoryStore: Block rdd_265_2 stored as values in memory (estimated size 5.3 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 11.063884 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 24.088537 ms
23/09/24 20:12:35 INFO MemoryStore: Block rdd_271_2 stored as values in memory (estimated size 5.5 KiB, free 5.0 GiB)
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 7.871598 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 10.212085 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 13.194184 ms
23/09/24 20:12:35 INFO CodeGenerator: Code generated in 4.406297 ms
23/09/24 20:12:35 INFO Executor: Finished task 2.0 in stage 72.0 (TID 103). 2887 bytes result sent to driver
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 108
23/09/24 20:12:37 INFO Executor: Running task 2.0 in stage 75.0 (TID 108)
23/09/24 20:12:37 INFO TorrentBroadcast: Started reading broadcast variable 118 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_118_piece0 stored as bytes in memory (estimated size 51.3 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO TorrentBroadcast: Reading broadcast variable 118 took 3 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_118 stored as values in memory (estimated size 139.3 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO BlockManager: Found block rdd_271_2 locally
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 10.831916 ms
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 10.381355 ms
23/09/24 20:12:37 INFO Executor: Finished task 2.0 in stage 75.0 (TID 108). 2790 bytes result sent to driver
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 112
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 113
23/09/24 20:12:37 INFO Executor: Running task 0.0 in stage 77.0 (TID 112)
23/09/24 20:12:37 INFO Executor: Running task 3.0 in stage 77.0 (TID 113)
23/09/24 20:12:37 INFO TorrentBroadcast: Started reading broadcast variable 119 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_119_piece0 stored as bytes in memory (estimated size 7.1 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO TorrentBroadcast: Reading broadcast variable 119 took 8 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_119 stored as values in memory (estimated size 15.1 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO BlockManager: Found block rdd_88_0 locally
23/09/24 20:12:37 INFO BlockManager: Found block rdd_88_3 locally
23/09/24 20:12:37 INFO Executor: Finished task 3.0 in stage 77.0 (TID 113). 3119 bytes result sent to driver
23/09/24 20:12:37 INFO Executor: Finished task 0.0 in stage 77.0 (TID 112). 3327 bytes result sent to driver
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 114
23/09/24 20:12:37 INFO Executor: Running task 2.0 in stage 80.0 (TID 114)
23/09/24 20:12:37 INFO TorrentBroadcast: Started reading broadcast variable 121 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_121_piece0 stored as bytes in memory (estimated size 51.9 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 118
23/09/24 20:12:37 INFO Executor: Running task 2.0 in stage 82.0 (TID 118)
23/09/24 20:12:37 INFO TorrentBroadcast: Reading broadcast variable 121 took 4 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_121 stored as values in memory (estimated size 140.2 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO TorrentBroadcast: Started reading broadcast variable 122 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_122_piece0 stored as bytes in memory (estimated size 51.9 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO TorrentBroadcast: Reading broadcast variable 122 took 4 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_122 stored as values in memory (estimated size 140.3 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 9.571446 ms
23/09/24 20:12:37 INFO BlockManager: Found block rdd_271_2 locally
23/09/24 20:12:37 INFO Executor: Finished task 2.0 in stage 82.0 (TID 118). 3130 bytes result sent to driver
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 33.262565 ms
23/09/24 20:12:37 INFO BlockManager: Found block rdd_271_2 locally
23/09/24 20:12:37 INFO Executor: Finished task 2.0 in stage 80.0 (TID 114). 3126 bytes result sent to driver
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 123
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 126
23/09/24 20:12:37 INFO Executor: Running task 1.0 in stage 81.0 (TID 123)
23/09/24 20:12:37 INFO Executor: Running task 4.0 in stage 81.0 (TID 126)
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 129
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Updating epoch to 12 and clearing cache
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 132
23/09/24 20:12:37 INFO Executor: Running task 7.0 in stage 81.0 (TID 129)
23/09/24 20:12:37 INFO TorrentBroadcast: Started reading broadcast variable 123 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:37 INFO Executor: Running task 10.0 in stage 81.0 (TID 132)
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_123_piece0 stored as bytes in memory (estimated size 53.6 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO TorrentBroadcast: Reading broadcast variable 123 took 4 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_123 stored as values in memory (estimated size 147.6 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 134
23/09/24 20:12:37 INFO Executor: Running task 0.0 in stage 83.0 (TID 134)
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Updating epoch to 13 and clearing cache
23/09/24 20:12:37 INFO TorrentBroadcast: Started reading broadcast variable 124 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 11, fetching them
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:39929)
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 11, fetching them
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Getting 1 (334.0 B) non-empty blocks including 0 (0.0 B) local and 1 (334.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Getting 1 (405.0 B) non-empty blocks including 0 (0.0 B) local and 1 (405.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Getting 1 (368.0 B) non-empty blocks including 0 (0.0 B) local and 1 (368.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 9.729214 ms
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Getting 1 (334.0 B) non-empty blocks including 1 (334.0 B) local and 0 (0.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_124_piece0 stored as bytes in memory (estimated size 53.6 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO TorrentBroadcast: Reading broadcast variable 124 took 29 ms
23/09/24 20:12:37 INFO MemoryStore: Block broadcast_124 stored as values in memory (estimated size 147.6 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 9.036517 ms
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 13, fetching them
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:39929)
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Getting 1 (334.0 B) non-empty blocks including 0 (0.0 B) local and 1 (334.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:37 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 17 ms
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 24.664294 ms
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 8.774772 ms
23/09/24 20:12:37 INFO MemoryStore: Block rdd_302_0 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO MemoryStore: Block rdd_283_10 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO MemoryStore: Block rdd_283_1 stored as values in memory (estimated size 4.8 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO MemoryStore: Block rdd_283_7 stored as values in memory (estimated size 4.6 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO MemoryStore: Block rdd_283_4 stored as values in memory (estimated size 4.7 KiB, free 5.0 GiB)
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 13.537211 ms
23/09/24 20:12:37 INFO Executor: Finished task 7.0 in stage 81.0 (TID 129). 4580 bytes result sent to driver
23/09/24 20:12:37 INFO Executor: Finished task 4.0 in stage 81.0 (TID 126). 4580 bytes result sent to driver
23/09/24 20:12:37 INFO Executor: Finished task 10.0 in stage 81.0 (TID 132). 4580 bytes result sent to driver
23/09/24 20:12:37 INFO Executor: Finished task 1.0 in stage 81.0 (TID 123). 4580 bytes result sent to driver
23/09/24 20:12:37 INFO CodeGenerator: Code generated in 21.634156 ms
23/09/24 20:12:37 INFO Executor: Finished task 0.0 in stage 83.0 (TID 134). 4580 bytes result sent to driver
23/09/24 20:12:37 INFO CoarseGrainedExecutorBackend: Got assigned task 152
23/09/24 20:12:37 INFO Executor: Running task 2.0 in stage 84.0 (TID 152)
23/09/24 20:12:37 INFO MapOutputTrackerWorker: Updating epoch to 15 and clearing cache
23/09/24 20:12:38 INFO TorrentBroadcast: Started reading broadcast variable 125 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_125_piece0 stored as bytes in memory (estimated size 75.0 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO TorrentBroadcast: Reading broadcast variable 125 took 6 ms
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_125 stored as values in memory (estimated size 233.4 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 12, fetching them
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:39929)
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Getting 12 (4.3 KiB) non-empty blocks including 4 (1441.0 B) local and 8 (2.9 KiB) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 8.851357 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 10.017059 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 12.289657 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 28.289496 ms
23/09/24 20:12:38 INFO MemoryStore: Block rdd_323_0 stored as values in memory (estimated size 7.1 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 11.820701 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 4.695554 ms
23/09/24 20:12:38 INFO MemoryStore: Block rdd_333_0 stored as values in memory (estimated size 4.0 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 10.530988 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 5.524569 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 4.91111 ms
23/09/24 20:12:38 INFO TorrentBroadcast: Started reading broadcast variable 120 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_120_piece0 stored as bytes in memory (estimated size 3.7 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO TorrentBroadcast: Reading broadcast variable 120 took 8 ms
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_120 stored as values in memory (estimated size 7.4 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 11.077188 ms
23/09/24 20:12:38 INFO Executor: Finished task 2.0 in stage 84.0 (TID 152). 7923 bytes result sent to driver
23/09/24 20:12:38 INFO CoarseGrainedExecutorBackend: Got assigned task 156
23/09/24 20:12:38 INFO Executor: Running task 2.0 in stage 91.0 (TID 156)
23/09/24 20:12:38 INFO TorrentBroadcast: Started reading broadcast variable 126 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_126_piece0 stored as bytes in memory (estimated size 75.1 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO TorrentBroadcast: Reading broadcast variable 126 took 4 ms
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_126 stored as values in memory (estimated size 234.1 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 16.197182 ms
23/09/24 20:12:38 INFO BlockManager: Found block rdd_333_0 locally
23/09/24 20:12:38 INFO Executor: Finished task 2.0 in stage 91.0 (TID 156). 7948 bytes result sent to driver
23/09/24 20:12:38 INFO CoarseGrainedExecutorBackend: Got assigned task 159
23/09/24 20:12:38 INFO Executor: Running task 1.0 in stage 92.0 (TID 159)
23/09/24 20:12:38 INFO CoarseGrainedExecutorBackend: Got assigned task 162
23/09/24 20:12:38 INFO CoarseGrainedExecutorBackend: Got assigned task 165
23/09/24 20:12:38 INFO Executor: Running task 4.0 in stage 92.0 (TID 162)
23/09/24 20:12:38 INFO CoarseGrainedExecutorBackend: Got assigned task 168
23/09/24 20:12:38 INFO CoarseGrainedExecutorBackend: Got assigned task 171
23/09/24 20:12:38 INFO Executor: Running task 10.0 in stage 92.0 (TID 168)
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Updating epoch to 16 and clearing cache
23/09/24 20:12:38 INFO TorrentBroadcast: Started reading broadcast variable 127 with 1 pieces (estimated total size 4.0 MiB)
23/09/24 20:12:38 INFO Executor: Running task 7.0 in stage 92.0 (TID 165)
23/09/24 20:12:38 INFO Executor: Running task 13.0 in stage 92.0 (TID 171)
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_127_piece0 stored as bytes in memory (estimated size 129.5 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO TorrentBroadcast: Reading broadcast variable 127 took 6 ms
23/09/24 20:12:38 INFO MemoryStore: Block broadcast_127 stored as values in memory (estimated size 379.7 KiB, free 5.0 GiB)
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Don't have map outputs for shuffle 15, fetching them
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Doing the fetch; tracker endpoint = NettyRpcEndpointRef(spark://MapOutputTracker@static.154.188.161.5.clients.your-server.de:39929)
23/09/24 20:12:38 INFO MapOutputTrackerWorker: Got the output locations
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Getting 2 (608.0 B) non-empty blocks including 1 (304.0 B) local and 1 (304.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Getting 2 (608.0 B) non-empty blocks including 1 (304.0 B) local and 1 (304.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 35.504156 ms
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Getting 2 (580.0 B) non-empty blocks including 0 (0.0 B) local and 2 (580.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Getting 2 (580.0 B) non-empty blocks including 0 (0.0 B) local and 2 (580.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 8.417668 ms
23/09/24 20:12:38 INFO CodeGenerator: Code generated in 7.826574 ms
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Getting 2 (608.0 B) non-empty blocks including 0 (0.0 B) local and 2 (608.0 B) host-local and 0 (0.0 B) remote blocks
23/09/24 20:12:38 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using user defined output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO FileOutputCommitter: File Output Committer Algorithm version is 1
23/09/24 20:12:38 INFO SQLHadoopMapReduceCommitProtocol: Using output committer class org.apache.parquet.hadoop.ParquetOutputCommitter
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO CodecConfig: Compression: SNAPPY
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Dictionary is on
23/09/24 20:12:38 INFO ParquetOutputFormat: Validation is off
23/09/24 20:12:38 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 20:12:38 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 20:12:38 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 20:12:38 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 20:12:38 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Dictionary is on
23/09/24 20:12:38 INFO ParquetOutputFormat: Validation is off
23/09/24 20:12:38 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 20:12:38 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 20:12:38 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 20:12:38 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 20:12:38 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Dictionary is on
23/09/24 20:12:38 INFO ParquetOutputFormat: Validation is off
23/09/24 20:12:38 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 20:12:38 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 20:12:38 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 20:12:38 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 20:12:38 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Dictionary is on
23/09/24 20:12:38 INFO ParquetOutputFormat: Validation is off
23/09/24 20:12:38 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 20:12:38 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 20:12:38 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 20:12:38 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 20:12:38 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet block size to 134217728
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Parquet dictionary page size to 1048576
23/09/24 20:12:38 INFO ParquetOutputFormat: Dictionary is on
23/09/24 20:12:38 INFO ParquetOutputFormat: Validation is off
23/09/24 20:12:38 INFO ParquetOutputFormat: Writer version is: PARQUET_1_0
23/09/24 20:12:38 INFO ParquetOutputFormat: Maximum row group padding size is 8388608 bytes
23/09/24 20:12:38 INFO ParquetOutputFormat: Page size checking is: estimated
23/09/24 20:12:38 INFO ParquetOutputFormat: Min row count for page size check is: 100
23/09/24 20:12:38 INFO ParquetOutputFormat: Max row count for page size check is: 10000
23/09/24 20:12:38 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 20:12:38 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 20:12:38 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 20:12:38 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 20:12:38 INFO ParquetWriteSupport: Initialized Parquet WriteSupport with Catalyst schema:
{
  "type" : "struct",
  "fields" : [ {
    "name" : "z_zid",
    "type" : "long",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_cluster",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "z_prediction",
    "type" : "double",
    "nullable" : false,
    "metadata" : {
      "ml_attr" : {
        "type" : "nominal",
        "num_vals" : 2
      }
    }
  }, {
    "name" : "z_score",
    "type" : "double",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_isMatch",
    "type" : "integer",
    "nullable" : false,
    "metadata" : { }
  }, {
    "name" : "fname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "lname",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "stNo",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add1",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "add2",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "city",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "areacode",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "state",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "dob",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "ssn",
    "type" : "string",
    "nullable" : true,
    "metadata" : { }
  }, {
    "name" : "z_source",
    "type" : "string",
    "nullable" : false,
    "metadata" : { }
  } ]
}
and corresponding Parquet message type:
message spark_schema {
  required int64 z_zid;
  required binary z_cluster (UTF8);
  required double z_prediction;
  optional double z_score;
  required int32 z_isMatch;
  optional binary fname (UTF8);
  optional binary lname (UTF8);
  optional binary stNo (UTF8);
  optional binary add1 (UTF8);
  optional binary add2 (UTF8);
  optional binary city (UTF8);
  optional binary areacode (UTF8);
  optional binary state (UTF8);
  optional binary dob (UTF8);
  optional binary ssn (UTF8);
  required binary z_source (UTF8);
}

       
23/09/24 20:12:38 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 20:12:38 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 20:12:38 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 20:12:38 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 20:12:38 INFO CodecPool: Got brand-new compressor [.snappy]
23/09/24 20:12:38 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 439
23/09/24 20:12:38 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 445
23/09/24 20:12:38 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 428
23/09/24 20:12:38 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 427
23/09/24 20:12:38 INFO InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 354
23/09/24 20:12:39 INFO FileOutputCommitter: Saved output of task 'attempt_202309242012384652336026329800635_0092_m_000007_165' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242012384652336026329800635_0092_m_000007
23/09/24 20:12:39 INFO FileOutputCommitter: Saved output of task 'attempt_202309242012388833230362668857158_0092_m_000013_171' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242012388833230362668857158_0092_m_000013
23/09/24 20:12:39 INFO SparkHadoopMapRedUtil: attempt_202309242012384652336026329800635_0092_m_000007_165: Committed
23/09/24 20:12:39 INFO FileOutputCommitter: Saved output of task 'attempt_202309242012381765300316199661560_0092_m_000001_159' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242012381765300316199661560_0092_m_000001
23/09/24 20:12:39 INFO FileOutputCommitter: Saved output of task 'attempt_202309242012383829576489690917426_0092_m_000010_168' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242012383829576489690917426_0092_m_000010
23/09/24 20:12:39 INFO SparkHadoopMapRedUtil: attempt_202309242012388833230362668857158_0092_m_000013_171: Committed
23/09/24 20:12:39 INFO SparkHadoopMapRedUtil: attempt_202309242012381765300316199661560_0092_m_000001_159: Committed
23/09/24 20:12:39 INFO SparkHadoopMapRedUtil: attempt_202309242012383829576489690917426_0092_m_000010_168: Committed
23/09/24 20:12:39 INFO Executor: Finished task 7.0 in stage 92.0 (TID 165). 10148 bytes result sent to driver
23/09/24 20:12:39 INFO Executor: Finished task 1.0 in stage 92.0 (TID 159). 10191 bytes result sent to driver
23/09/24 20:12:39 INFO FileOutputCommitter: Saved output of task 'attempt_202309242012387898842344152097089_0092_m_000004_162' to file:/root/zingg/models/100/trainingData/unmarked/_temporary/0/task_202309242012387898842344152097089_0092_m_000004
23/09/24 20:12:39 INFO SparkHadoopMapRedUtil: attempt_202309242012387898842344152097089_0092_m_000004_162: Committed
23/09/24 20:12:39 INFO Executor: Finished task 4.0 in stage 92.0 (TID 162). 10148 bytes result sent to driver
23/09/24 20:12:39 INFO Executor: Finished task 10.0 in stage 92.0 (TID 168). 10191 bytes result sent to driver
23/09/24 20:12:39 INFO Executor: Finished task 13.0 in stage 92.0 (TID 171). 10191 bytes result sent to driver
23/09/24 20:12:39 INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
23/09/24 20:12:39 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
tdown