Spark broadcast class message java.lang.stackoverflowerror
Background: a 167m tree class needs to be broadcast, so the stack memory is not enough
solution: add in spark submit: (at present, due to the small data level, it runs in local mode)
spark-submit \$
--class bp_beauty_op.beauty_op \$
--master local[*] \$
--driver-java-options "-Xss256m" \$
test-1.0-SNAPSHOT.jar
Or add spark.Driver.Extrajavaoptions = “- xss256m”
in spark-defaults.conf to test the running.