Tag Archives: java.lang.StackOverflowError

Spark Error: java.lang.StackOverflowError [How to Solve]

Spark broadcast class message java.lang.stackoverflowerror

Background: a 167m tree class needs to be broadcast, so the stack memory is not enough
solution: add in spark submit: (at present, due to the small data level, it runs in local mode)

spark-submit \$
--class bp_beauty_op.beauty_op \$
--master local[*] \$
--driver-java-options "-Xss256m" \$
test-1.0-SNAPSHOT.jar

Or add spark.Driver.Extrajavaoptions = “- xss256m”
in spark-defaults.conf to test the running.