Spark broadcast class message java.lang.stackoverflowerror
Background: a 167m tree class needs to be broadcast, so the stack memory is not enough
solution: add in spark submit: (at present, due to the small data level, it runs in local mode)
spark-submit \$
--class bp_beauty_op.beauty_op \$
--master local[*] \$
--driver-java-options "-Xss256m" \$
test-1.0-SNAPSHOT.jar
Or add spark.Driver.Extrajavaoptions = “- xss256m”
in spark-defaults.conf to test the running.
Read More:
- JPA @ManyToMany java.lang.StackOverflowError: null [How to Solve]
- NestedServletException, Handler dispatch failed; nested exception is java.lang.StackOverflowError [Solved]
- spark Program Error: ERROR01——java.lang.NullPointerException
- [Solved] Spark Error: org.apache.spark.SparkException: A master URL must be set in your configuration
- Java.lang.AbstractMethodError: org.mybatis.spring.transaction.SpringManagedTransaction.getTimeout()Ljava/lang/Integer; error resolution
- [Solved] .java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a runn
- Hive ERROR Failed with exception java.io.IOException:java.lang.IllegalArgumentException
- [Solved] java.lang.reflect.InaccessibleObjectException: Unable to make protected java.net.http.HttpRequest()…
- [Solved] Response Export error on submit request on future invoke, java.lang.OutOfMemoryError: Java heap space
- Idea Error: java: java.lang.OutOfMemoryError: GC overhead limit exceeded
- [How to Solve] java.lang.IllegalArgumentException: Request header is too large
- [Solved] jhat Analyzes dump File Error: java.lang.OutOfMemoryError
- [Solved] Hadoop error java.lang.nosuchmethoderror
- Tomcat startup error: java.lang.NoClassDefFoundError
- How to Solve spark Writes Files to odps Error
- Kettle Error Caused by: java.lang.ArrayIndexOutOfBoundsException
- [Solved] Weblogic startup error: java.lang.NoClassDefFoundError
- Spark ERROR client.TransportResponseHandler: Still have 1 requests outstanding when connection from
- [Solved] Android Error: java.lang.IllegalStateException: Not allowed to start service Intent