The spark task reports an error (though the calculated result is correct).
21/04/29 14:38:38 ERROR client.TransportResponseHandler: Still have 1 requests outstanding when connection from /172.16.4.156:37528 is closed
21/04/29 14:38:39 ERROR cluster.YarnScheduler: Lost executor 3 on cdh-slave1.test.com: Container marked as failed: container_1618397085347_0529_01_000004 on host: cdh-slave1.test.com. Exit status: 137. Diagnostics: [2021-04-29 14:38:39.112]Container killed on request. Exit code is 137
[2021-04-29 14:38:39.117]Container exited with a non-zero exit code 137.
[2021-04-29 14:38:39.119]Killed by external signal
Looking at the code, I found that hive is not used in the spark program, but hive support is set during spark initialization.
The program works fine after deleting this line of code.
Read More:
- [Solved] Spark Error: org.apache.spark.SparkException: A master URL must be set in your configuration
- Springcloud configuration center client 3355 startup Error: error on get request for“ http://localhost:8761/eureka/apps/ “: Connection
- Git: “error: RPC failed; curl 18 transfer closed with outstanding read data remaining”
- Spark Error: java.lang.StackOverflowError [How to Solve]
- Ajax prevents page requests from reporting errors
- Spark-shell startup error: Yarn application has already ended! It might have been killed or unable to launch application master
- [Solved] removeerror: ‘requests’ is a dependency of CONDA and cannot be removed from
- [Solved] org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow
- [Solved] error: value createSchemaRDD is not a member of org.apache.spark.sql.SQLContext
- How to Solve Spark Writes Hudi Error
- [Solved] Error: failed to create deliver client: orderer client failed to connect to orderer.example.com:7050
- [Solved] Error: Could not open client transport with JDBC Uri
- How to Solve spark Writes Files to odps Error
- [Solved] Fabric Start Network Error: Error: error getting endorser client for channel: endorser client failed to connect to
- WordPress update failed 429 too many requests (How to Fix)
- [Solved] Exception in thread “main“ org.apache.spark.SparkException: When running with master ‘yarn‘ either
- IDEA: How to Solve spark source code Modified Error
- [Solved] spark sql Error: Can‘t zip RDDs with unequal numbers of partitions
- spark Program Error: ERROR01——java.lang.NullPointerException
- [Solved] Sparksql error: Exception in thread “main” org.apache.spark.sql.catalyst.errors.package$TreeNodeException