Error when running the project to connect to Spark:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
22/10/08 21:02:10 INFO SparkContext: Running Spark version 3.0.0
22/10/08 21:02:10 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:380)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:120)
at test.wyh.wordcount.TestWordCount$.main(TestWordCount.scala:10)
at test.wyh.wordcount.TestWordCount.main(TestWordCount.scala)
22/10/08 21:02:10 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:380)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:120)
at test.wyh.wordcount.TestWordCount$.main(TestWordCount.scala:10)
at test.wyh.wordcount.TestWordCount.main(TestWordCount.scala)
Process finished with exit code 1
Solution:
Configure the following parameters:
-Dspark.master=local[*]
Restart IDEA.
Read More:
- [Solved] Exception in thread “main“ org.apache.spark.SparkException: When running with master ‘yarn‘ either
- [Solved] org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow
- [Solved] ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Could not pars
- [Solved] Sparksql error: Exception in thread “main” org.apache.spark.sql.catalyst.errors.package$TreeNodeException
- [Solved] error: value createSchemaRDD is not a member of org.apache.spark.sql.SQLContext
- Spark-shell startup error: Yarn application has already ended! It might have been killed or unable to launch application master
- [Solved] Spark Error: ERROR StatusLogger No log4j2 configuration file found
- How to Solve Spark Writes Hudi Error
- Spark Error: java.lang.StackOverflowError [How to Solve]
- How to Solve spark Writes Files to odps Error
- spark Program Error: ERROR01——java.lang.NullPointerException
- Spark ERROR client.TransportResponseHandler: Still have 1 requests outstanding when connection from
- HBase shell Find ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
- [Solved] HBase Error: ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
- IDEA: How to Solve spark source code Modified Error
- Spark-SQL Error: A JNI error has occurred, please check your installation and try again Exceptio
- [Solved] Flume Error: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
- apache 2.2 configuration error: couldn’t perform authentication. AuthType not set!: /
- [Solved] hbase Create Sheet Error: ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
- [Solved] hbase Create Sheet Error: ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing