How to connect pyspark with PostgreSQL
Method 1: the PostgreSQL jar package placed in the spark2/jars path does not work. (error)
Method 2: setting spark. Conf. set in spark. Conf. set (‘spark. Jars’, ‘/ usr/HDP/3.0.1.0-187/spark2/jars/postgresql-42.2.5. Jar’) does not work. (error)
Solution
When starting pyspark, add parameters after it
pyspark –jars / usr/hdp/3.0.1.0-187/spark2/jars/postgresql-42.2.5.jar
That’s fine
Read More:
- Used by: Java. Lang. classnotfoundexception: org. PostgreSQL. Driver
- java.lang.ClassNotFoundException : com.mysql.jdbc . driver project cannot connect to database, exception, lack of MySQL driver
- Mybatis exception error setting driver on unpoleddatasource. Cause: java.lang.ClassNotFoundException :
- [BUG-mybatis-plus] org.postgresql.util.PSQLException: ERROR: syntax error at or near “current_date“
- PHP connection PostgreSQL error call to undefined function PG_ connect()
- Solution in idea java.lang.ClassNotFoundException : org.springframework.web . context.ContextLoaderListener
- 【error】postgresql relation does not exist
- pg_ctl: no database directory specified and environment variable PGDATA unset ， centos 7 postgreSQL
- [PostgreSQL tutorial] · PostgreSQL error prompt: “error 42501” or “permission denied”
- java.lang.NoSuchMethodError: org.json.JSONObject.put(Java/lang/String; Java/util/Collection;) “209151;”
- An error occurred during win10 PostgreSQL installation: “failed to load SQL modules into the database cluster“
- Win10 installation PostgreSQL error running… And installation process
- [unity] Android package error java.lang.ClassNotFoundException Did not find class com.unity3d . player.MainActivity
- PostgreSQL sets the default value of the field to the current year
- Cause: java.sql.SQLException: Error setting driver on UnpooledDataSource. Cause: java.lang.ClassNot
- postgresql FATAL：no pg_hba.conf entry for host “192….“
- Caused by: java.lang.ClassNotFoundException: com.alibaba.nacos.api.naming.NamingMaintainService
- pyspark : NameError: name ‘spark’ is not defined
- Error in idea compilation: java.lang.OutOfMemoryError Java heap space and java.lang.StackOverflowError
- java.lang.NoClassDefFoundError: org/junit/platform/launcher/TestExecutionListener