Tag Archives: sqoop

Sqoop exports hive data to MySQL Error [How to Solve]

The following error is reported when exporting hive data from sqoop to MySQL

Export job failed!
	at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
	at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

This problem is caused by the field type problem or table structure problem between MySQL and hive (the fields are different or the table structure is different).

Solution: modify the type in MySQL table to be the same as that in hive table or modify the table structure

[Solved] Sqoop Mysqltohive error: Error: java.lang.RuntimeException: java.lang.RuntimeException…

Problem Description
Execute statementbin/sqoop import --connect jdbc:mysql://localhost:3306/gdcmxy --username root --password root --table 2019bigdata --fields-terminated-by '\t' --delete-target-dir --num-mappers 1 --hive-import --hive-database gdcmxy --hive-table 2019bigdata

Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

After seeing the error report, it was preliminarily judged that the main problem was the database connection problem. At first, I thought the firewall was not turned off, but considering that hadoo had certainly turned off the firewall before, it should have nothing to do with the firewall.

Cluster environment:

Solution:

Change the localhost of the execution statement to the IP address of the local machine bin/sqoop import -- connect JDBC: mysql://192.168.112.81:3306/gdcmxy --Username root -- password root -- table 2019bigdata -- fields terminated by '\ t' -- delete target dir -- num mappers 1 -- hive import -- hive database gdcmxy -- hive table 2019bigdata

successfully imported

Sqoop connection gbase data error [How to Solve]

1. When sqoop accesses the gbase database through the command, the connection error is as follows:;

Just add the — driver parameter to the command

sqoop list-tables -connect jdbc: gbase://10.100.111.48:8010/dm -Driver com.gbase.jdbc.driver – username gbase – password gbase2010531
run successfully!

[Solved] Sqoop Error: ERROR tool.ImportTool: Import failed: java.io.IOException 

21/11/08 12:13:10 ERROR tool.ImportTool: Import failed: java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
        at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:143)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:108)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:101)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1311)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
        at org.apache.hadoop.mapreduce.Job.connect(Job.java:1306)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1335)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1359)
        at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:200)
        at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:173)
        at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:270)
        at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
        at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

The above error occurs when running the sqoop script (the content of the script is to import MySQL data into HDFS). It is found that there is a lack of dependency. I still report an error after copying the two jar packages hadoop-mapreduce-client-common-2.8.5.jar and hadoop-mapreduce-client-core-2.8.5.jar to the Lib directory of sqoop, Then I copied all the jar packages in the hadoop-2.8.5/share/hadoop/mapreduce directory of Hadoop to solve the problem and run the script successfully. It was simple and violent.

Sqoop Error: Can‘t parse input data: ‘\N‘ [How to Solve]

Sqoop reports an error can’t parse input data: ‘\ n’

Problem Description: use sqoop to push the data in hive to Oracle. The program reports an error: can’t parse input data: ‘\ n’
. Null value cannot be recognized when derivative.

The reason for this exception is that when the table in hive is a partitioned table, the added fields do not use cascade, such as:

alter table app.suntest_user add  columns(name string comment 'name') cascade

When does not use the cascade keyword, hive does not update the history partition definition, resulting in differences between the columns in the table and the target table when transmitting history data.

For example:

False table suntest_ User created partition M08 in August, added a name column in September, and created partition M09, and added a name column to the table definition in the target database (Oracle). There is no problem exporting M09 partition data through the sqoop script, but an error will be reported when exporting M08 data: can’t parse input data: ‘\ n’

Solution:

You can rebuild the partition in August , or create a table for derivatives. The following is a table creation statement.

----- Create a table corresponding to the partition and use the temporary derivative 
	create table app.test_suntest_user as 
	select id,name from app.suntest_user where partition_month='M08' 

----Rebuild partition statement 
	--Rename partition 
	alter table app.suntest_user partition (partition_month='M08') rename to partition (partition_month='M08bak');
	--New Partition 
	insert overwrite table app.suntest_user partition(partition_month='M08')
	select id ,name from app.suntest_user where partition_month='M08bak';

The sqoop statement is as follows:

sqoop export --connect "jdbc:oracle:thin:@**********" \
--username "abc" \
--password "123" \
--input-null-string '\\N'  --input-null-non-string '\\N' \
--table "SUNTEST_USER" \
--export-dir /user/hive/warehouse/app/test_suntest_user\
--input-fields-terminated-by '\001' \

Sqoop Error: Error during export:Export job failed [How to Solve]

Sqoop：Error during export:Export job failed !
sqoop export data to mysql exception, usually are very normal today can not, there is nothing to report error information, is not very depressed.
21/08/25 15:03:05 ERROR tool.ExportTool: Error during export:
Export job failed!
at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:439)
at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

Several possibilities
1, conversion: the null problem requires conversion
–input-fields-terminated-by ‘\001’
–input-null-string ‘\N’
–input-null-non-string ‘\N’
2, table structure inconsistencies, hive table fields mysql table must have, mysql fields can be more not less
3, mysql table field length to less, change the structure on the good

Spoop export hive to MySQL

SQOOP Export: mysql> create a Hive table (????)
Like this:

 
I have modified it once before, and the method works together. However, when the server is changed and the MySQL library is newly created, method one fails to work, and method two takes effect.
 
Method one:
Create a new query window, or use the command to enter mysql> , the input

set character_set_database=utf8;
set character_set_server=utf8;

Show variables like ‘character%’; View the results:

SQOOP script with:?useUnicode=true& characterEncoding=utf-8
Import the data again without scrambling.
 
Method 2:
Modify the system file: /etc/my.cnf

vim /etc/my.cnf


Text:

init_connect=’SET collation_connection = utf8_unicode_ci’
init_connect=’SET NAMES utf8′
character-set-server=utf8
collation-server=utf8_unicode_ci
skip-character-set-client-handshake

 
 
 
 
Directing to /bin/systemctl Restart mysqld. Service (1) : Directing to /bin/systemctl Restart mysqld.
Sqoop plus?useUnicode=true& CharacterEncoding = UTF-8, Chinese garbled code problem resolved.
 
In the first method, the set modifies the configuration and restarts MySQL. This is probably the first reason why the configuration change does not work.
 
 

Sqoop import error communications link failure

1. Problem description

The

sqoop list-tables tool works, but there is a problem with the import tool. Strange!! 

[hadoop@master ~]$ sqoop import  --username test --password test  --connect jdbc:mysql://172.1.1.96:3306/test  -m 1 --delete-target-dir --table student_exam_score_20170802  --fields-terminated-by '\t'  --hive-import   --hive-database bond_edu --hive-table student_exam_score_20170802
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/11/08 11:31:13 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/11/08 11:31:13 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/11/08 11:31:13 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/11/08 11:31:13 INFO tool.CodeGenTool: Beginning code generation
18/11/08 11:31:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_exam_score_20170802` AS t LIMIT 1
18/11/08 11:31:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_exam_score_20170802` AS t LIMIT 1
18/11/08 11:31:14 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/hadoop-2.8.1/share/hadoop/mapreduce
Note: /tmp/sqoop-hadoop/compile/98d200728b6a446d170eba746e2c9a4c/student_exam_score_20170802.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/11/08 11:31:18 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/98d200728b6a446d170eba746e2c9a4c/student_exam_score_20170802.jar
18/11/08 11:31:19 INFO tool.ImportTool: Destination directory student_exam_score_20170802 is not present, hence not deleting.
18/11/08 11:31:19 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/11/08 11:31:19 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/11/08 11:31:19 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/11/08 11:31:19 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/11/08 11:31:19 INFO mapreduce.ImportJobBase: Beginning import of student_exam_score_20170802
18/11/08 11:31:20 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/11/08 11:31:20 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/11/08 11:31:20 INFO client.RMProxy: Connecting to ResourceManager at master/10.0.1.118:18040
18/11/08 11:31:24 INFO db.DBInputFormat: Using read commited transaction isolation
18/11/08 11:31:24 INFO mapreduce.JobSubmitter: number of splits:1
18/11/08 11:31:24 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1541640011445_0018
18/11/08 11:31:25 INFO impl.YarnClientImpl: Submitted application application_1541640011445_0018
18/11/08 11:31:25 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1541640011445_0018/
18/11/08 11:31:25 INFO mapreduce.Job: Running job: job_1541640011445_0018
18/11/08 11:31:37 INFO mapreduce.Job: Job job_1541640011445_0018 running in uber mode : false
18/11/08 11:31:37 INFO mapreduce.Job:  map 0% reduce 0%
18/11/08 11:31:45 INFO mapreduce.Job: Task Id : attempt_1541640011445_0018_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2222)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2017)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more
Caused by: java.net.SocketException: Network is unreachable (connect failed)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300)
        ... 26 more

18/11/08 11:31:51 INFO mapreduce.Job: Task Id : attempt_1541640011445_0018_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2222)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2017)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more
Caused by: java.net.SocketException: Network is unreachable (connect failed)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300)
        ... 26 more

18/11/08 11:31:57 INFO mapreduce.Job: Task Id : attempt_1541640011445_0018_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2222)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2017)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more
Caused by: java.net.SocketException: Network is unreachable (connect failed)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300)
        ... 26 more

18/11/08 11:32:05 INFO mapreduce.Job:  map 100% reduce 0%
18/11/08 11:32:06 INFO mapreduce.Job: Job job_1541640011445_0018 failed with state FAILED due to: Task failed task_1541640011445_0018_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

18/11/08 11:32:06 INFO mapreduce.Job: Counters: 8
        Job Counters 
                Failed map tasks=4
                Launched map tasks=4
                Other local map tasks=4
                Total time spent by all maps in occupied slots (ms)=20616
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=20616
                Total vcore-seconds taken by all map tasks=20616
                Total megabyte-seconds taken by all map tasks=21110784
18/11/08 11:32:06 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
18/11/08 11:32:06 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 46.2311 seconds (0 bytes/sec)
18/11/08 11:32:06 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
18/11/08 11:32:06 INFO mapreduce.ImportJobBase: Retrieved 0 records.
18/11/08 11:32:06 ERROR tool.ImportTool: Error during import: Import job failed!
[hadoop@master ~]$ sqoop list-tables  --username test --password test  --connect jdbc:mysql://172.1.1.96:3306/test 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/11/08 11:35:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/11/08 11:35:27 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/11/08 11:35:27 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
a_jw_coursejoin_detail_f
a_test
bk_fee_common
bk_student_rect
bk_subjectdet
bonus
dept
emp
exam_ticket
gg
gg_tets
id
incre_table
p_bk_real_lession_cost_detail
registration
salgrade
student_exam_score_20170802
student_exam_score_20170803
student_exam_score_20170906
student_exam_term
t
t_card_package
t_dictionary
t_sms_send
t_time
t_tracelog_cgi
t_tracelog_cgi_20180920
t_tracelog_cgi_20180921
t_tracelog_cgi_tmp
tbl_name
test_1
test_2
test_3
test_4
test_5
test_x
treenodes
tt1
tt2
tt3
tt4
user
z
[hadoop@master ~]$

2. Problem solving

2.1 mysql local (sqoop local)

this query data:

1. Change the url localhost to IP (just configure /etc/hosts file)

sqoop  import  --username test --password test  --connect jdbc:mysql://127.0.0.1:3306/test?autoReconnect=true  --delete-target-dir --table t --fields-terminated-by '\t'  -m 1 --hive-import   --hive-database test --hive-table t

2. Empower the user (ignore if you have permission) 

 grant all privileges on  test  to test@'%' IDENTIFIED BY 'test' WITH GRANT OPTION;
 flush PRIVILEGES;

Locahost import failed: 

[hadoop@master ~]$ sqoop  import  --username test --password test  --connect jdbc:mysql://127.0.0.1:3306/test?autoReconnect=true  --delete-target-dir --table t --fields-terminated-by '\t'  -m 1 --hive-import   --hive-database test --hive-table t         
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/11/08 11:53:58 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/11/08 11:53:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/11/08 11:53:58 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/11/08 11:53:58 INFO tool.CodeGenTool: Beginning code generation
18/11/08 11:53:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `t` AS t LIMIT 1
18/11/08 11:53:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `t` AS t LIMIT 1
18/11/08 11:53:59 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/hadoop-2.8.1/share/hadoop/mapreduce
Note: /tmp/sqoop-hadoop/compile/910a58c86a0bd49d993fb3c8e9860e5f/t.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/11/08 11:54:01 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/910a58c86a0bd49d993fb3c8e9860e5f/t.jar
18/11/08 11:54:03 INFO tool.ImportTool: Destination directory t is not present, hence not deleting.
18/11/08 11:54:03 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/11/08 11:54:03 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/11/08 11:54:03 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/11/08 11:54:03 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/11/08 11:54:03 INFO mapreduce.ImportJobBase: Beginning import of t
18/11/08 11:54:03 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/11/08 11:54:03 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/11/08 11:54:03 INFO client.RMProxy: Connecting to ResourceManager at master/10.0.1.118:18040
18/11/08 11:54:07 INFO db.DBInputFormat: Using read commited transaction isolation
18/11/08 11:54:07 INFO mapreduce.JobSubmitter: number of splits:1
18/11/08 11:54:07 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1541640011445_0023
18/11/08 11:54:07 INFO impl.YarnClientImpl: Submitted application application_1541640011445_0023
18/11/08 11:54:07 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1541640011445_0023/
18/11/08 11:54:07 INFO mapreduce.Job: Running job: job_1541640011445_0023
18/11/08 11:54:19 INFO mapreduce.Job: Job job_1541640011445_0023 running in uber mode : false
18/11/08 11:54:19 INFO mapreduce.Job:  map 0% reduce 0%
18/11/08 11:54:31 INFO mapreduce.Job: Task Id : attempt_1541640011445_0023_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.Util.getInstance(Util.java:408)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2097)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2022)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2038)
        ... 24 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300)
        ... 26 more

18/11/08 11:54:41 INFO mapreduce.Job: Task Id : attempt_1541640011445_0023_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.Util.getInstance(Util.java:408)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2097)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2022)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more
Caused by: java.sql.SQLException: Access denied for user 'test'@'localhost' (using password: YES)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:964)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:873)
        at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1710)
        at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2191)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2038)
        ... 24 more

18/11/08 11:54:51 INFO mapreduce.Job: Task Id : attempt_1541640011445_0023_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.Util.getInstance(Util.java:408)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:897)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:886)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2097)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2022)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:779)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:389)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more
Caused by: java.sql.SQLException: Access denied for user 'test'@'localhost' (using password: YES)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:964)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:873)
        at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1710)
        at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2191)
        at com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2038)
        ... 24 more

18/11/08 11:55:03 INFO mapreduce.Job:  map 100% reduce 0%
18/11/08 11:55:03 INFO mapreduce.Job: Job job_1541640011445_0023 failed with state FAILED due to: Task failed task_1541640011445_0023_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

18/11/08 11:55:03 INFO mapreduce.Job: Counters: 8
        Job Counters 
                Failed map tasks=4
                Launched map tasks=4
                Other local map tasks=4
                Total time spent by all maps in occupied slots (ms)=36276
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=36276
                Total vcore-seconds taken by all map tasks=36276
                Total megabyte-seconds taken by all map tasks=37146624
18/11/08 11:55:03 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
18/11/08 11:55:03 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 60.1829 seconds (0 bytes/sec)
18/11/08 11:55:03 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
18/11/08 11:55:03 INFO mapreduce.ImportJobBase: Retrieved 0 records.
18/11/08 11:55:03 ERROR tool.ImportTool: Error during import: Import job failed!
[hadoop@master ~]$

IP import successful: 

[hadoop@master ~]$ sqoop  import  --username test --password test  --connect jdbc:mysql://10.0.1.118:3306/test?autoReconnect=true  --delete-target-dir --table t --fields-terminated-by '\t'  -m 1 --hive-import   --hive-database test --hive-table t  
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/11/08 11:51:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/11/08 11:51:29 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/11/08 11:51:30 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/11/08 11:51:30 INFO tool.CodeGenTool: Beginning code generation
18/11/08 11:51:30 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `t` AS t LIMIT 1
18/11/08 11:51:30 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `t` AS t LIMIT 1
18/11/08 11:51:30 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/hadoop-2.8.1/share/hadoop/mapreduce
Note: /tmp/sqoop-hadoop/compile/8dc5c288374d370426b87eca124b15e0/t.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/11/08 11:51:33 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/8dc5c288374d370426b87eca124b15e0/t.jar
18/11/08 11:51:34 INFO tool.ImportTool: Destination directory t deleted.
18/11/08 11:51:34 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/11/08 11:51:34 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/11/08 11:51:34 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/11/08 11:51:34 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/11/08 11:51:34 INFO mapreduce.ImportJobBase: Beginning import of t
18/11/08 11:51:34 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/11/08 11:51:34 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/11/08 11:51:34 INFO client.RMProxy: Connecting to ResourceManager at master/10.0.1.118:18040
18/11/08 11:51:37 INFO db.DBInputFormat: Using read commited transaction isolation
18/11/08 11:51:37 INFO mapreduce.JobSubmitter: number of splits:1
18/11/08 11:51:38 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1541640011445_0022
18/11/08 11:51:38 INFO impl.YarnClientImpl: Submitted application application_1541640011445_0022
18/11/08 11:51:38 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1541640011445_0022/
18/11/08 11:51:38 INFO mapreduce.Job: Running job: job_1541640011445_0022
18/11/08 11:51:49 INFO mapreduce.Job: Job job_1541640011445_0022 running in uber mode : false
18/11/08 11:51:49 INFO mapreduce.Job:  map 0% reduce 0%
18/11/08 11:51:57 INFO mapreduce.Job:  map 100% reduce 0%
18/11/08 11:51:57 INFO mapreduce.Job: Job job_1541640011445_0022 completed successfully
18/11/08 11:51:57 INFO mapreduce.Job: Counters: 30
        File System Counters
                FILE: Number of bytes read=0
                FILE: Number of bytes written=160855
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=87
                HDFS: Number of bytes written=76
                HDFS: Number of read operations=4
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters 
                Launched map tasks=1
                Other local map tasks=1
                Total time spent by all maps in occupied slots (ms)=5208
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=5208
                Total vcore-seconds taken by all map tasks=5208
                Total megabyte-seconds taken by all map tasks=5332992
        Map-Reduce Framework
                Map input records=10
                Map output records=10
                Input split bytes=87
                Spilled Records=0
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=98
                CPU time spent (ms)=820
                Physical memory (bytes) snapshot=107737088
                Virtual memory (bytes) snapshot=2091540480
                Total committed heap usage (bytes)=18919424
        File Input Format Counters 
                Bytes Read=0
        File Output Format Counters 
                Bytes Written=76
18/11/08 11:51:57 INFO mapreduce.ImportJobBase: Transferred 76 bytes in 22.9794 seconds (3.3073 bytes/sec)
18/11/08 11:51:57 INFO mapreduce.ImportJobBase: Retrieved 10 records.
18/11/08 11:51:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `t` AS t LIMIT 1
18/11/08 11:51:57 INFO hive.HiveImport: Loading uploaded data into Hive
18/11/08 11:52:08 INFO hive.HiveImport: SLF4J: Class path contains multiple SLF4J bindings.
18/11/08 11:52:08 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/home/hadoop/apache-hive-2.1.1/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/11/08 11:52:08 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/11/08 11:52:08 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/11/08 11:52:08 INFO hive.HiveImport: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
18/11/08 11:52:09 INFO hive.HiveImport: 
18/11/08 11:52:09 INFO hive.HiveImport: Logging initialized using configuration in file:/home/hadoop/apache-hive-2.1.1/conf/hive-log4j2.properties Async: true
18/11/08 11:52:18 INFO hive.HiveImport: OK
18/11/08 11:52:18 INFO hive.HiveImport: Time taken: 1.476 seconds
18/11/08 11:52:19 INFO hive.HiveImport: Loading data to table test.t
18/11/08 11:52:19 INFO hive.HiveImport: OK
18/11/08 11:52:19 INFO hive.HiveImport: Time taken: 1.235 seconds
18/11/08 11:52:20 INFO hive.HiveImport: Hive import complete.
18/11/08 11:52:20 INFO hive.HiveImport: Export directory is contains the _SUCCESS file only, removing the directory.

2.2 mysql is not local (remote)

is the hadoop cluster (multiple nodes), many data nodes, must ensure N data node can access mysql database (N depends on the specified sqoop -m parameter N, greater than or equal to the -m) at the back of the parameters, sqoop run at the bottom of the MR, MR run on hadoop cluster, so the hadoop each node must be able to connect the mysql database, mysql available – h * * * * * u * * * * * – p * * * * * test.

my environment:

three virtual machines, two network CARDS each, one private (communication between clusters), one public bridge (communication with outside machines)

The

node

IP

master

10.0.1.118

slave – 1

10.0.1.227

slave – 2

10.0.1.226

slave-1 to connect to remote database :

slave-2 connects to remote database:

import remote 172.1.1.96 data table test.student_exam_score_20170802 to hive bond_edu.student_exam_score_20170802

[hadoop@master ~]$ sqoop import  --username test --password test  --connect jdbc:mysql://172.1.1.96:3306/test  -m 1 --delete-target-dir --table student_exam_score_20170802  --fields-terminated-by '\t'  --hive-import   --hive-database bond_edu --hive-table student_exam_score_20170802
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/11/08 11:22:28 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/11/08 11:22:28 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/11/08 11:22:28 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/11/08 11:22:28 INFO tool.CodeGenTool: Beginning code generation
18/11/08 11:22:29 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_exam_score_20170802` AS t LIMIT 1
18/11/08 11:22:29 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_exam_score_20170802` AS t LIMIT 1
18/11/08 11:22:29 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/hadoop-2.8.1/share/hadoop/mapreduce
Note: /tmp/sqoop-hadoop/compile/47ceab507f5a9890950cf26e7da06fdd/student_exam_score_20170802.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/11/08 11:22:32 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/47ceab507f5a9890950cf26e7da06fdd/student_exam_score_20170802.jar
18/11/08 11:22:34 INFO tool.ImportTool: Destination directory student_exam_score_20170802 is not present, hence not deleting.
18/11/08 11:22:34 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/11/08 11:22:34 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/11/08 11:22:34 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/11/08 11:22:34 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/11/08 11:22:34 INFO mapreduce.ImportJobBase: Beginning import of student_exam_score_20170802
18/11/08 11:22:34 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/11/08 11:22:34 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/11/08 11:22:35 INFO client.RMProxy: Connecting to ResourceManager at master/10.0.1.118:18040
18/11/08 11:22:39 INFO db.DBInputFormat: Using read commited transaction isolation
18/11/08 11:22:39 INFO mapreduce.JobSubmitter: number of splits:1
18/11/08 11:22:39 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1541640011445_0015
18/11/08 11:22:40 INFO impl.YarnClientImpl: Submitted application application_1541640011445_0015
18/11/08 11:22:40 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1541640011445_0015/
18/11/08 11:22:40 INFO mapreduce.Job: Running job: job_1541640011445_0015
18/11/08 11:22:53 INFO mapreduce.Job: Job job_1541640011445_0015 running in uber mode : false
18/11/08 11:22:53 INFO mapreduce.Job:  map 0% reduce 0%
18/11/08 11:23:03 INFO mapreduce.Job:  map 100% reduce 0%
18/11/08 11:23:03 INFO mapreduce.Job: Job job_1541640011445_0015 completed successfully
18/11/08 11:23:03 INFO mapreduce.Job: Counters: 30
        File System Counters
                FILE: Number of bytes read=0
                FILE: Number of bytes written=161078
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=87
                HDFS: Number of bytes written=420640
                HDFS: Number of read operations=4
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters 
                Launched map tasks=1
                Other local map tasks=1
                Total time spent by all maps in occupied slots (ms)=6613
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=6613
                Total vcore-seconds taken by all map tasks=6613
                Total megabyte-seconds taken by all map tasks=6771712
        Map-Reduce Framework
                Map input records=3956
                Map output records=3956
                Input split bytes=87
                Spilled Records=0
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=92
                CPU time spent (ms)=1370
                Physical memory (bytes) snapshot=106192896
                Virtual memory (bytes) snapshot=2092593152
                Total committed heap usage (bytes)=17985536
        File Input Format Counters 
                Bytes Read=0
        File Output Format Counters 
                Bytes Written=420640
18/11/08 11:23:03 INFO mapreduce.ImportJobBase: Transferred 410.7812 KB in 28.5838 seconds (14.3711 KB/sec)
18/11/08 11:23:03 INFO mapreduce.ImportJobBase: Retrieved 3956 records.
18/11/08 11:23:03 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student_exam_score_20170802` AS t LIMIT 1
18/11/08 11:23:03 INFO hive.HiveImport: Loading uploaded data into Hive
18/11/08 11:23:19 INFO hive.HiveImport: SLF4J: Class path contains multiple SLF4J bindings.
18/11/08 11:23:19 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/home/hadoop/apache-hive-2.1.1/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/11/08 11:23:19 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/11/08 11:23:19 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
18/11/08 11:23:19 INFO hive.HiveImport: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
18/11/08 11:23:22 INFO hive.HiveImport: 
18/11/08 11:23:22 INFO hive.HiveImport: Logging initialized using configuration in file:/home/hadoop/apache-hive-2.1.1/conf/hive-log4j2.properties Async: true
18/11/08 11:23:38 INFO hive.HiveImport: OK
18/11/08 11:23:38 INFO hive.HiveImport: Time taken: 4.372 seconds
18/11/08 11:23:39 INFO hive.HiveImport: Loading data to table bond_edu.student_exam_score_20170802
18/11/08 11:23:40 INFO hive.HiveImport: OK
18/11/08 11:23:40 INFO hive.HiveImport: Time taken: 1.812 seconds
18/11/08 11:23:41 INFO hive.HiveImport: Hive import complete.
18/11/08 11:23:41 INFO hive.HiveImport: Export directory is contains the _SUCCESS file only, removing the directory.
[hadoop@master ~]$

summary:

1. Sqoop remote import of mysql data into hive must ensure that N data nodes can connect to the remote database properly (N> or equal to the number of sqoop parallelism).

2. Sqoop local import users may need to use IP or modify the host file.

3. Sqoop list-tables does not require data nodes?(Doubtful??) .

sqoop-import ERROR tool.ImportTool: Import failed: No primary key could be found for table user_info

USES sqoop-import to import data from mysql into HDFS because mysql table has no primary key 

[walker001@walker001 ~]$ sqoop-import \
> --connect 'jdbc:mysql://192.168.220.129:3306/test?characterEncoding=UTF-8' \
> --username root \
> --password zwk95914 \
> --table user_info \
> --columns userId,userName,password,trueName,addedTime \
> --target-dir /sqoop/mysql 
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/04/05 11:16:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
20/04/05 11:16:29 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/04/05 11:16:29 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/04/05 11:16:29 INFO tool.CodeGenTool: Beginning code generation
Sun Apr 05 11:16:30 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
20/04/05 11:16:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_info` AS t LIMIT 1
20/04/05 11:16:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_info` AS t LIMIT 1
20/04/05 11:16:31 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/walker001/app/hadoop-2.8.2
注: /tmp/sqoop-walker001/compile/e8a05a19dfbef5b687215bb6f631fbd2/user_info.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
20/04/05 11:16:40 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-walker001/compile/e8a05a19dfbef5b687215bb6f631fbd2/user_info.jar
20/04/05 11:16:40 WARN manager.MySQLManager: It looks like you are importing from mysql.
20/04/05 11:16:40 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
20/04/05 11:16:40 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
20/04/05 11:16:40 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
20/04/05 11:16:40 ERROR tool.ImportTool: Import failed: No primary key could be found for table user_info. Please specify one with --split-by or perform a sequential import with '-m 1'.

<人力资源>

mysql> alter table user_info add primary key(userId);
Query OK, 0 rows affected (0.04 sec)
Records: 0  Duplicates: 0  Warnings: 0

add primary key 

[walker001@walker001 ~]$ sqoop-import --connect 'jdbc:mysql://192.168.220.129:3306/test?characterEncoding=UTF-8' --username root --password zwk95914 --table user_info --columns userId,userName,password,trueName,addedTime --target-dir /sqoop/mysql
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/walker001/app/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/04/05 11:19:46 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
20/04/05 11:19:47 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/04/05 11:19:47 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/04/05 11:19:47 INFO tool.CodeGenTool: Beginning code generation
Sun Apr 05 11:19:47 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
20/04/05 11:19:48 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_info` AS t LIMIT 1
20/04/05 11:19:48 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_info` AS t LIMIT 1
20/04/05 11:19:48 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/walker001/app/hadoop-2.8.2
注: /tmp/sqoop-walker001/compile/72faaf2287c7c39a8586ce10f0e78d74/user_info.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
20/04/05 11:19:52 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-walker001/compile/72faaf2287c7c39a8586ce10f0e78d74/user_info.jar
20/04/05 11:19:52 WARN manager.MySQLManager: It looks like you are importing from mysql.
20/04/05 11:19:52 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
20/04/05 11:19:52 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
20/04/05 11:19:52 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
20/04/05 11:19:52 INFO mapreduce.ImportJobBase: Beginning import of user_info
20/04/05 11:19:53 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
20/04/05 11:19:54 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
20/04/05 11:19:54 INFO client.RMProxy: Connecting to ResourceManager at walker001/192.168.220.129:8032
Sun Apr 05 11:20:06 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
20/04/05 11:20:07 INFO db.DBInputFormat: Using read commited transaction isolation
20/04/05 11:20:07 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`userId`), MAX(`userId`) FROM `user_info`
20/04/05 11:20:07 INFO db.IntegerSplitter: Split size: 0; Num splits: 4 from: 1 to: 2
20/04/05 11:20:07 INFO mapreduce.JobSubmitter: number of splits:2
20/04/05 11:20:08 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1586054209222_0001
20/04/05 11:20:09 INFO impl.YarnClientImpl: Submitted application application_1586054209222_0001
20/04/05 11:20:09 INFO mapreduce.Job: The url to track the job: http://walker001:8088/proxy/application_1586054209222_0001/
20/04/05 11:20:09 INFO mapreduce.Job: Running job: job_1586054209222_0001
20/04/05 11:20:33 INFO mapreduce.Job: Job job_1586054209222_0001 running in uber mode : false
20/04/05 11:20:33 INFO mapreduce.Job:  map 0% reduce 0%
20/04/05 11:20:56 INFO mapreduce.Job:  map 100% reduce 0%
20/04/05 11:20:58 INFO mapreduce.Job: Job job_1586054209222_0001 completed successfully
20/04/05 11:20:59 INFO mapreduce.Job: Counters: 30
        File System Counters
                FILE: Number of bytes read=0
                FILE: Number of bytes written=318188
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=213
                HDFS: Number of bytes written=68
                HDFS: Number of read operations=8
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=4
        Job Counters 
                Launched map tasks=2
                Other local map tasks=2
                Total time spent by all maps in occupied slots (ms)=39020
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=39020
                Total vcore-milliseconds taken by all map tasks=39020
                Total megabyte-milliseconds taken by all map tasks=39956480
        Map-Reduce Framework
                Map input records=2
                Map output records=2
                Input split bytes=213
                Spilled Records=0
                Failed Shuffles=0
                Merged Map outputs=0
                GC time elapsed (ms)=840
                CPU time spent (ms)=2990
                Physical memory (bytes) snapshot=204382208
                Virtual memory (bytes) snapshot=3781640192
                Total committed heap usage (bytes)=48259072
        File Input Format Counters 
                Bytes Read=0
        File Output Format Counters 
                Bytes Written=68
20/04/05 11:20:59 INFO mapreduce.ImportJobBase: Transferred 68 bytes in 65.254 seconds (1.0421 bytes/sec)
20/04/05 11:20:59 INFO mapreduce.ImportJobBase: Retrieved 2 records.
[walker001@walker001 ~]$ hadoop fs -ls /sqoop/mysql
Found 3 items
-rw-r--r--   2 walker001 supergroup          0 2020-04-05 11:20 /sqoop/mysql/_SUCCESS
-rw-r--r--   2 walker001 supergroup         35 2020-04-05 11:20 /sqoop/mysql/part-m-00000
-rw-r--r--   2 walker001 supergroup         33 2020-04-05 11:20 /sqoop/mysql/part-m-00001
[walker001@walker001 ~]$ hadoop fs -cat /sqoop/mysql/*
1,hello,123456,zhangsan,2017-09-01
2,hello2,123456,lisis,2019-09-01

after re-import