Tag Archives: Step on the pit

Network error — browser error [How to Solve]

Record it! After half a month of internship, when I changed the bug today (an interface for modifying personal information), it was obviously changed, and the test environment was running smoothly. When I opened the test environment demonstration, I click submit to modify and report an error network error as shown in the following figure

I opened F12 to view the request, and found that the status was CORS error. My first feeling was that there was a cross domain problem, but I should not, Just opened the console and reported an error, as shown in the figure below:

I still feel that it is cross domain, but it should not be cross domain ~
finally, I found that the top left corner of the browser is as shown in the figure

remember that the company’s domain name has a certificate, is it because there is no s, so I changed the address bar HTTP to HTTPS, The problem is solved ~
then the cross domain reason should be that http//: test.xxx.com visited the interface of HTTPS: test.xxx.com, while the domain name allowed to cross domain in the background is http//: test.xxx.com https://test.xxx.com , which leads to cross domain ~
lack of talent and learning, misunderstanding and mistakes, please give us some advice in the comments area~

Method threw ‘java.lang.NullPointerException‘ exception. Cannot evaluate com.sun.proxy.xxx

When I debug, I find that method thread 'Java. Lang. NullPointerException' exception. Cannot evaluate com. Sun. Proxy. $proxy0. Tostring() will appear, but it does not affect the normal operation of the program. It should be because the object generated during dynamic proxy may not have tostring() method, and the information is printed through toString in debug mode. So it also explains why the real proxy object can be returned in the invoke method

Step on the pit — error reported by sqoop tool.ExportTool : Error during export

@To live better

Step on the pit — error reported by sqoop tool.ExportTool : Error during export

The error printed on the console is

19/04/19 20:17:09 ERROR mapreduce.ExportJobBase: Export job failed!
19/04/19 20:17:09 ERROR tool.ExportTool: Error during export: 
Export job failed!
	at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
	at org.apache.sqoop.manager.MySQLManager.upsertTable(MySQLManager.java:145)
	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:73)
	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

finishing touch : this error indicates that the fields of MySQL and hive do not correspond or the data format is different
solution:
Step 1 : check whether the structure (field name and data type) of MySQL and hive tables are consistent.
Step 2 : check whether the data has been imported?If the data is not imported, please change the time type in MySQL and hive to string or varchar in the first step. If there is an import, but the imported data is incomplete or incorrect. It must be that your data type is inconsistent with the actual data. There are two situations. See the following three steps for details.
Step 3 : in MySQL to hive, please be sure to check whether your data contains the default line break for hive table creation ( lines terminated by '\ n' ). If yes, add hive delims replacement or hive drop import delims to the sqoop statement, as follows

#!/bin/bash
/usr/local/package/sqoop-1.4.7.bin/bin/sqoop import \
--connect jdbc:mysql://ip:3306/mysql_DB \
--username root \
--password 1q2w3e4r \
--table mysql_Table \
--delete-target-dir \
--hive-delims-replacement , \
--hive-import \
--hive-overwrite \
--hive-database hive_DB \
--hive-table hive_Table \
--hive-partition-key ymday \
--hive-partition-value 20190419 \
--fields-terminated-by '\t' \
-m 1

Step 4 : in MySQL to hive, please be sure to check whether your data contains the field separator commonly used in hive table creation ( Fields terminated by '\ t' ). If yes, the fields terminated by parameter cannot be used in the sqoop statement, and the code is as follows:

Step 5 : if there are many empty columns from hive to hive, sometimes the above error will be reported, then add input null string and input null non string in the sqoop statement, and the following code
remember ^ A, and use Ctrl + V and Ctrl + A in VI of Linux To generate, you can't copy the following ^ a
remember ^ a directly. In VI of Linux, you can't copy the following ^ a
remember ^ a directly. In VI of Linux, you can't copy the following ^ a directly

#!/bin/bash
/usr/local/package/sqoop-1.4.7.bin/bin/sqoop export \
--connect "jdbc:mysql://ip:3306/report_db?useUnicode=true&characterEncoding=utf-8" \
--username root \
--password 1q2w3e4r \
--table mysql_table \
--columns name,age \
--update-key name \
--update-mode allowinsert \
--input-null-string '\\N' \
--input-null-non-string '\\N' \
--export-dir "/hive/warehouse/dw_db.db/hive_table/ymday=20190419/*" \
--input-fields-terminated-by '^A' \
-m 1

====================================================================

@To live better

If you have any questions about the blog, please leave a message

Failed building wheel for psutil solution

PIP installation toroch failed to install psutil. The specific prompt is as follows:

problem:

as indicated in the error message, Python. H header file

is missing

solution:

install the corresponding version of the python-dev file, under ubuntu 

sudo apt-get install python3.6-dev

reference

https://github.com/giampaolo/psutil/issues/1143