Project scenario:
The need for programming to obtain data pedigrees is not common for data warehouses, and separate data pedigree work is only necessary when the scale of data reaches a significant level, or the number of reports with complex data production relationships increases to a significant level.
Until scale is reached, manual identification and management is more cost effective.
antlr is the open source syntax parser that automatically generates syntax trees based on input and displays them visually. antlr-Another Tool for Language Recognition, formerly PCCTS, provides a syntax tree for languages including Java, C++, and C# by description to automatically construct a custom language recognizer (recognizer), compiler (parser) and interpreter (translator) framework. ANTLR is available in v2 v3 v4, with most of the Chinese documentation being in v2 and the hive 1.1.0 version mentioning antlr 3.4 in the comments. antlr combines the above by allowing us to define lexical rules for recognizing character streams and syntactic parser rules for interpreting Token streams. ANTLR will then automatically generate the appropriate lexical/syntactic parser based on the syntax file provided by the user. The user can use them to compile the input text and convert it into other forms (e.g. AST-Abstract Syntax Tree).
Use ANTLR version 3.5.2 to parse Hivesql’s source code grammar file for HiveLexer.g, and place the code:
@lexer::header {
package org.apache.hadoop.hive.ql.parse
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.hive.conf.HiveConf
}
@lexer::members {
private Configuration hiveConf
public void setHiveConf(Configuration hiveConf) {
this.hiveConf = hiveConf
}
protected boolean allowQuotedId() {
if(hiveConf == null){
return false
}
String supportedQIds = HiveConf.getVar(hiveConf, HiveConf.ConfVars.HIVE_QUOTEDID_SUPPORT)
return !"none".equals(supportedQIds)
}
}
This paragraph needs to be annotated so that Java compiled with ANTLR does not contain this package.
Problem description
After this paragraph is annotated with/* * /, it is still compiled and an error is found:
[10:58:10] error(100): HiveLexer.g:42:1: syntax error: antlr: NoViableAltException(9@[])
[10:58:10] error(100): HiveLexer.g:42:2: syntax error: antlr: NoViableAltException(49@[])
[10:58:10] error(100): HiveLexer.g:42:7: syntax error: antlr: MissingTokenException(inserted [@-1,0:0='<missing ACTION>',<4>,42:6] at :)
[10:58:10] error(100): HiveLexer.g:42:8: syntax error: antlr: NoViableAltException(22@[])
[10:58:10] error(100): HiveLexer.g:42:9: syntax error: antlr: NoViableAltException(80@[])
[10:58:10] error(100): HiveLexer.g:42:9: syntax error: antlr: NoViableAltException(80@[])
[10:58:10] error(100): HiveLexer.g:47:1: syntax error: antlr: MissingTokenException(inserted [@-1,0:0='<missing SEMI>',<82>,47:0] at KW_TRUE)
Cause analysis:
Query data discovery
‘; ‘ is a terminator in comment and so error occurred
Solution:
Delete all the original annotation documents or delete them; Delete it.
Problem solved.
