Presto queries the data of Apache druid and inserts it into kudu. Occasionally, an error is reported: generic_INTERNAL_Error (65536)
java.lang.illegalargumentexception: unknown field druid.druid. Data source name. Field name: varchar
com.google.common.base.preconditions.checkargument (preconditions. Java: 216)
Check the Druid log and find the corresponding error:
unknown exception (org. Apache. Cite. Tools. Validationexception): org. Apache. Cite. Runtime. Calculecontextexception: from line 2, column 8 to line 2, column 12: column ‘field name’ not found in any table
in addition, errors are occasionally reported when using superset to obtain the data preview of the corresponding Druid data source
Problem analysis:
1. First, judge that it should be a druid problem. The same operation only occasionally reports an error. It is suspected that it is a druid broker problem
2. It is found that the results of querying the number of columns of the same data source through different brokers are inconsistent (SQL: select count (*) columncount from information_schema. Columns where table_neme = ‘data source name’). This may be because historical has not provided services to all segments when the broker is restarted by rolling (after restarting historical, the segment load is not completed, and the segment is still being loaded gradually), resulting in different schema information such as dimension indicators obtained through the segment at different times.
Solution: restart the broker after historically loading segments to ensure that the dimension indicators and other schema information obtained by the broker through segments are consistent. Subsequent restart of the broker should ensure that all segments in the cluster provide services normally.
Summary: the reason for error reporting is that the schema information obtained twice is inconsistent. The inconsistency may be caused by adding or deleting columns, or modifying column names and column data types. To solve the problem of inconsistent schema information, you can solve the current error reporting.