Tag Archives: Good

[Solved] Canal Error: CanalParseException: column size is not match，parse row data failed

1、 Background phenomenon

Background: there is a problem with the company’s flick task, and the data is not written to the result library.

So immediately check the flick task. On the web page, there are no exceptions, checkpoints and backpressure
then the problem is not my program. The spearhead is directed at environmental problems

2、 Environmental investigation

First, I checked the log printed by the task manager of Flink and found that the data was consumed for a certain period of time, and no data came in later
it means that the data is not sent to the Flink program, so there is a problem at the source
after checking Kafka, it is found that there is no message backlog and the consumption rate is normal
then the problem is not Kafka. Then it can only come from a more original place: canal

3、 Culprit canal

The operation and maintenance boss checked the canal log:

com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed.
Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed.
Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed.
Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: column size is not match for table:xxxx.xxx,22 vs 21

Obviously, this log says that a new field has been added to the database, which is inconsistent with the number of fields in the previous database. It causes an error in canal, and then the message is not sent

At that time, I thought that canal must support DDL compatibility, which must be a problem with one of canal’s settings
then I went to GitHub to find the document of canal
during the search, an issue was found. The content in the issue is similar to mine.
the key points are:

 canal.instance.filter.query.ddl = true

Semantically speaking, it is obvious that canal filters out the DDL statements of MySQL, so it is naturally impossible to perceive that MySQL has added a new field. In this way, when a new piece of data comes after adding a field, canal will report an error if it cannot match the number of fields.

Solution

canal.instance.filter.query.ddl = false

In this way, canal can receive DDL statements and adapt to the changes after adding new fields.

Service temporarily unavailable due to an ongoing leader election. Please refresh

After the Flink cluster is started, access the http://10.10.10.150 :44973

Report the following error

{“errors”:[“Service temporarily unavailable due to an ongoing leader electio

Please check the mapping relationship between/etc/hosts server name and IP, if not, add it

For example: 10.10.15.10       hhht_ mk_ it

Restart the Flink cluster.

Flink 1.1 error: no executorfactory found to execute the application

Error reported when migrating Flink to 1.1:

No ExecutorFactory found to execute the application

After investigation, the reason is: starting from flink1.11, the Flink streaming Java dependency on Flink clients has been removed, and the clients dependency needs to be added manually.

The POM file is as follows, add Flink clients_ 2.12, problem solving.

 

window.open () several ways to open windows

    opens Baidu in the current window and makes the URL appear in the search bar.
window.open("http://www.baidu.com/", "_self");
    Open Baidu
a new window

window.open("http://www.baidu.com/", "_blank");
    Open a new window and name it “hello”
window.open("", "hello");
    > In addition, there are several options for the second argument of the open function :
_top: If there are framesets on the page, the URL replaces the top layer of framesets. That is, if there are no framesets, the effect is equivalent to _self._parent: the page pointed to by the URL is loaded into the parent of the current frame. If there are no framesets, the effect is equivalent to _self._media: the page pointed to by the URL is loaded into the HTML code area contained in the Media Bar. If there is no Media Bar, add it to itself.

    If you want to add something else to the new window, you need a third parameter :
Channelmode: yes | no | 1 | 0 (window display for theater mode [full screen displays the current web page, including the toolbar], or channel mode [general show]). Directories: yes | no | 1 | 0 (whether to add directory buttons, such as under the IE there could be a “link” button on the top) fullscreen: yes | no | 1 | 0 (the browser handle full screen mode, and hide the title bar and menu, etc.) the menubar: Yes | no | 1 | 0 (whether to display the menu bar of the browser by default) resizeable: yes | no | 1 | 0 (window can resize) scrollbars: yes | no | 1 | 0) (whether to allow a horizontal or vertical scroll bar titlebar: yes | no | 1 | 0 (whether to add a title bar) toolbar: yes | no | 1 | 0 (whether to add the default browser toolbar) status: Yes | no | 1 | 0 (whether to display the status bar) location: yes | no | 1 | 0 (whether to display the search bar) copyhistory: yes | no | 1 | 0 (seems to have been abandoned, if as long as the toolbars, according to the history button will be displayed) height: the height of the window, the minimum value of 100 pixels width: the width of the window, the minimum value of 100 pixels left: the window of the distance relative to the screen on the left

window.open("http://www.baidu.com/", "_self","height=400,width=400,top=10,left=10,resizable=yes");

There is one last argument to the open function, which is about history, whether the window being opened has the same history URL as the window being opened, or whether it is a separate memory.