pandas parse_ Data exception, automatically skip

When processing raw data, the following error occurs:

id,name,date
0,a,2020/01/01
0,b,2020/01/01
0,c,2020/01/01
0,d,2020/01/01
0,e,2020/01/01
0,f,9999/01/01

It was treated with panda :

data = pandas.read_csv(file, sep=";", encoding="ISO-8859-1", parse_dates=["date"],  date_parser=lambda x: pandas.to_datetime(x, format="%d.%m.%Y"))

But the running time is wrong, which means out of bonds timestamp .

Our current approach is to skip the exception line,

The following line needs to be added

date_parser=lambda x: pd.to_datetime(x, errors="coerce")

There are three kinds of assignments for the errors parameter. The default value is’ raise ‘. An error will be reported if the parsing does not conform to the specification.

You can assign the errors parameter to “coerce” and set the time format of the error to NAT during parsing. If you don’t want to deal with the wrong time format, you can assign errors to ‘ignore’, so that the original format is the same.

errors{‘ignore’, ‘raise’, ‘coerce’}, default ‘raise’

If ‘raise’, then invalid parsing will raise an exception.If ‘coerce’, then invalid parsing will be set as NaT.If ‘ignore’, then invalid parsing will return the input.


Read More: