How to Solve Pytorch DataLoader Loading Error: UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xe5 in position 1023

The complete error reports are:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 301, in _on_run
    r = r.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe5 in position 1023: unexpected end of data

 

Solution:

This is not to solve the problem of Unicode decodeerror: 'UTF-8' codec can't decode byte 0xe5 in position 1023: unexpected end of data , but to solve the problem that the model cannot be iterated. The method is as follows:

Replace the data source in tensor format with numpy format, then convert it to tensor , and finally put it into dataloader

Unicode decodeerror will still be reported when moving from numpy to tensor, but the loaded data will not be encapsulated in the dataloader, resulting in the stop of the data cycle and the training of the model will not be affected.


Read More: