Tag Archives: python

[Solved] ParserError: NULL byte detected. This byte cannot be processed in Python‘s native csv library

ParserError: NULL byte detected. This byte cannot be processed in Python’s native csv library at the moment, so please pass in engine=’c’ instead


Error:

file_name = os.listdir(base_dir)[0]

col_list = [feature list]
col = col_list
#encoding
#data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding="GBK",usecols=range(len(col)))
    
data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding = 'unicode_escape', engine ='python')


#data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding = 'utf-8', engine ='python')

path = "D:\\test\\repo\\data.csv"

Solution:

engine =’c’

file_name = os.listdir(base_dir)[0]

#encoding
#data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding="GBK",usecols=range(len(col)))
    
data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding = 'unicode_escape', engine ='c')


#data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding = 'utf-8', engine ='python')

path = "D:\\test\\repo\\data.csv"

Full Error Messages:
—————————————————————————

Error                                     Traceback (most recent call last)
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _next_iter_line(self, row_num)
2967             assert self.data is not None
-> 2968             return next(self.data)
2969         except csv.Error as e:
Error: line contains NULL byte
During handling of the above exception, another exception occurred:
ParserError                               Traceback (most recent call last)
<ipython-input-12-c5d0c651c50e> in <module>
85                    ]
86
---> 87     data = inference_process(data_dir)
88     #print(data.head())
89     f=open("break_model1.pkl",'rb')
<ipython-input-12-c5d0c651c50e> in inference_process(base_dir)
18     #encoding
19 #     data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding="GBK",usecols=range(len(col)))
---> 20     data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding = 'unicode_escape', engine ='python')
21 #     data = pd.read_csv("D:\\test\\repo\\data.csv",sep = ',',encoding = 'utf-8', engine ='python')
22
D:\anaconda\lib\site-packages\pandas\io\parsers.py in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)
608     kwds.update(kwds_defaults)
609
--> 610     return _read(filepath_or_buffer, kwds)
611
612
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
460
461     # Create the parser.
--> 462     parser = TextFileReader(filepath_or_buffer, **kwds)
463
464     if chunksize or iterator:
D:\anaconda\lib\site-packages\pandas\io\parsers.py in __init__(self, f, engine, **kwds)
817             self.options["has_index_names"] = kwds["has_index_names"]
818
--> 819         self._engine = self._make_engine(self.engine)
820
821     def close(self):
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _make_engine(self, engine)
1048             )
1049         # error: Too many arguments for "ParserBase"
-> 1050         return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
1051
1052     def _failover_to_python(self):
D:\anaconda\lib\site-packages\pandas\io\parsers.py in __init__(self, f, **kwds)
2308                 self.num_original_columns,
2309                 self.unnamed_cols,
-> 2310             ) = self._infer_columns()
2311         except (TypeError, ValueError):
2312             self.close()
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _infer_columns(self)
2615             for level, hr in enumerate(header):
2616                 try:
-> 2617                     line = self._buffered_line()
2618
2619                     while self.line_pos <= hr:
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _buffered_line(self)
2809             return self.buf[0]
2810         else:
-> 2811             return self._next_line()
2812
2813     def _check_for_bom(self, first_row):
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _next_line(self)
2906
2907             while True:
-> 2908                 orig_line = self._next_iter_line(row_num=self.pos + 1)
2909                 self.pos += 1
2910
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _next_iter_line(self, row_num)
2989                     msg += ". " + reason
2990
-> 2991                 self._alert_malformed(msg, row_num)
2992             return None
2993
D:\anaconda\lib\site-packages\pandas\io\parsers.py in _alert_malformed(self, msg, row_num)
2946         """
2947         if self.error_bad_lines:
-> 2948             raise ParserError(msg)
2949         elif self.warn_bad_lines:
2950             base = f"Skipping line {row_num}: "
ParserError: NULL byte detected. This byte cannot be processed in Python's native csv library at the moment, so please pass in engine='c' instea

urllib.error.URLError: ＜urlopen error [Errno -3] Temporary failure in name resolution＞

When training the model, load some pre training models, such as VGg. The code is as follows

model = torchvision.models.vgg19(pretrained=True)

Train will display

Downloading: "https://download.pytorch.org/models/vgg19-dcbb9e9d.pth" to /root/.cache/torch/checkpoints/vgg19-dcbb9e9d.pth

Then an error occurred:

socket.gaierror: [Errno -3] Temporary failure in name resolution
and
urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

This is because the pre training model cannot be downloaded, so it needs to be downloaded from the Internet
therefore, it is more convenient to download the model first, find a way to connect to the Internet, and then input the link automatically https://download.pytorch.org/models/vgg19-dcbb9e9d.pth
Then put the downloaded. PTH model file under a fixed path, such as

/home/team/torch/models/pre_ model/vgg19-dcbb9e9d.pth

Finally, change the code to

model = torchvision.models.vgg19(pretrained=False)
pthfile = r'/home/team/torch/models/pre_model/vgg19-dcbb9e9d.pth'
model.load_state_dict(torch.load(pthfile))```

[Solved] Pygame.error: mixer not initialized & pygame.error: WASAPI can‘t find requested audio endpoint: Could not Find the Element.

When developing games using python, we will inevitably use the pyGame module, which has a sound function. Using this function, we can add sound effects to our games.

Problem Description:

To use the sound module, we must initialize our game at the beginning of the main function, so we add the following statement at the beginning of the main function to initialize the game.

# Game initialization
    pygame.init()

However, when I run the program, I find that the game window flashes back and an error message appears, as follows:

D:\Game\TankWar\venv\Scripts\python.exe D:/Game/TankWar/main.py
pygame 2.0.2 (SDL 2.0.16, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
  File "D:/Game/TankWar/main.py", line 41, in <module>
    is_quit_game = run_Game(config)
  File "D:/Game/TankWar/main.py", line 22, in run_Game
    sounds[key] = pygame.mixer.Sound(value)
pygame.error: mixer not initialized

Process finished with exit code 1

it says I didn’t initialize the mixer!!! We can’t help it. Let’s go according to his error report and initialize the mixer separately.

pygame.init()
pygame.mixer.init()

The following error message still appears, and the game window still flashes back.

D:\Game\TankWar\venv\Scripts\python.exe D:/Game/TankWar/main.py
pygame 2.0.2 (SDL 2.0.16, Python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
  File "D:/Game/TankWar/main.py", line 42, in <module>
    is_quit_game = run_Game(config)
  File "D:/Game/TankWar/main.py", line 17, in run_Game
    pygame.mixer.init()
pygame.error: WASAPI can't find requested audio endpoint: Could not find the element.

Process finished with exit code 1

Solution:

After repeated tests, I found that it can run normally sometimes, and the above error reports will appear sometimes. Finally, I found a big man’s article and solved this problem.

earphone problem

Because I use a desktop computer and have no audio connected, there has been no audio output device, which causes pyGame to not know where to output the sound (in this case, the audio device cannot be found), resulting in an error. After inserting the audio device (i.e. my headset), it’s solved…

[Solved] Python urllib sending request Error: urllib.error.urlerror: ＜urlopen error [SSL: certificate_verify_failed]….>

Error:urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:xxx)>

Solution:
Add the following codes before you use urllib.request.Request(url):

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

Problem analysis

This is because the website visited is HTTPS://, which requires SSL authentication, and using urllib directly will lead to local authentication failure (the specific reason is not found out), so SSL is used_create_unverified_Context turn off authentication

Error recurrence

When request = urllib.Request.Request (URL, data) is executed, an error is reported. Cancel the comments in the upper two lines to solve the problem

import json
import urllib


def baidu_search():
    url = "https://www.baidu.com/s?"
    data = {"wd": "AHA"}
    data = json.dumps(data).encode('GBK')
    # import ssl
    # ssl._create_default_https_context = ssl._create_unverified_context  # If these two lines are not added, the next line reports an error
    request = urllib.request.Request(url, data)
    response = urllib.request.urlopen(request)
    content = response.read()
    print(str(content))


if __name__ == '__main__':
    baidu_search()

Error: one input ui-file must be specified [How to Solve]

QT designer installed   After success, install PIP anyway   install PyQt5   And pip   Install pyqt5 designer is useless

So consider whether the installation path of QT designer is wrong. Therefore, searching designer.exe in the folder unexpectedly found 2 files.

The first path has been used before, but the UI file cannot be converted to py file. The link below was replaced.

OK after running the test

Hope to help you!

AttributeError: type object ‘Image‘ has no attribute ‘open‘

AttributeError: type object ‘Image‘ has no attribute ‘open‘

Cause analysis: the calling sequence of image is wrong, because the from PIL import image in the first line conflicts with Tkinter import * in the second line, and Tkinter also contains image class, so you use tkinter.image

Solution: from PIL import image as IMIM

Original call sequence

try:
    from PIL import Image
except ImportError:
    import Image

import tkinter as tk
from tkinter import *
from tkinter import filedialog

Original call sequence

import tkinter as tk
from tkinter import *
from tkinter import filedialog

try:
    from PIL import Image
except ImportError:
    import Image

(29)RuntimeError: cuda runtime error (999)

Problem Description: when running the improved yolov4 program, I accidentally encountered the top row of “PC” on the keyboard and one of the three keys around it, and the program directly reported an error:

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1595629403081/work/aten/src/THC/THCGeneral.cpp line=47 error=999 : unknown error
Traceback (most recent call last):
  File "/home/lidan/PycharmProjects/yolov4/B_mobilenet-yolov4-lite_378/train.py", line 280, in <module>
    net = net.cuda()
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 458, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 354, in _apply
    module._apply(fn)
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 354, in _apply
    module._apply(fn)
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 354, in _apply
    module._apply(fn)
  [Previous line repeated 3 more times]
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 376, in _apply
    param_applied = fn(param)
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 458, in <lambda>
    return self._apply(lambda t: t.cuda(device))
  File "/home/lidan/.conda/envs/ultralytics_yolov3/lib/python3.7/site-packages/torch/cuda/__init__.py", line 190, in _lazy_init
    torch._C._cuda_init()
RuntimeError: cuda runtime error (999) : unknown error at /opt/conda/conda-bld/pytorch_1595629403081/work/aten/src/THC/THCGeneral.cpp:47

In addition, some other pages of the computer are not easy to use, such as the web page cannot be displayed completely…

Solution: restart the computer and everything returns to normal

cv2.error: OpenCV(4.5.1) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-buil

catalogue

The following operations are carried out under the windows environment.  

one   Change single slash to double slash

2. The modified path is in English

3. Delete CV2. Imshow ()

When I use CV2 to save pictures to a new price asking folder, I report an error CV2. Error: opencv (4.5.1) C: \ users \ appveyor \ appdata \ local \ temp \ 1 \ PIP req build

The solution is as follows:

The following operations are carried out under the windows environment.  

one   Change single slash to double slash

Some people on the Internet say that the path reading error is caused by the number in the path. Correction method: change the single slash in the path into double slash.

For example:

cv2.imwrite(save_dir + '\\' + img_name, img)

However, the same error is reported later, so read the following comments for 2 operations:  

2. The modified path is in English

I reported a similar error here because Chinese appears in the quoted picture path. If you change the picture path to an all English path, there will be no problem.   

for i in ori_imgs_single:
    img_name = i.split('\\')[-1]
    img = cv2.imread(i)
    cv2.imwrite(save_dir + '\\' + img_name, img)
print('save OK!')

OK：

3. Delete CV2. Imshow ()

Maybe there is something wrong with my environment. If you still report an error, try deleting CV2. Imshow (). If I delete it, it will be OK:  

 

  OK：

 

numpy.core._exceptions.MemoryError: Unable to allocate xxx GB

This error was reported during the training of 15W pieces of data recently, but I only used 60% of my local memory. After solving it, record it.

After checking, it should be the problem of Python bits. At first, my local is 32 bits.  

If your Python is 32-bit, your pandas and numpy can only be 32-bit. When your memory usage exceeds 2G, the memory will be automatically terminated

Later, it was changed to 64 bit, and there was no error