Category Archives: Python

[Solved] Python install kenlm error: ERROR: Command errored out with exit status 1: …

Use pip install kenlm error:

python/kenlm.cpp:6381:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
tstate->exc_traceback = *tb;
^~~~~~~~~~~~~
curexc_traceback
error: command ‘gcc’ failed with exit status 1

ERROR: Command errored out with exit status 1: …

It seems to be the GCC Issue

Solution:
Use pypi-kenlm

pip install pypi-kenlm

IndexError: list index out of range [How to Solve]

IndexError: list index out of range

When training ctpn, some pictures report this error
the error code is as follows. Because the mean value is subtracted from three channels respectively when subtracting the mean value, some pictures are of two channels, and the length does not match, so an error is reported
solution: convert to RGB three channel diagram here

    vggMeans = [122.7717, 102.9801, 115.9465] 
    imageList = cv2.split(image.astype(np.float32))
    imageList[0] = imageList[0]-vggMeans[0]  
    imageList[1] = imageList[1]-vggMeans[1]
    imageList[2] = imageList[2]-vggMeans[2]
    image = cv2.merge(imageList) 

[Solved] PyInstaller Error: ValueError: too many values to unpack

Execution: pyinstaller .\checkattendance.spec

report errors:

for name, pth in format_binaries_and_datas(datas, workingdir=spec_dir):
File "d:\python\winpython-64bit-2.7.10.3\python-2.7.10.amd64\lib\site-packages\PyInstaller\building\utils.py", line 440, in format_binaries_and_datas
for src_root_path_or_glob, trg_root_dir in binaries_or_datas:
ValueError: too many values to unpack

When reading an error, there should be a problem with the format of the data in the data or binaries. Think of adding content to the data field of the spec file, such as data = [“config.Ini”, ‘.’]

According to the information, the list of data is a list of tuples, and the tuple is a tuple of two elements. The first element is the position when reading the file in Python code, and the second element represents the real directory of the data file to be read.

So the format should be data = [(“config.Ini”, ‘.’)]

After modification, execute pyinstaller .\checkattendance.spec

It can be packed smoothly. I hope it can help you with the same problem~

[Solved] Sklearn Call Error: DLL load failed while importing _arpack

DLL load failed while importing when sklearn is called_Arpack error

There is no effect after reinstalling sklearn and numpy with CONDA
some people in the Post said that they could delete some DLL files under Win32 to prevent error reporting and did not try

Knowing that CONDA may have its own problems (such as some DLL file configurations), I reinstalled sklearn with PIP

pip uninstall scipy
pip install scipy

The problem is solved (of course, the image should be configured in advance)

PS: I still reported an error when I just used the Jupiter notebook. I found that I had to restart it.

Reference Resources:
https://stackoverflow.com/questions/55201924/scikit-learn-dll-load-failed-in-anaconda

[CUDA Environment] Python Pytorch Error: CudaSetupArgument

@TOC
the probability of this problem is that the CUDA version used for compilation is inconsistent with the CUDA version running
first check the CUDA version of the system (that is, the CUDA version used for compilation)

nvcc -V

In my pytorch + CONDA environment, you can use CONDA list to view the cudatoolkit version in the virtual environment. At first, the CUDA version of my system is 9.0 and the cudatoolkit version is 10.2. Therefore, the version is inconsistent, so the error message shown in the title appears. Later, I switched the CUDA version of the system and the problem was solved
brief description of the specific version switching method:
echo $path view CUDA path information, add the path of cuda10.2 and link it to/usr/local/CUDA. The specific instructions are

ln -s /usr/local/cuda10.2 /usr/local/cuda

Then modify the system path as follows:

vim ~/.bashrc

Add code at the end

export PATH=/usr/local/cuda:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Press ESC, enter: WQ, press enter to exit, and then enter on the command line

source ~/.bashrc

Update path information
now enter from the command line

nvcc -V

You can view the CUDA version after switching

[Solved] Python project runs the open() function error: FileNotFoundError: [Errno 2] No such file or directory

Traceback (most recent call last):
  File "D:/Python/Practice/file_path_test01.py", line 10, in <module>
    open(path1, 'wb')
FileNotFoundError: [Errno 2] No such file or directory: './output/experiment_UNet_ResFourLayerConvBlock_ResTwoLayerConvBlock_None_fold-1_coarse_size-160_channel-8_depth-4_loss-dice_metric-dice_time-2021-11-20_16-14-52\\logs\\train\\events.out.tfevents.1637396155.DESKTOP-AHH47H9123456789012345678901234567890'

Reason 1: the parent folder of the file to be created does not exist. The open function will recreate the file when it does not exist, but cannot create the nonexistent folder.

Reason 2: the character length of file name + absolute path of file exceeds the limit of the operating system. Windows system has restrictions

Python server run code ModuleNotFoundError Error [How to Solve]

1. Problem description

A piece of Python code runs normally on the local ide. After it is deployed to the server for operation, a modulenotfounderror: no module named ‘xxx’ error occurs.

2. Cause of problem

The package of other files (self written package, not installed by PIP) is introduced into the code. What’s the problem  import that line.

The reason for the error is that the path on the server side is different from our local path.

3. Solution example

To solve this problem, you can add the following code at the top of your code:

import sys
import os
sys.path.append(os.path.dirname(sys.path[0]))

perhaps

import sys
import os

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, BASE_DIR)

4. The above codes of sys.Path.Append() and sys. Path.Insert() can ensure that you can switch to the directory where the currently executed script is located at any time, which can be adjusted according to the directory structure where your script is located.

Used in Python programs import xxx, the python parser will search XXX in the current directory, installed modules and third-party modules. If it fails to search, an error will be reported.

sys.path  The module can dynamically modify the system path. The path imported by this method will become invalid after the python program exits.

sys.path  It’s a list, so it’s easy to add a directory in it. After adding, the new directory will take effect immediately. In the future, every time import operation may check this directory.

1. sys.path.append()

In the sys.path temporary end of the list to add the search path, convenient and concise import other packages and modules. The path imported by this method will become invalid after the Python program exits.

Example:

import sys

sys.path.append('..') # Indicates to import the upper directory of the current file into the search path

sys.path.append('/home/model') # absolute path

from folderA.folderB.fileA import functionA

 

2. sys.path.insert()

You can define the search priority. The sequence number starts from 0, indicating the maximum priority, sys.Path.Insert() is also a temporary search path, which will become invalid after the program exits.

Example:

import sys

sys.path.insert(1, "./model")

 

[Solved] AttributeError: module ‘thread‘ has no attribute ‘start_new_thread‘

There is a package name thread in the project (the folder name in Python is also the package name), which conflicts with the thread library of the system. Just rename the folder in the project.

That is, change thread.py in the project to another name.

Right click thread.py → refactor → rename → in pycharm to change the name to be different from the system thread library.

[Solved] raise ContentTooShortError(urllib.error.ContentTooShortError: <urlopen error retrieval incomplete:

1. Problem description

The following error occurred during crawler batch download

 raise ContentTooShortError(
urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 0 out of 290758 bytes>

2. Cause of problem

Problem cause: urlretrieve download is incomplete

3. Solution

1. Solution I

Use the recursive method to solve the incomplete method of urlretrieve to download the file. The code is as follows:

def auto_down(url,filename):
    try:
        urllib.urlretrieve(url,filename)
    except urllib.ContentTooShortError:
        print 'Network conditions is not good.Reloading.'
        auto_down(url,filename)

However, after testing, urllib.ContentTooShortError appears in the downloaded file, and it will take too long to download the file again, and it will often try several times, or even more than a dozen times, and occasionally fall into a dead cycle. This situation is very unsatisfactory.

2. Solution II

Therefore, the socket module is used to shorten the time of each re-download and avoid falling into a dead cycle, so as to improve the operation efficiency
the following is the code:

import socket
import urllib.request
#Set the timeout period to 30s
socket.setdefaulttimeout(30)
#Solve the problem of incomplete download and avoid falling into an endless loop
try:
    urllib.request.urlretrieve(url,image_name)
except socket.timeout:
    count = 1
    while count <= 5:
        try:
            urllib.request.urlretrieve(url,image_name)                                                
            break
        except socket.timeout:
            err_info = 'Reloading for %d time'%count if count == 1 else 'Reloading for %d times'%count
            print(err_info)
            count += 1
    if count > 5:
        print("downloading picture fialed!")