Tag Archives: python

Pytorch torch.cuda.FloatTensor Error: RuntimeError: one of the variables needed for gradient computation has…

pytorch 1.9 Error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4, 512, 16, 16]], which is output 0 of ConstantPadNdBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True). #23
At first I thought it was the input z = torch.randn(batch_size, 128,1,1).to(device).
Solution:
pip install torch == 1.4 torchvision = 0.05

[Solved] Grid Search Error (GridSearchCV): ‘ascii‘ codec can‘t encode characters in position 18-20: ordinal not in r

Grid Search Error: UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 18-20: ordinal not in range(128)

E:\DLstudy\Scripts\python.exe E:/PycharmProjects/DLstudy/run/train_model.py
[INFO] tuning hyperparameters...
Traceback (most recent call last):
  File "E:\PycharmProjects\DLstudy\run\train_model.py", line 22, in <module>
    model.fit(trainX, trainY)
  File "E:\DLstudy\lib\site-packages\sklearn\model_selection\_search.py", line 820, in fit
    with parallel:
  File "E:\DLstudy\lib\site-packages\joblib\parallel.py", line 725, in __enter__
    self._initialize_backend()
  File "E:\DLstudy\lib\site-packages\joblib\parallel.py", line 735, in _initialize_backend
    n_jobs = self._backend.configure(n_jobs=self.n_jobs, parallel=self,
  File "E:\DLstudy\lib\site-packages\joblib\_parallel_backends.py", line 494, in configure
    self._workers = get_memmapping_executor(
  File "E:\DLstudy\lib\site-packages\joblib\executor.py", line 20, in get_memmapping_executor
    return MemmappingExecutor.get_memmapping_executor(n_jobs, **kwargs)
  File "E:\DLstudy\lib\site-packages\joblib\executor.py", line 42, in get_memmapping_executor
    manager = TemporaryResourcesManager(temp_folder)
  File "E:\DLstudy\lib\site-packages\joblib\_memmapping_reducer.py", line 531, in __init__
    self.set_current_context(context_id)
  File "E:\DLstudy\lib\site-packages\joblib\_memmapping_reducer.py", line 535, in set_current_context
    self.register_new_context(context_id)
  File "E:\DLstudy\lib\site-packages\joblib\_memmapping_reducer.py", line 560, in register_new_context
    self.register_folder_finalizer(new_folder_path, context_id)
  File "E:\DLstudy\lib\site-packages\joblib\_memmapping_reducer.py", line 590, in register_folder_finalizer
    resource_tracker.register(pool_subfolder, "folder")
  File "E:\DLstudy\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 191, in register
    self._send('REGISTER', name, rtype)
  File "E:\DLstudy\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 204, in _send
    msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 18-20: ordinal not in range(128)

Process finished with exit code 1

Solutioin:

Original error code:

model = GridSearchCV(LogisticRegression(), params, cv=3, n_jobs=-1)

Set parameter n_jobs = - 1 parameter can be deleted and changed to:

model = GridSearchCV(LogisticRegression(), params, cv=3)

After a look, this parameter indicates how many processors we need to work

 

n_jobs : int, default=None
        Number of jobs to run in parallel.
        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.
        ``-1`` means using all processors. See :term:`Glossary <n_jobs>`
        for more details.

If n_jobs = – 1 is specified, there is a step at the bottom to use ASCII for coding, but the coding fails every time
therefore, if we do not specify this parameter, one processor will be used by default.

If you really want to specify multiple processors

Then we need to modify the code of the path with the problem in our error message
for example, in our error messages:

  File "E:\DLstudy\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 204, in _send
    msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 18-20: ordinal not in range(128)

Note the location of my error is e:\dlstudy\lib\site packages\joblib\externals\rocky\backend\resource_Line 204 of tracker.py In the _send method, click
Source code of _send function:

  def _send(self, cmd, name, rtype):
        msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('ascii')
        if len(name) > 512:
            # posix guarantees that writes to a pipe of less than PIPE_BUF
            # bytes are atomic, and that PIPE_BUF >= 512
            raise ValueError('name too long')
        nbytes = os.write(self._fd, msg)
        assert nbytes == len(msg)

Change
msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('ascii')
to
msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('utf8')
That is, the encoding is changed to utf-8, and the changed code is as follows.

```python
  def _send(self, cmd, name, rtype):
        msg = '{0}:{1}:{2}\n'.format(cmd, name, rtype).encode('utf8')
        if len(name) > 512:
            # posix guarantees that writes to a pipe of less than PIPE_BUF
            # bytes are atomic, and that PIPE_BUF >= 512
            raise ValueError('name too long')
        nbytes = os.write(self._fd, msg)
        assert nbytes == len(msg)

Then run the code again
don’t worry, you will still report errors. Because we only modified the encoding method, but not the decoding method
the error information is as follows:

     .............(...)
    splitted = line.strip().decode('ascii').split(':')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 18: ordinal not in range(128)
Traceback (most recent call last):
  File "E:\DLstudy\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py", line 253, in main
    splitted = line.strip().decode('ascii').split(':')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 18: ordinal not in range(128)

Similarly, find the error path in the error message: e:\dlstudy\lib\sitepackages\joblib\externals\loky\backend\resource

There is an error in line 253 of the tracker.py file. We found the corresponding location:

......(...)
       with open(fd, 'rb') as f:
            while True:
                line = f.readline()
                if line == b'':  # EOF
                    break
                try:
                    splitted = line.strip().decode('ascii').split(':')
                    # name can potentially contain separator symbols (for
                    # instance folders on Windows)
                    cmd, name, rtype = (
                        splitted[0], ':'.join(splitted[1:-1]), splitted[-1])
......(...)

We just need to replace
line. Strip(). Decode ('ascii '). Split (': ')
with
line. Strip(). Decode (' 'utf8). Split (': ') ,
Run the file again to succeed.

[Solved] Pycharm from xx import xx Error: Unresolved reference

There is a problem: the related classes cannot be referenced, but these classes are indeed in the project

Analysis reason: import failed because the path does not correspond. Pycharm defaults to the source directory as the root directory of the project

Solution:

Search the corresponding item searchtest and select “sources”; Finally, be sure to “apply”

Set the folder where the package is placed as source, so that the module class of import can be found through these source folders as the root path, that is, find the imported things in these source folders

Or

[Solved] unknown error: DevToolsActivePort file doesn‘t exis

Scene

When using selenium, an error is reported one day, as shown in the title, and the code is as follows:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

option = Options()
option.add_argument('--headless')
option.add_argument('--no-sandbox')
option.add_argument('--disable-dev-shm-usage')

browser = webdriver.Chrome('./chromedriver',chrome_options=option)

browser.get('http://www.baidu.com/')
print(browser.title)
browser.quit()  

Solution:

First, ensure that the versions of Google Chrome and chrome River are consistent. If you are sure to be consistent, skip the following three steps.

    1. View Google Chrome version command:
google-chrome --version

I’m not sure about the command to view the chrome River version, but if you find the Google Chrome version, you can go to the following website to download the corresponding chrome river. If the versions are inconsistent, you need to download the consistent version from the download location

    1. google-chrome: http://dist.control.lth.se/public/CentOS-7/x86_64/google.x86_64/chromedriver: https://sites.google.com/a/chromium.org/chromedriver/downloads

Then the solution is to add an option

option.add_argument("--remote-debugging-port=9222")  # this

[Solved] CONDA ENV create and run Error: F environment.yml under win10

The error report description corresponds to the solution according to the serial number. Since everyone has different luck and problems when installing the software, read it as needed. (of course, I stumbled all the way, so friends who installed for the first time still suggest reading the error report description first, and then read it as needed)

Error reporting description

    1. the CMD console constantly displays the following warning information

    1. can’t find the version of Matplotlib = = 2.2.2 (if the corresponding version number of other packages can’t be found, it can also be handled as this) failed to build panda numpy

Solution

      1. the PIP command may be omitted in the environment.yml file. Just add the PIP command in the corresponding position of the file (the content indicated by the red arrow in the figure below)

      1. delete the following version number (as shown by the red arrow)

      1. failed to build panda numpy

 

      1. to be honest, in this environment, I ignored the error report, I found that it doesn’t seem to affect my subsequent operation. For example, I can open the gluon environment and open jupyter notepad in the gluon environment. (if there is any subsequent impact, I will continue to solve it and update the content)


Reference link:

        1. after installing miniconda3, run CONDA env create – F environment.yml and report an error miniconda installs numpy but Python can’t import it

Add a little knowledge. The command to delete the gluon environment is as follows:

conda remove -n gluon --all  

Resolve the error raise importerror, str (MSG) + ‘, please install the python TK package’ (valid for personal testing)

Solve the following similar error reports:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 42, in <module>
    raise ImportError, str(msg) + ', please install the python-tk package'
ImportError: /usr/lib/libtk8.5.so.0: invalid ELF header, please install the python-tk package

terms of settlement:

It appears that the library is corrupted
try sudo apt get remove Python TK ,
then sudo apt get clean download the package again,
sudo apt get install Python TK and then try importing again
this problem is resolved.

Another possibility is that you somehow messed up your apt/sources. List, and you installed a library for the wrong platform.

Reference link:
https://stackoverflow.com/questions/11043844/python-tk-package-not-recognized-in-python-2-7-3

Solve the runtimeerror in RNN: expected scalar type long but found float error

Project scenario:

Today, I saw the code of an RNN instance. I want to try to pass in RNN with my own data, but I can report an error.

Problem Description:

The error is runtimeerror: expected scalar type long but found float


Cause analysis:

The wrong input is as follows:

input=torch.tensor([ 0,  0,  0,  0,  0,  0,  0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
         0,  0,  0,  0,  0,  0])

The input in the example is as follows:

input=torch.tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0.,0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,0.]])

The reason is that I set dtype = torch.long when generating input


Solution:

Input = torch. Tensor (input, dtype = torch. Float)
specify dtype = torch. Float when generating the input tensor, and the resulting input will be of the following types

input=torch.tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0.,0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,0.]])

If it meets the requirements, no error will be reported
I didn’t expect to solve this problem. I spent most of the night. I really haven’t started yet. I don’t have enough skills.

Mayavi fails to get wglchoosepixelformatarb and other error reporting solutions

Development environment:

Win10 + Python 3.6 (virtual environment) + mayavi2 + pychar

Phenomenon:

This virtual environment used to work well in the past, but suddenly one day when drawing with mayavi, the following error appears:
error: in C: \ VPP \ standalone build \ VTK source \ rendering \ opengl2 \ vtkwin32openglrenderwindow.cxx, line 685

vtkWin32OpenGLRenderWindow (000001FACEF53880): failed to get wglChoosePixelFormatARB

ERROR: In C:\VPP\standalone-build\VTK-source\Rendering\OpenGL2\vtkWin32OpenGLRenderWindow.cxx, line 769

vtkWin32OpenGLRenderWindow (000001FACEF53880): failed to get valid pixel format.

ERROR: In C:\VPP\standalone-build\VTK-source\Rendering\OpenGL2\vtkOpenGLRenderWindow.cxx, line 785

vtkWin32OpenGLRenderWindow (000001FACEF53880): GLEW could not be initialized.

The program is a routine on the official website. The phenomenon is that after executing the program, if an error is reported, it exits after a few seconds.

Solution:

Consult relevant materials and initially locate the problem on environmental variables and drivers. Looking back on what I have done in recent days, I found that I have newly installed Master Lu and updated some drivers. So I tried to return the graphics card driver to the previous version, and the problem was successfully solved
the figure below shows that Master Lu has completed the rollback of the driver version


happy drawing again

Solve the problem of garbled code when Python connects to ADB

The following problems occurred when using pycharm to connect ADB:

  The following statements are used:

import os
os.system('adb version')

After converting the character set to GBK:

reason:

The computer could not find adb.exe and could not start ADB.

Solution: configure the ADB driver path to the environment variable path.

 

Running ADB in CMD succeeded:

Restart the computer

Successful operation in pychar:

Error condahtterror: http 000 connection failed

Error condahtterror: http 000 connection failed

CONDA create-n Python 36 Python =3.6 error condahtterror: http 000 connection failed for URL

CONDA create-n Python 36 python = = 3.6

ÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐÐ

C:\Users\Administrator>conda create -n python36 python==3.6
Collecting package metadata (current_repodata.json): failed

CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/win-64/current_repodata.json>
Elapsed: -

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
'https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/win-64'

terms of settlement

Modify the file. Condarc
in C: \ users \ administrator directory to read:

channels:
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
show_channel_urls: true

Hadoop reports an error. Cannot access scala.serializable and python MapReduce reports an error

Record the problems encountered when doing school Hadoop homework. The homework is more basic, that is, calling Hadoop through makefile to execute the MapReduce program written in advance

Error 1

An error occurred in the Hadoop wordcount code

java: cannot access scala.Serializable class file for scala.Serializable not found

An error is reported

Solution:
through this Q & A on stack overflow, I guess that the scala version is incompatible with the Hadoop version, so rollback to 2.7 will solve the problem

Error report 2

Attempting to run Python on Hadoop. But an error is reported. The error information is not detailed:
insert a picture description here

solution:
add the following at the beginning of the source code:

#!/usr/bin/env python
# -*-coding:utf-8 -*

(the problem with the coding format is really that I don’t know how to debug it.)

Bad file descriptor(C:\ci\zeromq 1602704446950\work\src\epoll.cpp:100)

Anaconda and jupyter notebook were installed on my new computer recently. When I opened the code with jupyter notebook, I found that it could not run. I turned back to CMD and found that the following errors interrupted jupyter Notebook:

I checked many answers. It is said that it was caused by using Chinese when registering a Windows account. Coincidentally, I happen to use Chinese, but I think it’s very troublesome to re create an account. I don’t believe in evil. I’ll see if there are other methods. I’ve tried and found it
according to others, it seems that when you install jupyterab or jupyter notebook, you will automatically install ipykernel, and then automatically install a higher version of pyzmq.
this problem is the problem of pyzmq version. Uninstall pyzmq 22.0.2, execute PIP install pyzmq = = 19.0.2, and install pyzmq 19.0.2.
as shown in the figure below,
paste a command to facilitate everyone to copy

pip uninstall pyzmq
pip install pyzmq==19.0.2 --user

After installing pyzmq version 19.0.2, I have the following error messages

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 
spyder 4.2.5 requires pyqt5<5.13, which is not installed. 
spyder 4.2.5 requires pyqtwebengine<5.13, which is not installed. 

But fortunately, it does not affect the use. You can open the code again by using jupyter notebook.