Tag Archives: Deep learning

[Solved] matplotlib.units.ConversionError: Failed to convert value(s) to axis units: ‘LiR‘

 

solve the problem


No handles with labels found to put in legend.

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5.py", line 508, in _draw_idle
    self.draw()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 670, in draw
    bbox, info, descent = textobj._get_layout(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 276, in _get_layout
    key = self.get_prop_tup(renderer=renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 831, in get_prop_tup
    x, y = self.get_unitless_position()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 813, in get_unitless_position
    x = float(self.convert_xunits(self._x))
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 180, in convert_xunits
    return ax.xaxis.convert_units(x)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1553, in convert_units
    f'units: {x!r}') from e
matplotlib.units.ConversionError: Failed to convert value(s) to axis units: 'LiR'
Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1550, in convert_units
    ret = self.converter.convert(x, self.units, self)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\category.py", line 52, in convert
    'Missing category information for StrCategoryConverter; '
ValueError: Missing category information for StrCategoryConverter; this might be caused by unintendedly mixing categorical and numeric data
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5.py", line 508, in _draw_idle
    self.draw()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 670, in draw
    bbox, info, descent = textobj._get_layout(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 276, in _get_layout
    key = self.get_prop_tup(renderer=renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 831, in get_prop_tup
    x, y = self.get_unitless_position()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 813, in get_unitless_position
    x = float(self.convert_xunits(self._x))
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 180, in convert_xunits
    return ax.xaxis.convert_units(x)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1553, in convert_units
    f'units: {x!r}') from e
matplotlib.units.ConversionError: Failed to convert value(s) to axis units: 'LiR'

Error:
matplotlib.units.Conversion error: failed to convert value to axis unit: ‘LiR’

Solution:
matplotlib version is low, update the matplotlib library to version 3.3.2 or higher!

How to Solve Yolox Training C Disk Full Issue

0. Problem description: COCO dataset training to half suddenly interrupted, look at the C disk shows red, there is not much memory (training, generated in AppData/Temp in the temporary file too much)
As shown in the figure: as the epoch increases, the file is getting bigger and bigger (the figure is still yolox-tiny), if we use yolox-x, the C drive is directly full!

1. Problem Cause.
YOLOX-main/yolox/evaluators/coco_evaluator.py in line 203 or so **tempfile.mkstemp()** after creating the file, no close() and remove() operations are performed
The following figure.

2. Solution methods
(1) Method 1
As shown above, add os.close(_) and os.remove(tmp) two lines of code, directly delete the file just created after use. Note: import os module at the beginning]
(2) Method 2
The problem is already known, you can use with…as… to create, automatically delete and close the file.
(3) Method 3
If you want to keep each temporary file, and do not want to C drive blow up, then directly change the save location to a custom path.
Code location: Anoconda/envs/using-environment/Lib/tempfile.py in line 159-185.
directly change the direct dirlist operation to the user-defined folder location, as follows:

(4) Method 4
Manually clean up files in temp at regular intervals
Note: VOC format dataset training, no temporary files are generated because it uses the with…as… file creation method. For details, please refer to the end of voc_evaluater.py

[Solved] MindSpore Error: “operation does not support the type kMetaTypeNone“

Environment:
Hardware Environment(Ascend/GPU/CPU): All
Software Environment:
MindSpore version (source or binary): 1.6.0 & Earlier versions
Python version (e.g., Python 3.7.5): 3.7.6
OS platform and distribution (e.g., Linux Ubuntu 16.04): Ubuntu
GCC/Compiler version (if compiled from source): gcc 9.4.0
python code examples

from mindspore import nn

class Net(nn.Cell):
    def __init__(self):
        super(Net, self).__init__()

    def construct(self, x):
        return self.y + x

net = Net()
output = net(1)

Error reporting information

Traceback (most recent call last):
  File "test_self.py", line 11, in <module>
    output = net(1)
  File "mindspore\nn\cell.py", line 477, in __call__
    out = self.compile_and_run(*args)
  File "mindspore\nn\cell.py", line 803, in compile_and_run
    self.compile(*inputs)
  File "mindspore\nn\cell.py", line 790, in compile
    _cell_graph_executor.compile(self, *inputs, phase=self.phase, auto_parallel_mode=self._auto_parallel_mode)
  File "mindspore\common\api.py", line 632, in compile
    result = self._graph_executor.compile(obj, args_list, phase, self._use_vm_mode())
RuntimeError: mindspore\ccsrc\frontend\operator\composite\multitype_funcgraph.cc:162 GenerateFromTypes] The 'add' operation does not support the type [kMetaTypeNone, Int64].
The supported types of overload function `add` is: [Tuple, Tuple], [RowTensor, Tensor], [Tensor, Tensor], [List, List], [Tensor, List], [List, Tensor], [String, String], [Tuple, Tensor], [kMetaTypeNone, kMetaTypeNone], [Number, Number], [Number, Tensor], [Tensor, Number], [Tensor, Tuple].

The function call stack (See file 'rank_0/om/analyze_fail.dat' for more details):
# 0 In file test_self.py(8)
        return self.y + x

 

Solution:

Since the execution error is caused by using an undefined variable, the solution is to define the variable in the network’s initialization function __init__(self):. Note that only variables defined as members of self can be used in the construct(self, x) method.

from mindspore import nn

class Net(nn.Cell):
    def __init__(self):
        super(Net, self).__init__()
        self.y = 1.0

    def construct(self, x):
        return self.y + x

net = Net()
output = net(1)

If you write self.y = 1.0 instead of y = 1.0, you will also get an error because the variable is undefined.

RuntimeError: No HIP GPUs are available [How to Solve]

Error reason:

The code uses GPU accelerated torch, but I downloaded the C + + version of torch. Use PIP list to find packages as follows:

 

Solution:

Uninstall torch and torchvision, and then use command pip install torch==1.7.0+cu110 torchvision==0.8.0+cu110 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html to install it.

[Solved] RuntimeError: Error(s) in loading state_dict for Model: Missing key(s) in state_dict

Error message:

Using the network weights trained by FCN, UNET and deeplab, an error is reported when loading the model:

RuntimeError: Error(s) in loading state_dict for Model: Missing key(s) in state_dict....

Training environment:

CPU:Intel E5
GPU: 3090*2
Pytorch1. ten

Solution:

Solve the mismatch problem
method 1 (invalid): the dictionary cannot match strictly. When the model is loaded, use the following code to add false to solve the mismatch problem

net.load_state_dict(t.load(ckpt_path),False)

Note: this method unlocks the strict matching and can ignore the error report. However, the model loaded by this method has problems in the actual segmentation effect due to the mismatch of parameters. Use it with caution!

Method 2: the network is the same, the only difference is that NN is used in the training process The dataparallel() method calls two graphics cards for training, so before loading the model weight, the model is also suitable for dataparallel packaging, which can solve the above error reporting problem.

net = nn.DataParallel(net)
net = net.to(device)

[Solved] RuntimeError: unexpected EOF, expected 73963 more bytes. The file might be corrupted.

RuntimeError: unexpected EOF, expected 73963 more bytes. The file might be corrupted.

Problem Description:

When the project executes Python script, when downloading the pre training model weight of pytorch, if the weight is not downloaded due to network instability and other reasons, an error will be reported runtimeerror: unexpected EOF, expected xxxxx more bytes The file might be corrupted.

Cause analysis:

This error indicates that the downloaded weight file may be damaged. You need to delete the damaged weight file and execute the code to download again.

Solution:

To find the location where the downloaded weight file is saved, this paper analyzes three situations:

1. Windows System & Anaconda Environment

The path of download is D:\Anaconda3\anaconda\envs\yypy36\Lib\site-packages\facexlib\weightsdetection_Resnet50_Final.pt, so you need go to this folder and delete the weight file as the screenshot below:
2. Windows system & Python environment:

The code automatically downloads the model weights file and saves it to the C:\Users\username/.cache\torch\checkpoints folder. Note that .cache may be a hidden file, you need to view the hidden file to see it, just delete the weight file.

3. Linux systems:
Linux system weights files are usually saved under: \home\username\.cache\torch. Note that .cache is a hidden folder and will only be shown if you press ctrl+Alt+H in winSCP; or, in the home directory, use ls -a to show it. root mode, the default path for downloaded weight files is under: /root/.cache/torch/checkpoints. Just delete the weight file.

In the above three cases, after deleting the weight file, execute the code again to download again.

Additional:

If the execution program downloads the code too slowly or the network stability is not good, we can directly download it manually from the website where the weight file is located and put it in the specified location. The Linux system can adopt WGet method.

wget -P Local path where the weights are saved Address of the weights

If the download is interrupted, WGet supports continuous transmission at breakpoints. Add a parameter - C :

wget -P Local path where weights are saved -c Address of weights

eg:

wget -P /home/20220222Proj/pretrained_models -c https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth

TensorFlow-gpu Error: failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected

Error Messages:

failed call to cuInit: CUDA_ERROR_NO_Device: no CUDA capable device is detected
this is also what I encountered when running a CNN SVM classifier program of tensorflow GPU today. It’s not the problem of the program. It’s our graphics card.

Solution:

import tensorflow as tf

config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.compat.v1.InteractiveSession(config=config)

Just add these lines of code to the head of the code, and you don’t need to write this code belowos.environ['CUDA_VISIBLE_DEVICES'] = '/gpu:0'

[Solved] AttributeError: ‘NoneType‘ object has no attribute ‘astype‘

Problem description

When running the code, an error is reported attributeerror: 'nonetype' object has no attribute 'asttype', as shown in the following figure:

Traceback (most recent call last):
  File "work/person_search-master/tools/demo.py", line 82, in <module>
    query_feat = net.inference(query_img, query_roi).view(-1, 1)
  File "/home/featurize/work/person_search-master/tools/../lib/models/network.py", line 178, in inference
    processed_img, scale = img_preprocessing(img)
  File "/home/featurize/work/person_search-master/tools/../lib/datasets/data_processing.py", line 49, in img_preprocessing
    processed_img = img.astype(np.float32)
AttributeError: 'NoneType' object has no attribute 'astype'

Solution:

According to the error message, the error is reported because the img is a ‘NoneType’ object, so the ‘astype’ property cannot be used.

In general, the above error occurs when the img does not exist, so

  • You need to make sure that the image exists in the appropriate path in the code.
  • You need to run the command python XXX.py in the correct directory to ensure that XXX.py searches for the image in the correct directory.

[Solved] torchvision Error: UserWarning: Failed to load image Python extension: Could not find module

Tochvision error: userwarning: failed to load image Python extension: could not find module

One reason is that the version of torchvision is too high. It is suspected that the new version of torchvision has its own pot. At first, according to the official website

pip3 install torch==1.10.1+cu102 torchvision==0.11.2+cu102 torchaudio===0.10.1+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html

After that, the error of image will be reported. You only need to reduce the version of torch vision. For example, you can enter it in your Anaconda prompt

conda activate ltorch # ltorch is the name of the virtual environment I created
pip install torchvision==0.10.1+cu102 -f https://download.pytorch.org/whl/cu102/torch_stable.html

It was 0.11.2, but I reduced it to 0.10.1. Import without error:

Of course, you can change the version according to the following link

https://download.pytorch.org/whl/cu102/torch_stable.html

Keras import package error: importerror: cannot import name ‘get_ config‘

ImportError: cannot import name ‘get_config’

Traceback (most recent call last):
  File "siameseNet.py", line 6, in <module>
    from keras.layers import Merge
  File "/usr/local/lib/python3.6/site-packages/keras/__init__.py", line 25, in <module>
    from keras import models
  File "/usr/local/lib/python3.6/site-packages/keras/models.py", line 19, in <module>
    from keras import backend
  File "/usr/local/lib/python3.6/site-packages/keras/backend.py", line 39, in <module>
    from tensorflow.python.eager.context import get_config
ImportError: cannot import name 'get_config'

Solution:

pip install keras == 2.1.0 --force-reinstall

[Solved] RuntimeError: scatter(): Expected dtype int64 for index


RuntimeError: scatter(): Expected dtype int64 for index

1. Error reporting reason:

Scatter requires the data to be of type Int64, and I wrote torch when defining tensor Tensor (x) should be written as torch Longtensor (x), specified as Int64 type.

2. Solutions

Find the definition method of the original data and change it
generally, dtype = NP int64; dtype=np.
in float32 (most definition functions have dtype attribute)
it is better to have the same number of bits of int and float

import numpy as np
a = np.random.randint(100, size=(10**6), dtype="int64")
print(a)
print(type(a[0]))