Category Archives: Python

[Solved] Error “incorrect padding” in decoding of Base64 module in Python

Problem description
decodes the field information encoded by Base64, which is normally decoded in the base64 encoding and decoding tool, but b64decode and A2B in the modules Base64 and binascii under python_ Decoding error in Base64 and other methods
the error information is as follows

---------------------------------------------------------------------------
Error                                     Traceback (most recent call last)
<ipython-input-11-787bc11958b4> in get_proxies(urls)
     14         try:
---> 15             raw = base64.b64decode(response)
     16         except Exception as r:

c:\program files\python3\lib\base64.py in b64decode(s, altchars, validate)
     86         raise binascii.Error('Non-base64 digit found')
---> 87     return binascii.a2b_base64(s)
     88 

Error: Incorrect padding

Solution

Base64 in Python is read in 4, so the field to be decoded should be a multiple of 4, not enough to add ‘=’

# The field a to be decoded is judged, and if it is a multiple of 4, it is unchanged, and if not, how much is missing, how much is made up
a = a + '=' * (4 - len(a) % 4) if len(a) % 4 != 0 else a

[Solved] Mindspot error: Error: runtimeerror:_kernel.cc:88 CheckParam] AddN output shape must be equal to input…

Mindspot rewrites withlosscell, traionestepcell interface

Error: runtimeerror:_kernel.cc:88 CheckParam] AddN output shape must be equal to input shape.Trace: In file add_impl.py(272)/ return F.addn((x, y))/

Solution: do not return multiple parameters in the construct method of withlosscell, otherwise an error will be reported in the gradoperation in the construct of traionestepcell.

“EncoderDecoder: ‘mit_b1 is not in the backbone registry‘“ [How to Solve]

Open source network:

https://github.com/NVlabs/SegFormer

When you train segformer, you report an error:

“EncoderDecoder: ‘mit_ b1 is not in the backbone registry'”

The direct cause of this exception is:

Find MIT in the dictionary_ B1, throw exception if not found:

    obj_type = args.pop('type')
    if isinstance(obj_type, str):
        obj_cls = registry.get(obj_type)
        if obj_cls is None:
            print(obj_type)
            raise KeyError(
                f'{obj_type} is not in the {registry.name} registry')

mit_ B1 is a class,

@BACKBONES.register_module()
class mit_b1(MixVisionTransformer):
    def __init__(self, **kwargs):
        super(mit_b1, self).__init__(
            patch_size=4, embed_dims=[64, 128, 320, 512], num_heads=[1, 2, 5, 8], mlp_ratios=[4, 4, 4, 4],
            qkv_bias=True, norm_layer=partial(nn.LayerNorm, eps=1e-6), depths=[2, 2, 2, 2], sr_ratios=[8, 4, 2, 1],
            drop_rate=0.0, drop_path_rate=0.1)

Self simulation exception:

Method 1, put the class MIT_ If B1 is commented out, the above exception will be reported, MIT_ B1 not registered

Method 2:

In mmseg/Models/builder.py

Add test code:

bbb= BACKBONES.get(‘mit_ b2’)

print(“bbb”,bbb)

Complete as follows:

import warnings

from mmcv.utils import Registry, build_from_cfg
from torch import nn

BACKBONES = Registry('backbone')
NECKS = Registry('neck')
HEADS = Registry('head')
LOSSES = Registry('loss')
SEGMENTORS = Registry('segmentor')

bbb= BACKBONES.get('mit_b2')

print("bbb",bbb)

The result BBB is empty, MIT_ B1 not registered

Method 3:

# from mmseg.models import BACKBONES
from mmseg.models.builder import BACKBONES

bbb= BACKBONES.get('mit_b1')

print("bbb",bbb)

The result BBB is empty, MIT_ B1 not registered

Find and quote MIT again_ B1 file

mix_transformer.py

You can register successfully. The code is as follows:

Create a registry under the root directory_ Demo.py, the test code is as follows:

# from mmseg.models import BACKBONES
from mmseg.models.backbones import mix_transformer
from mmseg.models.builder import BACKBONES

# from .mix_transformer import *


bbb= BACKBONES.get('mit_b1')

print("bbb2",bbb)

bbb2 < class ‘mmseg.models.backbones.mix_ transformer.mit_ b1’>

Normal test method:

In the root directory,

Create a registry_ Demo.py, the test code is as follows:

If the registration is successful, bbb2 is not empty. If the registration is not successful, bbb2 is empty.

from mmseg.models import BACKBONES

bbb= BACKBONES.get('mit_b1')

print("bbb2",bbb)

[Solved] Cv2.imshow Error: window.cpp:1274: error: (-2:Unspecified error) The function is not implemented.

Traceback (most recent call last):
File “/home/data/PJS/test_bed/img_show.py”, line 18, in
cv2.imshow(‘img’, img)
cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-9gwtlx3c/opencv/modules/highgui/src/window.cpp:1274: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function ‘cvShowImage’
Solution:

$ sudo apt install libgtk2.0-dev pkg-config
$ pip uninstall opencv-python opencv-contrib-python
$ pip install opencv-python

[Solved] Python Read bam File Error: &&OSError: no BGZF EOF marker; file may be truncated

1. Read FASTA file:

(1) Method 1: Bio Library

from Bio import SeqIO

fa_seq = SeqIO.read("res/sequence1.fasta", "fasta")

seq = str(fa_seq.seq)

seqs = [fa.seq for fa in SeqIO.parse("res/multi.fasta", "fasta")]

(2) Method 2: pysam Library

fa = pysam.FastaFile(‘filename’)

fa.references

fa.lengths

seq = fa.fetch(reference=’chr21’, start=a, end=b)

seq = fa.fetch(reference=’chr21’)

2. Convert sequence to string: str (FA. SEQ)


print("G Counts: ", fa.count("G"))

print("reverse: ", fa[::-1])

print("Reverse complement: ", fa.complement())

3、pysam read bam file error:
File “pysam/libcalignmentfile.pyx”, line 742, in pysam.libcalignmentfile.AlignmentFile.__cinit__
File “pysam/libcalignmentfile.pyx”, line 952, in pysam.libcalignmentfile.AlignmentFile._open
File “pysam/libchtslib.pyx”, line 365, in pysam.libchtslib.HTSFile.check_truncation
OSError: no BGZF EOF marker; file may be truncated
The file can be read this way, i.e. by adding ignore_truncation=True:

bam = pysam.AlignmentFile(‘**.bam’, "rb", ignore_truncation=True)

4. Cram format file: it has the characteristics of high compression and has a higher compression rate than BAM. Most files in cram format may be used in the future

5. Comparison data (BAM/cram/SAM), variation data: (VCF/BCF)

6. Get each read

for read in bam:
    read.reference_name # The name of the chromosome to which the reference sequence is compared.

    read.pos # the position of the read alignment

    read.mapq # the quality value of the read comparison

    read.query_qualities # read sequence base qualities

    read.query_sequence # read sequence bases

    read.reference_length # 在reference genome上read比对的长度

7. Read cram file

cf = pysam.AlignmentFile(‘*.cram’, ‘rc’)

8. Read SAM file

samfile = pysam.AlignmentFile(‘**.sam’, ‘r’)

9. Get a region in Bam file

for r in pysam.AlignmentFile(‘*.bam’, ‘rb’).fetch(‘chr21’, 300, 310):
    pass  # This is done provided that the *.bam file is indexed

Python scatter chart error: TypeError: object of type ‘NoneType’ has no len()

housing.plot(kind="scatter", x="longitude", y="latitude", alpha=0.4,
             s=housing["population"]/100, label="population", figsize=(10,7),
             c="median_house_value", cmap=plt.get_cmap("jet"), colorbar=True,
             sharex=False)
plt.legend()

TypeError: object of type ‘NoneType’ has no len()

After removing label =’population ‘, the picture can be displayed

but a new warning appears

No handles with labels found to put in legend.

I didn’t finish the adjustment for two hours until I updated Matplotlib:) it is OK.

[Solved] Original error was: No module named ‘numpy.core._multiarray_umath‘

Original error was: No module named ‘numpy.core._ multiarray_ umath’

Problem description analysis and solution

Problem description

Numpy is clearly installed and frequently used without errors, but the following errors occur:
no module named 'numpy.core_ multiarray_ umath

Analyze problems

    1. the model was not saved successfully due to an error in saving the model. At this time, delete the saved model, and the numpy version is too low </ OL>

 

Solution:

Display version PIP show numpy

Upgrade version PIP install -- upgrade numpy

Solve the problem by uninstalling and reloading:

pip uninstall numpy
 
pip install --upgrade numpy
#or
pip  install numpy -i https://pypi.tuna.tsinghua.edu.cn/simple

How to Solve Pytorch DataLoader Loading Error: UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xe5 in position 1023

The complete error reports are:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 301, in _on_run
    r = r.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe5 in position 1023: unexpected end of data

 

Solution:

This is not to solve the problem of Unicode decodeerror: 'UTF-8' codec can't decode byte 0xe5 in position 1023: unexpected end of data , but to solve the problem that the model cannot be iterated. The method is as follows:

Replace the data source in tensor format with numpy format, then convert it to tensor , and finally put it into dataloader

Unicode decodeerror will still be reported when moving from numpy to tensor, but the loaded data will not be encapsulated in the dataloader, resulting in the stop of the data cycle and the training of the model will not be affected.

from keras.preprocessing.text import Tokenizer error: AttributeError: module ‘tensorflow.compat.v2‘ has..

Error from keras.preprocessing.text import tokenizer: attributeerror: module ‘tensorflow. Compat. V2’ has

Import the vocabulary mapper tokenizer in keras in NLP code

from keras.preprocessing.text import Tokenizer

Execute code and report error:

AttributeError: module 'tensorflow.compat.v2' has no attribute '__ internal__'

Baidu has been looking for the same error for a long time, but it sees a similar problem. Just change the above code to:

from tensorflow.keras.preprocessing.text import Tokenizer

That’s it!

AttributeError: DatetimeProperties object has no attribute

1.Question

AttributeError: ‘DatetimeProperties’ object has no attribute ‘weekday_ name’

Simple test, run the following code:

import pandas as pd

# Create dates
dates = pd.Series(pd.date_range("7/26/2021", periods=3, freq="D"))
# Check the day of the week
print(dates.dt.weekday_name)
# Show only values
print(dates.dt.weekday)

2.Solution

weekday_ Change name to day_ name()

import pandas as pd

# Create dates
dates = pd.Series(pd.date_range("7/26/2021", periods=3, freq="D"))
# Check the day of the week
print(dates.dt.day_name())
# Show only values
print(dates.dt.weekday)

For example:

Type