Category Archives: Python

python minio client Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certific

Built minio service, support https, python call reported error.

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='xx.xx.xx.xxx', port=9000): Max retries exceeded with url: /allstruct?location= (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1108)')))

Ignore the certificate error issue and try out the demo script


import os
from minio import Minio
import urllib3
from urllib.parse import urlparse
import certifi
from minio.commonconfig import REPLACE, CopySource
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)


minio_endpoint = os.getenv("MINIO_ENDPOINT", "https://xxx.xxx.xxx.xxx:9000")
secure = True

minio_endpoint = urlparse(minio_endpoint)


if minio_endpoint.scheme == 'https':
    secure = True

ok_http_client=urllib3.PoolManager(
            timeout=urllib3.util.Timeout(connect=10, read=10),
            maxsize=10,
            cert_reqs='CERT_NONE',
            ca_certs= os.environ.get('SSL_CERT_FILE') or certifi.where(),
            retries=urllib3.Retry(
                total=5,
                backoff_factor=0.2,
                status_forcelist=[500, 502, 503, 504]
            )
        )


minioClient = Minio(minio_endpoint.netloc,
                    access_key='username',
                    secret_key='password',
                    http_client=ok_http_client,
                    secure=secure)

print(minioClient.list_buckets())

dtype=np.int error: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`.

There is a line of code c = np.fromfile(“b.dat”, dtype=np.int, sep=”,”) when running Error,The content is as follows:

DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20 .0-notes.html#deprecations
c = np.fromfile(“b.dat”, dtype=np.int, sep=”,”)

Reason: numpy1.20 has deprecated np.int, checked my version

The numpy version has reached 1.23.2

Solution: Change to np.int16/np.int32 directly to specific bytes or change to np.int_

Run again, problem-solved!

 

Ubuntu22.04 OpenHarmony Execute hb set Error [How to Solve]

Error message

Recently using Ubuntu22.04 to build OpenHarmony compilation environment, When executing the hb set command to select the development board, the following error appears:
Insert picture description here
The detailed log is as follows:

fangye@fangye-virtual-machine:~/niobeu4_src$ hb set
[OHOS ERROR] Traceback (most recent call last):
[OHOS ERROR] File "/home/fangye/.local/lib/python3.10/site-packages/hb/__main__.py", line 84, in main
[OHOS ERROR] status = args.command(args)
[OHOS ERROR] File "/home/fangye/niobeu4_src/build/lite/hb_internal/set/set.py", line 45, in exec_command
[OHOS ERROR] return set_product() == 0
[OHOS ERROR] File "/home/fangye/niobeu4_src/build/lite/hb_internal/set/set.py", line 62, in set_product
[OHOS ERROR] product_info = Product.product_menuconfig()
[OHOS ERROR] File "/home/fangye/niobeu4_src/build/lite/hb_internal/common/product.py", line 260, in product_menuconfig
[OHOS ERROR] product = menu.list_promt('product', 'Which product do you need" alt="" />

 

Solution:

Modify the python script reported error: modify from collections import Mapping tofrom collections.abc import Mapping:

vim ~/.local/lib/python3.10/site-packages/prompt_toolkit/styles/from_dict.py

在这里插入图片描述

onnx error: ImportError: /home/dy/anaconda3/envs/torch/lib/python3.6/site-packages/onnx…

onnx error:


import onnx



Traceback (most recent call last):
  File "torch2onnx.py", line 3, in 
    import onnx
  File "/home/dy/anaconda3/envs/torch/lib/python3.6/site-packages/onnx/__init__.py", line 5, in 
    from .onnx_cpp2py_export import ONNX_ML
ImportError: /home/dy/anaconda3/envs/torch/lib/python3.6/site-packages/onnx/onnx_cpp2py_export.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN6google8protobuf7Message17CopyWithSizeCheckEPS1_RKS1_

Solution:

git clone https://github.com/onnx/onnx.git

cd onnx

git submodule update –init –recursive

# Optional: prefer lite proto

export CMAKE_ARGS=”-DONNX_USE_PROTOBUF_SHARED_LIBS=ON”

export CMAKE_ARGS=-DONNX_USE_LITE_PROTO=ON

pip install -e .

[Solved] Python sqlite3.OperationalError: near “?“: syntax error

code

con = sqlite3.connect('db.sqlite3')
cur = con.cursor()
sql = 'SELECT id FROM ?WHERE source_id = ?'
cur.execute(sql, ('article_table', 41945,))
one = cur.fetchone()
con.close()

report errors

sqlite3.OperationalError: near "?": syntax error

reason

sqlite3 placeholder (?) Cannot be used for column or table names.
Placeholders are used to insert or retrieve values of data from the database in order to prevent SQL injection.

Solution:

con = sqlite3.connect('db.sqlite3')
cur = con.cursor()
sql = 'SELECT id FROM article_table WHERE source_id = ?'
cur.execute(sql, (41945,))
one = cur.fetchone()
con.close()

[Solved] raise KeyError(key) from err KeyError: ‘Dates‘

Description of error reporting:

Today, when reading Excel data and processing the data, an error is reported as follows:

Error reason:

The Excel table data read in by pandas is not aligned. Please check the Excel table data read in
I print out the dataframe after reading it into Excel

Solution:

Delete Sheet2 and Sheet3 in the Excel table, so that Pandas will align after reading the Excel table data

The root cause of such problems is that the data is not aligned

Debug method: 1. Print out the dataframe data and check the format of the data after it is read in. 2. Then adjust the read data

[Solved] RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm

Problem

After training to a certain number of iterations, an error is reported:
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)

Possible causes

  • The shape dimension does not match
  • Variables are not on the same device
  • pytorch and cuda versions do not match

Solution

Add os.environ['CUDA_VISIBLE_DEVICES'] = '0' at the beginning of the train.py file, and set device='cuda'.
But there is a strange phenomenon: if you do not set the visible gpu, but specify device='cuda:0', it will also report an error.

[Solved] RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place

yolov5 Error: RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place

Solution:
In model/yolo.py file

        for mi, s in zip(m.m, m.stride):  # from
            b = mi.bias.view(m.na, -1)  # conv.bias(255) to (3,85)
            b[:, 4] += math.log(8/(640/s) ** 2)  # obj (8 objects per 640 image)
            b[:, 5:] += math.log(0.6/(m.nc - 0.99)) if cf is None else torch.log(cf/cf.sum())  # cls
            mi.bias = torch.nn.Parameter(b.view(-1), requires_grad=True)

Add with torch.no_grad(): as follows

        for mi, s in zip(m.m, m.stride):  # from
            b = mi.bias.view(m.na, -1)  # conv.bias(255) to (3,85)
            with torch.no_grad():
                b[:, 4] += math.log(8/(640/s) ** 2)  # obj (8 objects per 640 image)
                b[:, 5:] += math.log(0.6/(m.nc - 0.99)) if cf is None else torch.log(cf/cf.sum())  # cls
            mi.bias = torch.nn.Parameter(b.view(-1), requires_grad=True)

——> The root cause is to add:

with torch.no_grad():

[Solved] AttributeError: module ‘time‘ has no attribute ‘clock‘

preface:

When the blogger runs the previous code, he encounters this problem because time method is not available in the higher version of Python.

Running code:

Error:

Traceback (most recent call last):
  File "/media/gis/DISCK_21/code/CoANet-main/process/create_crops.py", line 139, in <module>
    main()
  File "/media/gis/DISCK_21/code/CoANet-main/process/create_crops.py", line 118, in main
    start = time.clock()
AttributeError: module 'time' has no attribute 'clock'

Solution:

start = time.time()

[Solved] cv2.error: OpenCV(4.6.0) D:\a\opencv-python\opencv-python\opencv\modules\……

Project Scenarios

Run the opencv official website example code facedetect.py

#!/usr/bin/env python

'''
face detection using haar cascades

USAGE:
    facedetect.py [--cascade <cascade_fn>] [--nested-cascade <cascade_fn>] [<video_source>]
'''

# Python 2/3 compatibility
from __future__ import print_function

import numpy as np
import cv2 as cv

# local modules
from video import create_capture
from common import clock, draw_str


def detect(img, cascade):
    rects = cascade.detectMultiScale(img, scaleFactor=1.3, minNeighbors=4, minSize=(30, 30),
                                     flags=cv.CASCADE_SCALE_IMAGE)
    if len(rects) == 0:
        return []
    rects[:,2:] += rects[:,:2]
    return rects

def draw_rects(img, rects, color):
    for x1, y1, x2, y2 in rects:
        cv.rectangle(img, (x1, y1), (x2, y2), color, 2)

def main():
    import sys, getopt

    args, video_src = getopt.getopt(sys.argv[1:], '', ['cascade=', 'nested-cascade='])
    try:
        video_src = video_src[0]
    except:
        video_src = 0
    args = dict(args)
    cascade_fn = args.get('--cascade', "data/haarcascades/haarcascade_frontalface_alt.xml")
    nested_fn = args.get('--nested-cascade', "data/haarcascades/haarcascade_eye.xml")

    
    cascade = cv.CascadeClassifier(cv.samples.findFile(cascade_fn))
    nested = cv.CascadeClassifier(cv.samples.findFile(nested_fn))
    cam = create_capture(video_src, fallback='synth:bg={}:noise=0.05'.format(cv.samples.findFile('data/lena.jpg')))#此处的data文件夹是从opencv示例simple包中复制到该项目中。

    
    while True:
        _ret, img = cam.read()
        gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
        gray = cv.equalizeHist(gray)

        t = clock()
        rects = detect(gray, cascade)
        vis = img.copy()
        draw_rects(vis, rects, (0, 255, 0))
        if not nested.empty():
            for x1, y1, x2, y2 in rects:
                roi = gray[y1:y2, x1:x2]
                vis_roi = vis[y1:y2, x1:x2]
                subrects = detect(roi.copy(), nested)
                draw_rects(vis_roi, subrects, (255, 0, 0))
        dt = clock() - t

        draw_str(vis, (20, 20), 'time: %.1f ms' % (dt*1000))
        cv.imshow('facedetect', vis)

        if cv.waitKey(5) == 27:
            break

    print('Done')

if __name__ == '__main__':
    print(__doc__)
    main()
    cv.destroyAllWindows()

Problem description

Running the above code reports the following error:

[ WARN:[email protected]] global D:\a\opencv-python\opencv-python\opencv\modules\core\src\utils\samples.cpp (61) cv::samples::findFile cv::samples::findFile('data/haarcascades/haarcascade_frontalface_alt.xml') => ''
Traceback (most recent call last):
  File "E:\Pycharm\Pycharm7\facedetect.py", line 81, in <module>
    main()
  File "E:\Pycharm\Pycharm7\facedetect.py", line 48, in main
    cascade = cv.CascadeClassifier(cv.samples.findFile(cascade_fn))
cv2.error: OpenCV(4.6.0) D:\a\opencv-python\opencv-python\opencv\modules\core\src\utils\samples.cpp:64: error: (-2:Unspecified error) OpenCV samples: Can't find required data file: data/haarcascades/haarcascade_frontalface_alt.xml in function 'cv::samples::findFile'

Cause analysis:

According to the error location:

cascade = cv.CascadeClassifier(cv.samples.findFile(cascade_fn))

It can be found that the file paths of “haarcascade_frontalface_alt.xml” and “data/haarcascades/haarcascade_eye.xml” are not correct.


Solution:

“haarcascade_frontalface_alt.xml” and “data/haarcascades/haarcascade_eye.xml” are under the cv2 package installed in python, just change the path in the code to a relative path.

The code is changed as follows:

cascade_fn = args.get('--cascade', "D:/Python/yingyong/Lib/site-packages/cv2/data/haarcascades/haarcascade_frontalface_alt.xml")
    nested_fn  = args.get('--nested-cascade', "D:/Python/yingyong/Lib/site-packages/cv2/data/haarcascades/haarcascade_eye.xml")

Run again to run normally.