Category Archives: Python

Python Pandas Error: KeyError: 0 [How to Solve]

Keyerror: error reported by 0

The following are error codes

I call my own library function and use apply to realize vlookup in Excel. The following is the code

data2 = super_function.vlook_up(data1, ['material group', 'material description'], data, ['material group', 'material group description'])

Error message

KeyError: 0

Error reporting reason

This kind of error reporting is due to the index problem. As a result, some numbers were deleted during the original data processing, resulting in the index starting from 6 instead of 0.

Solution:

Just reassign the index

data1.index = list(range(len(data1)))

result

Run successfully.

No module named ‘google.rpc‘ [How to Solve]

The occurrence of this error is also inexplicable. The error is reported when importing the KFP module after installing the KFP module, but there is no error in its own installation process.

Open KFP to locate the error position:

According to the past experience, after all the troubleshooting, I still can’t find the cause of the error. At this time, I have to open the local Google installation directory. The screenshot is as follows:

It was found that there was no RPC directory. The reason was found. Because I really didn’t want to study more, I created a new virtual environment and installed it. After that, I copied all the modules in Google and copied them to my current Google directory, as shown below:

Re import, solve the problem.

[Solved] Python Error: An attempt has been made to start a new process before the current process has finished …

This error usually occurs in Windows systems using multiple processes. For example, execute the following code in pychar:

import torch
import torch.utils.data as Data
import numpy as np
from sklearn.datasets import load_iris

iris_x, irisy = load_iris(return_X_y=True)
print("iris_x.dtype:", iris_x.dtype)
print("irisy:", irisy.dtype)

## transform the training set x into a tensor, and the training set y into a tensor
train_xt = torch.from_numpy(iris_x.astype(np.float32))
train_yt = torch.from_numpy(irisy.astype(np.int64))
print("train_xt.dtype:", train_xt.dtype)
print("train_yt.dtype:", train_yt.dtype)

## After converting the training set into a tensor, use TensorDataset to collate X and Y together
train_data = Data.TensorDataset(train_xt, train_yt)
## Define a data loader to batch the training dataset
train_loader = Data.DataLoader(
    dataset=train_data, ## the dataset to use
    batch_size=10, # # Batch sample size
    shuffle=True, # Break up the data before each iteration
    num_workers=2, # [Note: 2 processes are used here]
)

## Check if the dimensionality of the samples of a batch of the training dataset is correct
for step, (b_x, b_y) in enumerate(train_loader):
    if step > 0:
        break
## Output the dimensions of the training image and the labels, and the data type
print("b_x.shape:", b_x.shape)
print("b_y.shape:", b_y.shape)
print("b_x.dtype:", b_x.dtype)
print("b_y.dtype:", b_y.dtype)


## --------- -The correct result is as follows -------- --

# iris_x.dtype: float64
# irisy: int32
# train_xt.dtype: torch.float32
# train_yt.dtype: torch.int64
# b_x.shape: torch.Size([10, 4])
# b_y.shape: torch.Size([10])
# b_x.dtype: torch.float32
# b_y.dtype: torch.int64

The following errors will be reported. (no error will be reported when running in jupyter notebook under the same environment. I don’t know why…)

RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

 

Solution 1:

Remove the statement setting up multiple processes. In this example, comment or delete the following line.

num_workers=2,  # [Note: 2 processes are used here]

Solution 2:

Move the code part of calling multiple processes to [if _name_ = = ‘_main_’:].

if __name__ == '__main__':
    ##  Check if the dimensionality of the samples of a batch of the training dataset is correct
    for step, (b_x, b_y) in enumerate(train_loader):
        if step > 0:
            break
        ## Output the dimensions of the training image and the dimensions of the labels, and the data type
    print("b_x.shape:", b_x.shape)
    print("b_y.shape:", b_y.shape)
    print("b_x.dtype:", b_x.dtype)
    print("b_y.dtype:", b_y.dtype)

However, in pychart, the part before [for step, (b_x, b_y) in enumerate (train_loader):] will be executed twice.

## ——————————The result of running in Pycharm is as follows——————————
iris_x.dtype: float64
irisy: int32
train_xt.dtype: torch.float32
train_yt.dtype: torch.int64
iris_x.dtype: float64
irisy: int32
train_xt.dtype: torch.float32
train_yt.dtype: torch.int64
b_x.shape: torch.Size([10, 4])
b_y.shape: torch.Size([10])
b_x.dtype: torch.float32
b_y.dtype: torch.int64

[Solved] Huawei OBS Python SDK download picture error: nosuchkey

Background:

In the past, Huawei OBS was used to download pictures (that is, to view pictures through a browser), and the address was used to access OBS directly.
for example,
endpoint: obs-example-domain.cn
picture name: QCX% 2F1% 2f20210804% 2f2db3c4bb-0c2c-4c3c-84e0-7e131c1e8db61628047890560.jpg

Access address:

http://obs-example-domain.cn/qcx%2F1%2F20210804%2F2db3c4bb -0c2c-4c3c-84e0-7e131c1e8db61628047890560. jpg

WGet can download pictures from the above address.

However, if you use Python SDK to access, an error will be reported:


AK = 'PLAU4DD8EYVXSA****UL'
SK = 'MdNZCKgSwt9Qgq6ZXtaF7wtZOd8********xEiv'
server = "http://obs-example-domain.cn"
bucketName = 'qcx'
obsClient = ObsClient(access_key_id=AK, secret_access_key=SK, server=server)
name = "qcx%2F1%2F20210804%2F2db3c4bb-0c2c-4c3c-84e0-7e131c1e8db61628047890560.jpg"
resp = obsClient.getObject(bucketName, name, loadStreamInMemory=True)
print(resp.body)

Output: the specified key does not exist

In the above procedures: name = QCX% 2F1% 2f20210804% 2f2db3c4bb-0c2c-4c3c-84e0-7e131c1e8db61628047890560 jpg

Solution:

The picture name above is actually URLEncode. The original string urlcode can obtain:

qcx/1/20210804/2db3c4bb-0c2c-4c3c-84e0-7e131c1e8db61628047890560. jpg

Change the name in the above code to the picture name after URLDecode, that is:

name = "qcx/1/20210804/2db3c4bb-0c2c-4c3c-84e0-7e131c1e8db61628047890560.jpg"

You can get the picture correctly.

The function completion code

Picture browsing completed line:

Browser request img URL -> nginx -> API (with SK AK) – > obs -> Response API – > nginx -> browser

Python 2.7 (only this version is available on the server and cannot be upgraded to 3.X)

#coding=utf-8
from BaseHTTPServer import BaseHTTPRequestHandler
import urllib
import cgi
import os
import urllib
# print urllib.unquote('%E4%B8%89%E7%94%9F%E4%B8%89%E4%B8%96')

from obs import ObsClient

AK = 'PLAU4DD8EYVXSA****UL'
SK = 'MdNZCKgSwt9Qgq6ZXtaF7wtZOd8********xEiv'
server = "http://obs.cn-dchlw-1.digitalgd.com.cn"
bucketName = 'qcxx'
obsClient = ObsClient(access_key_id=AK, secret_access_key=SK, server=server)

cwd = os.getcwd()

class ObsHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        path = urllib.unquote(self.path)
        buf = ""
        query_list = path.split("/")  #auto urldecode
        #print(query_list)
        if query_list[1] == "obs":
            name = "/".join(query_list[3:]).replace("?","")   # for online
            #name = urllib.unquote(name)
            resp = obsClient.getObject(bucketName, name, loadStreamInMemory=True)
            if resp.status < 300: 
                self.send_response(200) 
                buf = resp.body.buffer
            else: 
                self.send_response(400)
        self.end_headers()
        self.wfile.write(buf)
       
 
def StartServer():
    from BaseHTTPServer import HTTPServer
    sever = HTTPServer(("",12000),ObsHandler)
    sever.serve_forever()
  
  
if __name__=='__main__':
    StartServer()

[Solved] Ubuntu using blender script error: Numpy cannot be found

When rendering with blender script on Ubuntu 16, use the command blender — background — Python * Py, an error is reported and numpy cannot be found. But I installed numpy in CONDA environment, so I was puzzled.

Later, I learned that blender has its own Python interpreter. When running my py script, the built-in Python interpreter does not install the numpy extension library, so an error is reported.

Solution:

Find the Python interpreter directory for blender in your environment.
Open blender, shift+F4 and go to blender's Python interpreter
You can see the version of Python that comes with it, then use whereis python for that version and find the interpreter's directory
Use sudo apt-get install python version-numpy to install the third-party library for blender's own Python interpreter

pandas.DataFrame() Initializes NULL Error: DataFrame [How to Solve]

An error occurred while initializing an empty dataframe

Traceback (most recent call last):
    result_format = pd.DataFrame(index=index, columns=columns)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 435, in __init__
    mgr = init_dict(data, index, columns, dtype=dtype)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\construction.py", line 239, in init_dict
    val = construct_1d_arraylike_from_scalar(np.nan, len(index), nan_dtype)
  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\dtypes\cast.py", line 1440, in construct_1d_arraylike_from_scalar
    dtype = dtype.dtype
AttributeError: type object 'object' has no attribute 'dtype'

Solution:
result_format = pd.DataFrame(index=index, columns=columns, dtype=object)

[Solved] Python-selenium locates an element cannot be clicked error: ElementClickInterceptedException

Problem Description:

prompt: elementclickinterceptedexception: Message: element click
when performing selenium UI automation test, you may encounter the situation that elements can be located but cannot be clicked. The following error is:

When writing an automatic program, you will encounter a new window pop-up, and the program always locates the element on the first window page by default. In this way, the element will not be located, and the program will report an error.

For example, when locating an element, the UI is covered by the pop-up window in the list. If an element cannot be located, the program reports an error
click the code of the located element:

web_driver.find_element(By.CSS_SELECTOR,'#funtype_click').click()

Cause analysis:

prompt: elementclickinterceptedexception
when locating an element, the UI is covered by the pop-up window in the list. If an element cannot be located, the element click has been blocked</ font>


Solution:

Error code:

# Click on the function category
web_driver.find_element(By.CSS_SELECTOR,'#funtype_click').click()

Resolved Code:

# Functional Category
funtype_click = web_driver.find_element(By.CSS_SELECTOR,'#funtype_click')
web_driver.execute_script('arguments[0].click()',funtype_click)

[Solved] git review Error: UnicodeDecodeError: ‘gbk‘ codec can‘t decode

The error message is similar to:

UnicodeDecodeError: ‘gbk’ codec can’t decode byte 0xac in position 44: illegal multibyte sequence

Solution:
find the python installation directory or directly search the subprocess Py, find the installation place. Python 38-32\lib\ has a file subprocess Py, encoding = none is changed to encoding = ‘UTF-8’.

[Solved] python3.10 Error: cannot import name ‘Iterable‘ from ‘collections‘

Generally, errors will be reported when using the pyechards library for higher versions of Python:
cannot import name ‘iteratable’ from ‘collections’

Solution:
go to the directory where the pyecarts library is installed, enter the pyecarts file, open render, and continue to open engine Py file, where you can find the following code:

from collections import Iterable

Change to the following code:

from collections.abc import Iterable

Call through the following command, and no error will be reported again

from collections.abc import Iterable

Solve the error of panda index unalignable Boolean series provided as indexer

Complete error reports are as follows: pandas core.indexing.IndexingError: Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match).

Solution:

There are two indexing methods, one of which can be selected:

Solution 1

dataframe[pd.Series([True, True, True], index=dataframe.index)]

Solution 2:

dataframe.loc[pd.Series([True, True], index=['a', 'b']).index]

Problem analysis

An error will appear in the following code:

import pandas as pd
import numpy as np


dataframe = pd.DataFrame(data=np.random.random(size=(3, 5)), index=['a', 'b', 'c'])

Error codes are as follows:

dataframe[pd.Series([True, True], index=['a', 'b'])]

You can see that the original dataframe includes 3 lines, but there are only two true values here, so an error is reported. The correct method is:

dataframe[pd.Series([True, True, True], index=['a', 'b', 'c'])]
dataframe.loc[pd.Series([True, True], index=['a', 'b']).index]