Tag Archives: Machine learning

MindSpore Error: [ERROR] MD:unexpected error.Not a valid index

ERROR:[MD]:unexpected error. Not a valid index


problem phenomenon: a single card does not report an error, and the training process can be correctly executed. However, when switching to distributed training, an error in the diagram is reported. After troubleshooting, the cause of the error is found to be the wrong use of distributed sampling
the error reporting method is as follows:

the order of distributed sampling and random sampling needs to be changed. The correct way is to perform random sampling first and then distributed sampling

The correct modification is as follows:

distributed training can be performed correctly after modification

[Solved] mindinsight modelart Error: RuntimeError: An attempt has been made to start a new process before…

 

Question:

Mindinsight uses error reporting on modelart.

After adding the summary and training some epoch normally, the operation will report an error:

Solution:

When using SummaryCollector, you need to put the code block in if__name__ == __main__:

The official mindspire tutorial has been updated. You can refer to the writing method of the latest tutorial: collect summary data – mindspire master document

Codes like this:

def train():
  summary_collector = SummaryCollector(summary_dir='./summary_dir')

  ...

  model.train(...., callback=[summary_collector])

if __name__ == '__main__':
    train()

 

[Solved] ValueError: Error when checking input: expected conv2d_input to have 4 dimensions

Error Messages:

ValueError: Error when checking input: expected conv2d_input to have 4 dimensions, but got array with shape (150, 150, 3)
Codes:

image = mpimg.imread("./ima/b.jpg")
image = image/255
classe = model.predict(image, batch_size=1)

Reason:
the input format is incorrect
solution:
standardize the dataset

Specific solutions:

image = mpimg.imread("./ima/b.jpg")
image = image.reshape(image.shape(1,150,150,3)/255
classe = model.predict(image, batch_size=1)

[Solved] Error: ‘attrition‘ is not an exported object from ‘namespace:rsample‘

Error: ‘attrition’ is not an exported object from ‘namespace:rsample’


# Import package and library

# load required packages
library(rsample)
library(dplyr)
library(h2o)
library(DALEX)

# initialize h2o session
h2o.no_progress()
h2o.init()
##  Connection successful!
## 
## R is connected to the H2O cluster: 
##     H2O cluster uptime:         4 hours 30 minutes 
##     H2O cluster timezone:       America/New_York 
##     H2O data parsing timezone:  UTC 
##     H2O cluster version:        3.18.0.11 
##     H2O cluster version age:    1 month and 17 days  
##     H2O cluster name:           H2O_started_from_R_bradboehmke_gny210 
##     H2O cluster total nodes:    1 
##     H2O cluster total memory:   1.01 GB 
##     H2O cluster total cores:    4 
##     H2O cluster allowed cores:  4 
##     H2O cluster healthy:        TRUE 
##     H2O Connection ip:          localhost 
##     H2O Connection port:        54321 
##     H2O Connection proxy:       NA 
##     H2O Internal Security:      FALSE 
##     H2O API Extensions:         XGBoost, Algos, AutoML, Core V3, Core V4 
##     R Version:                  R version 3.5.0 (2018-04-23)

#Data preprocessing and processing to h2o format;

Error: ‘attrition’ is not an exported object from ‘namespace:rsample’

#

# classification data
df <- rsample::attrition %>% 
  mutate_if(is.ordered, factor, ordered = FALSE) %>%
  mutate(Attrition = recode(Attrition, "Yes" = "1", "No" = "0") %>% factor(levels = c("1", "0")))

# convert to h2o object
df.h2o <- as.h2o(df)

# create train, validation, and test splits
set.seed(123)
splits <- h2o.splitFrame(df.h2o, ratios = c(.7, .15), destination_frames = c("train","valid","test"))
names(splits) <- c("train","valid","test")

# variable names for resonse & features
y <- "Attrition"
x <- setdiff(names(df), y) 

Solution:

You can use the attrition dataset of DALEX package directly;

Remove resample:

#

# classification data
df <- attrition %>% 
  mutate_if(is.ordered, factor, ordered = FALSE) %>%
  mutate(Attrition = recode(Attrition, "Yes" = "1", "No" = "0") %>% factor(levels = c("1", "0")))

# convert to h2o object
df.h2o <- as.h2o(df)

# create train, validation, and test splits
set.seed(123)
splits <- h2o.splitFrame(df.h2o, ratios = c(.7, .15), destination_frames = c("train","valid","test"))
names(splits) <- c("train","valid","test")

# variable names for resonse & features
y <- "Attrition"
x <- setdiff(names(df), y) 

Full Error Messages:

> # classification data
> df <- rsample::attrition %>% 
+     mutate_if(is.ordered, factor, ordered = FALSE) %>%
+     mutate(Attrition = recode(Attrition, "Yes" = "1", "No" = "0") %>% factor(levels = c("1", "0")))
Error: 'attrition' is not an exported object from 'namespace:rsample'
>

torch.max Example (How to Use)

torch.max(input, dim)

pred = torch.max(input, dim)

Returns the maximum value per row (dim = 1) or column (dim = 0).

_, pred = torch.max(input, dim)

Only the position of the maximum value in each row (dim = 1) or column (dim = 0) is returned.

Example:

import torch

# Construct a 5x3 randomly initialized matrix
x = torch.rand(5, 3)
print('input: ', x)
print('-'*10)
y1 = torch.max(x, 1)
print('max by row: ', y1)
print('-'*10)
y2 = torch.max(x, 0)
print('max by col: ', y2)
print('-'*10)
_, y3 = torch.max(x, 1)
print('max index by row: ', y3)
print('-'*10)
_, y4 = torch.max(x, 0)
print('max index by col: ', y4)

Output result:

input:  tensor([[0.5504, 0.3160, 0.2448],
        [0.8694, 0.3295, 0.2085],
        [0.5530, 0.9984, 0.3531],
        [0.2874, 0.1025, 0.9419],
        [0.0867, 0.4234, 0.8334]])
----------
max by row:  torch.return_types.max(
values=tensor([0.5504, 0.8694, 0.9984, 0.9419, 0.8334]),
indices=tensor([0, 0, 1, 2, 2]))
----------
max by col:  torch.return_types.max(
values=tensor([0.8694, 0.9984, 0.9419]),
indices=tensor([1, 2, 3]))
----------
max index by row:  tensor([0, 0, 1, 2, 2])
----------
max index by col:  tensor([1, 2, 3])

[Solved] Decision tree error: Graphviz’s executables not found

Decision tree code

from sklearn.datasets import load_iris
import pydotplus
from IPython.display import Image
from sklearn import tree
#Training Model
iris=load_iris()
clf=tree.DecisionTreeClassifier()
clf=clf.fit(iris.data,iris.target)

#Drawing
dot_data=tree.export_graphviz(decision_tree=clf,
                              out_file=None,
                              rounded=True,
                              filled=True,
                              feature_names=iris.feature_names)
graph=pydotplus.graph_from_dot_data(dot_data)
Image(graph.create_png())

Installation

Install pydotplus

pip install pydotplus

Running error: graphviz’s executables not found

Installing graphviz

pip install graphviz

Graphviz’s executables not found

After consulting the data, graphviz belongs to an independent software, which needs to be downloaded and installed on the official website

Website: http://www.graphviz.org/Download

Add environment variable

There are two methods, and the second one is recommended for personal test

Method 1: add the path of graphviz’s bin directory in environment variable.

Method 2: OS method

CMD

python

Enter Python environment

imoprt os
os.environ["PATH"]+= os.pathsep+"C:/Program Files/Graphviz/bin"  # change to your path of Graphviz

Done!

Operation results

If it is used in jupyter notebook like me, you need to restart jupyter notebook

[Solved] matplotlib.units.ConversionError: Failed to convert value(s) to axis units: ‘LiR‘

 

solve the problem


No handles with labels found to put in legend.

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5.py", line 508, in _draw_idle
    self.draw()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 670, in draw
    bbox, info, descent = textobj._get_layout(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 276, in _get_layout
    key = self.get_prop_tup(renderer=renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 831, in get_prop_tup
    x, y = self.get_unitless_position()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 813, in get_unitless_position
    x = float(self.convert_xunits(self._x))
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 180, in convert_xunits
    return ax.xaxis.convert_units(x)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1553, in convert_units
    f'units: {x!r}') from e
matplotlib.units.ConversionError: Failed to convert value(s) to axis units: 'LiR'
Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1550, in convert_units
    ret = self.converter.convert(x, self.units, self)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\category.py", line 52, in convert
    'Missing category information for StrCategoryConverter; '
ValueError: Missing category information for StrCategoryConverter; this might be caused by unintendedly mixing categorical and numeric data
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5.py", line 508, in _draw_idle
    self.draw()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 670, in draw
    bbox, info, descent = textobj._get_layout(renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 276, in _get_layout
    key = self.get_prop_tup(renderer=renderer)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 831, in get_prop_tup
    x, y = self.get_unitless_position()
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\text.py", line 813, in get_unitless_position
    x = float(self.convert_xunits(self._x))
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 180, in convert_xunits
    return ax.xaxis.convert_units(x)
  File "D:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1553, in convert_units
    f'units: {x!r}') from e
matplotlib.units.ConversionError: Failed to convert value(s) to axis units: 'LiR'

Error:
matplotlib.units.Conversion error: failed to convert value to axis unit: ‘LiR’

Solution:
matplotlib version is low, update the matplotlib library to version 3.3.2 or higher!

[Solved] R Language Error: Discrete value supplied to continuous scale

Error: discrete value supplied to continuous scale

#Simulation data

set.seed(123)
my_df1 <- data.frame(a=rnorm(100), b=runif(100), c=rep(1:10, 10))
my_df2 <- data.frame(a=rnorm(100), b=runif(100), c=factor(rep(LETTERS[1:5], 20)))

#Error: Discrete value supplied to continuous scale

ggplot() +
  geom_point(data=my_df1, aes(x=a, y=b, color=c)) +
  geom_polygon(data=my_df2, aes(x=a, y=b, color=c), alpha=0.5)

#Solution:

Use the fill parameter;

ggplot() +
  geom_point(data=my_df1, aes(x=a, y=b, color=c)) +
  geom_polygon(data=my_df2, aes(x=a, y=b, fill=c), alpha=0.5)

[Solved] Neo4j error: Import-Module & neo4j Neo.ClientError.Security.Unauthorized

Error reporting record

1. Import module: failed to load the specified module. Solution:

Solution: find the neo4j PS1 file in the bin directory, open it, find the import module, and change the relative path to the absolute path, that is,

"D:\Neo4j\neo4j-community-3.5.25\bin\Neo4j-Management.psd1"

2. When changing the password, neo4j Neo.ClientError.Security.Unauthorized

Solution: open neo4j.conf in the conf folder and delete the comment in front of #dbms.security.auth_enabled=false

[Solved] graphviz Error: failed to execute [‘dot’, ‘-Tsvg’], make sure the Graphviz executables are on your systems’ PATH

 

graphviz error: failed to execute [‘dot’, ‘-Tsvg’], make sure the Graphviz executables are on your systems’ PATH

1. Download graphviz

Go to https://graphviz.org/download/ to download graphviz (64-bit system must choose 64-bit Installer, can not choose ZIP)

2. Installation

In the following interface, you must select the red option (if not selected, you can only manually add environment variables later), and then you can finish all the way.

3. Test

win + R, type cmd, open the cmd window, use dot -version test, can display the version information is installed successfully
(Do not blindly open CMD, if the open is C:\WINDOWS\system32> that can not be measured)

Note: if not, restart anaconda.

TensorFlow-gpu Error: failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected

Error Messages:

failed call to cuInit: CUDA_ERROR_NO_Device: no CUDA capable device is detected
this is also what I encountered when running a CNN SVM classifier program of tensorflow GPU today. It’s not the problem of the program. It’s our graphics card.

Solution:

import tensorflow as tf

config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.compat.v1.InteractiveSession(config=config)

Just add these lines of code to the head of the code, and you don’t need to write this code belowos.environ['CUDA_VISIBLE_DEVICES'] = '/gpu:0'

[Solved] Error: package or namespace load failed for ‘ggplot2’ in loadNamespace(i, c(lib.loc, .libPaths()), v

Error Messages:
> library(ggplot2)
Error: package or namespace load failed for ‘ggplot2’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):

Loaded namespace ‘ellipsis’ 0.3.1, but what is needed is >= 0.3.2

Solution:
In Rstudio, find the packages module, remove the package that reported the error and then re-install it using install.packages for the corresponding installation.