Original code:
from keras.preprocessing import image
img = image.load_img(img_path,target_size=(224,224))
x = image.img_to_array(img)report errors:
AttributeError: module 'keras.preprocessing.image' has no attribute 'load_img'Reason: The keras version has been updated.
Solution:
from keras.utils import image_utils
img = image_utils.load_img(img_path,target_size=(224,224))
x = image_utils.img_to_array(img)
The environment is TensorFlow2.9
The code to be run on github The TensorFlow environment is 1.x
Many apis cannot be called
Original code
train_summary_op = tf.summary.merge([loss_summary])
Error reporting in method 1
train_summary_op = tf.compat.v1.summary.merge([loss_summary])
display
TypeError: Tensors in list passed to 'inputs' of 'MergeSummary' Op have types [bool] that do not match expected type string.
If it can’t be solved, I will try
import tensorflow as tf
Modified as
import tensorflow._api.v2.compat.v1 as tf
tf.disable_v2_behavior()
Finally, solve the problem successfully
The reason for the above error when compiling custom functions is that tf2.x’s keras.compile does not support specific values by default
Questions
When using the wrapping method to customize the loss function of the keras model and need to calculate accuracy metrics such as precision or recall, or need to extract the specific values of the inputs y_true and y_prd (operations such as y_true.numpy()), an error message appears:
OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.
Or
AttributeError: 'Tensor' object has no attribute 'numpy'
Solution:
Pass in parameters in the compile function:
run_eagerly=True
Reason:
Tf2.x enables eager mode by default, namely eager execution, that is, dynamic calculation graph. Compared with the static calculation graph of tf1.x, the advantage of eager mode is that it is convenient for debugging, which can easily print tensor values and evaluate results; and Numpy interacts well, and the conversion between tensor and ndarray is convenient and even universal. The tradeoff is that it runs significantly slower. After the static calculation graph is defined, it is almost always executed with C++ code on the tensorflow core, so the calculation efficiency is higher and the speed is faster.
Even so, run_eagerly defaults to False in the model.compile method, which means that the logic of the model is encapsulated in tf.function, which achieves faster computational efficiency (the autograph mechanism converts the dynamic computational graph through the @tf.function wrapper). is a static computation graph). But the @tf.function wrapper requires the function to use basic tf operations, not other operations in python or even functions from other packages, so the first error occurs when calling functions such as sklearn.metrics’ accuracy_score or imblearn.metrcis’ geometric_mean_score function. The second error occurs when using the y_true.numpy() method. The fundamental reason is that the model.compile method does not support the above operations after the static calculation graph converted by the @tf.function wrapper, although tf2.x enables the use of dynamic calculation graphs by default.
After passing run_eagerly=True to the model.compile method, the dynamic calculation graph is used to run, and the above operations can be performed normally. The disadvantage is that the dynamic calculation graph has the disadvantage of low operation efficiency.
When training your own dataset, you often report errors:
tensorflow2.3 InvalidArgumentError: jpeg::Uncompress failed
[[{{node decode_image/DecodeImage}}]] [Op:IteratorGetNext]
Solution:
check whether the picture is damaged before training:
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import os
num_skipped = 0
for folder_name in ("Fruit apples", "Fruit bananas", "Fruit oranges"):
folder_path = os.path.join(".\data\image_data", folder_name)
for fname in os.listdir(folder_path):
fpath = os.path.join(folder_path, fname)
try:
fobj = open(fpath, mode="rb")
is_jfif = tf.compat.as_bytes("JFIF") in fobj.peek(10)
finally:
fobj.close()
if not is_jfif:
num_skipped += 1
# Delete corrupted image
os.remove(fpath)
print("Deleted %d images" % num_skipped)
Delete the damaged picture and train again to solve the problem
if an error is prompted again, use:
# Determine if an image is corrupt from local
def is_valid_image(path):
'''
Check if the file is corrupt
'''
try:
bValid = True
fileObj = open(path, 'rb') # Open in binary form
buf = fileObj.read()
if not buf.startswith(b'\xff\xd8'): # whether to start with \xff\xd8
bValid = False
elif buf[6:10] in (b'JFIF', b'Exif'): # ASCII code of "JFIF"
if not buf.rstrip(b'\0\r\n').endswith(b'\xff\xd9'): # whether it ends with \xff\xd9
bValid = False
else:
try:
Image.open(fileObj).verify()
except Exception as e:
bValid = False
print(e)
except Exception as e:
return False
return bValid
num_skipped = 0
for folder_name in ("fruit-apple", "fruit-banana", "fruit-orange"):
#os.path.join() joins two or more pathname components
folder_path = os.path.join(". \data\image_data", folder_name)
# os.listdir(path) lists the subdirectories under this directory
for fname in os.listdir(folder_path):
fpath = os.path.join(folder_path, fname)
flag1 = is_valid_image(fpath)
if not flag1:
print(flag1)
print(fpath)#Print the path and name of the error file
Adjust the error file and train again to solve the problem.
Today, I debugged the code and found this error. The complete contents are as follows:
RuntimeError: Cannot clone object < keras. wrappers. scikit_ learn. KerasClassifier object at 0x000003A783733F33>, as the constructor either does not set or modifies parameter layers
No error is reported when the code is found. It is normal
The code is then investigated:
from keras.wrappers.scikit_learn import KerasRegressorChange to:
from tensorflow.keras.wrappers.scikit_learn import KerasRegressorProblem solved!
When using tensorflow.keras, this error is often reported during model training:
tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]
tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]
According to my own experience, there are several reasons for this errory:
1. The input image_size and input_shape does not match or is not defined when the model is built. Note that the input_shape must be defined when defining the first layer of the convolutional layer, e.g.
model = keras.models.Sequential([
# Input image [None,224,224,3]
# Convolution layer 1: 32 5*5*3 filters, step size set to 1, fill set to same
# Output [None,32,32,3]
keras.layers.Conv2D(32, kernel_size=5, strides=1, padding='same', data_format='channels_last',
activation='relu', input_shape=(224, 224, 3)),2. There is also train_generator and validate_generator related parameters must be consistent, such as batch_size, target_size, class_mode, etc.
3. The configuration limit itself, try to change the batch_size to a smaller size, or even to 1
4. The last program did not finish completely, finish all python programs to see.
Error Messages:
failed call to cuInit: CUDA_ERROR_NO_Device: no CUDA capable device is detected
this is also what I encountered when running a CNN SVM classifier program of tensorflow GPU today. It’s not the problem of the program. It’s our graphics card.
Solution:
import tensorflow as tf
config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.compat.v1.InteractiveSession(config=config)
Just add these lines of code to the head of the code, and you don’t need to write this code belowos.environ['CUDA_VISIBLE_DEVICES'] = '/gpu:0'
windows system:
original_keras_version = f.attrs[‘keras_version’].decode(‘utf8’)
1. error:
load_weights_from_hdf5_group
original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'
2. Cause analysis
When installing tensorflow, the default installed h5py is 3.1.0, and an error is reported because the TF you installed does not support an excessively high version of h5py
3. Solutions
1. Uninstall h5py3 Version 1.0, installing h5py2.0 Version 10.0.2. Restart the compiler
pip install h5py==2.10.0
**
Error (s) in loading state_dict for GeneratorResNet
**
cause of the problem: check whether we use dataparallel for multi GPU during training. The model generated by this method will automatically add key: module
observe the error message:
you can find that the key values in the model are more modules
Solution:
1. Delete the module
gentmps=torch.load("./saved_models/generator_%d.pth" % opt.epoch)
distmps = torch.load("./saved_models/discriminator_%d.pth" % opt.epoch)
from collections import OrderedDict
new_gens = OrderedDict()
new_diss = OrderedDict()
for k, v in gentmps.items():
name = k.replace('module.','') # remove 'module.'
new_gens[name] = v #The value corresponding to the key value of the new dictionary is a one-to-one value.
for k, v in distmps.items():
name = k.replace('module.','') # remove 'module.'
new_diss[name] = v #The value corresponding to the key value of the new dictionary is a one-to-one value.
generator.load_state_dict(new_gens)
discriminator.load_state_dict(new_diss)
Background of the problem
Getting started with Python
try to use pyldavis for simple topic extraction;
Problems and related codes
Pyldavis has been installed, as shown in the figure (Annotated):
but an error occurs when entering the following statement:
error code segment:
import pyLDAvis
import pyLDAvis.sklearn
pyLDAvis.enable_notebook()
pyLDAvis.sklearn.prepare(lda,tf,tf_vectorizer)
Screenshot of specific error reporting:
Solution:
Some bloggers suggested that the installation was not successful. I installed it as an administrator. I operated it, but it was useless
in fact, the final solution is very simple. I found that the kernel used by my Jupiter notebook is wrong. I used a virtual environment I set up before. Just change it to the default environment (my one is named python3).
1. Error information
When building the neural network model, you can call plot in keras_ The model module draws a schematic diagram of the model to facilitate the adjustment of the model structure:
from tensorflow.keras.models import Model
from tensorflow.keras.utils import plot_model
model = Model(dense_inputs+sparse_inputs, output_layer)
plot_model(model, "fm_model.png", show_shapes=True)
As a result, the following error messages appear:
(‘Failed to import pydot. You must pip install pydot and install graphviz (ht
tps://graphviz.gitlab.io/download/ ), ‘, ‘for pydotprint to work.’)
Understand the error message: the installation is complete without pydot and graphviz packages
2. Solutions
2.1 installation of graphviz package
pip install graphviz
2.2 download and install graphviz Exe file and install
In Windows Environment
Download address: https://graphviz.gitlab.io/download/
2.3 configuring environment variables for graphviz
2.4 installing pydot package
pip install pydot-ng
2.5 restart development tools
Restart the IDE or other development tools (Jupiter notebook) with immediate effect.
3. Summary
1. Installing pydot and graphviz packages directly according to the error message does not work
2. You need to go to the website to download the corresponding EXE file or zip file. After installation, specify the environment variables
3. Don’t forget to restart your ide or other development tools
When Python 2 installs tensorflow, test after the installation is completed:
import tensorflow as tf
Will report an error:
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
File “/home/zhaokai/.local/lib/python2.7/site-packages/tensorflow/__init__.py”, line 28, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File “/home/zhaokai/.local/lib/python2.7/site-packages/tensorflow/python/__init__.py”, line 52, in <module>
from tensorflow.core.framework.graph_pb2 import *
File “/home/zhaokai/.local/lib/python2.7/site-packages/tensorflow/core/framework/graph_pb2.py”, line 7, in <module>
from google.protobuf import descriptor as _descriptor
File “/home/zhaokai/.local/lib/python2.7/site-packages/google/protobuf/descriptor.py”, line 113
class DescriptorBase(metaclass=DescriptorMetaclass):
^
SyntaxError: invalid syntax
The solution is to re-install protobuf:
pip install protobuf==3.17.3then Import tensorflow again.