Tag Archives: hard

The function of flatten layer in deep learning

The official account of WeChat

Flatten layer is implemented in Keras.layers.core . flatten() class.

effect:

Flatten layer is used to “flatten” the input, that is, to make the multi-dimensional input one-dimensional. It is often used in the transition from convolution layer to fully connected layer. Flatten does not affect the size of the batch.

example:

from keras.models import Sequential
from keras.layers.core import Flatten
from keras.layers.convolutional import Convolution2D
from keras.utils.vis_utils import plot_model


model = Sequential()
model.add(Convolution2D(64,3,3,border_mode="same",input_shape=(3,32,32)))
# now：model.output_shape==(None,64,32,32)

model.add(Flatten())
# now: model.output_shape==(None,65536)

plot_model(model, to_file='Flatten.png', show_shapes=True)

In order to better understand the function of flatten layer, I visualize this neural network, as shown in the figure below:

Error in testing keras


nning Keras under Python is reporting an error — it tells me that one way to fix this problem is to uninstall Numpy repeatedly until I can’t find it, and then reinstall this version. Installation keras
no problem, baidu, a lot of reasons, all have been adjusted, still not solve the
I finally find a suitable way to
find their numpy location
below is I save position (for reference only, everyone is different)

where will use the red pen circle below are deleted, then reinstall the


this is solved!

Exception ignored in: bound method basesession__ del__ Of

Error message

Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x000000001AB286D8>>
Traceback (most recent call last):
  File "python3.5.2\lib\site-packages\tensorflow\python\client\session.py", line xxx, in __del__
TypeError: 'NoneType' object is not callable

Reason: When the Python garbage collection mechanism collects the Session object, it finds that c_api_util or Tf_session has been collected, resulting in a null pointer. The following comment is the reason given when an exception is thrown in the source code of TensorFlow

# At shutdown, `c_api_util` or `tf_session` may have been garbage
# collected, causing the above method calls to fail. In this case,
# silently leak since the program is about to terminate anyway.

The solution
Method one:

import keras.backend as K

# your code

K.clear_session()

Method 2:

import gc

# your code

gc.collect()

reference
https://github.com/tensorflow/tensorflow/issues/3388
https://www.cnblogs.com/kaituorensheng/p/4449457.html

InternalError: Failed to create session. Error and solution

InternalError: Failed to create session. Errors and solutions
Introduction Error message solution

preface
This error occurred at the beginning of training Keras (using the TensorFlow as Backend) model.
Python version: 3.5.2
Keras version: 2.1.3
TensorFlow version: 1.9.0
Error message

InternalErrorTraceback (most recent call last)
< ipython-input-4-d4cc2ca313a3> in < module>
10 model.compile(loss= ‘mse’, optimizer= ‘Adam’)
11 # fit network
— > 12 history = model.fit(X_train, y_train, epochs=3000, batch_size=16, validation_data=(x_test, y_test), verbose=2, shuffle=False)
13 #history = model.fit(X,y, epochs=3000, batch_size=16, Verbose = 2, shuffle = False)
14 # plot history
/usr/local/lib/python3.5/dist – packages/keras/models. Py fit in (self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
963 initial_epoch=initial_epoch,
964 steps_per_epoch=steps_per_epoch,
— > 965 validation_steps = validation_steps)

967 966 def the evaluate (self, x = None, y = None,
/usr/local/lib/python3.5/dist – packages/keras/engine/training. Py fit in (self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
1667 initial_epoch=initial_epoch,
1668 steps_per_epoch=steps_per_epoch,
-> 1669 validation_steps = validation_steps)

1671 1670 def the evaluate (self, x = None, y = None,
/usr/local/lib/python3.5/dist – packages/keras/engine/training. Py in _fit_loop (self, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)
1204 ins_batch[i] = ins_batch[i].toarray()
1205
-> 1206 outs = f (ins_batch)
1207 if not isinstance (outs, a list) :
1208 outs = [outs]
/usr/local/lib/python3.5/dist – packages/keras/backend/tensorflow_backend py in the call (self, inputs)
2471 feed_dict[tensor] = value
2472 fetches = self.outputs + [self.updates_op] + self.fetches
-> 2473 session = get_session()
2474 updated = session.run(fetches=fetches, Feed_dict = feed_dict,
2475 * * self session_kwargs)
/usr/local/lib/python3.5/dist – packages/keras/backend/tensorflow_backend py in get_session ()
config = 174 Tf. ConfigProto (intra_op_parallelism_threads = num_thread,
175 allow_soft_placement = True)
– & gt; 176 _SESSION = tf. The Session (config = config)
177 Session = _SESSION 178 if not _MANUAL_VAR_INIT:

/usr/local/lib/python3.5/dist – packages/tensorflow/python/client/Session. Py in init (self, Target, graph, config)
1561
1562 “”
-& gt; 1563 super(Session, self).init(target, graph, config=config)
1564 # NOTE(mrry): Create these on first __enter__ to avoid a reference cycle.
1565 self._default_graph_context_manager = None
/usr/local/lib/python3.5/dist – packages/tensorflow/python/client/session. Py in init (self, target, graph, the config).
631 if self _created_with_new_api:
632 # pylint: Disable = protected – access
– & gt; 633 self._session = tf_session.TF_NewSession(self._graph._c_graph, opts)
634 # pylint: enable=protected-access
635 else:
InternalError: Failed to create session.

The solution


found that GPU memory has been occupied by other programs.
try to close these programs, then reruns the code (no need to restart kernel) and find the problem solved!

[Keras] ImportError: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to

problem description

what happened was, when I was happily building a model with Keras, I found that there was an official function for drawing the model plot_model(), so I happily called the function keras.utils.plot_model(model, 'model.png', show_shapes=True), and the result was the following error: 

InvocationException: GraphViz's executables not found
...
ImportError: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

I thought, was it because I didn’t install , pydot and graphviz?So I happily turned on PyCharm and loaded my virtual environment with pydot and graphviz, but it was still reporting an error.

then I shut myself up for a while. After I closed myself up, I went to baidu and Google to find a solution. Sure enough, many people met the same problem as me. According to the big guy [1], this is an environmental problem, the solution is to PIP install graphviz or something, or manually download the graphviz installation package on the official website, and then add the system environment variable Path, such as C:\Program Files (x86)\Graphviz2.38\bin something.

I thought this solution made sense, and I happily tried it out, and still got an error.

and then I shut myself down again and went back to looking for a solution. Some people [2] say that the order of installation is very important. First install pydot, then install graphviz; Someone [3] said to use keras.utils.vis_utils.plot_model instead of keras.utils.plot_model. In short, the feeling is not reliable.

finally, you guessed it, I uninstalled PyCharm and pydot in pydot ( PIP install pydot0) in PyCharm and pydot) in the pydot). Success!

then I thought about what was causing the error for a while, I tried to delete the system variable of graphviz, but the error didn’t occur. Oh, calculate, calculate, just work!

environment

  • Windows 10
  • Python 3.6
  • TensorFlow 2.0 Beta (built-in Keras)

effect

Reference

  1. Rob. (August 25, 2018). Graphviz and Pydotplus not working. Retrieved from https://datascience.stackexchange.com/questions/37428/graphviz-and-pydotplus-not-working
  2. web_ninja. (August 26, 13). Why is pydot unable to find GraphViz’s Executables in Windows 8?Retrieved from https://stackoverflow.com/questions/18438997/why-is-pydot-unable-to-find-graphvizs-executables-in-windows-8
  3. XifengGuo. (November, 6, 2017). pydot issue. Retrieved from https://github.com/XifengGuo/CapsNet-Keras/issues/7