Tag Archives: deploy

[Solved] TensorFlow severing Container Creat Error: failed: Out of range: Read less bytes than requested

0. Preface

Recently, I was doing tensorflow severing model deployment, and I took a yolov5 trained pt model for experiment. Everything was fine for the first few days, but today I encountered an error when creating the tensorflow severing container, the environment, commands, models, configuration files have not changed, and suddenly this error appeared.

1. Problem description

There was an error when creating the tensorflow searching container. The environment, commands, models, and configuration files did not change. Suddenly, this error occurred.

1. Error summary (F is easy to search):

  • E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
  • Failed to start server. Error: Unknown: 1 servable(s) did not become available: {{{name: yolov5_saved_model version: 1} due to error: Out of range: Read less bytes than requested}, }
  • 2022-07-30 11:15:09.097717: I tensorflow_serving/core/basic_manager.cc:279] Unload all remaining servables in the manager.

2. Error screenshot:

3. The complete error code is as follows:

(see Appendix)

2. Problem analysis:

The error is caused by the damage of the file. Specifically, I think of two possibilities: the interruption or error of the uploaded file, and the damage of the intact file due to disk problems.

Mine is the second kind

Reference https://github.com/tensorflow/tensorflow/issues/21544 Just found out. 👇

3. Solution:

Delete the model and configuration file and upload it again.

If there is no backup in advance, or you don’t know whether the file is damaged, you can re convert the model to generate a saved model model and create a new configuration file.

The following figure is the file structure and screenshot of the file I uploaded again.

models
|—-model1
|—- —-1
|—- —- —-assets
|—- —- —- —-variables
|—- —- —- —-variables. data-00000-of-00001
|—- —- —-variables. index
|—- —- —-saved_ model. pb
|—-model. config

reference resources

Similar problems:
https://github.com/tensorflow/tensorflow/issues/21544
https://bytemeta.vip/repo/Breta01/handwriting-ocr/issues/104

appendix

ubuntu% docker run --rm -p 8500:8500 --mount type=bind,source=/media/userdata/zhangxw/TFSever_Test2/yolo_cow/models,target=/models/models -t tensorflow/serving:latest-gpu --model_config_file=/models/models/model.config --allow_version_labels_for_unavailable_models=true &
[1] 8565
ubuntu% 2022-07-30 11:10:08.016885: I external/org_tensorflow/tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2022-07-30 11:10:08.052728: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2022-07-30 11:10:08.052746: I tensorflow_serving/model_servers/server_core.cc:591]  (Re-)adding model: yolov5_saved_model
2022-07-30 11:10:08.153403: I tensorflow_serving/core/basic_manager.cc:740] Successfully reserved resources to load servable {name: yolov5_saved_model version: 1}
2022-07-30 11:10:08.153468: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: yolov5_saved_model version: 1}
2022-07-30 11:10:08.153495: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: yolov5_saved_model version: 1}
2022-07-30 11:10:08.153563: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:10:08.221053: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 67495 microseconds.
2022-07-30 11:10:08.221079: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:11:08.221228: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 1
2022-07-30 11:11:08.221377: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:11:08.281805: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 60430 microseconds.
2022-07-30 11:11:08.281849: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:12:08.281994: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 2
2022-07-30 11:12:08.282124: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:12:08.316331: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 34210 microseconds.
2022-07-30 11:12:08.316359: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:13:08.316498: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 3
2022-07-30 11:13:08.316644: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:13:08.348579: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 31936 microseconds.
2022-07-30 11:13:08.348606: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:14:08.348747: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 4
2022-07-30 11:14:08.348881: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:14:08.381418: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 32539 microseconds.
2022-07-30 11:14:08.381447: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested

ubuntu% 2022-07-30 11:15:08.381606: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 5
2022-07-30 11:15:08.381749: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:15:08.413642: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 31895 microseconds.
2022-07-30 11:15:08.413673: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:15:08.413680: I tensorflow_serving/util/retrier.cc:46] Retrying of Loading servable: {name: yolov5_saved_model version: 1} exhausted max_num_retries: 5
2022-07-30 11:15:08.413698: I tensorflow_serving/core/loader_harness.cc:155] Encountered an error for servable version {name: yolov5_saved_model version: 1}: Out of range: Read less bytes than requested
2022-07-30 11:15:08.413705: E tensorflow_serving/core/aspired_versions_manager.cc:388] Servable {name: yolov5_saved_model version: 1} cannot be loaded: Out of range: Read less bytes than requested
Failed to start server. Error: Unknown: 1 servable(s) did not become available: {{{name: yolov5_saved_model version: 1} due to error: Out of range: Read less bytes than requested}, }
2022-07-30 11:15:09.097717: I tensorflow_serving/core/basic_manager.cc:279] Unload all remaining servables in the manager.

[1]  + exit 255   docker run --rm -p 8500:8500 --mount  -t tensorflow/serving:latest-gpu
ubuntu%
```![Please add a picture description](https://img-blog.csdnimg.cn/1f8990d3771e4eeea98e3eb1bddd1f24.png)

[Solved] C++ reason ncnn model error: Segmentation fault (core dumped)

The reasoning code is as follows:

#include "net.h"
#include <iostream>
#include <fstream>
#include <algorithm>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <stdio.h>
#include <vector>
#include <opencv2/opencv.hpp>

using namespace std;

void pretty_print(const ncnn::Mat& m)
{
    for (int q=0; q<m.c; q++)
    {
        const float* ptr = m.channel(q);
        for (int y=0; y<m.h; y++)
        {
            for (int x=0; x<m.w; x++)
            {
                printf("%f ", ptr[x]);
            }
            ptr += m.w;
            printf("\n");
        }
        printf("------------------------\n");
    }
}
//main fuction
int main(){
    string img_path = "person1.jpeg";
    cv::Mat img = cv::imread(img_path, cv::IMREAD_COLOR);
    cv::Mat img2;
    int input_width = 512;//Input size specified when going onnx
    int input_height = 512;
    
    cv::resize(img, img2, cv::Size(input_width, input_height));
  

    // Load the converted and quantized alexnet network
    ncnn::Net net;
    // net.opt.num_threads=1;
    net.load_param("cps_simplif.param");
    net.load_model("cps_simplif.bin");
    // Convert opencv mat to ncnn mat
    ncnn::Mat input = ncnn::Mat::from_pixels(img2.data, ncnn::Mat::PIXEL_BGR, img2.cols, img2.rows);
    const float mean_vals[3] = {0.485, 0.456, 0.406};
    const float norm_vals[3] = {0.229, 0.224, 0.225}; //[0.485, 0.456, 0.406]),std=np.array([0.229, 0.224, 0.225]
    input.substract_mean_normalize(mean_vals, norm_vals);
    // ncnn forward calculation
    ncnn::Extractor extractor = net.create_extractor();
    extractor.input("input.1", input);
    ncnn::Mat output0;
    extractor.extract("1035", output0);

    //ncnn::mat ->>>>> cv::mat
    cv::Mat a(input_height,input_width, CV_8UC3);
    output0.to_pixels(a.data, ncnn::Mat::PIXEL_BGR2RGB);
    
    cv::imwrite("ncnninfer.png", a);
    
    // pretty_print(output0);
    // pretty_print(output1);

    cout<<"done"<<endl;
    return 0;
}

Only segmentation fault (core dumped) is reported after running

No core file is generated. Check through ulimit -c, the size of core file is unlimited and there is no problem. This method solves the problem of not generating core file

With core file

Pass under the terminal

apt-get update
apt-get install gdb

Install GDB

Installation completed, passed

gdb ./Execution file name   core

The core here can be modified according to its own core file name.
the error is found at:

Corresponding to the reasoning code above

output0.to_pixels(a.data, ncnn::Mat::PIXEL_BGR2RGB);

It is the code to realize the conversion from ncnn:: mat to cv:mat. After debugging, it is found that the reasoning results of ncnn are all Nan, which leads to the conversion failure

Adjust the previous input, normalization is not done well.

[Solved] Kafka Error: Discovered coordinator XXXXX:9092 (id: 2147483647 rack: null) for group itstyle.

Error information:

Discovered coordinator DESKTOP-NRTTBDM:9092 (id: 2147483647 rack: null) for group itstyle.

reason:

The host of Kafka running on windows is the machine name, not the IP address

So it will lead to error reporting

Desktop-nrttbdm is the host name of the server where the Kafka instance is located
and 9092 is the port of Kafka, that is, the connection address of Kafka.

Solution

Modify the hosts file directly

The windows hosts file is located in

C:\Windows\System32\drivers\etc\hosts

Open it with administrator’s permission and append the corresponding relationship between IP and host name

Add the

172.18.0.52 DESKTOP-NRTTBDM

Restart the service again

Problem solved!