Xtykutl

We are trying to run a semantic segmentation model on android using deeplabv3 and mobilenetv2.We followed the official tensorflow lite conversion procedure using TOCO and tflite_convert with the help of bazel.The source frozen graph was obtained from the official TensorFlow DeepLab Model Zoo.

We were able to successfully convert the model with the following command:-

CUDA_VISIBLE_DEVICES="0" toco --output_file=toco256.tflite
--graph_def_file=path/to/deeplab/deeplabv3_mnv2_pascal_trainval/frozen_inference_graph.pb
--input_arrays=ImageTensor --output_arrays=SemanticPredictions --input_shapes=1,256,256,3 --inference_input_type=QUANTIZED_UINT8 --inference_type=FLOAT --mean_values=128 --std_dev_values=127 --allow_custom_ops --post_training_quantize

The size of the tflite file was around 2.25 Mb.But when we tried to test the model using the official benchmark tool, it failed with the following error report :-

bazel run -c opt tensorflow/contrib/lite/tools/benchmark:benchmark_model -- --graph=`realpath toco256.tflite`

INFO: Analysed target //tensorflow/contrib/lite/tools/benchmark:benchmark_model (0 packages loaded).

INFO: Found 1 target...

Target //tensorflow/contrib/lite/tools/benchmark:benchmark_model up-to-date:

  bazel-bin/tensorflow/contrib/lite/tools/benchmark/benchmark_model

INFO: Elapsed time: 0.154s, Critical Path: 0.00s

INFO: 0 processes.

INFO: Build completed successfully, 1 total action

INFO: Running command line: bazel-bin/tensorflow/contrib/lite/tools/benchmark/benchmark_model '--graph=path/to/deeplab/venINFO: Build completed successfully, 1 total action

STARTING!

Num runs: [50]

Inter-run delay (seconds): [-1]

Num threads: [1]

Benchmark name: 

Output prefix: 

Warmup runs: [1]

Graph: path/to/venv/tensorflow/toco256.tflite]

Input layers: 

Input shapes: 

Use nnapi : [0]

Loaded model path/to/venv/tensorflow/toco256.tflite

resolved reporter

Initialized session in 45.556ms

Running benchmark for 1 iterations 

tensorflow/contrib/lite/kernels/pad.cc:96 op_context.dims != 4 (3 != 4)

Node number 24 (PAD) failed to prepare.



Failed to invoke!

Aborted (core dumped)

We also tried the same command without including the 'allow_custom_ops' and 'post_training_quantize' options and even used the same input size as 1,513,513,3; but the result was the same.

This issue seems to be similar to the following github issue:
(https://github.com/tensorflow/tensorflow/issues/21266). However in the latest version of TensorFlow the issue is supposed to be fixed.

Model: http://download.tensorflow.org/models/deeplabv3_mnv2_pascal_trainval_2018_01_29.tar.gz
Tensorflow version: 1.11
Bazel version: 0.17.2
OS: Ubuntu 18.04

Also the android application was not able to load the model properly (tflite interpretr)

So, how can we convert a segmentation model properly to a tflite format which can be used for inference on an android device?

UPDATE:-

Using tensorflow 1.12, we got a new error :

$ bazel run -c opt tensorflow/lite/tools/benchmark:benchmark_model -- --graph=`realpath /path/to/research/deeplab/venv/tensorflow/toco256.tflite`



    tensorflow/lite/kernels/depthwise_conv.cc:99 params->depth_multiplier * SizeOfDimension(input, 3) != SizeOfDimension(filter, 3) (0 != 32)

    Node number 30 (DEPTHWISE_CONV_2D) failed to prepare.

Also,while using a newer version of the same model(3 Mb .pb file) with depth_multiplier=0.5 from the tensorflow deeplab model zoo, we got a different error:-

F tensorflow/lite/toco/graph_transformations/propagate_fixed_sizes.cc:116] Check failed: dim_x == dim_y (3 vs. 32)Dimensions must match

In this case we used the same aforementioned command for tflite conversion ;but we were not even able to produce a 'tflite' file as output.It seems to be an issue with depth multiplier values.(Even we tried giving the depth_multiplier parameter as argument at the time of conversion).

I have same issue. from https://github.com/tantara/JejuNet I see that he was successfully convert model to tflite. I PM him for help, but unfortunately no response right now.