How to get faster prediction time

[ghost]

We are working on license plate recognition.We used a model converted from keras h5. But it is taking long to get loaded. The time for prediction is also high.

[zaidalyafeai]

@Aimantech, what is the architecture of the model and its size ? Are you converting a well know model or you are training form scratch ?

[ghost]

Training from scratch. We are following this keras example https://github.com/keras-team/keras/blob/master/examples/image_ocr.py. Size of the model is 17MB.

[ghost]

Model summary:

[zaidalyafeai]

@Aimantech , you said the loading time is long but compared to what? Are you loading the model from a server or local machine? If you are loading from a server then the speed of your internet connection is a factor. On the other hand, the prediction time should be fast as the model is not that huge.

[ghost]

Model is loading from github repo. Other tensorflow.js examples, including your's, works well. So internet speed may not be the problem. This warning is thrown when the model is being loaded:

Orthogonal initializer is being called on a matrix with more than 2000 (786432) elements: Slowness may result.

[zaidalyafeai]

@Aimantech, can you check with less parameters for the GRU layers. There might be a bug there. You don't have to retrain the model. Just convert an empty model.

[ghost]

You meant pruning?

[ghost]

`

@Aimantech , you said the loading time is long but compared to what? Are you loading the model from a server or local machine? If you are loading from a server then the speed of your internet connection is a factor. On the other hand, the prediction time should be fast as the model is not that huge.

The same model, when run with python keras takes about 0.2s. But in tensorflow.js the time for prediction is about 1.5min. In both case the backend was cpu.

[zaidalyafeai]

@Aimantech, not sure about the prediction time when the backend is cpu. I have not experimented with that. Maybe you could repost the question in https://groups.google.com/a/tensorflow.org/forum/#!forum/tfjs to get a precise answer from the developers of tfjs.