Official TF object detection API models dont support batching

[past wedge]

Hii everyone !!

I am working on a project which require object-detection, no classification needed as i am detecting a single object.

So for this I used TF object detection API, selected EfficientDet-d0 model and trained it on my custom dataset...
And later found out all of these model (all that i have checked) doesn't support batching which happens to be a necessity for my usecase..

So i have 2 paths from here either to completely change model and train a new one for which i need help from you guys, as if anyone aware of other TF based models which support batching and other paths is to use queues of inputs (obvious way) which i am not soo sure if it increase the inference time or effect the performance of pipeline overall..

https://tfhub.dev/tensorflow/collections/object_detection/1

[past wedge]

why these models cant support basic feature like batching

@calm hazel

[calm hazel]

Models in TF hub are usually saved in a SavedModel format. Some support batches but I havent encountered one that doesn’t.

[past wedge]

also I read somewhere batch inference only beneficial on GPU/TPU. Is it right??

[calm hazel]

You should be able to do it one CPU too. The difference is only gonna be the time it takes to process the data. Models work faster in general on GPU/TPU

[past wedge]

I think I have train a new models..

Anyone know any good APIs or way I can train a object detection TF model, I don't think I should build a architecture and train it from scratch??

[calm hazel]

I did send you those Keras resources (given that Keras is integrated as part of TF)

Reposting here
https://keras.io/guides/keras_cv/object_detection_keras_cv/
https://keras.io/examples/vision/retinanet/
https://keras.io/examples/vision/object_detection_using_vision_transformer/