Image Embedding Operator with Resnet50

Authors: derekdqc, shiyu22

Overview

This Operator generates feature vectors from the pytorch pretrained Resnet50 model[1], which is trained on imagenet dataset.

Resnet models were proposed in “Deep Residual Learning for Image Recognition”[2], this model was the winner of ImageNet challenge in 2015. "The fundamental breakthrough with ResNet was it allowed us to train extremely deep neural networks with 150+layers successfully. Prior to ResNet training very deep neural networks were difficult due to the problem of vanishing gradients"[3].

Interface

__init__(self, model_name: str, framework: str = 'pytorch')

Args:

model_name (str):

The model name for embedding, for example 'resnet50'.

framework (str):

The framework of the model, the default is 'pytorch'.

__call__(self, img_tensor: torch.Tensor)

Args:

img_tensor (torch.Tensor):

The image tensor.

Returns:

(Tuple[('cnn', numpy.ndarray)])

The embedding of image.

Requirements

You can get the required python package by requirements.txt.

How it works

The towhee/resnet50-image-embedding Operator implements the function of image embedding, which can add to the pipeline, for example, it's the key Operator named embedding_model within image_embedding_resnet50 pipeline, and it is the red box in the picture below.