resnet-image-embedding/README.md

# Image Embedding Operator with Resnet50

Authors: derekdqc, shiyu22

## Overview

This Operator generates feature vectors from the pytorch pretrained **Resnet50** model[1], which is trained on [imagenet dataset](https://image-net.org/download.php).

**Resnet** models were proposed in “[Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385)”[2], this model was the winner of ImageNet challenge in 2015. "The fundamental breakthrough with ResNet was it allowed us to train extremely deep neural networks with 150+layers successfully. Prior to ResNet training very deep neural networks were difficult due to the problem of vanishing gradients"[3].

## Interface

`__init__(self, model_name: str, framework: str = 'pytorch')`

**Args:**

- model_name (`str`):

  The model name for embedding, for example 'resnet50'.

- framework (`str`):

  The framework of the model, the default is 'pytorch'.

`__call__(self, img_tensor: torch.Tensor)`

**Args:**

​    img_tensor (`torch.Tensor`):

​        The image tensor.

**Returns:**

​    (`Tuple[('feature_vector', numpy.ndarray)]`)

​         The embedding of image.

## Requirements

You can get the required python package by [requirements.txt](./requirements.txt).

## How it works

The `towhee/resnet50-image-embedding` Operator implements the function of image embedding, which can add to the pipeline, for example, it's the key Operator named embedding_model within [image-embedding-resnet50](https://hub.towhee.io/towhee/image-embedding-resnet50) pipeline, and it is the red box in the picture below.

![img](./readme_res/operator.png)

## Reference

[1].https://pytorch.org/hub/pytorch_vision_resnet/

[2].https://arxiv.org/abs/1512.03385

[3].https://towardsdatascience.com/understanding-and-coding-a-resnet-in-keras-446d7ff84d33