Image Embedding Pipeline with Resnet50

Authors: derekdqc, shiyu22

Overview

This pipeline is used to extract the feature vector of a given image.

The pipeline performs two major steps. First, normalize the image, and then use resnet50 model to generate the vector.

Interface

Pipeline Input Arguments:

• img(PIL.Image):

  The image to be encoded. PIL.Image is the currently supported image type.

Pipeline Returns:

• (Tuple[('cnn', numpy.ndarray)])

The pipeline returns a tuple.

• The feature_vector field contains the embedding of the image.

How to use

1. Install Towhee

$ pip3 install towhee

You can refer to Getting Started with Towhee for more details. If you have any questions, you can submit an issue to the towhee repository.

2. Run it with Towhee

>>> from towhee import pipeline
>>> from PIL import Image

>>> img = Image.open('path/to/your/image') # for example, './test_data/test.jpg'
>>> embedding_pipeline = pipeline('towhee/image-embedding-resnet50')
>>> embedding = embedding_pipeline(img)

How it works

This pipeline includes two main operators: transform image (implemented as towhee/transform-image) and image embedding (implemented as towhee/resnet50-image-embedding). The transform image operator will first convert the original image into a normalized format, such as with 512x512 resolutions. Then, the normalized image will be encoded via image embedding operator, and finally we get a feature vector of the given image. (The basic concepts of pipeline and operator are introduced in Towhee architecture.)