Image Preprocessing Operator

Authors: derekdqc

Overview

This operator uses PyTorch to transform the image, such as cropping, PIL.Image and Tensor conversion, normalization and other operations on the image.

In computer vision (CV) directions, image transformations are usually an indispensable part, which can be used to pre-process images and enhance data. And transforms are common image transformations, they can be chained together using Compose in Pytorch.

Interface

__init__(self, size: int)

Args:

• size:
  • the size of the output image
  • supported types: int

__call__(self, img_path: Union[Image.Image, torch.Tensor, str])

Args:

• img_path:
  • the input image to be processed
  • supported type: PIL.image or str (path of the image)

Returns:

The Operator returns a tuple Tuple[('img_transformed', torch.Tensor)] containing following fields:

• img_transformed:
  • the tensor of the transformed image
  • data type: torch.Tensor

Requirements

You can get the required python package by requirements.txt.

How it works

The towhee/transform-image Operator is used for image transformation and is an important part of data preprocessing. It can be added to the pipeline and is usually used as the first custom operator of the pipeline. For example, it's the first Operator named processing within image-embedding-resnet50 pipeline.

Reference

• https://pytorch.org/vision/stable/transforms.html

More Resources

• Scalar Quantization and Product Quantization - Zilliz blog: A hands-on dive into scalar quantization (integer quantization) and product quantization with Python.
• Combine AI Models for Image Search using ONNX and Milvus - Zilliz blog: Use ONNX to process multiple models of CV (computer vision), and combine multiple models with Milvus for similar vector retrieval to get similar images.
• Demystifying Color Histograms: A Guide to Image Processing and Analysis - Zilliz blog: Mastering color histograms is indispensable for anyone involved in image processing and analysis. By understanding the nuances of color distributions and leveraging advanced techniques, practitioners can unlock the full potential of color histograms in various imaging projects and research endeavors.
• Using Vector Search to Better Understand Computer Vision Data - Zilliz blog: How Vector Search improves your understanding of Computer Vision Data
• What are Vision Transformers (ViT)? - Zilliz blog: Vision Transformers (ViTs) are neural network models that use transformers to perform computer vision tasks like object detection and image classification.
• Understanding ImageNet: A Key Resource for Computer Vision and AI Research: The large-scale image database with over 14 million annotated images. Learn how this dataset supports advancements in computer vision.
• Image Embeddings for Enhanced Image Search - Zilliz blog: Image Embeddings are the core of modern computer vision algorithms. Understand their implementation and use cases and explore different image embedding models.
• Batch vs Layer Normalization - Zilliz blog: A guide to equip developers with the knowledge to unlock efficiency and enhance model performance.