Deploying yolort on TVM

This article is an introductory tutorial to deploy PyTorch YOLOv5 models with Relay VM.

For us to begin with, PyTorch should be installed. TorchVision is also required since we will be using it as our model zoo.

A quick solution is to install via pip

pip install torch==1.10.1
pip install torchvision==0.11.2

or please refer to official site https://pytorch.org/get-started/locally/

PyTorch versions should be backwards compatible but should be used with the proper TorchVision version.

Currently, Only test TVM with PyTorch 1.7.x and 1.10.x, other versions may be unstable.

And this notebook is running on macOS M1.

---

Copyright © Most of the codes is copied from the TVM tutorial.

import tvm
from tvm import relay
from tvm.runtime.vm import VirtualMachine

import numpy as np
import cv2

# PyTorch imports
import torch
from torch import nn
import torchvision

Load pre-trained yolov5n from yolort and do tracing

in_size = 640
input_shape = (in_size, in_size)

from yolort.models import yolov5n
from yolort.relay import get_trace_module

model_func = yolov5n(pretrained=True, size=(in_size, in_size))
script_module = get_trace_module(model_func, input_shape=input_shape)

/Users/zhiqiang/miniconda3/envs/tvm/lib/python3.8/site-packages/torch/_jit_internal.py:668: LightningDeprecationWarning: The `LightningModule.loaded_optimizer_states_dict` property is deprecated in v1.4 and will be removed in v1.6.
  if hasattr(mod, name):
/Users/zhiqiang/miniconda3/envs/tvm/lib/python3.8/site-packages/torch/_jit_internal.py:668: LightningDeprecationWarning: The `LightningModule.model_size` property was deprecated in v1.5 and will be removed in v1.7. Please use the `pytorch_lightning.utilities.memory.get_model_size_mb`.
  if hasattr(mod, name):
/Users/zhiqiang/miniconda3/envs/tvm/lib/python3.8/site-packages/torch/_jit_internal.py:669: LightningDeprecationWarning: The `LightningModule.model_size` property was deprecated in v1.5 and will be removed in v1.7. Please use the `pytorch_lightning.utilities.memory.get_model_size_mb`.
  item = getattr(mod, name)
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/yolo_module.py:113: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for img in inputs:
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/transform.py:179: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  images = [img for img in images]
/Users/zhiqiang/miniconda3/envs/tvm/lib/python3.8/site-packages/torch/nn/functional.py:3701: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  (torch.floor((input.size(i + 2).float() * torch.tensor(scale_factors[i], dtype=torch.float32)).float()))
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/transform.py:282: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  img_h, img_w = _get_shape_onnx(img)
/Users/zhiqiang/miniconda3/envs/tvm/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/anchor_utils.py:45: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  anchors = torch.as_tensor(self.anchor_grids, dtype=torch.float32, device=device).to(dtype=dtype)
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/anchor_utils.py:46: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  strides = torch.as_tensor(self.strides, dtype=torch.float32, device=device).to(dtype=dtype)
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/box_head.py:402: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  strides = torch.as_tensor(self.strides, dtype=torch.float32, device=device).to(dtype=dtype)
/Users/zhiqiang/Coding/yolov5-rt-stack/yolort/models/box_head.py:333: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for head_output, grid, shift, stride in zip(head_outputs, grids, shifts, strides):

script_module.graph

graph(%self.1 : __torch__.yolort.relay.trace_wrapper.TraceWrapper,
      %x : Float(1, 3, 640, 640, strides=[1228800, 409600, 640, 1], requires_grad=0, device=cpu)):
  %model : __torch__.yolort.models.yolo_module.YOLOv5 = prim::GetAttr[name="model"](%self.1)
  %4973 : (Tensor, Tensor, Tensor) = prim::CallMethod[name="forward"](%model, %x)
  %4970 : Float(0, 4, strides=[4, 1], requires_grad=0, device=cpu), %4971 : Float(0, strides=[1], requires_grad=0, device=cpu), %4972 : Long(0, strides=[1], requires_grad=0, device=cpu) = prim::TupleUnpack(%4973)
  %3751 : (Float(0, 4, strides=[4, 1], requires_grad=0, device=cpu), Float(0, strides=[1], requires_grad=0, device=cpu), Long(0, strides=[1], requires_grad=0, device=cpu)) = prim::TupleConstruct(%4970, %4971, %4972)
  return (%3751)

Download a test image and pre-process

from yolort.utils import get_image_from_url

img_source = "https://huggingface.co/spaces/zhiqwang/assets/resolve/main/bus.jpg"
# img_source = "https://huggingface.co/spaces/zhiqwang/assets/resolve/main/zidane.jpg"
img = get_image_from_url(img_source)

img = img.astype("float32")
img = cv2.resize(img, (in_size, in_size))

img = np.transpose(img / 255.0, [2, 0, 1])
img = np.expand_dims(img, axis=0)

Import the graph to Relay

input_name = "input0"
shape_list = [(input_name, (1, 3, *input_shape))]
mod, params = relay.frontend.from_pytorch(script_module, shape_list)

Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using reduce.cpu for min based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for minimum based on highest priority (10)
Using injective.cpu for floor based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using reduce.cpu for min based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for minimum based on highest priority (10)
Using injective.cpu for floor based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for ceil based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for round based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for ceil based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for round based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for ceil based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for round based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for ceil based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for round based on highest priority (10)

Compile with Relay VM

Note: Currently only CPU target is supported. For x86 target, it is highly recommended to build TVM with Intel MKL and Intel OpenMP to get best performance, due to the existence of large dense operator in torchvision rcnn models.

# Add "-libs=mkl" to get best performance on x86 target.
# For x86 machine supports AVX512, the complete target is
# "llvm -mcpu=skylake-avx512 -libs=mkl"
target = "llvm"

with tvm.transform.PassContext(opt_level=3):
    vm_exec = relay.vm.compile(mod, target=target, params=params)

Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sqrt based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for negative based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for arange based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for meshgrid based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for copy based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for arange based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for meshgrid based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for copy based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for arange based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for meshgrid based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for copy based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for repeat based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for less based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for minimum based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using conv2d_nchw.x86 for nn.conv2d based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for image.resize2d based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using pool.cpu for nn.max_pool2d based on highest priority (10)
Using pool.cpu for nn.max_pool2d based on highest priority (10)
Using pool.cpu for nn.max_pool2d based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for image.resize2d based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for image.resize2d based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using conv2d_NCHWc.x86 for nn.contrib_conv2d_NCHWc based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using transpose.generic for transpose based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for power based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using transpose.generic for transpose based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for power based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using injective.cpu for layout_transform based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using transpose.generic for transpose based on highest priority (10)
Using injective.cpu for sigmoid based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for power based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for reshape based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for take based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for greater based on highest priority (10)
Using argwhere.generic for argwhere based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for adv_index based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for adv_index based on highest priority (10)
Using injective.cpu for ndarray_size based on highest priority (10)
Using injective.cpu for equal based on highest priority (10)
Using injective.cpu for zeros based on highest priority (10)
Using reduce.cpu for min based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using reduce.cpu for max based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using concatenate.cpu for concatenate based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using injective.cpu for shape_of based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for arange based on highest priority (10)
Using injective.cpu for expand_dims based on highest priority (10)
Using nms.generic for vision.non_max_suppression based on highest priority (10)
[12:46:54] /Users/zhiqiang/Coding/tvm/src/te/schedule/bound.cc:119: not in feed graph consumer = hybrid(hybrid_nms, 0x60000a817250)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for shape_of based on highest priority (10)
Using injective.cpu for slice_like based on highest priority (10)
Using injective.cpu for where based on highest priority (10)
Using injective.cpu for greater_equal based on highest priority (10)
Using injective.cpu for where based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for dyn.strided_slice based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for strided_slice based on highest priority (10)
Using injective.cpu for adv_index based on highest priority (10)
Using injective.cpu for split based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for squeeze based on highest priority (10)
Using injective.cpu for subtract based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for stack based on highest priority (10)
Using injective.cpu for adv_index based on highest priority (10)
Using injective.cpu for adv_index based on highest priority (10)
[12:46:54] /Users/zhiqiang/Coding/tvm/src/relay/transforms/let_list.h:54: Warning: letlist not used
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for cast based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using injective.cpu for add based on highest priority (10)
Using injective.cpu for divide based on highest priority (10)
Using reduce.cpu for prod based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using reduce.cpu for prod based on highest priority (10)
Using injective.cpu for multiply based on highest priority (10)
Using reduce.cpu for prod based on highest priority (10)
[12:46:55] /Users/zhiqiang/Coding/tvm/src/relay/transforms/let_list.h:54: Warning: letlist not used
[12:46:55] /Users/zhiqiang/Coding/tvm/src/relay/transforms/let_list.h:54: Warning: letlist not used

Inference with Relay VM

ctx = tvm.cpu()
vm = VirtualMachine(vm_exec, ctx)
vm.set_input("main", **{input_name: img})
tvm_res = vm.run()

%%timeit
vm.set_input("main", **{input_name: img})
tvm_res = vm.run()

65.2 ms ± 1.31 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Get boxes with score larger than 0.6

score_threshold = 0.6
boxes = tvm_res[0].asnumpy().tolist()
valid_boxes = []
for i, score in enumerate(tvm_res[1].asnumpy().tolist()):
    if score > score_threshold:
        valid_boxes.append(boxes[i])
    else:
        break

print(f"Get {len(valid_boxes)} valid boxes")

Get 4 valid boxes

Verify the Inference Output on TVM backend

with torch.no_grad():
    torch_res = script_module(torch.from_numpy(img))

for i in range(len(torch_res)):
    torch.testing.assert_allclose(torch_res[i], tvm_res[i].asnumpy(), rtol=1e-4, atol=1e-4)

print("Exported model has been tested with TVM Runtime, and the result looks good!")

Exported model has been tested with TVM Runtime, and the result looks good!

View this document as a notebook: https://github.com/zhiqwang/yolov5-rt-stack/blob/main/notebooks/export-relay-inference-tvm.ipynb

---