I would like to implement Instance segmentation in TensorFlow 2+ based on CondInst + CenterNet. CondInst is currently implemented only in PyTorch. I saw the implementation in pytorch and I’m not sure if this is possible in tensorflow …
The paper is interesting
Has anyone worked on it? Any tips?
June 21, 2021, 3:53pm
You need to write your CondConv custom layer like this (not personally tested):
This file has been truncated.
import tensorflow as tf
from config import WEIGHT_DECAY
from tensorflow.keras import layers
def conv2d(kernel_size, stride, filters, kernel_regularizer=tf.keras.regularizers.l2(WEIGHT_DECAY), padding="same", use_bias=False,
return layers.Conv2D(kernel_size=kernel_size, strides=stride, filters=filters, kernel_regularizer=kernel_regularizer, padding=padding,
use_bias=use_bias, kernel_initializer=kernel_initializer, **kwargs)
def __init__(self, out_channels, dropout_rate, temperature=30, **kwargs):
self.avgpool = layers.GlobalAveragePooling2D()
self.dropout = layers.Dropout(rate=dropout_rate)
self.fc = layers.Dense(units=out_channels)
self.softmax = layers.Softmax()
self.temperature = temperature
Centernet models are available in the Model Gardens research section:
Thanks for suggestion, really appreciate it.
Actually this implementation that you posted is more like Mixture of Experts than CondConv: see discussion github prstrive/CondConv-tensorflow/issues/1
And instead of doing the CondConv on previous convolution outputs (the number of outputs is always the same), we need to do it on instances of detected objects (the number of objects is different for every image).
June 22, 2021, 9:13am
You can create your CondConv custom layer from the original impl:
This file has been truncated.
# -*- coding: utf-8 -*-
from skimage import color
from torch import nn
import torch.nn.functional as F
from detectron2.structures import ImageList
from detectron2.modeling.proposal_generator import build_proposal_generator
from detectron2.modeling.backbone import build_backbone
from detectron2.modeling.meta_arch.build import META_ARCH_REGISTRY
from detectron2.structures.instances import Instances
from detectron2.structures.masks import PolygonMasks, polygons_to_bitmask
from .dynamic_mask_head import build_dynamic_mask_head
from .mask_branch import build_mask_branch
from adet.utils.comm import aligned_bilinear
Or request the off the shelf model at:
07:14AM - 06 Jun 20 UTC
This issue contains **all open requests for paper implementations requested by t
… he community**.
We cannot guarantee that we can fulfill community requests for specific paper implementations.
If you'd like to contribute, **please add a comment to the relevant GitHub issue to express your interest in providing your paper implementation**.
Awesome external contributors will be nominated for [Google Open Source Peer Bonus](https://opensource.google/docs/growing/peer-bonus/).
Please also see our [contribution guidelines](https://github.com/tensorflow/models/wiki/Research-paper-code-contribution) and [paper selection criteria](https://github.com/tensorflow/models/wiki/Research-paper-code-contribution#model-selection).
## Computer Vision
| Paper | Conference | GitHub issue | Note |
| ResNeXt: [Aggregated Residual Transformations for Deep Neural Networks](https://arxiv.org/abs/1611.05431) | CVPR 2017 | #6752 | |
| DenseNet: [Densely Connected Convolutional Networks](https://arxiv.org/abs/1608.06993) | CVPR 2017 | #8278 | |
| [Density estimation using Real NVP](https://arxiv.org/abs/1605.08803) | ICLR 2017 | #7848 | Need to migrate [TF 1 code](https://github.com/tensorflow/models/tree/master/research/real_nvp) to TF 2 |
| [Spatiotemporal Contrastive Video Representation Learning](https://arxiv.org/abs/2008.03800) | CVPR 2021 | #9993 | In progress (Internally) |