I recently put the finishing touches on my Faster R-CNN self-learning exercise. My goal was to replicate the model from scratch using only the paper. That was a bit ambitious and I had to eventually relent and peek at some existing implementations to understand a few things the paper is unclear on. The repo is here: GitHub - trzy/FasterRCNN: Clean and readable implementations of Faster R-CNN in PyTorch and TensorFlow 2 with Keras.
I wrote both a PyTorch and a TensorFlow implementation. I’d like to think they are pretty clean, readable, and easy to use. I also documented some of my struggles and takeaways in the README.md file.
One thing that continues to bother me is the need for an additional tf.stop_gradient() in the regression loss functions surrounding a tf.less statement. The function itself is differentiable. The PyTorch version doesn’t need this. I might make a post about it on one of the other sub-forums because I stumbled upon the solution by accident. Without the explicit stop_gradient, the model still learns, but achieves significantly lower precision. Would love to learn about how others would approach debugging such an issue.