Training weak learners in using video or image dataset on TensorFlow

Hi I have a problem statement where I have to train an action recognizer using videos.
but due to non availability of good dataset there is a slight misclassification.
Is there any setting or function where I can train the misclassified or weak learners by assigning them more weights.
Its sounds similar like Xgboost but I want something which can work on Deeplearning frame works and on Image and Video data set