I wanted to mention a talk I presented today titled “You Don’t Know TensorFlow”. In this talk, I presented some under-appreciated and under-used features of TensorFlow. In particular, I focused on two things: (1) handling variable-length sequences in TensorFlow and (2) XLA.
For the first one, I showed the use of
tf.padded_batch() to speed up the training time of a simple sequence model from 250 seconds down to 147 seconds!
For the second one, I started with “operator fusion” and moved all the way up to speed up text generation workflows by ~100x (with lots of parts discarded in the interest of time).
The materials can be found at the below links
Code and slides: