Can BERT be used as an alternative to the Universal Sentence Encoder?

General Discussion
, , ,
#1

BERT is available in TFJS for question answering but can also it be used as a sentence encoder? USE is getting pretty old and my impression is that BERT is better for the same task. Is it?

2 Likes
#2

I don’t know if you need exactly BERT for your project but if you want something ready we have many USE models available in TensorFlow Hub

Here you can find conversion commands for TFjs:

3 Likes
#3

Thanks. Do you know how those USE models compare to the one in the tfjs-models on TensorFlow GitHub that has already been converted?

1 Like
#4

Here are some resources:

1 Like
#5

If you click the TFHub link in that Readme you can see that it is the USE light on TFHub

1 Like
#6

I’m getting the idea that I should stay with USE (and forget about BERT) but in deciding which TFHub USE version, instead of finding any performance comparisons, I find the sentence “We apply this model to the STS benchmark for semantic similarity, and the results can be seen in the example notebook made available.”

So, if I want to use benchmarks to decide if it is worth trying different sizes of USE do I need to run this notebook to generate my own benchmarks? The paper by Cer et al lists some benchmarks but not for different sizes of USE. I don’t even find a table listing the number of parameters for each of the USE variants.

1 Like
#7

This is a great repo for comparison between different embeddings, language models, techniques, etc. for sentence embeddings:

They also have a notebook for USE metrics so you can compare that with the rest of that metrics table.

2 Likes