Bert-based QA can't answer yes/no questions

General Discussion
, ,
#1

The TFJS BERT-based QA demo (https://storage.googleapis.com/tfjs-models/demos/mobilebert-qna/index.html) can answer “Where was Tesla born?” but not “Was Tesla born in the Austrian Empire?”.

In general, it doesn’t seem to be able to answer yes/no questions. Why is that?

1 Like
#2

I think that these models could be still finetuned on BoolQ style QA datasets :

2 Likes
#3

Is there a smaller version that might be ported to TensorFlow.js?

#4

BoolQ Is also part of SuperGlue and we have some fine-tuning scripts in model garden:

Check there what kind of nlp models in model garden could be finetuned with these scripts.

1 Like
#5

Also can you check this version of a demo I created a while back as a demo that may work slightly better. If not no worries, but worth a try.

Also @pyu May have some thoughts here as he was heavily involved in creating the model conversion on the TensorFlow.js side of things!

2 Likes
#7

I tried “Is TensorFlow an end-to-end open source platform for machine learning?” with the default passage and it didn’t work. “What is TensorFlow?” did work fine.

2 Likes
#9

@pyu May have some thoughts here who worked on this model.