I am following this example to use BERT for sentiment classification.
text_input = tf.keras.layers.Input(shape=(), dtype=tf.string) preprocessor = hub.KerasLayer( "https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3") encoder_inputs = preprocessor(text_input) encoder = hub.KerasLayer( "https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4", trainable=True) outputs = encoder(encoder_inputs) pooled_output = outputs["pooled_output"] # [batch_size, 768]. sequence_output = outputs["sequence_output"] # [batch_size, seq_length, 768]. embedding_model = tf.keras.Model(text_input, pooled_output)sentences = tf.constant(["(your text here)"])print(embedding_model(sentences))
The sequence length by default seems to 128 from looking at the output shape from encoder_inputs. However, I’m not sure how to change this? Ideally I’d like to use to a larger sequence length.
There’s an example of modifying sequence length from the preprocessor page, but I’m not sure how to incorporate this into the functional model definition I have above? I would greatly appreciate any help with this.