Embeddings sharing the same parameter

General Discussion
,
#1

Hi everyone!

I have a model in which I have 3 parallel encoders and one decoder (see the image attached). I would like to implement the model in pure TensorFlow/Keras (there is an implementation in Lingvo, which is obscure given my limited knowledge).

If colors of embeddings are the same, it means that embeddings share parameters between them. I would like to ask how to achieve this in Tensorflow/keras?

neighbors_model

#3

Hi @acraev, You can define an embedding layer and then reuse it by passing the encoder outputs. For example

embedding = Embedding(10, 6, input_length=1, name='embedding')
target = embedding(enocder1)
context = embedding(encoder2)

Thank You.

1 Like
#4

Actually, thank you!

Is it that simple? I mean, if I have to create the second embedding, it would just be a matter of defining a new embedding, like embedding2 = Embedding(10, 6, input_length=1, name=‘embedding2’) ?

Sorry then for stupid question, I guess and thank you for pointing this out!

#6

Hi @acraev, Yes, for using second embedding to have to define the new one. Thank You

1 Like