Possible error in MobileBert embedding convolution

[jaemo91]

google-research/mobilebert/modeling.py

Lines 229 to 235 in 5dce46b

	if config.trigram_input:
	inputs = self.word_embedding_output
	self.embedding_output = tf.concat(
	[tf.pad(inputs[:, 1:], ((0, 0), (0, 1), (0, 0))),
	inputs,
	tf.pad(inputs[:, :-1], ((0, 0), (1, 0), (0, 0)))],
	axis=2)

(This gives you [1=0=pad],[2=1=0],[3=2=1] style concatenation)
It seems the concatenation is being done backwards and it should be done like below:

        if config.trigram_input:
          inputs = self.word_embedding_output
          self.embedding_output = tf.concat(
              [tf.pad(inputs[:, :-1], ((0, 0), (1, 0), (0, 0))),
               inputs,
               tf.pad(inputs[:, 1:], ((0, 0), (0, 1), (0, 0)))],
              axis=2)

(This gives you [pad=0=1],[0=1=2],[1=2=3] style concatenation)

Given that this change will affect all the models already generated, I don't think it would be a good idea to force an update, but I was wondering if this was intentional and if you were aware of this issue.