From the end of last year, big changes have witnessed in NLP research.
Embracing an unprecedented growth, I started to study new exciting results and advances.
In doing so, I noticed I’m not listed as contributor of repo which my PR accessed.
How did I come to a repository?
When I’m stuck, I would prefer to code, than to go deep in theory. (It must be so.. too much to understand 🤒)
It was BERT released by Google AI I felt keenly the necessity of implementing, because not only couldn’t understand the way they figured out positional encoding formula, but how it actually works.
What does it mean to “scale” dot product in Attention? (Now I know it’s far from my section 😂)
Figure 1. Scaled Dot Product. Adopted from tensorflow blog
What was the code error?
Meanwhile, I found out a small error in tokenization process, which was changing a token into [MASK], enabled bidirectional representation.
I’ve made PR, and got merged. But I was not in contributors. Why?
Figure 2. Merged Pull request Adopted from graykode project
Actually I happened to know there can be couple of reasons github doesn’t include my name as contributor.
Well, if contributors tab has more than 100 people, in which case it shows you up only if you are in the top 100 contributors because displaying too many contributors can make webpages down.
Somethimes, however, it doesn’t that problem. Why not?
Two possibilities are there.
Second, if you happend to commit using a different git email that what is in your GitHub profile, it will not be attached to your Github user, and “doesn’t show up” as you.