Draw self-attention
Hard · Whiteboard
Draw scaled dot-product self-attention for a sequence of n tokens. Write the formula, explain why we divide by √d_k, and give the time and memory cost in n.
Create a free account to answer it and get scored on what a strong answer covers.