A latent mistake about the semantic representation of the message within predicted label computation in LLM classification fine-tuning

[Jack1447]

I found a technical issue in the book regarding sequence classification with LLMs fine-tuning. The text suggests using hidden_states[:, -1, :] to extract the last token's representation.

Because of right-padding, the token at index [-1] is usually a vector corresponding to the [PAD] token, which may be replaced with the last valid token dynamically using the attention_mask?