Interpretability and Analysis

Attention Weight Matrices

Head-specific Patterns

Layer-wise Analysis

Attention Rollout

Embedding Space Structure

Layer-wise Representations

Probing Tasks

Geometric Properties

Syntactic Patterns

Semantic Patterns

Positional Patterns

Multi-Head Specialization

Attention Entropy

Attention Distance

Head Importance Scoring

Layer Importance Analysis