Sparsity in LLM
#Day4 of Being an Imposter đŸ˜› Sparsity in #LLMs refers to the fraction of parameters that are active during an […]
Blog, Study Room
Blog
Blog
PLE – Per Layer Embedding in LLMs
Day 3 of Being an imposter đŸ˜› PLE (Per Layer Embedding) is a surprisingly similar approach to MoE,Instead of doing
Blog, Little Questions, Loud Thoughts
Patient… or Just Told to Be Patient?*
There is a quiet irony in the word patient. The one who feels the pain is called the patient—and the
Blog
MoE – Mixture of Experts in LLMs
Day 2 of Being an imposter đŸ˜› MoE (Mixture of Experts) was a leap beyond thought, that is now being
Blog, Study Room
RoPE in LLM
Day 1 of being an imposter đŸ˜› RoPE (Rotary Positional Embedding) is crazy good way to reduce the dimensional space,