ASCI Logo
ASCI.Academy
LearnPracticeCompeteCareer
Sign InGet Started

Search ASCI Academy

Universal search across Courses, Lessons, Skills, Projects, Challenges, Competitions, Jobs, Internships, and Users

deepseek r1-reasoning-models

Curriculum Index

Current LevelInitiate
0/ 500 XP
Gravit Certificate:
100% Required
  • 1.1 MLA KV Cache Compression Ratio
•
1
2
3
4
5
6
7
8
9
10
Module 1: Multi-Head Latent Attention (MLA) Architecture•Lesson 1

1.1 MLA KV Cache Compression Ratio

### MLA Compression Ratio
Traditional MHA requires storing $2 \times n_{heads} \times d_{head}$ parameters per token. MLA compresses keys and values into a single latent vector of dimension $d_c$:

compression = (2 * n_heads * d_head) / d_c
Python 3.11
•
Interactive Challenge
+75 XP
Challenge Objective

Compute the MLA compression ratio for 128 heads of dim 128 versus latent dim 512 and print 'MLA_RATIO: 64.0'.

solution.py
UTF-84 SpacesAuto-saved
cpython3.11:~/workspace
CPython 3.11.8 (WASM)
$python3 -u solution.py
Click "Compile & Run" (or press Ctrl + Enter) to compile AST, verify bytecode, and execute the solution.