Dense
Meta · Unknown
Llama 3.2 3B
Dense decoder architecture with GQA attention mechanism.
Llama 3.2 3B decoder block architecture: Attention: GQA. Normalization: RMSNorm. FFN: SwiGLU. Position encoding: RoPE. Scale: 3B, 128K context, 24 layers. Decoder type: Dense.
GQA·SwiGLU
3B|128K context|GQA|Dense
Architecture Specifications
Parameters3B
Context Window128K
Decoder TypeDense
AttentionGQA
Release DateUnknown
CategoryEfficient & Small
OrganizationMeta
Key Features
Grouped Query AttentionLayer mix: 28 GQAKV cache: 112 KiB/token
Enterprise AI platform
Compare, evaluate, and deploy LLM architectures at scale
Colaberry AI provides architecture specifications, benchmark comparisons, and deployment guidance for enterprise AI teams.