Model,Configuration,Attn (bits),FFN (bits),Emb (bits),Size (MB),Attn Layers,FFN Layers,Total Quantized
DeepSeek-R1-Distill-Qwen-1.5B,aggressive,4,8,8,6779.05,112,84,197
DeepSeek-R1-Distill-Qwen-1.5B,uniform_8bit,8,8,8,6779.05,112,84,197
DeepSeek-R1-Distill-Qwen-1.5B,very_aggressive,4,4,8,6779.05,112,84,197
Phi-3-mini-128k-instruct,aggressive,4,8,8,14576.26,64,64,129
Phi-3-mini-128k-instruct,uniform_8bit,8,8,8,14576.26,64,64,129
Phi-3-mini-128k-instruct,very_aggressive,4,4,8,14576.26,64,64,129
Falcon-E-3B-Base,aggressive,4,8,8,512.51,0,0,1
Falcon-E-3B-Base,uniform_8bit,8,8,8,512.51,0,0,1
Falcon-E-3B-Base,very_aggressive,4,4,8,512.51,0,0,1