| Model,Configuration,Attn (bits),FFN (bits),Emb (bits),Size (MB),Attn Layers,FFN Layers,Total Quantized |
| DeepSeek-R1-Distill-Qwen-1.5B,aggressive,4,8,8,6779.05,112,84,197 |
| DeepSeek-R1-Distill-Qwen-1.5B,uniform_8bit,8,8,8,6779.05,112,84,197 |
| DeepSeek-R1-Distill-Qwen-1.5B,very_aggressive,4,4,8,6779.05,112,84,197 |
| Phi-3-mini-128k-instruct,aggressive,4,8,8,14576.26,64,64,129 |
| Phi-3-mini-128k-instruct,uniform_8bit,8,8,8,14576.26,64,64,129 |
| Phi-3-mini-128k-instruct,very_aggressive,4,4,8,14576.26,64,64,129 |
| Falcon-E-3B-Base,aggressive,4,8,8,512.51,0,0,1 |
| Falcon-E-3B-Base,uniform_8bit,8,8,8,512.51,0,0,1 |
| Falcon-E-3B-Base,very_aggressive,4,4,8,512.51,0,0,1 |
|
|
