Update README.md

README.md

@@ -219,7 +219,7 @@ We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-h
 | Benchmark                        |                        |                             |                                 |
 |----------------------------------|------------------------|-----------------------------|---------------------------------|
 |                                  | google/gemma-3-12b-it  | pytorch/gemma-3-12b-it-INT4 | pytorch/gemma-3-12b-it-AWQ-INT4 |
-| mmlu_abstract_algebra            | 43                     | 41                          | 42                              |
+| professional_law                 | TODO                   | 54.24                       | TODO                            |
 
 
 <details>
@@ -250,7 +250,7 @@ lm_eval --model hf --model_args pretrained=$MODEL --tasks mmlu --device cuda:0 -
 | Benchmark                        |                        |                             |                                 |
 |----------------------------------|------------------------|-----------------------------|---------------------------------|
 |                                  | google/gemma-3-12b-it  | pytorch/gemma-3-12b-it-INT4 | pytorch/gemma-3-12b-it-AWQ-INT4 |
-| Peak Memory (GB)                 | 24.50	                | 8.68 (65% reduction)        | TODO                            |
+| Peak Memory (GB)                 | 24.50	                | 8.57 (65% reduction)        | 12.71 (48% reduction)           |
 
 
 <details>
@@ -308,12 +308,14 @@ print(f"Peak Memory Usage: {mem:.02f} GB")
 ## Results (H100 machine)
 
 
-| Benchmark (Latency)              |                        |                             |                                 |
-|----------------------------------|------------------------|-----------------------------|---------------------------------|
-|                                  | google/gemma-3-12b-it  | pytorch/gemma-3-12b-it-INT4 | pytorch/gemma-3-12b-it-AWQ-INT4 |
-| latency (batch_size=1)           | 3.73s	                | TODO (TODO% reduction)      | TODO                            |
-| latency (batch_size=256)         | TODO	                | TODO (TODO% reduction)      | TODO                            |
+| Benchmark (Latency)              |                        |                                |                                 |
+|----------------------------------|------------------------|--------------------------------|---------------------------------|
+|                                  | google/gemma-3-12b-it  | jerryzh168/gemma-3-12b-it-INT4 | pytorch/gemma-3-12b-it-AWQ-INT4 |
+| latency (batch_size=1)           | 3.73s	                | 2.76 (1.35x speedup)           | 2.76s (1.35x speedup)           |
+| latency (batch_size=256)         | 13.63s	                | 14.32 (0.95x speedup)          | 14.30s (0.95x speedup)          |
+
 
+Note: jerryzh168/gemma-3-12b-it-INT4 is the H100 optimized checkpoint for INT4
 <details>
 <summary> Reproduce Model Performance Results </summary>