Spaces:
Running
on
Zero
Running
on
Zero
added research capibilities
Browse files
Documentation/Technical_Report.md
CHANGED
|
@@ -1,12 +1,23 @@
|
|
| 1 |
# Documentation for Gemma Chat Demo
|
| 2 |
|
| 3 |
-
## Model Research
|
|
|
|
|
|
|
| 4 |
|
| 5 |
### Model Overview
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
### Performance Analysis
|
| 8 |
|
| 9 |
-
###
|
| 10 |
|
| 11 |
## User Research
|
| 12 |
|
|
|
|
| 1 |
# Documentation for Gemma Chat Demo
|
| 2 |
|
| 3 |
+
## Model Choice and Research
|
| 4 |
+
|
| 5 |
+
Basing off of the Geema 3 technical report, I will analyze and compare different model sizes to determine the most suitable one for deployment in this project. For this portion, I will take the model's size, performance benchmarks, inference efficiency and memory requirements into consideration. The goal of this analysis is to strike a balance between computational cost and model quality.
|
| 6 |
|
| 7 |
### Model Overview
|
| 8 |
|
| 9 |
+
The Gemma 3 family consists of four model sizes, each with increasing capabilities and resource requirements:
|
| 10 |
+
|
| 11 |
+
| Model | Parameters | Vision Encoder | Total Size | Context Length | Key Capabilities |
|
| 12 |
+
|-------|------------|---------------|------------|----------------|------------------|
|
| 13 |
+
| Gemma 3-1B | 698M language + 302M embeddings | None | 1B | 32K tokens | Basic text generation; no vision capabilities, extrmely lightweight |
|
| 14 |
+
| Gemma 3-4B | 3.2B language + 675M embeddings + 417M vision | SigLIP | 4.3B | 128K tokens | Multimodal with good balance of performance and efficiency |
|
| 15 |
+
| Gemma 3-12B | 10.8B language + 1B embeddings + 417M vision | SigLIP | 12.2B | 128K tokens | Strong performance across all tasks with reasonable resource needs |
|
| 16 |
+
| Gemma 3-27B | 25.6B language + 1.4B embeddings + 417M vision | SigLIP | 27.4B | 128K tokens | Best performance; comparable to Gemini 1.5 Pro on benchmarks |
|
| 17 |
+
|
| 18 |
### Performance Analysis
|
| 19 |
|
| 20 |
+
### HuggingFace Spaces Resources
|
| 21 |
|
| 22 |
## User Research
|
| 23 |
|