Upload model
Browse filesThis view is limited to 50 files because it contains too many changes.
See raw diff
- .gitattributes +1 -0
- README.md +39 -0
- chat_template.jinja +3 -0
- config.json +69 -0
- configuration_deepseek.py +199 -0
- generation_config.json +9 -0
- model-00001-of-00139.safetensors +3 -0
- model-00002-of-00139.safetensors +3 -0
- model-00003-of-00139.safetensors +3 -0
- model-00004-of-00139.safetensors +3 -0
- model-00005-of-00139.safetensors +3 -0
- model-00006-of-00139.safetensors +3 -0
- model-00007-of-00139.safetensors +3 -0
- model-00008-of-00139.safetensors +3 -0
- model-00009-of-00139.safetensors +3 -0
- model-00010-of-00139.safetensors +3 -0
- model-00011-of-00139.safetensors +3 -0
- model-00012-of-00139.safetensors +3 -0
- model-00013-of-00139.safetensors +3 -0
- model-00014-of-00139.safetensors +3 -0
- model-00015-of-00139.safetensors +3 -0
- model-00016-of-00139.safetensors +3 -0
- model-00017-of-00139.safetensors +3 -0
- model-00018-of-00139.safetensors +3 -0
- model-00019-of-00139.safetensors +3 -0
- model-00020-of-00139.safetensors +3 -0
- model-00021-of-00139.safetensors +3 -0
- model-00022-of-00139.safetensors +3 -0
- model-00023-of-00139.safetensors +3 -0
- model-00024-of-00139.safetensors +3 -0
- model-00025-of-00139.safetensors +3 -0
- model-00026-of-00139.safetensors +3 -0
- model-00027-of-00139.safetensors +3 -0
- model-00028-of-00139.safetensors +3 -0
- model-00029-of-00139.safetensors +3 -0
- model-00030-of-00139.safetensors +3 -0
- model-00031-of-00139.safetensors +3 -0
- model-00032-of-00139.safetensors +3 -0
- model-00033-of-00139.safetensors +3 -0
- model-00034-of-00139.safetensors +3 -0
- model-00035-of-00139.safetensors +3 -0
- model-00036-of-00139.safetensors +3 -0
- model-00037-of-00139.safetensors +3 -0
- model-00038-of-00139.safetensors +3 -0
- model-00039-of-00139.safetensors +3 -0
- model-00040-of-00139.safetensors +3 -0
- model-00041-of-00139.safetensors +3 -0
- model-00042-of-00139.safetensors +3 -0
- model-00043-of-00139.safetensors +3 -0
- model-00044-of-00139.safetensors +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model:
|
| 4 |
+
- deepseek-ai/DeepSeek-V3.1-Terminus
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
## Model Details
|
| 8 |
+
|
| 9 |
+
This model is a mixed int8 model with group_size 128 and symmetric quantization of deepseek-ai/DeepSeek-V3.1-Terminus generated by intel/auto-round via RTN(no algorithm tuning). Please refer to Section Generate the model for more details. Please follow the license of the original model.
|
| 10 |
+
|
| 11 |
+
## Generate the Model
|
| 12 |
+
|
| 13 |
+
```bash
|
| 14 |
+
auto-round --model_name deepseek-ai/DeepSeek-V3.1-Terminus --iters 0 --bits 8
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
## Ethical Considerations and Limitations
|
| 18 |
+
|
| 19 |
+
The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
|
| 20 |
+
|
| 21 |
+
Therefore, before deploying any applications of the model, developers should perform safety testing.
|
| 22 |
+
|
| 23 |
+
## Caveats and Recommendations
|
| 24 |
+
|
| 25 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|
| 26 |
+
|
| 27 |
+
Here are a couple of useful links to learn more about Intel's AI software:
|
| 28 |
+
|
| 29 |
+
- [Intel Neural Compressor](https://github.com/intel/neural-compressor)
|
| 30 |
+
|
| 31 |
+
## Disclaimer
|
| 32 |
+
|
| 33 |
+
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.
|
| 34 |
+
|
| 35 |
+
## Cite
|
| 36 |
+
|
| 37 |
+
@article{cheng2023optimize, title={Optimize weight rounding via signed gradient descent for the quantization of llms}, author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi}, journal={arXiv preprint arXiv:2309.05516}, year={2023} }
|
| 38 |
+
|
| 39 |
+
[arxiv](https://arxiv.org/abs/2309.05516) [github](https://github.com/intel/auto-round)
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% if not thinking is defined %}{% set thinking = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, system_prompt='', is_first_sp=true, is_last_user=false) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '
|
| 2 |
+
|
| 3 |
+
' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{ bos_token }}{{ ns.system_prompt }}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{%- set ns.is_first = false -%}{%- set ns.is_last_user = true -%}{{'<|User|>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['tool_calls'] is defined and message['tool_calls'] is not none %}{%- if ns.is_last_user %}{{'<|Assistant|></think>'}}{%- endif %}{%- set ns.is_last_user = false -%}{%- set ns.is_first = false %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls'] %}{%- if not ns.is_first %}{%- if message['content'] is none %}{{'<|tool▁calls▁begin|><|tool▁call▁begin|>'+ tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments'] + '<|tool▁call▁end|>'}}{%- else %}{{message['content'] + '<|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments'] + '<|tool▁call▁end|>'}}{%- endif %}{%- set ns.is_first = true -%}{%- else %}{{'<|tool▁call▁begin|>'+ tool['function']['name'] + '<|tool▁sep|>' + tool['function']['arguments'] + '<|tool▁call▁end|>'}}{%- endif %}{%- endfor %}{{'<|tool▁calls▁end|><|end▁of▁sentence|>'}}{%- endif %}{%- if message['role'] == 'assistant' and (message['tool_calls'] is not defined or message['tool_calls'] is none) %}{%- if ns.is_last_user %}{{'<|Assistant|>'}}{%- if message['prefix'] is defined and message['prefix'] and thinking %}{{'<think>'}} {%- else %}{{'</think>'}}{%- endif %}{%- endif %}{%- set ns.is_last_user = false -%}{%- if ns.is_tool %}{{message['content'] + '<|end▁of▁sentence|>'}}{%- set ns.is_tool = false -%}{%- else %}{%- set content = message['content'] -%}{%- if '</think>' in content %}{%- set content = content.split('</think>', 1)[1] -%}{%- endif %}{{content + '<|end▁of▁sentence|>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_last_user = false -%}{%- set ns.is_tool = true -%}{{'<|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}{%- endif %}{%- endfor -%}{%- if add_generation_prompt and ns.is_last_user and not ns.is_tool %}{{'<|Assistant|>'}}{%- if not thinking %}{{'</think>'}}{%- else %}{{'<think>'}}{%- endif %}{% endif %}
|
config.json
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"DeepseekV3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"attn_module_list_cfg": [],
|
| 8 |
+
"auto_map": {
|
| 9 |
+
"AutoConfig": "configuration_deepseek.DeepseekV3Config",
|
| 10 |
+
"AutoModel": "modeling_deepseek.DeepseekV3Model",
|
| 11 |
+
"AutoModelForCausalLM": "modeling_deepseek.DeepseekV3ForCausalLM"
|
| 12 |
+
},
|
| 13 |
+
"bos_token_id": 0,
|
| 14 |
+
"dtype": "bfloat16",
|
| 15 |
+
"eos_token_id": 1,
|
| 16 |
+
"ep_size": 1,
|
| 17 |
+
"first_k_dense_replace": 3,
|
| 18 |
+
"hidden_act": "silu",
|
| 19 |
+
"hidden_size": 7168,
|
| 20 |
+
"initializer_range": 0.02,
|
| 21 |
+
"intermediate_size": 18432,
|
| 22 |
+
"kv_lora_rank": 512,
|
| 23 |
+
"max_position_embeddings": 163840,
|
| 24 |
+
"model_type": "deepseek_v3",
|
| 25 |
+
"moe_intermediate_size": 2048,
|
| 26 |
+
"moe_layer_freq": 1,
|
| 27 |
+
"n_group": 8,
|
| 28 |
+
"n_routed_experts": 256,
|
| 29 |
+
"n_shared_experts": 1,
|
| 30 |
+
"norm_topk_prob": true,
|
| 31 |
+
"num_attention_heads": 128,
|
| 32 |
+
"num_experts_per_tok": 8,
|
| 33 |
+
"num_hidden_layers": 61,
|
| 34 |
+
"num_key_value_heads": 128,
|
| 35 |
+
"num_nextn_predict_layers": 1,
|
| 36 |
+
"q_lora_rank": 1536,
|
| 37 |
+
"qk_nope_head_dim": 128,
|
| 38 |
+
"qk_rope_head_dim": 64,
|
| 39 |
+
"quantization_config": {
|
| 40 |
+
"autoround_version": "0.8.0.dev",
|
| 41 |
+
"bits": 8,
|
| 42 |
+
"data_type": "int",
|
| 43 |
+
"group_size": 128,
|
| 44 |
+
"iters": 0,
|
| 45 |
+
"packing_format": "auto_round:auto_gptq",
|
| 46 |
+
"quant_method": "auto-round",
|
| 47 |
+
"sym": true
|
| 48 |
+
},
|
| 49 |
+
"rms_norm_eps": 1e-06,
|
| 50 |
+
"rope_scaling": {
|
| 51 |
+
"beta_fast": 32,
|
| 52 |
+
"beta_slow": 1,
|
| 53 |
+
"factor": 40,
|
| 54 |
+
"mscale": 1.0,
|
| 55 |
+
"mscale_all_dim": 1.0,
|
| 56 |
+
"original_max_position_embeddings": 4096,
|
| 57 |
+
"type": "yarn"
|
| 58 |
+
},
|
| 59 |
+
"rope_theta": 10000,
|
| 60 |
+
"routed_scaling_factor": 2.5,
|
| 61 |
+
"scoring_func": "sigmoid",
|
| 62 |
+
"tie_word_embeddings": false,
|
| 63 |
+
"topk_group": 4,
|
| 64 |
+
"topk_method": "noaux_tc",
|
| 65 |
+
"transformers_version": "4.56.2",
|
| 66 |
+
"use_cache": true,
|
| 67 |
+
"v_head_dim": 128,
|
| 68 |
+
"vocab_size": 129280
|
| 69 |
+
}
|
configuration_deepseek.py
ADDED
|
@@ -0,0 +1,199 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from transformers.configuration_utils import PretrainedConfig
|
| 2 |
+
from transformers.utils import logging
|
| 3 |
+
|
| 4 |
+
logger = logging.get_logger(__name__)
|
| 5 |
+
|
| 6 |
+
DEEPSEEK_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
|
| 7 |
+
class DeepseekV3Config(PretrainedConfig):
|
| 8 |
+
r"""
|
| 9 |
+
This is the configuration class to store the configuration of a [`DeepseekV3Model`]. It is used to instantiate an DeepSeek
|
| 10 |
+
model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
|
| 11 |
+
defaults will yield a similar configuration to that of the DeepSeek-V3.
|
| 12 |
+
|
| 13 |
+
Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
|
| 14 |
+
documentation from [`PretrainedConfig`] for more information.
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
Args:
|
| 18 |
+
vocab_size (`int`, *optional*, defaults to 129280):
|
| 19 |
+
Vocabulary size of the Deep model. Defines the number of different tokens that can be represented by the
|
| 20 |
+
`inputs_ids` passed when calling [`DeepseekV3Model`]
|
| 21 |
+
hidden_size (`int`, *optional*, defaults to 4096):
|
| 22 |
+
Dimension of the hidden representations.
|
| 23 |
+
intermediate_size (`int`, *optional*, defaults to 11008):
|
| 24 |
+
Dimension of the MLP representations.
|
| 25 |
+
moe_intermediate_size (`int`, *optional*, defaults to 1407):
|
| 26 |
+
Dimension of the MoE representations.
|
| 27 |
+
num_hidden_layers (`int`, *optional*, defaults to 32):
|
| 28 |
+
Number of hidden layers in the Transformer decoder.
|
| 29 |
+
num_nextn_predict_layers (`int`, *optional*, defaults to 1):
|
| 30 |
+
Number of nextn predict layers in the DeepSeekV3 Model.
|
| 31 |
+
num_attention_heads (`int`, *optional*, defaults to 32):
|
| 32 |
+
Number of attention heads for each attention layer in the Transformer decoder.
|
| 33 |
+
n_shared_experts (`int`, *optional*, defaults to None):
|
| 34 |
+
Number of shared experts, None means dense model.
|
| 35 |
+
n_routed_experts (`int`, *optional*, defaults to None):
|
| 36 |
+
Number of routed experts, None means dense model.
|
| 37 |
+
routed_scaling_factor (`float`, *optional*, defaults to 1.0):
|
| 38 |
+
Scaling factor or routed experts.
|
| 39 |
+
topk_method (`str`, *optional*, defaults to `gready`):
|
| 40 |
+
Topk method used in routed gate.
|
| 41 |
+
n_group (`int`, *optional*, defaults to None):
|
| 42 |
+
Number of groups for routed experts.
|
| 43 |
+
topk_group (`int`, *optional*, defaults to None):
|
| 44 |
+
Number of selected groups for each token(for each token, ensuring the selected experts is only within `topk_group` groups).
|
| 45 |
+
num_experts_per_tok (`int`, *optional*, defaults to None):
|
| 46 |
+
Number of selected experts, None means dense model.
|
| 47 |
+
moe_layer_freq (`int`, *optional*, defaults to 1):
|
| 48 |
+
The frequency of the MoE layer: one expert layer for every `moe_layer_freq - 1` dense layers.
|
| 49 |
+
first_k_dense_replace (`int`, *optional*, defaults to 0):
|
| 50 |
+
Number of dense layers in shallow layers(embed->dense->dense->...->dense->moe->moe...->lm_head).
|
| 51 |
+
\--k dense layers--/
|
| 52 |
+
norm_topk_prob (`bool`, *optional*, defaults to False):
|
| 53 |
+
Whether to normalize the weights of the routed experts.
|
| 54 |
+
scoring_func (`str`, *optional*, defaults to 'softmax'):
|
| 55 |
+
Method of computing expert weights.
|
| 56 |
+
aux_loss_alpha (`float`, *optional*, defaults to 0.001):
|
| 57 |
+
Auxiliary loss weight coefficient.
|
| 58 |
+
seq_aux = (`bool`, *optional*, defaults to True):
|
| 59 |
+
Whether to compute the auxiliary loss for each individual sample.
|
| 60 |
+
num_key_value_heads (`int`, *optional*):
|
| 61 |
+
This is the number of key_value heads that should be used to implement Grouped Query Attention. If
|
| 62 |
+
`num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if
|
| 63 |
+
`num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When
|
| 64 |
+
converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed
|
| 65 |
+
by meanpooling all the original heads within that group. For more details checkout [this
|
| 66 |
+
paper](https://arxiv.org/pdf/2305.13245.pdf). If it is not specified, will default to
|
| 67 |
+
`num_attention_heads`.
|
| 68 |
+
hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
|
| 69 |
+
The non-linear activation function (function or string) in the decoder.
|
| 70 |
+
max_position_embeddings (`int`, *optional*, defaults to 2048):
|
| 71 |
+
The maximum sequence length that this model might ever be used with.
|
| 72 |
+
initializer_range (`float`, *optional*, defaults to 0.02):
|
| 73 |
+
The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
|
| 74 |
+
rms_norm_eps (`float`, *optional*, defaults to 1e-06):
|
| 75 |
+
The epsilon used by the rms normalization layers.
|
| 76 |
+
use_cache (`bool`, *optional*, defaults to `True`):
|
| 77 |
+
Whether or not the model should return the last key/values attentions (not used by all models). Only
|
| 78 |
+
relevant if `config.is_decoder=True`.
|
| 79 |
+
pad_token_id (`int`, *optional*):
|
| 80 |
+
Padding token id.
|
| 81 |
+
bos_token_id (`int`, *optional*, defaults to 1):
|
| 82 |
+
Beginning of stream token id.
|
| 83 |
+
eos_token_id (`int`, *optional*, defaults to 2):
|
| 84 |
+
End of stream token id.
|
| 85 |
+
tie_word_embeddings (`bool`, *optional*, defaults to `False`):
|
| 86 |
+
Whether to tie weight embeddings
|
| 87 |
+
rope_theta (`float`, *optional*, defaults to 10000.0):
|
| 88 |
+
The base period of the RoPE embeddings.
|
| 89 |
+
rope_scaling (`Dict`, *optional*):
|
| 90 |
+
Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports two scaling
|
| 91 |
+
strategies: linear and dynamic. Their scaling factor must be a float greater than 1. The expected format is
|
| 92 |
+
`{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
|
| 93 |
+
`max_position_embeddings` to the expected new maximum.
|
| 94 |
+
attention_bias (`bool`, defaults to `False`, *optional*, defaults to `False`):
|
| 95 |
+
Whether to use a bias in the query, key, value and output projection layers during self-attention.
|
| 96 |
+
attention_dropout (`float`, *optional*, defaults to 0.0):
|
| 97 |
+
The dropout ratio for the attention probabilities.
|
| 98 |
+
|
| 99 |
+
```python
|
| 100 |
+
>>> from transformers import DeepseekV3Model, DeepseekV3Config
|
| 101 |
+
|
| 102 |
+
>>> # Initializing a Deepseek-V3 style configuration
|
| 103 |
+
>>> configuration = DeepseekV3Config()
|
| 104 |
+
|
| 105 |
+
>>> # Accessing the model configuration
|
| 106 |
+
>>> configuration = model.config
|
| 107 |
+
```"""
|
| 108 |
+
|
| 109 |
+
model_type = "deepseek_v3"
|
| 110 |
+
keys_to_ignore_at_inference = ["past_key_values"]
|
| 111 |
+
|
| 112 |
+
def __init__(
|
| 113 |
+
self,
|
| 114 |
+
vocab_size=129280,
|
| 115 |
+
hidden_size=7168,
|
| 116 |
+
intermediate_size=18432,
|
| 117 |
+
moe_intermediate_size = 2048,
|
| 118 |
+
num_hidden_layers=61,
|
| 119 |
+
num_nextn_predict_layers=1,
|
| 120 |
+
num_attention_heads=128,
|
| 121 |
+
num_key_value_heads=128,
|
| 122 |
+
n_shared_experts = 1,
|
| 123 |
+
n_routed_experts = 256,
|
| 124 |
+
ep_size = 1,
|
| 125 |
+
routed_scaling_factor = 2.5,
|
| 126 |
+
kv_lora_rank = 512,
|
| 127 |
+
q_lora_rank = 1536,
|
| 128 |
+
qk_rope_head_dim = 64,
|
| 129 |
+
v_head_dim = 128,
|
| 130 |
+
qk_nope_head_dim = 128,
|
| 131 |
+
topk_method = 'noaux_tc',
|
| 132 |
+
n_group = 8,
|
| 133 |
+
topk_group = 4,
|
| 134 |
+
num_experts_per_tok = 8,
|
| 135 |
+
moe_layer_freq = 1,
|
| 136 |
+
first_k_dense_replace = 3,
|
| 137 |
+
norm_topk_prob = True,
|
| 138 |
+
scoring_func = 'sigmoid',
|
| 139 |
+
hidden_act="silu",
|
| 140 |
+
max_position_embeddings=4096,
|
| 141 |
+
initializer_range=0.02,
|
| 142 |
+
rms_norm_eps=1e-6,
|
| 143 |
+
use_cache=True,
|
| 144 |
+
pad_token_id=None,
|
| 145 |
+
bos_token_id=0,
|
| 146 |
+
eos_token_id=1,
|
| 147 |
+
tie_word_embeddings=False,
|
| 148 |
+
rope_theta=10000.0,
|
| 149 |
+
rope_scaling=None,
|
| 150 |
+
attention_bias=False,
|
| 151 |
+
attention_dropout=0.0,
|
| 152 |
+
**kwargs,
|
| 153 |
+
):
|
| 154 |
+
self.vocab_size = vocab_size
|
| 155 |
+
self.max_position_embeddings = max_position_embeddings
|
| 156 |
+
self.hidden_size = hidden_size
|
| 157 |
+
self.intermediate_size = intermediate_size
|
| 158 |
+
self.moe_intermediate_size = moe_intermediate_size
|
| 159 |
+
self.num_hidden_layers = num_hidden_layers
|
| 160 |
+
self.num_nextn_predict_layers = num_nextn_predict_layers
|
| 161 |
+
self.num_attention_heads = num_attention_heads
|
| 162 |
+
self.n_shared_experts = n_shared_experts
|
| 163 |
+
self.n_routed_experts = n_routed_experts
|
| 164 |
+
self.ep_size = ep_size
|
| 165 |
+
self.routed_scaling_factor = routed_scaling_factor
|
| 166 |
+
self.kv_lora_rank = kv_lora_rank
|
| 167 |
+
self.q_lora_rank = q_lora_rank
|
| 168 |
+
self.qk_rope_head_dim = qk_rope_head_dim
|
| 169 |
+
self.v_head_dim = v_head_dim
|
| 170 |
+
self.qk_nope_head_dim = qk_nope_head_dim
|
| 171 |
+
self.topk_method = topk_method
|
| 172 |
+
self.n_group = n_group
|
| 173 |
+
self.topk_group = topk_group
|
| 174 |
+
self.num_experts_per_tok = num_experts_per_tok
|
| 175 |
+
self.moe_layer_freq = moe_layer_freq
|
| 176 |
+
self.first_k_dense_replace = first_k_dense_replace
|
| 177 |
+
self.norm_topk_prob = norm_topk_prob
|
| 178 |
+
self.scoring_func = scoring_func
|
| 179 |
+
# for backward compatibility
|
| 180 |
+
if num_key_value_heads is None:
|
| 181 |
+
num_key_value_heads = num_attention_heads
|
| 182 |
+
|
| 183 |
+
self.num_key_value_heads = num_key_value_heads
|
| 184 |
+
self.hidden_act = hidden_act
|
| 185 |
+
self.initializer_range = initializer_range
|
| 186 |
+
self.rms_norm_eps = rms_norm_eps
|
| 187 |
+
self.use_cache = use_cache
|
| 188 |
+
self.rope_theta = rope_theta
|
| 189 |
+
self.rope_scaling = rope_scaling
|
| 190 |
+
self.attention_bias = attention_bias
|
| 191 |
+
self.attention_dropout = attention_dropout
|
| 192 |
+
|
| 193 |
+
super().__init__(
|
| 194 |
+
pad_token_id=pad_token_id,
|
| 195 |
+
bos_token_id=bos_token_id,
|
| 196 |
+
eos_token_id=eos_token_id,
|
| 197 |
+
tie_word_embeddings=tie_word_embeddings,
|
| 198 |
+
**kwargs,
|
| 199 |
+
)
|
generation_config.json
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"bos_token_id": 0,
|
| 4 |
+
"do_sample": true,
|
| 5 |
+
"eos_token_id": 1,
|
| 6 |
+
"temperature": 0.6,
|
| 7 |
+
"top_p": 0.95,
|
| 8 |
+
"transformers_version": "4.56.2"
|
| 9 |
+
}
|
model-00001-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:02da1cad8fb36d6f663b20ab8b6c3e1b0e2670e2ae1d9e5d16e09fc57b5f15ed
|
| 3 |
+
size 4993274912
|
model-00002-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:89b957b17257c9bd2fbd15e8655afc7c8309eeb00f7ef434825642e698c5763d
|
| 3 |
+
size 4988132080
|
model-00003-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4e914297ccfe075db11b00902f3dfee925a0c52d42d051f126f8034a2632633f
|
| 3 |
+
size 4988132744
|
model-00004-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cf2d9b42a376639d127dcfb99b3cca1424d54083733ce14cd6047686a76e7982
|
| 3 |
+
size 4988009688
|
model-00005-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8b1043a4d7097bb5642681bafe863f7407789a5f7937c7bc6be28ca9a6798b5e
|
| 3 |
+
size 4988132712
|
model-00006-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ade5192820fc93309f7999a8956218ba847a2b10b50bbc04f58afaa6f4aedba7
|
| 3 |
+
size 4988010048
|
model-00007-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8982d1b0d247efd40f3964a2a911b0ce148392f7641085a9e01b45704352ceb7
|
| 3 |
+
size 4988132352
|
model-00008-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e74d361f79c62cdab1ecf6982d68a188c8508f3be73ed76ea7810a9716541687
|
| 3 |
+
size 4988010408
|
model-00009-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3aa48000fb47bcb618f5405936d7af0dbde410fd226265edaed3cff6adf5577a
|
| 3 |
+
size 4988131992
|
model-00010-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5a2119cbadc09f7350554c9a7b109cc3b1c52b28532e62a5a9b15f1b110ec147
|
| 3 |
+
size 4988132744
|
model-00011-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9fc8e3f331d07242400c9f0a268dbe0e8a02a06be1257e12b0f1f93687c978c1
|
| 3 |
+
size 4988009776
|
model-00012-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7bc5eb9a155daccb8141309a362060c916340f0e4636dc7d4c0acf115942c193
|
| 3 |
+
size 4988132624
|
model-00013-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b0d5eab2358d3219c45e997d976dee6520b4555918b7cc31a4e8b256f61d2b13
|
| 3 |
+
size 4988010136
|
model-00014-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ba0612afe9c14aca40bfeb69a79f7f469672d2111300055454870f784246fcfd
|
| 3 |
+
size 4988132264
|
model-00015-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:505b87753109197fbbb6365f7e68b237af4372e9a12ea883327b028249c0d0df
|
| 3 |
+
size 4988010520
|
model-00016-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5ce7df2b6e981fdda5e666d17d497a6c0b0f1c628ec1da77cc099160b893630e
|
| 3 |
+
size 4988131880
|
model-00017-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5e2159de4bd60879fcdda9715451f81a1b9689053ecfa305ae38c87beb75a31a
|
| 3 |
+
size 4988132752
|
model-00018-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1780e14bf288b3db0a9120178688f28f3e338afc5313ed4d483774819b24b51a
|
| 3 |
+
size 4988010568
|
model-00019-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:29523ae75f51c287da6721c169cbc913fc0beb8b46b63bdc8863a812708b8e01
|
| 3 |
+
size 4988133544
|
model-00020-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6bf35328fa19d8aaa358e370329f35435a529fa9489c85cdf35de49d69b90ca6
|
| 3 |
+
size 4988011192
|
model-00021-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:84019346977303f158325a6197a2391d524d6f77301bcbf62731fdd32f9dad90
|
| 3 |
+
size 4988133184
|
model-00022-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:341ece3986cb523fa9648752a6265b60a38051d0e5387a0959135cb2a1699013
|
| 3 |
+
size 4972989744
|
model-00023-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:29e2f42cb595dcc2ac6607258149e9d7f85139aff58f3fcc5ea5daf0838772c4
|
| 3 |
+
size 4988130128
|
model-00024-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:49450f342667776c3fdbf6ab6f189ab8bc413b0eff833da0a6b74c4a7768e5d5
|
| 3 |
+
size 4988133744
|
model-00025-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b11f7725320b4a0f4b0da48baf1277d5a4330db7501f8b596e4dcf0ca699e4f5
|
| 3 |
+
size 4988010920
|
model-00026-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:38239858239757f48dd36489d04ebcc788bcebb93b605037216e9fd291bb3c62
|
| 3 |
+
size 4988133456
|
model-00027-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9a4a5e5376d7eb7c4c6500e766599e81136d5f8a513ad69ef23c2db0d5dd77bb
|
| 3 |
+
size 4988011280
|
model-00028-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8dcf68c62194797b6a136521cf4e943f6daa717cc67a79a5537467a860f1f4a8
|
| 3 |
+
size 4988133096
|
model-00029-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac8af64f04b7739be438dbcea0ad43c5b74d5545ce364ac4123b5bbfc3d2e524
|
| 3 |
+
size 4988133744
|
model-00030-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5716aa08cdbdf600e2aa090d1ffd81189d1b54e54c9f86ecae3cee556715ea6d
|
| 3 |
+
size 4988010640
|
model-00031-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8436d2983ceed0201d03365604aa0e2b3eb414f3b8c72432c4980049dd3954c9
|
| 3 |
+
size 4988133736
|
model-00032-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4c20b70d4b7c0d27730be4119c5eccc19dfd707ce4c262af3a24d11edd458ead
|
| 3 |
+
size 4988011000
|
model-00033-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:817707f438edc4cbea8a56a9f0f4e8b378794f18933498352d5dda0c640dcb0c
|
| 3 |
+
size 4988133376
|
model-00034-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aaf13ab931438a52ff881570cd2c378ca4088ef0d166e59f9b05fc5f0521b3a5
|
| 3 |
+
size 4988011360
|
model-00035-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4947f251a5ada44941dabdd520a4016dbb563f06e8704b79ace98b1ad2356058
|
| 3 |
+
size 4988133016
|
model-00036-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ef3ef47fdbaea5412a0b39bdbe68d9154c1dc4cf3624f8dd3cfc20f2a1ceedc
|
| 3 |
+
size 4988133744
|
model-00037-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac3d8718a229864baebb6e0abfc1ba1939cf9e804220b3ef72db8c524698b40a
|
| 3 |
+
size 4988010728
|
model-00038-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ffbe8e39bee19d9978a47100c1822de9b7c5426190f3a8164e64340aa4622dda
|
| 3 |
+
size 4988133648
|
model-00039-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fad347cff1247c1420f85b7d9705cf3050b2dcb2e2484ca0c8d9c8b4034a088b
|
| 3 |
+
size 4988011088
|
model-00040-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c72b52327092f716ee657771a35afc04e05ea97001f49b9c552a813f02eef736
|
| 3 |
+
size 4988133288
|
model-00041-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1eae13a637f2f2e148e646d1e37b98d8cb76f1ee5abc3d37d0811ad81fdcbafc
|
| 3 |
+
size 4988011456
|
model-00042-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7fcef061b248966c23012ec880a453657bcad80ae77340701a49a1f749381bc4
|
| 3 |
+
size 4988132920
|
model-00043-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:49a70bab8afb14873e3d573561645076ffe99dac5df1c7390e9598fc7ed6769e
|
| 3 |
+
size 4988133744
|
model-00044-of-00139.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fb1c2f52769572afbb8744a8ec055d8c69c1b473d87b609a1f39e8e38596aab5
|
| 3 |
+
size 4988010816
|