metadata
base_model:
- yamatazen/Twilight-SCE-12B-v2
- yamatazen/EtherealAurora-12B-v2
- DreadPoor/Irix-12B-Model_Stock
- yamatazen/LorablatedStock-12B
library_name: transformers
tags:
- mergekit
- merge
Anora-12b
☕ Support My Work
If you like my work, consider buying me a coffee to support future merges, GPU time, experiments.
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Model Stock merge method using DreadPoor/Irix-12B-Model_Stock as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
base_model: DreadPoor/Irix-12B-Model_Stock
dtype: bfloat16
merge_method: model_stock
modules:
default:
slices:
- sources:
- layer_range: [0, 40]
model: yamatazen/Twilight-SCE-12B-v2
- layer_range: [0, 40]
model: yamatazen/EtherealAurora-12B-v2
- layer_range: [0, 40]
model: yamatazen/LorablatedStock-12B
- layer_range: [0, 40]
model: DreadPoor/Irix-12B-Model_Stock