---
{}
---
# Model with No Refusal Tokens

This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on UltraChat SFT, CoCoNoT refusals, and CoCoNoT's contrast data as SFT data. Note that this model is not the model found in the paper the original models are not able to be released due to corporate legalities.

## Suggusted Method For Generations

For generating a output from this model, please refer to the code found in [repo](https://github.com/neelsjain/refusal-tokens) in the `coconot_eval` folder. However, for this model, `model.generate` or `pipeline` are also sufficient.

<!-- ## Training procedure -->

<!-- ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- total_eval_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.1129        | 0.9996 | 1890 | 0.9726          |


### Framework versions

- Transformers 4.44.0.dev0
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1 -->