EXL3 quantization of rnj-1-instruct, 8 bits per weight, including output layers.

Model	Q4	Q6	Q8	FP16
rnj-1-instruct-exl3-4bpw	88.4	89.0	87.8	87.2
rnj-1-instruct-exl3-6bpw	87.2	86.6	86.6	86.0
rnj-1-instruct-exl3-8bpw-h8	86.6	87.2	86.6	86.6

Safetensors

Model size

5B params

Tensor type

F16

I16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for isogen/rnj-1-instruct-exl3-8bpw-h8

Base model

Finetuned

Quantized

(25)

this model