SeanScripts

/

pixtral-12b-nf4

like 20

The example fails on transformers-4.51.1:

Traceback (most recent call last):                                                                                                          
  File "./pixtral-4bit-test.py", line 24, in <module>                             
    generate_ids = model.generate(**inputs, max_new_tokens=512)
...
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same