DocOwl2 using flash attention mandatorily

by nicozck - opened Dec 9, 2024

Dec 9, 2024

DocOwl2 cannot be loaded without flash_attn because the implementation of the compressor mandatorily uses flash attention.

This issue causes DocOwl2 to not run on many non-NVIDIA devices. Please consider adding an option to disable or enable flash attention.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment