ValueError: Model type llama4 not supported.
With a new mlx_vlm (0.1.21) I'm getting "ValueError: Model type llama4 not supported.".
python -m mlx_vlm.generate --model mlx-community/meta-llama-Llama-4-Scout-17B-16E-4bit --max-tokens 100 --temperature 0.0 --prompt "Please read off the read noise for a gain of 50" --image ~/Desktop/IMG_1.jpg
Fetching 17 files: 100%|██████████████████████████████████████████████████████████████| 17/17 [00:00<00:00, 12716.81it/s]
ERROR:root:Model type llama4 not supported.
Traceback (most recent call last):
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/utils.py", line 64, in get_model_and_args
arch = importlib.import_module(f"mlx_vlm.models.{model_type}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mlx_vlm.models.llama4'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/generate.py", line 156, in
main()
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/generate.py", line 97, in main
model, processor, config = get_model_and_processors(args.model, args.adapter_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/generate.py", line 86, in get_model_and_processors
model, processor = load(
^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/utils.py", line 272, in load
model = load_model(model_path, lazy, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/utils.py", line 152, in load_model
model_class, model_type = get_model_and_args(config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_vlm/utils.py", line 68, in get_model_and_args
raise ValueError(msg)
I get the same with mlx_lm too...
[ERROR:root:Model type llama4 not supported.
Traceback (most recent call last):
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 148, in _get_classes
arch = importlib.import_module(f"mlx_lm.models.{model_type}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mlx_lm.models.llama4'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/jdavies/PycharmProjects/pythonProject/llama4_mlx.py", line 3, in
model, tokenizer = load("mlx-community/meta-llama-Llama-4-Scout-17B-16E-4bit")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 785, in load
model, config = load_model(model_path, lazy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 720, in load_model
model_class, model_args_class = get_model_classes(config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 152, in _get_classes
raise ValueError(msg)
ValueError: Model type llama4 not supported.](ERROR:root:Model type llama4 not supported.
Traceback (most recent call last):
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 71, in _get_classes
arch = importlib.import_module(f"mlx_lm.models.{model_type}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mlx_lm.models.llama4'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/jdavies/PycharmProjects/pythonProject/llama4_mlx.py", line 3, in
model, tokenizer = load("mlx-community/meta-llama-Llama-4-Scout-17B-16E-4bit")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 256, in load
model, config = load_model(model_path, lazy)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 191, in load_model
model_class, model_args_class = get_model_classes(config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/miniconda3/lib/python3.12/site-packages/mlx_lm/utils.py", line 75, in _get_classes
raise ValueError(msg)
ValueError: Model type llama4 not supported.)
You need to use the llama4 branch for now
Can I use two RTX 3090s and two RTX 3080s together for a total of 72GB VRAM?