Llama.cpp Now Supports Qwen2-VL (Vision Language Model)

Testing right now to get it running.

if i git clone the Qwen/Qwen2-VL-2B-Instruct repo in /whatever/Qwen/Qwen2-VL-2B-Instruct/ and make a gguf out of it with convert_hf_to_gguf.py everything is fine and i get a Qwen-Qwen2-VL-2B-Instruct-F16.gguf

But when i try to convert the vision encoder to GGUF format with qwen2_vl_surgery.py :
python examples/llava/qwen2_vl_surgery.py "/whatever/Qwen/Qwen2-VL-2B-Instruct/"

i can’t, python throws an error :

v.blk.0.ln1.weight

[to_gguf_name] merger.mlp.2.bias –> mm.2.bias
Traceback (most recent call last):

File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py”, line 406, in hf_raise_for_status
response.raise_for_status()
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/requests/models.py”, line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/Qwen2-VL-2B-Instruct/resolve/main/processor_config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/utils/hub.py”, line 403, in cached_file
resolved_file=hf_hub_download(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py”, line 114, in _inner_fn
return fn(*args, **kwargs)
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 862, in hf_hub_download
return _hf_hub_download_to_cache_dir(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 969, in _hf_hub_download_to_cache_dir
_raise_on_head_call_error(head_call_error, force_download, local_files_only)
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 1484, in _raise_on_head_call_error
raise head_call_error
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 1376, in _get_metadata_or_catch_error
metadata=get_hf_file_metadata(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py”, line 114, in _inner_fn
return fn(*args, **kwargs)
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 1296, in get_hf_file_metadata
r=_request_wrapper(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 277, in _request_wrapper
response=_request_wrapper(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py”, line 301, in _request_wrapper
hf_raise_for_status(response)
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py”, line 454, in hf_raise_for_status
raise _format(RepositoryNotFoundError, message, response) from e
huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-675de5d3-319c681e02ab26174d878b71;82815812-2512-468e-bba8-8beac819dd0c)

Repository Not Found for url: https://huggingface.co/Qwen2-VL-2B-Instruct/resolve/main/processor_config.json.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/whatever/llama.cpp/examples/llava/qwen2_vl_surgery.py”, line 158, in
main(args)
File “/whatever/llama.cpp/examples/llava/qwen2_vl_surgery.py”, line 142, in main
processor: Qwen2VLProcessor=AutoProcessor.from_pretrained(model_name)
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/models/auto/processing_auto.py”, line 254, in from_pretrained
processor_config_file=get_file_from_repo(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/utils/hub.py”, line 557, in get_file_from_repo
return cached_file(
File “/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/utils/hub.py”, line 426, in cached_file
raise EnvironmentError(
OSError: Qwen2-VL-2B-Instruct is not a local folder and is not a valid model identifier listed on ‘https://huggingface.co/models’
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=`”>

(venv) ali0une@Debian:~/compil/llama.cpp$ python examples/llava/qwen2_vl_surgery.py "/whatever/Qwen/Qwen2-VL-2B-Instruct"
model_name:  /whatever/Qwen/Qwen2-VL-2B-Instruct
`Qwen2VLRotaryEmbedding` can now be fully parameterized by passing the model config through the `config` argument. All other arguments will be removed in v4.46
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  2.17it/s]
Qwen2VLVisionConfig {
  "depth": 32,
  "embed_dim": 1280,
  "hidden_act": "quick_gelu",
  "hidden_size": 1536,
  "in_channels": 3,
  "in_chans": 3,
  "mlp_ratio": 4,
  "model_type": "qwen2_vl",
  "num_heads": 16,
  "patch_size": 14,
  "spatial_merge_size": 2,
  "spatial_patch_size": 14,
  "temporal_patch_size": 2,
  "transformers_version": "4.47.0"
}

[to_gguf_name] vision_model.blocks.0.norm1.weight --> v.blk.0.ln1.weight

...

[to_gguf_name] merger.mlp.2.bias --> mm.2.bias
Traceback (most recent call last):

  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
    response.raise_for_status()
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/Qwen2-VL-2B-Instruct/resolve/main/processor_config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/utils/hub.py", line 403, in cached_file
    resolved_file=hf_hub_download(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 862, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 969, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1484, in _raise_on_head_call_error
    raise head_call_error
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1376, in _get_metadata_or_catch_error
    metadata=get_hf_file_metadata(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1296, in get_hf_file_metadata
    r=_request_wrapper(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 277, in _request_wrapper
    response=_request_wrapper(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 301, in _request_wrapper
    hf_raise_for_status(response)
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 454, in hf_raise_for_status
    raise _format(RepositoryNotFoundError, message, response) from e
huggingface_hub.errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-675de5d3-319c681e02ab26174d878b71;82815812-2512-468e-bba8-8beac819dd0c)

Repository Not Found for url: https://huggingface.co/Qwen2-VL-2B-Instruct/resolve/main/processor_config.json.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/whatever/llama.cpp/examples/llava/qwen2_vl_surgery.py", line 158, in 
    main(args)
  File "/whatever/llama.cpp/examples/llava/qwen2_vl_surgery.py", line 142, in main
    processor: Qwen2VLProcessor=AutoProcessor.from_pretrained(model_name)
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/models/auto/processing_auto.py", line 254, in from_pretrained
    processor_config_file=get_file_from_repo(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/utils/hub.py", line 557, in get_file_from_repo
    return cached_file(
  File "/whatever/llama.cpp/venv/lib/python3.10/site-packages/transformers/utils/hub.py", line 426, in cached_file
    raise EnvironmentError(
OSError: Qwen2-VL-2B-Instruct is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=`

But it just works without specifying the path to the downloaded HF model repo :
(venv) ali0une@Debian:~/compil/llama.cpp$ python examples/llava/qwen2_vl_surgery.py

i get the /whatever/llama.cpp/qwen-qwen2-vl-2b-instruct-vision.gguf

Note: This article have been indexed to our site. We do not claim legitimacy, ownership or copyright of any of the content above. To see the article at original source Click Here

Related Posts
Samsung Galaxy A22 4G receives January security update thumbnail

Samsung Galaxy A22 4G receives January security update

Primeiro só foi implementado no mercado asiático, mas agora o último patch de segurança de Janeiro de 2022 está a ser lançado nos mercados europeus e outros. De acordo com o último lançamento, esta actualização do patch de segurança de Janeiro de 2022 está disponível para o Samsung Galaxy A22 4G! A empresa sul-coreana Samsung…
Read More
The 5GHz “Problem” for Wi-Fi Networks: DFS (2018) thumbnail

The 5GHz “Problem” for Wi-Fi Networks: DFS (2018)

Wi-Fi networking provides us with 2 bands for the operation of wireless LAN networks: the 2.4Ghz band and the 5GHz band. The 2.4GHz band has a reputation of being something of a “sewer” of a band, due to its limited number of usable channels, the number of Wi-Fi devices already using the band, and the
Read More
When WhatsApp crashed, it was Telegram again that made a profit! thumbnail

When WhatsApp crashed, it was Telegram again that made a profit!

Anlık mesajlaşma uygulaması Telegram, Pazartesi günü Facebook kesintisi sırasında 70 milyondan fazla yeni kullanıcı kazandığını duyurdu. Dünyanın dört bir yanındaki insanların yaklaşık altı saat boyunca önemli mesajlaşma servislerinden yoksun kaldığı günde kar eden yine Telegram oldu. WhatsApp ne zaman bir sorunla karşılaşsa işin kaymağını Telegram yiyor diyebiliriz. Yaşanan Facebook, Instagram ve WhatsApp çöküntüsünün ardından tekrar…
Read More
Китайский производитель литографического оборудования отрицает нарушение патентов ASML thumbnail

Китайский производитель литографического оборудования отрицает нарушение патентов ASML

12.02.2022 [10:41],  Павел Котов Ранее на этой неделе стало известно, что нидерландский производитель литографических сканеров ASML обвинил своего китайского конкурента Dongfang Jingyuan в нарушении патентов. Азиатский стартап опубликовал ответное заявление, в резких выражениях отвергнув все обвинения в незаконной деятельности и подчеркнув, что вся её работа базируется исключительно на добросовестных исследованиях. Компания также намекнула, что не…
Read More
Index Of News
Total
0
Share