from transformers import Qwen3VLForConditionalGeneration

model = Qwen3VLForConditionalGeneration.from_pretrained(
"datalab-to/chandra",
trust_remote_code=True,
torch_dtype=torch.float16,
device_map="auto"
).eval()

even I faced the same issue try using Qwen3VLForConditionalGeneration this instead of automodel that helped me

Hi @Rajneel18
If possible can you share sample inference script? This is what i used

    model.processor = AutoProcessor.from_pretrained("datalab-to/chandra")

    img_path = "/projects/data/teams/data_team/indic_ocr_eval/synth_data/hindi/images/level_1/Devanagari_Anek_Devanagari_5_n_pages_1_1.png"
    PIL_IMAGE = load_image(img_path)

    batch = [
    BatchInputItem(
        image=PIL_IMAGE,
        prompt_type="ocr_layout"
    )
    ]

    result = generate_hf(batch, model)[0]
    markdown = parse_markdown(result.raw)

I am getting cuda related error which could due to issue due tokeniser

torch.AcceleratorError: CUDA error: device-side assert triggered
Search for `cudaErrorAssert' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions

Hi @carbene101
This worked for me too:

model = Qwen3VLForConditionalGeneration.from_pretrained(
"datalab-to/chandra",
trust_remote_code=True,
dtype="auto",
device_map="auto"
).eval()

full code:

from transformers import AutoModel, AutoProcessor, Qwen3VLForConditionalGeneration
from chandra.model.hf import generate_hf
from chandra.model.schema import BatchInputItem
from chandra.output import parse_markdown

from PIL import Image
PIL_IMAGE = Image.open("path to your image").convert("RGB")

model = Qwen3VLForConditionalGeneration.from_pretrained(
"datalab-to/chandra",
trust_remote_code=True,
dtype="auto",
device_map="auto"
).eval()

model.processor = AutoProcessor.from_pretrained("datalab-to/chandra")

batch = [
BatchInputItem(
image=PIL_IMAGE,
prompt_type="ocr_layout"
)
]

result = generate_hf(batch, model)[0]
markdown = parse_markdown(result.raw)
print(markdown)

Hi Team and @carbene101 , thanks for releasing the model, but I am facing the issue as ValueError: Image features and image tokens do not match: tokens: 630, features 630 when running this script

from transformers import AutoModel, AutoProcessor, Qwen3VLForConditionalGeneration
from chandra.model.hf import generate_hf
from chandra.model.schema import BatchInputItem
from chandra.output import parse_markdown

from PIL import Image

image_file = '../../201_300_1_21.jpg'

PIL_IMAGE = Image.open(image_file).convert("RGB")

model = Qwen3VLForConditionalGeneration.from_pretrained(
"datalab-to/chandra",
trust_remote_code=True,
dtype="auto",
device_map="auto"
).eval()

model.processor = AutoProcessor.from_pretrained("datalab-to/chandra")

batch = [
BatchInputItem(
image=PIL_IMAGE,
prompt_type="ocr_layout"
)
]

result = generate_hf(batch, model)[0]
markdown = parse_markdown(result.raw)
print(markdown)

Using the Qwen3VL model loader works for me. Try this out! 😄

#working
import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
from chandra.model.hf import generate_hf
from chandra.model.schema import BatchInputItem
from chandra.output import parse_markdown
from PIL import Image

model_name = "datalab-to/chandra"

Load generation-capable model

model = Qwen3VLForConditionalGeneration.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained(model_name)
model.processor = processor

Prepare your input

PIL_IMAGE = Image.open("your_file_here.jpg").convert("RGB")
#PIL_IMAGE = PIL_IMAGE.resize((600,800))
batch = [
BatchInputItem(image=PIL_IMAGE, prompt_type="ocr_layout")
]

Generate

result = generate_hf(batch, model)[0]
markdown = parse_markdown(result.raw)
print(markdown)

Hi @sushruthb , thanks for the help but I am getting same error ValueError: Image features and image tokens do not match: tokens: 630, features 630.
Please share me the requirements file if possible for you.
Mine versions are
torch:2.9.0
transformers:4.57.1
chandra-ocr:0.1.8
pillow:12.0.0