jaeyong2/Chunk-Generator-Preview

Chunk-Generator

example

from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "jaeyong2/Chunk-Generator-Preview"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = """
You are an AI that receives a document as input and divides it into chunks for application to RAG.

You divide the document into chunks, and you must divide it according to the following conditions.
1. You must set the chunk index according to the position in the document.
2. When chunks are combined, they must contain the same information as the document.
3. When creating chunks, if there is information outside the chunk (e.g., pronouns like "it", "he", or "she"), fill in the necessary information from the document.
4. Therefore, there should be no unknown information within the chunk.

Output format: JSON
json
{
    "chunk1":"...",
    "chunk2":"...",
    ...
}
""".strip()

content ="""
On this day in 1784, at the Maryland State House in Annapolis, the Continental Congress ratifies the Treaty of Paris. The document, negotiated in part by future President John Adams, contained terms for ending the Revolutionary War and established the United States as a sovereign nation. The treaty outlined America's fishing rights off the coast of Canada, defined territorial boundaries in North America formerly held by the British and forced an end to reprisals against British loyalists. Two other future presidents, Thomas Jefferson and James Monroe, were among the delegates who ratified the document on January 14, 1874.
Thomas Jefferson had planned to travel to Paris to join Adams, John Jay and Benjamin Franklin for the beginning of talks with the British in 1782. However, after a delay in his travel plans, Jefferson received word that a cessation of hostilities had been announced by King George III the previous December. Jefferson arrived in Paris in late February after the treaty had already been negotiated by Adams, Franklin and Jay.
Adams' experience and skill in diplomacy prompted Congress to authorize him to act as the United States' representative in negotiating treaty terms with the British. Following his role in ending the Revolutionary War and his participation in drafting the Declaration of Independence, Adams succeeded George Washington as the second president of the United States in 1797.
""".strip()

system = {"role":"system", "content":prompt}
user = {"role":"user", "content":content}
messages = [system, user]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False # Switches between thinking and non-thinking modes. Default is True.
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

# conduct text completion
generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=32768
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 
content = tokenizer.decode(output_ids, skip_special_tokens=True).strip("\n")

result

content: {
    "chunk1": "On this day in 1784, at the Maryland State House in Annapolis, the Continental Congress ratifies the Treaty of Paris. The document, negotiated in part by future President John Adams, contained terms for ending the Revolutionary War and established the United States as a sovereign nation. The treaty outlined America's fishing rights off the coast of Canada, defined territorial boundaries in North America formerly held by the British and forced an end to reprisals against British loyalists. Two other future presidents, Thomas Jefferson and James Monroe, were among the delegates who ratified the document on January 14, 1874.",
    "chunk2": "Thomas Jefferson had planned to travel to Paris to join Adams, John Jay and Benjamin Franklin for the beginning of talks with the British in 1782. However, after a delay in his travel plans, Jefferson received word that a cessation of hostilities had been announced by King George III the previous December. Jefferson arrived in Paris in late February after the treaty had already been negotiated by Adams, Franklin and Jay.",
    "chunk3": "Adams' experience and skill in diplomacy prompted Congress to authorize him to act as the United States' representative in negotiating treaty terms with the British. Following his role in ending the Revolutionary War and his participation in drafting the Declaration of Independence, Adams succeeded George Washington as the second president of the United States in 1797."
}

License

• Qwen/Qwen3-0.6B : https://huggingface.co/Qwen/Qwen3-0.6B/blob/main/LICENSE

Acknowledgement

This research is supported by TPU Research Cloud program.