How can I make Bloom stop generating when it should?
How can I make Bloom stop generating when it should?
I want bloom stop generating when end of the response, do not generate other things until max_new_token.
Excuse my poor English
There is a stop parameter. It will stop generating when any of the given tokens in the list are generated:
payload = {
"inputs": promt,
"parameters": {
"do_sample": True,
"top_p": X,
"max_new_tokens": 150,
"temperature": X,
"stop": ['.', 'The next day']
}
response = hfrequests.post("https://api-inference.huggingface.co/models/bigscience/bloom", headers=headers, json=payload)
There is a stop parameter. It will stop generating when any of the given tokens in the list are generated:
payload = {
"inputs": promt,
"parameters": {
"do_sample": True,
"top_p": X,
"max_new_tokens": 150,
"temperature": X,
"stop": ['.', 'The next day']
}
response = hfrequests.post("https://api-inference.huggingface.co/models/bigscience/bloom", headers=headers, json=payload)
I want to let bloom stop when finish the content, I give a () at begin and end of example, it will let bloom put ) when finish the content