Summarization is slow when use with pipeline
It takes a few minutes to summarize a paragraph with around 1000 tokens. Is it normal? Is there any way I could increase the speed? I tried to run it on GPU but the speed was not improved
Running it on GPU should already drastically improve the speed. Would you mind sharing your code here?
Running it on GPU should already drastically improve the speed. Would you mind sharing your code here?
Hi,
I now run it on Google Colab with high RAM T4 GPU in this way
summarizer_bart = pipeline("summarization", model="facebook/bart-large-cnn",device=0)
summarizer_bart(tg, max_length=256, min_length=64, do_sample=False, truncation=True)
where len(tg) is 3000
It still takes 4-5 minutes or even longer. Any way I can improve? Many thanks!
Yes for some reason it now takes way longer then it used to.