Fine tuning the LLM backbone
#76
by antogrk - opened
I'm working on a text-only task (which ultimately will be expanded to a multimodal task in the future). I was wondering if it's possible to fine-tune only the language model that's used as the backbone of the model on only textual data. Also, is it possible to apply LoRA on the LLM and train only the linear layers of it?
It would be very helpful if you could provide a very basic script of how the fine-tuning can be done for a causal language modeling task.
antogrk changed discussion status to closed
antogrk changed discussion status to open