Oct-25-2019, 04:39 AM
Sorry if this is a stupid question. I'm just starting out in NLP and am working with gpt-2 for text generation.
My situation is that I have to generate text in a particular field for eg. family businesses, which pretrained gpt-2 is unlikely to have much "training" with. Besides the topic, I also need to generate the text in the style of one particular writer (for eg. incorporating their turns of phrase etc) This particular writer hasn't written much about the family business topic unfortunately, but has written quite a lot on other topics .
It occurred to me that I can take gpt-2, finetune it on a large corpus of material on family businesses, and then finetune the new model on the written material of the particular writer. Or go the other way around - finetune it on the writer's corpus first, and then on family business material.
Would either of these be the right way to achieve my objective of creating content on family businesses in the style of this particular writer?
Any suggestions on what sort of stuff I should keep in mind while doing this?
Any help is much appreciated.
My situation is that I have to generate text in a particular field for eg. family businesses, which pretrained gpt-2 is unlikely to have much "training" with. Besides the topic, I also need to generate the text in the style of one particular writer (for eg. incorporating their turns of phrase etc) This particular writer hasn't written much about the family business topic unfortunately, but has written quite a lot on other topics .
It occurred to me that I can take gpt-2, finetune it on a large corpus of material on family businesses, and then finetune the new model on the written material of the particular writer. Or go the other way around - finetune it on the writer's corpus first, and then on family business material.
Would either of these be the right way to achieve my objective of creating content on family businesses in the style of this particular writer?
Any suggestions on what sort of stuff I should keep in mind while doing this?
Any help is much appreciated.