--- title: modeling.summarization keywords: fastai sidebar: home_sidebar summary: "This module contains custom models, loss functions, custom splitters, etc... summarization tasks." description: "This module contains custom models, loss functions, custom splitters, etc... summarization tasks." nb_path: "nbs/02e_modeling-summarization.ipynb" ---
torch.cuda.set_device(1)
print(f'Using GPU #{torch.cuda.current_device()}: {torch.cuda.get_device_name()}')
path = Path('./')
cnndm_df = pd.read_csv(path/'cnndm_sample.csv'); len(cnndm_df)
cnndm_df.head(2)
pretrained_model_name = "facebook/bart-large-cnn"
hf_arch, hf_config, hf_tokenizer, hf_model = BLURR_MODEL_HELPER.get_hf_objects(pretrained_model_name,
model_cls=BartForConditionalGeneration)
hf_arch, type(hf_config), type(hf_tokenizer), type(hf_model)
hf_batch_tfm = HF_SummarizationBatchTransform(hf_arch, hf_tokenizer, max_length=[256, 130])
blocks = (HF_TextBlock(hf_batch_tfm=hf_batch_tfm), noop)
dblock = DataBlock(blocks=blocks,
get_x=ColReader('article'),
get_y=ColReader('highlights'),
splitter=RandomSplitter())
dls = dblock.dataloaders(cnndm_df, bs=2)
b = dls.one_batch()
len(b), b[0]['input_ids'].shape, b[1].shape
dls.show_batch(dataloaders=dls, max_n=2)
Here we create a summarization specific subclass of HF_BaseModelCallback
in order to include custom, summarization specific, metrics, and also handle the pre-calculated loss during training
We add a custom param splitter to give us a bit more depth in applying discriminative learning rates for summarization.
Even though we don't really need a loss function, we have to provide a custom loss class/function for fastai to function properly (e.g. one with a decodes
and activation
methods). Why? Because these methods will get called in methods like show_results
to get the actual predictions.
text_gen_kwargs = { **hf_config.task_specific_params['summarization'], **{'max_length': 130, 'min_length': 30} }
text_gen_kwargs
model = HF_BaseModelWrapper(hf_model)
model_cb = HF_SummarizationModelCallback(text_gen_kwargs=text_gen_kwargs)
learn = Learner(dls,
model,
opt_func=ranger,
loss_func=HF_MaskedLMLoss(),
cbs=[model_cb],
splitter=partial(summarization_splitter, arch=hf_arch))#.to_fp16()
learn.create_opt()
learn.freeze()
b = dls.one_batch()
preds = learn.model(b[0])
len(preds),preds[0], preds[1].shape
len(b), len(b[0]), b[0]['input_ids'].shape, len(b[1]), b[1].shape
print(len(learn.opt.param_groups))
learn.lr_find(suggestions=True)
learn.fit_one_cycle(1, lr_max=4e-5)
test_article = """
About 10 men armed with pistols and small machine guns raided a casino in Switzerland and made off
into France with several hundred thousand Swiss francs in the early hours of Sunday morning, police said.
The men, dressed in black clothes and black ski masks, split into two groups during the raid on the Grand Casino
Basel, Chief Inspector Peter Gill told CNN. One group tried to break into the casino's vault on the lower level
but could not get in, but they did rob the cashier of the money that was not secured, he said. The second group
of armed robbers entered the upper level where the roulette and blackjack tables are located and robbed the
cashier there, he said. As the thieves were leaving the casino, a woman driving by and unaware of what was
occurring unknowingly blocked the armed robbers' vehicles. A gunman pulled the woman from her vehicle, beat
her, and took off for the French border. The other gunmen followed into France, which is only about 100
meters (yards) from the casino, Gill said. There were about 600 people in the casino at the time of the robbery.
There were no serious injuries, although one guest on the Casino floor was kicked in the head by one of the
robbers when he moved, the police officer said. Swiss authorities are working closely with French authorities,
Gill said. The robbers spoke French and drove vehicles with French license plates. CNN's Andreena Narayan
contributed to this report.
"""
res = learn.blurr_predict(test_article)
print(hf_tokenizer.decode(res[0][:20]))
That doesn't look much like a human-generated summary. Let's use huggingface's PreTrainedModel.generate
method to create something more human-like.
b = dls.valid.one_batch()
test_input_ids = b[0]['input_ids'][0].unsqueeze(0).to(learn.model.hf_model.device)
test_trg_ids = b[1][0].unsqueeze(0).to(learn.model.hf_model.device)
gen_text = learn.model.hf_model.generate(test_input_ids, num_beams=4, max_length=130, min_length=30)
print('=== Target ===')
print(f'{hf_tokenizer.decode(test_trg_ids[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)}\n')
print('=== Prediction ===')
print(hf_tokenizer.decode(gen_text[0], skip_special_tokens=True, clean_up_tokenization_spaces=True))
We'll add a blurr_summarize
method to Learner
that uses huggingface's PreTrainedModel.generate
to create our predictions. For the full list of arguments you can pass in see here. You can also check out their "How To Generate" notebook for more information about how it all works.
outputs = learn.blurr_summarize(test_article, num_return_sequences=3)
for idx, o in enumerate(outputs):
print(f'=== Prediction {idx+1} ===\n{o}\n')
Much nicer!!! Now, we can update our @typedispatched show_results
to use this new method.
learn.show_results(learner=learn)
learn.export(fname='summarize_export.pkl')
inf_learn = load_learner(fname='summarize_export.pkl')
inf_learn.blurr_summarize(test_article)
The tests below to ensure the core training code above works for all pretrained summarization models available in huggingface. These tests are excluded from the CI workflow because of how long they would take to run and the amount of data that would be required to download.
Note: Feel free to modify the code below to test whatever pretrained summarization models you are working with ... and if any of your pretrained summarization models fail, please submit a github issue (or a PR if you'd like to fix it yourself)
try: del learn; torch.cuda.empty_cache()
except: pass
BLURR_MODEL_HELPER.get_models(task='ConditionalGeneration')
pretrained_model_names = [
('facebook/bart-large-cnn',BartForConditionalGeneration),
('t5-small', T5ForConditionalGeneration),
#('google/pegasus-cnn_dailymail', PegasusForConditionalGeneration), ... don't fit on my 1080TI :(
]
path = Path('./')
cnndm_df = pd.read_csv(path/'cnndm_sample.csv')
#hide_output
bsz = 2
inp_seq_sz = 128; trg_seq_sz = 130
test_results = []
for model_name, model_cls in pretrained_model_names:
error=None
print(f'=== {model_name} ===\n')
hf_arch, hf_config, hf_tokenizer, hf_model = BLURR_MODEL_HELPER.get_hf_objects(model_name,
model_cls=model_cls)
print(f'architecture:\t{hf_arch}\ntokenizer:\t{type(hf_tokenizer).__name__}\nmodel:\t\t{type(hf_model).__name__}\n')
# 1. build your DataBlock
def add_t5_prefix(inp): return f'summarize: {inp}' if (hf_arch == 't5') else inp
hf_batch_tfm = HF_SummarizationBatchTransform(hf_arch, hf_tokenizer, max_length=[inp_seq_sz, trg_seq_sz])
blocks = (HF_TextBlock(hf_batch_tfm=hf_batch_tfm), noop)
dblock = DataBlock(blocks=blocks,
get_x=Pipeline([ColReader('article'), add_t5_prefix]),
get_y=ColReader('highlights'),
splitter=RandomSplitter())
dls = dblock.dataloaders(cnndm_df, bs=bsz)
# 2. build your Learner
text_gen_kwargs = {}
if (hf_arch in ['bart', 't5']):
text_gen_kwargs = {
**hf_config.task_specific_params['summarization'],
**{'max_length': 30, 'min_length': 10}
}
model = HF_BaseModelWrapper(hf_model)
model_cb = HF_SummarizationModelCallback(text_gen_kwargs=text_gen_kwargs)
learn = Learner(dls,
model,
opt_func=ranger,
loss_func=HF_MaskedLMLoss(),
cbs=[model_cb],
splitter=partial(summarization_splitter, arch=hf_arch))#.to_fp16()
learn.create_opt()
learn.freeze()
# 3. Run your tests
b = dls.one_batch()
try:
print('*** TESTING DataLoaders ***\n')
test_eq(len(b), 2)
test_eq(len(b[0]['input_ids']), bsz)
test_eq(b[0]['input_ids'].shape, torch.Size([bsz, inp_seq_sz]))
test_eq(len(b[1]), bsz)
print('*** TESTING One pass through the model ***')
preds = learn.model(b[0])
test_eq(preds[1].shape[0], bsz)
test_eq(preds[1].shape[2], hf_config.vocab_size)
print('*** TESTING Training/Results ***')
learn.fit_one_cycle(1, lr_max=1e-3)
test_results.append((hf_arch, type(hf_tokenizer).__name__, type(hf_model).__name__, 'PASSED', ''))
learn.show_results(learner=learn, max_n=2)
except Exception as err:
test_results.append((hf_arch, type(hf_tokenizer).__name__, type(hf_model).__name__, 'FAILED', err))
finally:
# cleanup
del learn; torch.cuda.empty_cache()