--- title: data.seq2seq.language_modeling keywords: fastai sidebar: home_sidebar summary: "This module contains the bits required to use the fastai DataBlock API and/or mid-level data processing pipelines to organize your data for causal and masked language modeling tasks. This includes things like training BERT from scratch or fine-tuning a particular pre-trained LM on your own corpus." description: "This module contains the bits required to use the fastai DataBlock API and/or mid-level data processing pipelines to organize your data for causal and masked language modeling tasks. This includes things like training BERT from scratch or fine-tuning a particular pre-trained LM on your own corpus." nb_path: "nbs/01zb_data-seq2seq-language-modeling.ipynb" ---
torch.cuda.set_device(1)
print(f'Using GPU #{torch.cuda.current_device()}: {torch.cuda.get_device_name()}')
Using GPU #1: GeForce GTX 1080 Ti
wiki_path = untar_data(URLs.WIKITEXT_TINY)
wiki_path.ls()
(#2) [Path('/home/wgilliam/.fastai/data/wikitext-2/train.csv'),Path('/home/wgilliam/.fastai/data/wikitext-2/test.csv')]
train_df = pd.read_csv(wiki_path/'train.csv', header=None)
valid_df = pd.read_csv(wiki_path/'test.csv', header=None)
print(len(train_df), len(valid_df))
train_df.head()
615 47
0 | |
---|---|
0 | \n = 2013 – 14 York City F.C. season = \n \n The 2013 – 14 season was the <unk> season of competitive association football and 77th season in the Football League played by York City Football Club , a professional football club based in York , North Yorkshire , England . Their 17th @-@ place finish in 2012 – 13 meant it was their second consecutive season in League Two . The season ran from 1 July 2013 to 30 June 2014 . \n Nigel Worthington , starting his first full season as York manager , made eight permanent summer signings . By the turn of the year York were only above the relegation z... |
1 | \n = Big Boy ( song ) = \n \n " Big Boy " <unk> " I 'm A Big Boy Now " was the first single ever recorded by the Jackson 5 , which was released by Steeltown Records in January 1968 . The group played instruments on many of their Steeltown compositions , including " Big Boy " . The song was neither a critical nor commercial success , but the Jackson family were delighted with the outcome nonetheless . \n The Jackson 5 would release a second single with Steeltown Records before moving to Motown Records . The group 's recordings at Steeltown Records were thought to be lost , but they were re... |
2 | \n = The Remix ( Lady Gaga album ) = \n \n The Remix is a remix album by American recording artist Lady Gaga . Released in Japan on March 3 , 2010 , it contains remixes of the songs from her first studio album , The Fame ( 2008 ) , and her third extended play , The Fame Monster ( 2009 ) . A revised version of the track list was prepared for release in additional markets , beginning with Mexico on May 3 , 2010 . A number of recording artists have produced the songs , including Pet Shop Boys , Passion Pit and The Sound of Arrows . The remixed versions feature both uptempo and <unk> composit... |
3 | \n = New Year 's Eve ( Up All Night ) = \n \n " New Year 's Eve " is the twelfth episode of the first season of the American comedy television series Up All Night . The episode originally aired on NBC in the United States on January 12 , 2012 . It was written by Erica <unk> and was directed by Beth McCarthy @-@ Miller . The episode also featured a guest appearance from Jason Lee as Chris and Reagan 's neighbor and Ava 's boyfriend , Kevin . \n During Reagan ( Christina Applegate ) and Chris 's ( Will <unk> ) first New Year 's Eve game night , Reagan 's competitiveness comes out causing Ch... |
4 | \n = Geopyxis carbonaria = \n \n Geopyxis carbonaria is a species of fungus in the genus Geopyxis , family <unk> . First described to science in 1805 , and given its current name in 1889 , the species is commonly known as the charcoal loving elf @-@ cup , dwarf <unk> cup , <unk> <unk> cup , or pixie cup . The small , <unk> @-@ shaped fruitbodies of the fungus are reddish @-@ brown with a whitish fringe and measure up to 2 cm ( 0 @.@ 8 in ) across . They have a short , tapered stalk . Fruitbodies are commonly found on soil where brush has recently been burned , sometimes in great numbers .... |
train_df['is_valid'] = False
valid_df['is_valid'] = True
df = pd.concat([train_df, valid_df])
df.head()
0 | is_valid | |
---|---|---|
0 | \n = 2013 – 14 York City F.C. season = \n \n The 2013 – 14 season was the <unk> season of competitive association football and 77th season in the Football League played by York City Football Club , a professional football club based in York , North Yorkshire , England . Their 17th @-@ place finish in 2012 – 13 meant it was their second consecutive season in League Two . The season ran from 1 July 2013 to 30 June 2014 . \n Nigel Worthington , starting his first full season as York manager , made eight permanent summer signings . By the turn of the year York were only above the relegation z... | False |
1 | \n = Big Boy ( song ) = \n \n " Big Boy " <unk> " I 'm A Big Boy Now " was the first single ever recorded by the Jackson 5 , which was released by Steeltown Records in January 1968 . The group played instruments on many of their Steeltown compositions , including " Big Boy " . The song was neither a critical nor commercial success , but the Jackson family were delighted with the outcome nonetheless . \n The Jackson 5 would release a second single with Steeltown Records before moving to Motown Records . The group 's recordings at Steeltown Records were thought to be lost , but they were re... | False |
2 | \n = The Remix ( Lady Gaga album ) = \n \n The Remix is a remix album by American recording artist Lady Gaga . Released in Japan on March 3 , 2010 , it contains remixes of the songs from her first studio album , The Fame ( 2008 ) , and her third extended play , The Fame Monster ( 2009 ) . A revised version of the track list was prepared for release in additional markets , beginning with Mexico on May 3 , 2010 . A number of recording artists have produced the songs , including Pet Shop Boys , Passion Pit and The Sound of Arrows . The remixed versions feature both uptempo and <unk> composit... | False |
3 | \n = New Year 's Eve ( Up All Night ) = \n \n " New Year 's Eve " is the twelfth episode of the first season of the American comedy television series Up All Night . The episode originally aired on NBC in the United States on January 12 , 2012 . It was written by Erica <unk> and was directed by Beth McCarthy @-@ Miller . The episode also featured a guest appearance from Jason Lee as Chris and Reagan 's neighbor and Ava 's boyfriend , Kevin . \n During Reagan ( Christina Applegate ) and Chris 's ( Will <unk> ) first New Year 's Eve game night , Reagan 's competitiveness comes out causing Ch... | False |
4 | \n = Geopyxis carbonaria = \n \n Geopyxis carbonaria is a species of fungus in the genus Geopyxis , family <unk> . First described to science in 1805 , and given its current name in 1889 , the species is commonly known as the charcoal loving elf @-@ cup , dwarf <unk> cup , <unk> <unk> cup , or pixie cup . The small , <unk> @-@ shaped fruitbodies of the fungus are reddish @-@ brown with a whitish fringe and measure up to 2 cm ( 0 @.@ 8 in ) across . They have a short , tapered stalk . Fruitbodies are commonly found on soil where brush has recently been burned , sometimes in great numbers .... | False |
model_cls = AutoModelForCausalLM
pretrained_model_name = "gpt2"
hf_arch, hf_config, hf_tokenizer, hf_model = BLURR.get_hf_objects(pretrained_model_name, model_cls=model_cls)
# some tokenizers like gpt and gpt2 do not have a pad token, so we add it here mainly for the purpose
# of setting the "labels" key appropriately (see below)
if (hf_tokenizer.pad_token is None): hf_tokenizer.pad_token = '[PAD]'
hf_tokenizer.pad_token, hf_tokenizer.pad_token_id
Using pad_token, but it is not set yet.
('[PAD]', 50256)
class
HF_CausalLMBeforeBatchTransform
[source]
HF_CausalLMBeforeBatchTransform
(hf_arch
,hf_config
,hf_tokenizer
,hf_model
,max_length
=None
,padding
=True
,truncation
=True
,is_split_into_words
=False
,ignore_token_id
=-100
,tok_kwargs
={}
,text_gen_kwargs
={}
, **kwargs
) ::HF_BeforeBatchTransform
Handles everything you need to assemble a mini-batch of inputs and targets, as well as
decode the dictionary produced as a byproduct of the tokenization process in the encodes
method.
Our HF_CausalLMBeforeBatchTransform
allows us to update the input's labels
and our targets appropriately given a causal LM task.
The labels
argument allows you to forgo calculating the loss yourself by letting huggingface return it for you should you choose to do that. Padding tokens are set to -100 by default (e.g., CrossEntropyLossFlat().ignore_index
) and prevent cross entropy loss from considering token prediction for tokens it should ... i.e., the padding tokens. For more information on the meaning of this argument, see the huggingface glossary entry for "Labels"
before_batch_tfm = HF_CausalLMBeforeBatchTransform(hf_arch, hf_config, hf_tokenizer, hf_model)
blocks = (HF_Seq2SeqBlock(before_batch_tfm=before_batch_tfm), noop)
dblock = DataBlock(blocks=blocks, get_x=ColReader(0), splitter=ColSplitter(col='is_valid'))
dls = dblock.dataloaders(df, bs=4)
b = dls.one_batch()
b[0]['input_ids'].shape, b[0]['labels'].shape, b[1].shape
(torch.Size([4, 1024]), torch.Size([4, 1024]), torch.Size([4, 1024]))
explode_types(b)
{tuple: [dict, torch.Tensor]}
dls.show_batch(dataloaders=dls, max_n=2, trunc_at=500)
text | target | |
---|---|---|
0 | \n = Bob Dylan = \n \n Bob Dylan ( / <unk> / ; born Robert Allen Zimmerman, May 24, 1941 ) is an American singer @-@ songwriter, artist and writer. He has been influential in popular music and culture for more than five decades. Much of his most celebrated work dates from the 1960s when his songs chronicled social unrest, although Dylan repudiated suggestions from journalists that he was a spokesman for his generation. Nevertheless, early songs such as " Blowin'in the Wind " and " The Times They Are a @-@ <unk>'" became anthems for the American civil rights and anti @-@ war movements. After he left his initial base in the American folk music revival, his six @-@ minute single " Like a Rolling Stone " altered the range of popular music in 1965. His mid @-@ 1960s recordings, backed by rock musicians, reached the top end of the United States music charts while also attracting <unk> and criticism from others in the folk movement. \n Dylan's lyrics have incorporated various political, social, philosophical, and literary influences. They defied existing pop music conventions and appealed to the burgeoning counterculture. Initially inspired by the performances of Little Richard and the songwriting of Woody Guthrie, Robert Johnson, and Hank Williams, Dylan has amplified and <unk> musical genres. His recording career, spanning more than 50 years, has explored the traditions in American song — from folk, blues, and country to gospel, rock and roll, and rockabilly to English, Scottish, and Irish folk music, embracing even jazz and the Great American Songbook. Dylan performs with guitar, keyboards, and harmonica. Backed by a changing lineup of musicians, he has toured steadily since the late 1980s on what has been dubbed the Never Ending Tour. His accomplishments as a recording artist and performer have been central to his career, but songwriting is considered his greatest contribution. \n Since 1994, Dylan has published six books of drawings and paintings, and his work has been exhibited in major art galleries. As a musician, Dylan has sold more than 100 million records, making him one of the best @-@ selling artists of all time. He has also received numerous awards including eleven Grammy Awards, a Golden Globe Award, and an Academy Award. Dylan has been inducted into the Rock and Roll Hall of Fame, Minnesota Music Hall of Fame, Nashville Songwriters Hall of Fame, and Songwriters Hall of Fame. The Pulitzer Prize jury in 2008 awarded him a special citation for " his profound impact on popular music and American culture, marked by lyrical compositions of extraordinary poetic power. " In May 2012, Dylan received the Presidential Medal of Freedom from President Barack Obama. \n \n = = Life and career = = \n \n \n = = = Origins and musical beginnings = = = \n \n Bob Dylan was born Robert Allen Zimmerman ( Hebrew name <unk> <unk> <unk> <unk> [ Shabtai <unk> ben <unk> ] ) in St Mary's Hospital on May 24, 1941, in Duluth, Minnesota, and raised in Hibbing, Minnesota, on the <unk> Range west of Lake Superior. He has a younger brother, David. Dylan's paternal grandparents, <unk> and Anna Zimmerman, emigrated from Odessa, in the Russian Empire ( now Ukraine ), to the United States following the anti @-@ Semitic <unk> of 1905. His maternal grandparents, Ben and Florence Stone, were Lithuanian Jews who arrived in the United States in 1902. In his autobiography, Chronicles : Volume One, Dylan wrote that his paternal grandmother's maiden name was <unk> and her family originated from <unk> district of <unk> Province in northeastern Turkey. \n Dylan's parents, <unk> Zimmerman and <unk> " Beatty " Stone, were part of a small but close @-@ knit Jewish community. They lived in Duluth until Robert was six, when his father had <unk> and the family returned to his mother's hometown, Hibbing, where they lived for the rest of Robert's childhood. In his early years he listened to the radio — first to blues and country stations from Shreveport, Louisiana and later, when he was a teenager, to rock and roll. He formed several bands while attending Hibbing High School. In the Golden <unk>, he performed covers of songs by Little Richard and Elvis Presley. Their performance of Danny & the Juniors'" Rock and Roll Is Here to Stay " at their high school talent show was so loud that the principal cut the microphone. In 1959, his high school <unk> carried the caption " Robert Zimmerman : to join'Little Richard '. " The same year, as <unk> <unk> [ sic ], he performed two dates with Bobby <unk>, playing piano and clapping. \n Zimmerman moved to Minneapolis in September 1959 and | \n = Bob Dylan = \n \n Bob Dylan ( / <unk> / ; born Robert Allen Zimmerman, May 24, 1941 ) is an American singer @-@ songwriter, artist and writer. He has been influential in popular music and culture for more than five decades. Much of his most celebrated work dates from the 1960s when his songs chronicled social unrest, although Dylan repudiated suggestions from journalists that he was a spokesman for his generation. Nevertheless, early songs such as " Blowin'in the Wind " and " The Times They Are a @-@ <unk>'" became anthems for the American civil rights and anti @-@ war movements. After he left his initial base in the American folk music revival, his six @-@ minute single " Like a Rolling Stone " altered the range of popular music in 1965. His mid @-@ 1960s recordings, backed by rock musicians, reached the top end of the United States music charts while also attracting <unk> and criticism from others in the folk movement. \n Dylan's lyrics have incorporated various political, social, philosophical, and literary influences. They defied existing pop music conventions and appealed to the burgeoning counterculture. Initially inspired by the performances of Little Richard and the songwriting of Woody Guthrie, Robert Johnson, and Hank Williams, Dylan has amplified and <unk> musical genres. His recording career, spanning more than 50 years, has explored the traditions in American song — from folk, blues, and country to gospel, rock and roll, and rockabilly to English, Scottish, and Irish folk music, embracing even jazz and the Great American Songbook. Dylan performs with guitar, keyboards, and harmonica. Backed by a changing lineup of musicians, he has toured steadily since the late 1980s on what has been dubbed the Never Ending Tour. His accomplishments as a recording artist and performer have been central to his career, but songwriting is considered his greatest contribution. \n Since 1994, Dylan has published six books of drawings and paintings, and his work has been exhibited in major art galleries. As a musician, Dylan has sold more than 100 million records, making him one of the best @-@ selling artists of all time. He has also received numerous awards including eleven Grammy Awards, a Golden Globe Award, and an Academy Award. Dylan has been inducted into the Rock and Roll Hall of Fame, Minnesota Music Hall of Fame, Nashville Songwriters Hall of Fame, and Songwriters Hall of Fame. The Pulitzer Prize jury in 2008 awarded him a special citation for " his profound impact on popular music and American culture, marked by lyrical compositions of extraordinary poetic power. " In May 2012, Dylan received the Presidential Medal of Freedom from President Barack Obama. \n \n = = Life and career = = \n \n \n = = = Origins and musical beginnings = = = \n \n Bob Dylan was born Robert Allen Zimmerman ( Hebrew name <unk> <unk> <unk> <unk> [ Shabtai <unk> ben <unk> ] ) in St Mary's Hospital on May 24, 1941, in Duluth, Minnesota, and raised in Hibbing, Minnesota, on the <unk> Range west of Lake Superior. He has a younger brother, David. Dylan's paternal grandparents, <unk> and Anna Zimmerman, emigrated from Odessa, in the Russian Empire ( now Ukraine ), to the United States following the anti @-@ Semitic <unk> of 1905. His maternal grandparents, Ben and Florence Stone, were Lithuanian Jews who arrived in the United States in 1902. In his autobiography, Chronicles : Volume One, Dylan wrote that his paternal grandmother's maiden name was <unk> and her family originated from <unk> district of <unk> Province in northeastern Turkey. \n Dylan's parents, <unk> Zimmerman and <unk> " Beatty " Stone, were part of a small but close @-@ knit Jewish community. They lived in Duluth until Robert was six, when his father had <unk> and the family returned to his mother's hometown, Hibbing, where they lived for the rest of Robert's childhood. In his early years he listened to the radio — first to blues and country stations from Shreveport, Louisiana and later, when he was a teenager, to rock and roll. He formed several bands while attending Hibbing High School. In the Golden <unk>, he performed covers of songs by Little Richard and Elvis Presley. Their performance of Danny & the Juniors'" Rock and Roll Is Here to Stay " at their high school talent show was so loud that the principal cut the microphone. In 1959, his high school <unk> carried the caption " Robert Zimmerman : to join'Little Richard '. " The same year, as <unk> <unk> [ sic ], he performed two dates with Bobby <unk>, playing piano and clapping. \n Zimmerman moved to Minneapolis in September 1959 and |
1 | \n = George Calvert, 1st Baron Baltimore = \n \n George Calvert, 1st Baron Baltimore ( <unk> – 15 April 1632 ) was an English politician and <unk>. He achieved domestic political success as a Member of Parliament and later Secretary of State under King James I. He lost much of his political power after his support for a failed marriage alliance between Prince Charles and the Spanish House of Habsburg royal family. Rather than continue in politics, he resigned all of his political offices in 1625 except for his position on the Privy Council and declared his Catholicism publicly. He was created Baron Baltimore in the Irish peerage upon his resignation. Baltimore Manor was located in County Longford, Ireland. \n Calvert took an interest in the British colonization of the Americas, at first for commercial reasons and later to create a refuge for English Catholics. He became the proprietor of Avalon, the first sustained English settlement on the southeastern peninsula on the island of Newfoundland ( off the eastern coast of modern Canada ). <unk> by its cold and sometimes inhospitable climate and the <unk> of the settlers, Sir George looked for a more suitable spot further south and sought a new royal charter to settle the region, which would become the state of Maryland. Calvert died five weeks before the new Charter was sealed, leaving the settlement of the Maryland colony to his son Cecil, ( 1605 @-@ <unk> ). His second son Leonard Calvert, ( <unk> @-@ 1647 ), was the first colonial governor of the Province of Maryland. Historians have long recognized George Calvert as the founder of Maryland, in spirit if not in fact. \n \n = = Family and early life = = \n \n Little is known of the ancestry of the Yorkshire branch of the <unk>. At George Calvert's <unk>, it was claimed that his family originally came from Flanders ( a Dutch @-@ speaking area today across the English Channel in modern @-@ day Kingdom of Belgium ). Calvert's father, ( an earlier ) Leonard was a country gentleman who had achieved some prominence as a tenant of Lord <unk>, and was wealthy enough to marry a " <unk> " of a noble line, Alicia or Alice <unk> ( or sometimes spelled : " <unk> " ). He established his family on the estate of the later @-@ built Kiplin Hall, near <unk> in <unk>, of Yorkshire. George Calvert was born at Kiplin in late <unk> ( birth month and day yet to be researched ). His mother Alicia / Alice died on 28 November <unk>, when he was fifteen years old. His father then married Grace <unk> ( or sometimes spelled : " <unk> " ), Alicia's first cousin. \n In 1569, Sir Thomas <unk> had described <unk> as a territory where all gentlemen were " evil in religion ", by which he meant predominately Roman Catholic ; it appears Leonard Calvert was no exception. During the reign of Queen Elizabeth I, continuing the changes wrought in the previous century by her father, King Henry VIII making the Monarch, the supreme authority of the Christian Church in England, continuing the pace Protestant Reformation from the Continent of Europe, with the political, spiritual and <unk> separation from the Roman Catholic Church and the Pope / <unk> in Rome, therefore the Royal Government exerted authority over the matters of religious faith, practices and the Church. Acts <unk> compulsory religious <unk> were enacted by Parliament and enforced through penal laws. The Acts of <unk> and the Uniformity Act of <unk> also included an oath of allegiance to the Queen ( <unk> Maria, wife of Charles I ) and an implicit denial of the Pope's ( then Pope Paul IV ) authority over the English Church. This oath was required of any common " subject " ( <unk> for citizen ), who wished to hold high office, attend university, or take advantage of opportunities controlled by the state ( king / kingdom ). \n The Calvert household suffered the intrusion of the <unk> @-@ era religious laws. From the year of George's birth onwards, his father, Leonard Calvert was subjected to repeated harassment by the Yorkshire authorities, who in 1580 extracted a promise of conformity from him, compelling his attendance at the Church of England services. In 1592, when George was twelve, the authorities denounced one of his tutors for teaching " from a <unk> <unk> " and instructed Leonard and Grace to send George and his brother Christopher to a Protestant tutor, and, if necessary, to present the children before the commission " once a month to see how they perfect in learning ". As a result, the boys were sent to a Protestant tutor called Mr. <unk> at <unk>. The senior Calvert had to give a " bond of conformity " ; he was banned from | \n = George Calvert, 1st Baron Baltimore = \n \n George Calvert, 1st Baron Baltimore ( <unk> – 15 April 1632 ) was an English politician and <unk>. He achieved domestic political success as a Member of Parliament and later Secretary of State under King James I. He lost much of his political power after his support for a failed marriage alliance between Prince Charles and the Spanish House of Habsburg royal family. Rather than continue in politics, he resigned all of his political offices in 1625 except for his position on the Privy Council and declared his Catholicism publicly. He was created Baron Baltimore in the Irish peerage upon his resignation. Baltimore Manor was located in County Longford, Ireland. \n Calvert took an interest in the British colonization of the Americas, at first for commercial reasons and later to create a refuge for English Catholics. He became the proprietor of Avalon, the first sustained English settlement on the southeastern peninsula on the island of Newfoundland ( off the eastern coast of modern Canada ). <unk> by its cold and sometimes inhospitable climate and the <unk> of the settlers, Sir George looked for a more suitable spot further south and sought a new royal charter to settle the region, which would become the state of Maryland. Calvert died five weeks before the new Charter was sealed, leaving the settlement of the Maryland colony to his son Cecil, ( 1605 @-@ <unk> ). His second son Leonard Calvert, ( <unk> @-@ 1647 ), was the first colonial governor of the Province of Maryland. Historians have long recognized George Calvert as the founder of Maryland, in spirit if not in fact. \n \n = = Family and early life = = \n \n Little is known of the ancestry of the Yorkshire branch of the <unk>. At George Calvert's <unk>, it was claimed that his family originally came from Flanders ( a Dutch @-@ speaking area today across the English Channel in modern @-@ day Kingdom of Belgium ). Calvert's father, ( an earlier ) Leonard was a country gentleman who had achieved some prominence as a tenant of Lord <unk>, and was wealthy enough to marry a " <unk> " of a noble line, Alicia or Alice <unk> ( or sometimes spelled : " <unk> " ). He established his family on the estate of the later @-@ built Kiplin Hall, near <unk> in <unk>, of Yorkshire. George Calvert was born at Kiplin in late <unk> ( birth month and day yet to be researched ). His mother Alicia / Alice died on 28 November <unk>, when he was fifteen years old. His father then married Grace <unk> ( or sometimes spelled : " <unk> " ), Alicia's first cousin. \n In 1569, Sir Thomas <unk> had described <unk> as a territory where all gentlemen were " evil in religion ", by which he meant predominately Roman Catholic ; it appears Leonard Calvert was no exception. During the reign of Queen Elizabeth I, continuing the changes wrought in the previous century by her father, King Henry VIII making the Monarch, the supreme authority of the Christian Church in England, continuing the pace Protestant Reformation from the Continent of Europe, with the political, spiritual and <unk> separation from the Roman Catholic Church and the Pope / <unk> in Rome, therefore the Royal Government exerted authority over the matters of religious faith, practices and the Church. Acts <unk> compulsory religious <unk> were enacted by Parliament and enforced through penal laws. The Acts of <unk> and the Uniformity Act of <unk> also included an oath of allegiance to the Queen ( <unk> Maria, wife of Charles I ) and an implicit denial of the Pope's ( then Pope Paul IV ) authority over the English Church. This oath was required of any common " subject " ( <unk> for citizen ), who wished to hold high office, attend university, or take advantage of opportunities controlled by the state ( king / kingdom ). \n The Calvert household suffered the intrusion of the <unk> @-@ era religious laws. From the year of George's birth onwards, his father, Leonard Calvert was subjected to repeated harassment by the Yorkshire authorities, who in 1580 extracted a promise of conformity from him, compelling his attendance at the Church of England services. In 1592, when George was twelve, the authorities denounced one of his tutors for teaching " from a <unk> <unk> " and instructed Leonard and Grace to send George and his brother Christopher to a Protestant tutor, and, if necessary, to present the children before the commission " once a month to see how they perfect in learning ". As a result, the boys were sent to a Protestant tutor called Mr. <unk> at <unk>. The senior Calvert had to give a " bond of conformity " ; he was banned from |