--- title: utils keywords: fastai sidebar: home_sidebar summary: "Various utility functions used by the blurr package." description: "Various utility functions used by the blurr package." nb_path: "nbs/00_utils.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %}
torch.cuda.set_device(1)
print(f'Using GPU #{torch.cuda.current_device()}: {torch.cuda.get_device_name()}')
Using GPU #1: GeForce GTX 1080 Ti
{% endraw %} {% raw %}
{% endraw %} {% raw %}

str_to_class[source]

str_to_class(classname)

converts string representation to class

{% endraw %} {% raw %}
{% endraw %} {% raw %}

class Singleton[source]

Singleton()

{% endraw %}

Singleton functions as python decorator. Use this above any class to turn that class into a singleton (see here for more info on the singleton pattern).

{% raw %}
@Singleton
class TestSingleton: pass

a = TestSingleton()
b = TestSingleton()
test_eq(a,b)
{% endraw %}

ModelHelper

{% raw %}
{% endraw %} {% raw %}

Singleton object at 0x7f2843081f90>[source]

Singleton object at 0x7f2843081f90>(*args, **kwargs)

{% endraw %}

ModelHelper is a Singleton (there exists only one instance, and the same instance is returned upon subsequent instantiation requests). You can get at via the BLURR_MODEL_HELPER constant below.

{% raw %}
mh = ModelHelper()
mh2 = ModelHelper()
test_eq(mh, mh2)
{% endraw %}

Provide global helper constant

Users of this library can simply use BLURR_MODEL_HELPER to access all the ModelHelper capabilities without having to fetch an instance themselves.

{% raw %}
{% endraw %} {% raw %}

ModelHelper.get_architectures[source]

ModelHelper.get_architectures()

Used to get all the architectures supported by your Transformers install

{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_architectures())
['albert', 'albert_fast', 'auto', 'bart', 'bart_fast', 'bert', 'bert_fast', 'bert_generation', 'bert_japanese', 'bertweet', 'blenderbot', 'camembert', 'camembert_fast', 'ctrl', 'deberta', 'distilbert', 'distilbert_fast', 'dpr', 'dpr_fast', 'electra', 'electra_fast', 'encoder_decoder', 'flaubert', 'fsmt', 'funnel', 'funnel_fast', 'gpt2', 'gpt2_fast', 'herbert', 'herbert_fast', 'layoutlm', 'layoutlm_fast', 'longformer', 'longformer_fast', 'lxmert', 'lxmert_fast', 'marian', 'mbart', 'mbart_fast', 'mmbt', 'mobilebert', 'mobilebert_fast', 'openai', 'openai_fast', 'pegasus', 'pegasus_fast', 'phobert', 'prophetnet', 'rag', 'reformer', 'reformer_fast', 'retribert', 'retribert_fast', 'roberta', 'roberta_fast', 'squeezebert', 'squeezebert_fast', 't5', 't5_fast', 'transfo_xl', 'utils_base', 'utils_fast', 'xlm', 'xlm_prophetnet', 'xlm_roberta', 'xlm_roberta_fast', 'xlnet', 'xlnet_fast']
{% endraw %}

We'll also create an enum for downstream tasks

{% raw %}
{% endraw %} {% raw %}
print(L(HF_ARCHITECTURES))
[<HF_ARCHITECTURES.albert: 1>, <HF_ARCHITECTURES.albert_fast: 2>, <HF_ARCHITECTURES.auto: 3>, <HF_ARCHITECTURES.bart: 4>, <HF_ARCHITECTURES.bart_fast: 5>, <HF_ARCHITECTURES.bert: 6>, <HF_ARCHITECTURES.bert_fast: 7>, <HF_ARCHITECTURES.bert_generation: 8>, <HF_ARCHITECTURES.bert_japanese: 9>, <HF_ARCHITECTURES.bertweet: 10>, <HF_ARCHITECTURES.blenderbot: 11>, <HF_ARCHITECTURES.camembert: 12>, <HF_ARCHITECTURES.camembert_fast: 13>, <HF_ARCHITECTURES.ctrl: 14>, <HF_ARCHITECTURES.deberta: 15>, <HF_ARCHITECTURES.distilbert: 16>, <HF_ARCHITECTURES.distilbert_fast: 17>, <HF_ARCHITECTURES.dpr: 18>, <HF_ARCHITECTURES.dpr_fast: 19>, <HF_ARCHITECTURES.electra: 20>, <HF_ARCHITECTURES.electra_fast: 21>, <HF_ARCHITECTURES.encoder_decoder: 22>, <HF_ARCHITECTURES.flaubert: 23>, <HF_ARCHITECTURES.fsmt: 24>, <HF_ARCHITECTURES.funnel: 25>, <HF_ARCHITECTURES.funnel_fast: 26>, <HF_ARCHITECTURES.gpt2: 27>, <HF_ARCHITECTURES.gpt2_fast: 28>, <HF_ARCHITECTURES.herbert: 29>, <HF_ARCHITECTURES.herbert_fast: 30>, <HF_ARCHITECTURES.layoutlm: 31>, <HF_ARCHITECTURES.layoutlm_fast: 32>, <HF_ARCHITECTURES.longformer: 33>, <HF_ARCHITECTURES.longformer_fast: 34>, <HF_ARCHITECTURES.lxmert: 35>, <HF_ARCHITECTURES.lxmert_fast: 36>, <HF_ARCHITECTURES.marian: 37>, <HF_ARCHITECTURES.mbart: 38>, <HF_ARCHITECTURES.mbart_fast: 39>, <HF_ARCHITECTURES.mmbt: 40>, <HF_ARCHITECTURES.mobilebert: 41>, <HF_ARCHITECTURES.mobilebert_fast: 42>, <HF_ARCHITECTURES.openai: 43>, <HF_ARCHITECTURES.openai_fast: 44>, <HF_ARCHITECTURES.pegasus: 45>, <HF_ARCHITECTURES.pegasus_fast: 46>, <HF_ARCHITECTURES.phobert: 47>, <HF_ARCHITECTURES.prophetnet: 48>, <HF_ARCHITECTURES.rag: 49>, <HF_ARCHITECTURES.reformer: 50>, <HF_ARCHITECTURES.reformer_fast: 51>, <HF_ARCHITECTURES.retribert: 52>, <HF_ARCHITECTURES.retribert_fast: 53>, <HF_ARCHITECTURES.roberta: 54>, <HF_ARCHITECTURES.roberta_fast: 55>, <HF_ARCHITECTURES.squeezebert: 56>, <HF_ARCHITECTURES.squeezebert_fast: 57>, <HF_ARCHITECTURES.t5: 58>, <HF_ARCHITECTURES.t5_fast: 59>, <HF_ARCHITECTURES.transfo_xl: 60>, <HF_ARCHITECTURES.utils_base: 61>, <HF_ARCHITECTURES.utils_fast: 62>, <HF_ARCHITECTURES.xlm: 63>, <HF_ARCHITECTURES.xlm_prophetnet: 64>, <HF_ARCHITECTURES.xlm_roberta: 65>, <HF_ARCHITECTURES.xlm_roberta_fast: 66>, <HF_ARCHITECTURES.xlnet: 67>, <HF_ARCHITECTURES.xlnet_fast: 68>]
{% endraw %} {% raw %}

ModelHelper.get_config[source]

ModelHelper.get_config(arch)

Used the locate the name of the configuration class for a given architecture

{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_config('bert'))
<class 'transformers.configuration_bert.BertConfig'>
{% endraw %} {% raw %}

ModelHelper.get_tokenizers[source]

ModelHelper.get_tokenizers(arch)

Used to get the available huggingface tokenizers for a given architecture. Note: There may be multiple tokenizers and so this returns a list.

{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_tokenizers('electra'))
[<class 'transformers.tokenization_electra.ElectraTokenizer'>]
{% endraw %} {% raw %}

ModelHelper.get_tasks[source]

ModelHelper.get_tasks(arch=None)

Get the type of tasks for which there is a custom model for (optional: by architecture). There are a number of customized models built for specific tasks like token classification, question/answering, LM, etc....

{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_tasks())
print('')
print(BLURR_MODEL_HELPER.get_tasks('bart'))
['CausalLM', 'Classification', 'ConditionalGeneration', 'Generation', 'LMHead', 'LMHeadModel', 'MaskedLM', 'MultipleChoice', 'NextSentencePrediction', 'PreTraining', 'QuestionAnswering', 'QuestionAnsweringSimple', 'Seq2SeqLM', 'SequenceClassification', 'TokenClassification']

['ConditionalGeneration', 'QuestionAnswering', 'SequenceClassification']
{% endraw %}

We'll create an enum for tasks as well, one for all tasks and another for tasks available via huggingface's AutoModel capabilities

{% raw %}
{% endraw %} {% raw %}
print('--- all tasks ---')
print(L(HF_TASKS_ALL))
print('\n--- auto only ---')
print(L(HF_TASKS_AUTO))
--- all tasks ---
[<HF_TASKS_ALL.CausalLM: 1>, <HF_TASKS_ALL.Classification: 2>, <HF_TASKS_ALL.ConditionalGeneration: 3>, <HF_TASKS_ALL.Generation: 4>, <HF_TASKS_ALL.LMHead: 5>, <HF_TASKS_ALL.LMHeadModel: 6>, <HF_TASKS_ALL.MaskedLM: 7>, <HF_TASKS_ALL.MultipleChoice: 8>, <HF_TASKS_ALL.NextSentencePrediction: 9>, <HF_TASKS_ALL.PreTraining: 10>, <HF_TASKS_ALL.QuestionAnswering: 11>, <HF_TASKS_ALL.QuestionAnsweringSimple: 12>, <HF_TASKS_ALL.Seq2SeqLM: 13>, <HF_TASKS_ALL.SequenceClassification: 14>, <HF_TASKS_ALL.TokenClassification: 15>]

--- auto only ---
[<HF_TASKS_AUTO.CausalLM: 1>, <HF_TASKS_AUTO.LMHead: 2>, <HF_TASKS_AUTO.MaskedLM: 3>, <HF_TASKS_AUTO.MultipleChoice: 4>, <HF_TASKS_AUTO.PreTraining: 5>, <HF_TASKS_AUTO.QuestionAnswering: 6>, <HF_TASKS_AUTO.Seq2SeqLM: 7>, <HF_TASKS_AUTO.SequenceClassification: 8>, <HF_TASKS_AUTO.TokenClassification: 9>]
{% endraw %} {% raw %}
HF_TASKS_ALL.Classification
<HF_TASKS_ALL.Classification: 2>
{% endraw %} {% raw %}

ModelHelper.get_models[source]

ModelHelper.get_models(arch=None, task=None)

The transformer models available for use (optional: by architecture | task)

{% endraw %} {% raw %}
print(L(BLURR_MODEL_HELPER.get_models()))
[<class 'transformers.modeling_transfo_xl.AdaptiveEmbedding'>, <class 'transformers.modeling_albert.AlbertForMaskedLM'>, <class 'transformers.modeling_albert.AlbertForMultipleChoice'>, <class 'transformers.modeling_albert.AlbertForPreTraining'>, <class 'transformers.modeling_albert.AlbertForQuestionAnswering'>, <class 'transformers.modeling_albert.AlbertForSequenceClassification'>, <class 'transformers.modeling_albert.AlbertForTokenClassification'>, <class 'transformers.modeling_albert.AlbertModel'>, <class 'transformers.modeling_albert.AlbertPreTrainedModel'>, <class 'transformers.modeling_auto.AutoModel'>, <class 'transformers.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.modeling_auto.AutoModelForMaskedLM'>, <class 'transformers.modeling_auto.AutoModelForMultipleChoice'>, <class 'transformers.modeling_auto.AutoModelForPreTraining'>, <class 'transformers.modeling_auto.AutoModelForQuestionAnswering'>, <class 'transformers.modeling_auto.AutoModelForSeq2SeqLM'>, <class 'transformers.modeling_auto.AutoModelForSequenceClassification'>, <class 'transformers.modeling_auto.AutoModelForTokenClassification'>, <class 'transformers.modeling_auto.AutoModelWithLMHead'>, <class 'transformers.modeling_bart.BartForConditionalGeneration'>, <class 'transformers.modeling_bart.BartForQuestionAnswering'>, <class 'transformers.modeling_bart.BartForSequenceClassification'>, <class 'transformers.modeling_bart.BartModel'>, <class 'transformers.modeling_bert.BertForMaskedLM'>, <class 'transformers.modeling_bert.BertForMultipleChoice'>, <class 'transformers.modeling_bert.BertForNextSentencePrediction'>, <class 'transformers.modeling_bert.BertForPreTraining'>, <class 'transformers.modeling_bert.BertForQuestionAnswering'>, <class 'transformers.modeling_bert.BertForSequenceClassification'>, <class 'transformers.modeling_bert.BertForTokenClassification'>, <class 'transformers.modeling_bert_generation.BertGenerationDecoder'>, <class 'transformers.modeling_bert_generation.BertGenerationEncoder'>, <class 'transformers.modeling_bert.BertLMHeadModel'>, <class 'transformers.modeling_bert.BertLayer'>, <class 'transformers.modeling_bert.BertModel'>, <class 'transformers.modeling_bert.BertPreTrainedModel'>, <class 'transformers.modeling_blenderbot.BlenderbotForConditionalGeneration'>, <class 'transformers.modeling_ctrl.CTRLLMHeadModel'>, <class 'transformers.modeling_ctrl.CTRLModel'>, <class 'transformers.modeling_ctrl.CTRLPreTrainedModel'>, <class 'transformers.modeling_camembert.CamembertForCausalLM'>, <class 'transformers.modeling_camembert.CamembertForMaskedLM'>, <class 'transformers.modeling_camembert.CamembertForMultipleChoice'>, <class 'transformers.modeling_camembert.CamembertForQuestionAnswering'>, <class 'transformers.modeling_camembert.CamembertForSequenceClassification'>, <class 'transformers.modeling_camembert.CamembertForTokenClassification'>, <class 'transformers.modeling_camembert.CamembertModel'>, <class 'transformers.modeling_utils.Conv1D'>, <class 'transformers.modeling_dpr.DPRContextEncoder'>, <class 'transformers.modeling_dpr.DPRPretrainedContextEncoder'>, <class 'transformers.modeling_dpr.DPRPretrainedQuestionEncoder'>, <class 'transformers.modeling_dpr.DPRPretrainedReader'>, <class 'transformers.modeling_dpr.DPRQuestionEncoder'>, <class 'transformers.modeling_dpr.DPRReader'>, <class 'transformers.modeling_deberta.DebertaForSequenceClassification'>, <class 'transformers.modeling_deberta.DebertaModel'>, <class 'transformers.modeling_deberta.DebertaPreTrainedModel'>, <class 'transformers.modeling_distilbert.DistilBertForMaskedLM'>, <class 'transformers.modeling_distilbert.DistilBertForMultipleChoice'>, <class 'transformers.modeling_distilbert.DistilBertForQuestionAnswering'>, <class 'transformers.modeling_distilbert.DistilBertForSequenceClassification'>, <class 'transformers.modeling_distilbert.DistilBertForTokenClassification'>, <class 'transformers.modeling_distilbert.DistilBertModel'>, <class 'transformers.modeling_distilbert.DistilBertPreTrainedModel'>, <class 'transformers.modeling_electra.ElectraForMaskedLM'>, <class 'transformers.modeling_electra.ElectraForMultipleChoice'>, <class 'transformers.modeling_electra.ElectraForPreTraining'>, <class 'transformers.modeling_electra.ElectraForQuestionAnswering'>, <class 'transformers.modeling_electra.ElectraForSequenceClassification'>, <class 'transformers.modeling_electra.ElectraForTokenClassification'>, <class 'transformers.modeling_electra.ElectraModel'>, <class 'transformers.modeling_electra.ElectraPreTrainedModel'>, <class 'transformers.modeling_encoder_decoder.EncoderDecoderModel'>, <class 'transformers.modeling_fsmt.FSMTForConditionalGeneration'>, <class 'transformers.modeling_fsmt.FSMTModel'>, <class 'transformers.modeling_flaubert.FlaubertForMultipleChoice'>, <class 'transformers.modeling_flaubert.FlaubertForQuestionAnswering'>, <class 'transformers.modeling_flaubert.FlaubertForQuestionAnsweringSimple'>, <class 'transformers.modeling_flaubert.FlaubertForSequenceClassification'>, <class 'transformers.modeling_flaubert.FlaubertForTokenClassification'>, <class 'transformers.modeling_flaubert.FlaubertModel'>, <class 'transformers.modeling_flaubert.FlaubertWithLMHeadModel'>, <class 'transformers.modeling_funnel.FunnelBaseModel'>, <class 'transformers.modeling_funnel.FunnelForMaskedLM'>, <class 'transformers.modeling_funnel.FunnelForMultipleChoice'>, <class 'transformers.modeling_funnel.FunnelForPreTraining'>, <class 'transformers.modeling_funnel.FunnelForQuestionAnswering'>, <class 'transformers.modeling_funnel.FunnelForSequenceClassification'>, <class 'transformers.modeling_funnel.FunnelForTokenClassification'>, <class 'transformers.modeling_funnel.FunnelModel'>, <class 'transformers.modeling_gpt2.GPT2DoubleHeadsModel'>, <class 'transformers.modeling_gpt2.GPT2ForSequenceClassification'>, <class 'transformers.modeling_gpt2.GPT2LMHeadModel'>, <class 'transformers.modeling_gpt2.GPT2Model'>, <class 'transformers.modeling_gpt2.GPT2PreTrainedModel'>, <class 'transformers.modeling_layoutlm.LayoutLMForMaskedLM'>, <class 'transformers.modeling_layoutlm.LayoutLMForTokenClassification'>, <class 'transformers.modeling_layoutlm.LayoutLMModel'>, <class 'transformers.modeling_longformer.LongformerForMaskedLM'>, <class 'transformers.modeling_longformer.LongformerForMultipleChoice'>, <class 'transformers.modeling_longformer.LongformerForQuestionAnswering'>, <class 'transformers.modeling_longformer.LongformerForSequenceClassification'>, <class 'transformers.modeling_longformer.LongformerForTokenClassification'>, <class 'transformers.modeling_longformer.LongformerModel'>, <class 'transformers.modeling_longformer.LongformerSelfAttention'>, <class 'transformers.modeling_lxmert.LxmertEncoder'>, <class 'transformers.modeling_lxmert.LxmertForPreTraining'>, <class 'transformers.modeling_lxmert.LxmertForQuestionAnswering'>, <class 'transformers.modeling_lxmert.LxmertModel'>, <class 'transformers.modeling_lxmert.LxmertPreTrainedModel'>, <class 'transformers.modeling_lxmert.LxmertVisualFeatureEncoder'>, <class 'transformers.modeling_lxmert.LxmertXLayer'>, <class 'transformers.modeling_mbart.MBartForConditionalGeneration'>, <class 'transformers.modeling_mmbt.MMBTForClassification'>, <class 'transformers.modeling_mmbt.MMBTModel'>, <class 'transformers.modeling_marian.MarianMTModel'>, <class 'transformers.modeling_mobilebert.MobileBertForMaskedLM'>, <class 'transformers.modeling_mobilebert.MobileBertForMultipleChoice'>, <class 'transformers.modeling_mobilebert.MobileBertForNextSentencePrediction'>, <class 'transformers.modeling_mobilebert.MobileBertForPreTraining'>, <class 'transformers.modeling_mobilebert.MobileBertForQuestionAnswering'>, <class 'transformers.modeling_mobilebert.MobileBertForSequenceClassification'>, <class 'transformers.modeling_mobilebert.MobileBertForTokenClassification'>, <class 'transformers.modeling_mobilebert.MobileBertLayer'>, <class 'transformers.modeling_mobilebert.MobileBertModel'>, <class 'transformers.modeling_mobilebert.MobileBertPreTrainedModel'>, <class 'transformers.modeling_mmbt.ModalEmbeddings'>, <class 'transformers.modeling_openai.OpenAIGPTDoubleHeadsModel'>, <class 'transformers.modeling_openai.OpenAIGPTForSequenceClassification'>, <class 'transformers.modeling_openai.OpenAIGPTLMHeadModel'>, <class 'transformers.modeling_openai.OpenAIGPTModel'>, <class 'transformers.modeling_openai.OpenAIGPTPreTrainedModel'>, <class 'transformers.modeling_pegasus.PegasusForConditionalGeneration'>, <class 'transformers.modeling_utils.PreTrainedModel'>, <class 'transformers.modeling_bart.PretrainedBartModel'>, <class 'transformers.modeling_fsmt.PretrainedFSMTModel'>, <class 'transformers.modeling_prophetnet.ProphetNetDecoder'>, <class 'transformers.modeling_prophetnet.ProphetNetEncoder'>, <class 'transformers.modeling_prophetnet.ProphetNetForCausalLM'>, <class 'transformers.modeling_prophetnet.ProphetNetForConditionalGeneration'>, <class 'transformers.modeling_prophetnet.ProphetNetModel'>, <class 'transformers.modeling_prophetnet.ProphetNetPreTrainedModel'>, <class 'transformers.modeling_rag.RagModel'>, <class 'transformers.modeling_rag.RagSequenceForGeneration'>, <class 'transformers.modeling_rag.RagTokenForGeneration'>, <class 'transformers.modeling_reformer.ReformerAttention'>, <class 'transformers.modeling_reformer.ReformerForMaskedLM'>, <class 'transformers.modeling_reformer.ReformerForQuestionAnswering'>, <class 'transformers.modeling_reformer.ReformerForSequenceClassification'>, <class 'transformers.modeling_reformer.ReformerLayer'>, <class 'transformers.modeling_reformer.ReformerModel'>, <class 'transformers.modeling_reformer.ReformerModelWithLMHead'>, <class 'transformers.modeling_retribert.RetriBertModel'>, <class 'transformers.modeling_retribert.RetriBertPreTrainedModel'>, <class 'transformers.modeling_roberta.RobertaForCausalLM'>, <class 'transformers.modeling_roberta.RobertaForMaskedLM'>, <class 'transformers.modeling_roberta.RobertaForMultipleChoice'>, <class 'transformers.modeling_roberta.RobertaForQuestionAnswering'>, <class 'transformers.modeling_roberta.RobertaForSequenceClassification'>, <class 'transformers.modeling_roberta.RobertaForTokenClassification'>, <class 'transformers.modeling_roberta.RobertaModel'>, <class 'transformers.modeling_squeezebert.SqueezeBertForMaskedLM'>, <class 'transformers.modeling_squeezebert.SqueezeBertForMultipleChoice'>, <class 'transformers.modeling_squeezebert.SqueezeBertForQuestionAnswering'>, <class 'transformers.modeling_squeezebert.SqueezeBertForSequenceClassification'>, <class 'transformers.modeling_squeezebert.SqueezeBertForTokenClassification'>, <class 'transformers.modeling_squeezebert.SqueezeBertModel'>, <class 'transformers.modeling_squeezebert.SqueezeBertModule'>, <class 'transformers.modeling_squeezebert.SqueezeBertPreTrainedModel'>, <class 'transformers.modeling_t5.T5ForConditionalGeneration'>, <class 'transformers.modeling_t5.T5Model'>, <class 'transformers.modeling_t5.T5PreTrainedModel'>, <class 'transformers.modeling_transfo_xl.TransfoXLLMHeadModel'>, <class 'transformers.modeling_transfo_xl.TransfoXLModel'>, <class 'transformers.modeling_transfo_xl.TransfoXLPreTrainedModel'>, <class 'transformers.modeling_xlm.XLMForMultipleChoice'>, <class 'transformers.modeling_xlm.XLMForQuestionAnswering'>, <class 'transformers.modeling_xlm.XLMForQuestionAnsweringSimple'>, <class 'transformers.modeling_xlm.XLMForSequenceClassification'>, <class 'transformers.modeling_xlm.XLMForTokenClassification'>, <class 'transformers.modeling_xlm.XLMModel'>, <class 'transformers.modeling_xlm.XLMPreTrainedModel'>, <class 'transformers.modeling_xlm_prophetnet.XLMProphetNetDecoder'>, <class 'transformers.modeling_xlm_prophetnet.XLMProphetNetEncoder'>, <class 'transformers.modeling_xlm_prophetnet.XLMProphetNetForCausalLM'>, <class 'transformers.modeling_xlm_prophetnet.XLMProphetNetForConditionalGeneration'>, <class 'transformers.modeling_xlm_prophetnet.XLMProphetNetModel'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForCausalLM'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForMaskedLM'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForMultipleChoice'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForQuestionAnswering'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForSequenceClassification'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForTokenClassification'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaModel'>, <class 'transformers.modeling_xlm.XLMWithLMHeadModel'>, <class 'transformers.modeling_xlnet.XLNetForMultipleChoice'>, <class 'transformers.modeling_xlnet.XLNetForQuestionAnswering'>, <class 'transformers.modeling_xlnet.XLNetForQuestionAnsweringSimple'>, <class 'transformers.modeling_xlnet.XLNetForSequenceClassification'>, <class 'transformers.modeling_xlnet.XLNetForTokenClassification'>, <class 'transformers.modeling_xlnet.XLNetLMHeadModel'>, <class 'transformers.modeling_xlnet.XLNetModel'>, <class 'transformers.modeling_xlnet.XLNetPreTrainedModel'>]
{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_models(arch='bert'))
[<class 'transformers.modeling_bert.BertForMaskedLM'>, <class 'transformers.modeling_bert.BertForMultipleChoice'>, <class 'transformers.modeling_bert.BertForNextSentencePrediction'>, <class 'transformers.modeling_bert.BertForPreTraining'>, <class 'transformers.modeling_bert.BertForQuestionAnswering'>, <class 'transformers.modeling_bert.BertForSequenceClassification'>, <class 'transformers.modeling_bert.BertForTokenClassification'>, <class 'transformers.modeling_bert.BertLMHeadModel'>, <class 'transformers.modeling_bert.BertLayer'>, <class 'transformers.modeling_bert.BertModel'>, <class 'transformers.modeling_bert.BertPreTrainedModel'>]
{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_models(task='TokenClassification'))
[<class 'transformers.modeling_albert.AlbertForTokenClassification'>, <class 'transformers.modeling_auto.AutoModelForTokenClassification'>, <class 'transformers.modeling_bert.BertForTokenClassification'>, <class 'transformers.modeling_camembert.CamembertForTokenClassification'>, <class 'transformers.modeling_distilbert.DistilBertForTokenClassification'>, <class 'transformers.modeling_electra.ElectraForTokenClassification'>, <class 'transformers.modeling_flaubert.FlaubertForTokenClassification'>, <class 'transformers.modeling_funnel.FunnelForTokenClassification'>, <class 'transformers.modeling_layoutlm.LayoutLMForTokenClassification'>, <class 'transformers.modeling_longformer.LongformerForTokenClassification'>, <class 'transformers.modeling_mobilebert.MobileBertForTokenClassification'>, <class 'transformers.modeling_roberta.RobertaForTokenClassification'>, <class 'transformers.modeling_squeezebert.SqueezeBertForTokenClassification'>, <class 'transformers.modeling_xlm.XLMForTokenClassification'>, <class 'transformers.modeling_xlm_roberta.XLMRobertaForTokenClassification'>, <class 'transformers.modeling_xlnet.XLNetForTokenClassification'>]
{% endraw %} {% raw %}
print(BLURR_MODEL_HELPER.get_models(arch='bert', task='TokenClassification'))
[<class 'transformers.modeling_bert.BertForTokenClassification'>]
{% endraw %} {% raw %}

ModelHelper.get_classes_for_model[source]

ModelHelper.get_classes_for_model(model_name_or_cls)

Get tokenizers, config, and model for a given model name / class

{% endraw %} {% raw %}
config, tokenizers, model = BLURR_MODEL_HELPER.get_classes_for_model('RobertaForSequenceClassification')

print(config)
print(tokenizers[0])
print(model)
<class 'transformers.configuration_roberta.RobertaConfig'>
<class 'transformers.tokenization_roberta.RobertaTokenizer'>
<class 'transformers.modeling_roberta.RobertaForSequenceClassification'>
{% endraw %} {% raw %}
config, tokenizers, model = BLURR_MODEL_HELPER.get_classes_for_model(DistilBertModel)

print(config)
print(tokenizers[0])
print(model)
<class 'transformers.configuration_distilbert.DistilBertConfig'>
<class 'transformers.tokenization_distilbert.DistilBertTokenizer'>
<class 'transformers.modeling_distilbert.DistilBertModel'>
{% endraw %} {% raw %}

ModelHelper.get_model_architecture[source]

ModelHelper.get_model_architecture(model_name_or_enum)

Get the architecture for a given model name / enum

{% endraw %} {% raw %}
BLURR_MODEL_HELPER.get_model_architecture('RobertaForSequenceClassification')
'roberta'
{% endraw %}

Methods for loading pre-trained (configs, tokenizer, model) hugginface classes

{% raw %}

ModelHelper.get_hf_objects[source]

ModelHelper.get_hf_objects(pretrained_model_name_or_path, task=None, config=None, tokenizer_cls=None, model_cls=None, config_kwargs={}, tokenizer_kwargs={}, model_kwargs={}, cache_dir=None)

Returns the architecture (str), config (obj), tokenizer (obj), and model (obj) given at minimum a pre-trained model name or path. Specify a task to ensure the right "AutoModelFor" is used to create the model.

Optionally, you can pass a config (obj), tokenizer (class), and/or model (class) (along with any related kwargs for each) to get as specific as you want w/r/t what huggingface objects are returned.

{% endraw %} {% raw %}
arch, config, tokenizer, model = BLURR_MODEL_HELPER.get_hf_objects("bert-base-cased-finetuned-mrpc",
                                                                   task=HF_TASKS_AUTO.LMHead)

print(arch)
print(type(config))
print(type(tokenizer))
print(type(model))
bert
<class 'transformers.configuration_bert.BertConfig'>
<class 'transformers.tokenization_bert.BertTokenizer'>
<class 'transformers.modeling_bert.BertForMaskedLM'>
{% endraw %} {% raw %}
arch, tokenizer, config, model = BLURR_MODEL_HELPER.get_hf_objects("fmikaelian/flaubert-base-uncased-squad",
                                                                   task=HF_TASKS_AUTO.QuestionAnswering)

print(arch)
print(type(config))
print(type(tokenizer))
print(type(model))
flaubert
<class 'transformers.tokenization_flaubert.FlaubertTokenizer'>
<class 'transformers.configuration_flaubert.FlaubertConfig'>
<class 'transformers.modeling_flaubert.FlaubertForQuestionAnsweringSimple'>
{% endraw %} {% raw %}
arch, tokenizer, config, model = BLURR_MODEL_HELPER.get_hf_objects("bert-base-cased-finetuned-mrpc",
                                                                   config=None,
                                                                   tokenizer_cls=BertTokenizer, 
                                                                   model_cls=BertForNextSentencePrediction)
print(arch)
print(type(config))
print(type(tokenizer))
print(type(model))
bert
<class 'transformers.tokenization_bert.BertTokenizer'>
<class 'transformers.configuration_bert.BertConfig'>
<class 'transformers.modeling_bert.BertForNextSentencePrediction'>
{% endraw %}

Cleanup