You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The basic_usage.Rmd vignette still has to do a manual tokenize step. That shouldn’t be necessary.
Right now we only auto-tokenize with {luz} fit/predict (and the callback telling it to do that). We need a clean way to tokenize when we use pretrained berts more directly, like we do here.
If that worked, we could create an (untokenized) dataset, then use it in the model, at which point it would be updated to match the model (or we call a helper or whatever). And then tidy_bert_output() could accept a dataset_bert_pretrained for its second argument.
The text was updated successfully, but these errors were encountered:
The basic_usage.Rmd vignette still has to do a manual tokenize step. That shouldn’t be necessary.
Right now we only auto-tokenize with {luz} fit/predict (and the callback telling it to do that). We need a clean way to tokenize when we use pretrained berts more directly, like we do here.
If that worked, we could create an (untokenized) dataset, then use it in the model, at which point it would be updated to match the model (or we call a helper or whatever). And then tidy_bert_output() could accept a dataset_bert_pretrained for its second argument.
The text was updated successfully, but these errors were encountered: