TypeError: convert_examples_to_features() got an unexpected keyword argument 'sep_token_extra'

[Mahhos]

I am getting this error when I run:
if args['do_train']: train_dataset = load_and_cache_examples(task, tokenizer) global_step, tr_loss = train(train_dataset, model, tokenizer) logger.info(" global_step = %s, average loss = %s", global_step, tr_loss

The error is raised in pad_token_segment_id=4 if args['model_type'] in ['xlnet'] else 0) line of load_and_cache_examples(task, tokenizer, evaluate) function. Do you have any idea what is wrong?
I am just running the cells in the run_model.ipynb.

[ThilinaRajapakse]

I can't reproduce this. Are you using the latest version of the repo?

[Mahhos]

I downloaded the repo and run jupyter notebook. data_prep works well. Then I ran run_model.ipynb and it keeps raising this error.
TypeError: convert_examples_to_features() got an unexpected keyword argument 'sep_token_extra

[Mahhos]

I just figured out that the generated files do not have the ordering as train_df = pd.DataFrame({ 'id':range(len(train_df)), 'label':train_df[0], 'alpha':['a']*train_df.shape[0], 'text': train_df[1].replace(r'\n', ' ', regex=True) }) and they are in alphabetical order. I was thinking that it might be a python version issue. I am using python 3.7.4. Do you know what might be wrong?

[ThilinaRajapakse]

It might be an issue with pandas. Try updating pandas to the latest version.

[Mahhos]

I resolved the ordering issue. But I am still getting the same error. TypeError: convert_examples_to_features() got an unexpected keyword argument 'sep_token_extra. May I know the exact python version you used?

[ThilinaRajapakse]

Python 3.7. I can't remember the exact version, although I can check later.

The function that is throwing the error is in the utils.py file. Please check whether this keyword is given in the parameter lost.

Also, please do consider switching to the Simple Transformers library linked in the readme unless there is a reason for using this particular repo. This repo relies on an outdated version of the Hugging Face library.