SimpleTransformers 错误：VersionConflict：tokenizers==0.9.4？我该如何解决？

问题描述

我正在尝试从他们在 google colab 上的网站上执行 simpletransformers 示例。

示例：

from simpletransformers.classification import ClassificationModel,ClassificationArgs
import pandas as pd
import logging


logging.basicConfig(level=logging.INFO)
transformers_logger = logging.getLogger("transformers")
transformers_logger.setLevel(logging.WARNING)

# Preparing train data
train_data = [
    ["aragorn was the heir of Isildur",1],["Frodo was the heir of Isildur",0],]
train_df = pd.DataFrame(train_data)
train_df.columns = ["text","labels"]

# Preparing eval data
eval_data = [
    ["Theoden was the king of Rohan",["Merry was the king of Rohan",]
eval_df = pd.DataFrame(eval_data)
eval_df.columns = ["text","labels"]

# Optional model configuration
model_args = ClassificationArgs(num_train_epochs=1)

# Create a ClassificationModel
model = ClassificationModel(
    "roberta","roberta-base",args=model_args
)

# Train the model
model.train_model(train_df)

# Evaluate the model
result,model_outputs,wrong_predictions = model.eval_model(eval_df)

# Make predictions with the model
predictions,raw_outputs = model.predict(["Sam was a Wizard"])

但它给了我以下错误：

VersionConflict: tokenizers==0.9.4 是正常的需要此模块的功能，但发现标记器== 0.10.0。尝试：点子如果您正在工作，请安装转换器 -U 或 pip install -e '.[dev]' 与 git master

我尝试过 !pip install transformers -U 甚至 !pip install tokenizers==0.9.4，但一直出现相同的错误。我之前执行过这段代码，它运行起来很有趣，但现在却出现了上述错误。

解决方法

我把这个放在这里以防有人遇到同样的问题。我得到了创作者本人的帮助。

Workaround:
Install tokenizers==0.9.4 before install simpletransformers

In Colab for example;

!pip install tokenizers==0.9.4
!pip install simpletransformers

https://github.com/ThilinaRajapakse/simpletransformers/issues/950