Skip to content

Apparently not possible to create models that are both multi class and multi label #636

Description

@peter-clark-moj

I've been looking for ages for a way to train multi label - multi class setfit models and I've reached the conclusion that this is either not supported or there is a desperate need for it to be properly documented. Can I ask if this is a feature that could be considered? I include a minimal example bellow of my failed attempts.

from datasets import Dataset
from setfit import SetFitModel, Trainer, TrainingArguments
import pandas as pd

dfA = [["test",["A","x"]],["not",["B","y"]],["yes",["C","z"]]]
dfB = [["test",["A",True]], ["not",["B",True]],["yes",["C",False]]]


args = TrainingArguments(num_epochs=1)

for t in ["multi-output","one-vs-rest","classifier-chain"]:
    model = SetFitModel.from_pretrained(
        "sentence-transformers/paraphrase-mpnet-base-v2",
        multi_target_strategy=t)
    for df in [dfA,dfB]:
        try:
            dataset = Dataset.from_pandas(pd.DataFrame(df,
                                                    columns=["text","label"]))
            trainer = Trainer(
                model=model,
                args=args,
                train_dataset=dataset
            )
            trainer.train()
        except:
            print("fail")

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions