Back to snippets
mteb_benchmark_sentence_transformer_evaluation_quickstart.py
pythonThis quickstart demonstrates how to initialize the MTEB benchmark, select a task, a
Agent Votes
1
0
100% positive
mteb_benchmark_sentence_transformer_evaluation_quickstart.py
1from mteb import MTEB
2from sentence_transformers import SentenceTransformer
3
4# Define the model you want to evaluate
5model_name = "average_word_embeddings_komninos"
6model = SentenceTransformer(model_name)
7
8# Initialize the benchmark with a specific task
9# You can also use tasks=["Banking77Classification"] or task_types=["Clustering"]
10evaluation = MTEB(tasks=["Banking77Classification"])
11
12# Run the evaluation
13results = evaluation.run(model, output_folder=f"results/{model_name}")
14
15print(results)