add results for tencent/EVIE-8B and tencent/EVIE-4.5B - #715
Conversation
Model Results ComparisonReference models: Results for
|
| task_name | tencent/EVIE-4.5B | Max result | Model with max result | In Training Data |
|---|---|---|---|---|
| Vidore3NuclearRetrieval | .564 | .549 | vultr/VultronRetrieverCore-Qwen3.5-4.5B | False |
| Vidore3TelecomRetrieval | .735 | .733 | webAI-Official/webAI-ColVec1.1-8b | False |
| Average | .650 | .641 | nan | - |
Model have high performance on these tasks: Vidore3TelecomRetrieval,Vidore3NuclearRetrieval
Results for tencent/EVIE-8B
| task_name | tencent/EVIE-8B | Max result | Model with max result | In Training Data |
|---|---|---|---|---|
| Vidore3NuclearRetrieval | .566 | .549 | vultr/VultronRetrieverCore-Qwen3.5-4.5B | False |
| Vidore3TelecomRetrieval | .743 | .733 | webAI-Official/webAI-ColVec1.1-8b | False |
| Average | .654 | .641 | nan | - |
Model have high performance on these tasks: Vidore3TelecomRetrieval,Vidore3NuclearRetrieval
|
Hi @KennethEnevoldsen, thanks again for running and adding the private Nuclear/Telecom results! 🙏 This PR is marked ready for review and closes #5451. Whenever you (or @Samoed) have a moment, would it be possible to merge it so the ViDoRe V3 entry is complete (8 public domains + the 2 private tasks)? Happy to help if anything else is needed on our side. Thank you! |
|
Sorry @officea1t I see that I wasn't quite clear in my communication. I was waiting for a review from you guys stating that you were happy to merge it (I assume you are from the message) |
Note see embeddings-benchmark/mteb#5451 for discussion on implementation
closes embeddings-benchmark/mteb#5451
Checklist
mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here