FAIR for Machine Learning Models

This rubric consists of assessment metrics that evaluate the FAIR maturity of ML models. The metrics are proposed based on relevant and well-established initiatives. The metrics of this rubric rely on a hybrid assessment method since they contain both manual and automated assessment metrics. In this way, the results from the automated (conducted via F-UJI) and the manual assessments are included in the same (FAIRshake) evaluation rubric and form the overall FAIR assessment score for an ML model.

License: https://creativecommons.org/licenses/by/4.0/

Tags: FAIR machine learning model FAIR assessment NFDI4DataScience

URL(s):

View Assessments

Associated Metrics (27)

Global, unique identifier

yesnomaybe

ML-F1-01S: ML model is assigned a globally unique identifier. Test: 1) Identifier is not resolvab...

Persistent identifier

yesnomaybe

ML-F1-02S: ML model is assigned a persistent identifier. Test: 1) Identifier follows a defined pe...

ML model versions

yesnobut

ML-F1.1-01S: Different versions of ML model are assigned different identifiers. Test: 1) Identifi...

Core descriptive elements

yesnobut

ML-F2-01M: Metadata includes descriptive core elements to support data findability. Test: 1) Som...

ML model identifier in the metadata

yesnobut

ML-F3-01M: Metadata includes the identifier of the ML model it describes. Test: 1) Metadata conta...

FAIR Metadata

yesnobut

ML-F4-01M: Metadata is offered in such a way that it can be retrieved by machines. Test: 1) Metad...

Access conditions

yesnobut

ML-A1-01M: Metadata contains access level and access conditions of the ML model. Test: 1) Informa...

Standards protocols for metadata access

yesnobut

ML-A1-02M: ML model metadata is accessible through a standardized communication protocol. Test: 1...

Standard protocols for data access

yesnobut

ML-A1-03S: ML model is accessible through a standardized communication protocol. Test: 1) Metadat...

Model authentication and authentication

yesnobut

ML-A1.1-01S: ML model is accessible through an access protocol that supports authentication and auth...

Metadata preservation

url

ML-A2-01M: Metadata remains available even if the ML model is no longer available. Test: 1) Progr...

Create New Metric

Associated Digital Objects (6)

WordPair-CNN

any

Code repository for discourse relation prediction using word pair CNNs.

German Zeroshot

This model has GBERT Large as base model and fine-tuned it on xnli de dataset.

Zero-Shot Classification Transformers PyTorch JAX xnli multilingual bert text-classification nli de Inference Endpoints

German BERT large

A German BERT language model trained collaboratively by the makers of the original German BERT (aka ...

Fill-Mask Transformers PyTorch TensorFlow Safetensors 4 datasets German Inference Endpoints

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Proc...

XLM-RoBERTa (base-sized model)

XLM-RoBERTa model pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. It was...

Fill-Mask Transformers PyTorch TensorFlow JAX ONNX Safetensors 94 languages xlm-roberta exbert Inference Endpoints arxiv: 1911.02116 License: mit

RoBERTa

The RoBERTa model was proposed in RoBERTa: A Robustly Optimized BERT Pretraining Approach by Yinhan ...

Text Classification Token Classification Fill-Mask Question Answering

Create New Digital Object