FAIR for Machine Learning Models

This rubric consists of assessment metrics that evaluate the FAIR maturity of ML models. The metrics are proposed based on relevant and well-established initiatives. The metrics of this rubric rely on a hybrid assessment method since they contain both manual and automated assessment metrics. In this way, the results from the automated (conducted via F-UJI) and the manual assessments are included in the same (FAIRshake) evaluation rubric and form the overall FAIR assessment score for an ML model.

License: https://creativecommons.org/licenses/by/4.0/

Tags: FAIR machine learning model FAIR assessment NFDI4DataScience

URL(s):

View Assessments

Associated Metrics (27)

Formal knowledge representation language

yesnobut

ML-I1-01M: Metadata is represented using a formal knowledge representation language. Test: 1) Par...

Semantic resources

yesnobut

ML-I2-01M: Metadata uses semantic resources. Test: 1) Vocabulary namespace URIs can be identified...

Qualified references

text

ML-I3-01M: Metadata includes links between the ML model and its related entities. Test: 1) Relate...

ML model content specification

yesnobut

ML-R1-01M: Metadata specifies the content of the ML model. Test: 1) Minimal information about ava...

License information

yesnobut

ML-R1.1-01M: Metadata includes license information under which ML model can be reused. Test: 1) Lic...

Provenance information

yesnobut

ML-R1.2-01M: Metadata includes provenance information about ML model creation or generation. Test...

ML model type

yesnobut

ML-R1.3-01M: Metadata includes information about ML model type. Test: 1) The model type is descri...

Dataset training, validation and testing

yesnobut

ML-R1.4-01M: Metadata includes information about the dataset used for ML model training, testing and...

ML model dataset collection

yesnobut

ML-R1.4-02M: Metadata includes information about ML model dataset collection process. Test: 1) T...

ML model source code

yesnobut

ML-R1.5-01M: Metadata includes information about ML model source code. Test: 1) The code to downl...

ML model hyperparameters

yesnobut

ML-R1.6-01M: Metadata describes ML model hyperparameters.

Create New Metric

Associated Digital Objects (6)

WordPair-CNN

any

Code repository for discourse relation prediction using word pair CNNs.

German Zeroshot

This model has GBERT Large as base model and fine-tuned it on xnli de dataset.

Zero-Shot Classification Transformers PyTorch JAX xnli multilingual bert text-classification nli de Inference Endpoints

German BERT large

A German BERT language model trained collaboratively by the makers of the original German BERT (aka ...

Fill-Mask Transformers PyTorch TensorFlow Safetensors 4 datasets German Inference Endpoints

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

Deep Neural Networks (DNNs) are known to be vulnerable to backdoor attacks. In Natural Language Proc...

XLM-RoBERTa (base-sized model)

XLM-RoBERTa model pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. It was...

Fill-Mask Transformers PyTorch TensorFlow JAX ONNX Safetensors 94 languages xlm-roberta exbert Inference Endpoints arxiv: 1911.02116 License: mit

RoBERTa

The RoBERTa model was proposed in RoBERTa: A Robustly Optimized BERT Pretraining Approach by Yinhan ...

Text Classification Token Classification Fill-Mask Question Answering

Create New Digital Object