Output

Output Assessment JSON format¶

Version : 1.0.0

The structure of the JSON output from the assessment is as following

{

  "components": [

  …

  ],

  "overall\_score": {

  "description": "Formula: score of each principle / total score",

  "score": 51.04

  }

}

The first element of the JSON is a collection of the class \<Component>.

The second element is an overall score of the Research Object. This score is calculated based on the partial scores. For the moment we used the policy defined in the description file.

Property	Mandatory	Expected value	Definition
description	Yes	String	Description of the policy used to calculate the overall score
score	Yes	Number	Value of the score

Component¶

A Research Object is composed of a collection of components/resources. Each component represents a digital object of the Research Object.

In a ro-crate metadata file, the first component is the Research Object itself, it means, the assessment of the metadata of the Research Object using the tool F-UJI. Note that this structure is the same in the rest of the RO resources

{

  "name": "Global Water Watch using OpenEO on C-Scale",

  "identifier": "https://w3id.org/ro-id/813ac793-f4ba-46d1-9d5e-6552227de7a0",

  "type": "ro-crate",

  "tool-used": [

  "F-uji",

  "ro-crate-metadata"

  ],

  "checks": [

  …

  ],

  "score": {

  …

  }

}

Property	Mandatory	Expected value	Definition
name	Yes	String	Title of the RO present in the ro-crate metadata file.
identifier	Yes	String	String that identifies the RO in the metadata file. Normally is assigned by the tool used to publish the RO
type	Yes	Enumerate(ro-crate	software
tool-used	Yes	Enumerate(ro-crate-metaF-UJI	somef
checks	Yes	\<Check>	It represents a collection of tests run for each principle.
score	Yes	\<Score>	It represents the partial scores of a component/resource

Check¶

Over a component, a set of tests are run to test each FAIR principle.

The following output is an example of a check element. There is a check element for each principle analyzed.

{

  "principle\_id": "F1.2",

  "category\_id": "Findable",

  "title": "Data is assigned a persistent identifier.",

  "status": "pass",

  "sources": [

  …

  ],

  "score": 1,

  "total\_score": 1

}

The fields of the check element are described in the following table (based on F-UJI notation):

Property	Mandatory	Expected value	Definition
principle_id	Yes	String	It is the identifier of the principle. We have followed the F-UJI notation
category	Yes	Enumerate(Findable	Accessible
title	Yes	String	It is the title of the principle. You can find their description in the section Research Object metadata
status	Yes	Enumerate(pass	fail)

source	Yes	[\<Source>]	It is the score of the assessment. It is the maximum between the scores given by the tools used.
score	Yes	Number	It represents the partial scores of a component/resource
total_score	Yes	Number	Is the maximum value of the score. Note that you can pass the test even if you don't have the maximum score (same happens with the F-UJI tool).

Source¶

Tests and scores obtained by the tests depend on the nature of the component. This nature determines the tool used to make the assessment.

The following output is an example of the sources element:

{

  "source": "F-UJI",

  "id": "F-UJI",

  "total\_passed\_tests": 0,

  "total\_tests\_run": 2,

  "explanation": [

    "FAIL: Identifier follows a defined persistent identifier syntax",

    "FAIL: Persistent identifier is resolvable"

  ],

  "assessment": "fail",

  "score": 0,

  "total\_score": 1

  },

  {

  "source": "ro-crate",

  "score": 1,

  "total\_tests\_run": 2,

  "total\_passed\_tests": 2,

  "total\_score": 1,

  "assessment": "pass",

  "explanations": [

    "PASS: Identifier follows a defined persistent identifier syntax",

    "PASS: Persistent identifier is resolvable"

  ]

}

The fields of the check element are the following:

Property	Mandatory	Expected value	Definition
source	Yes	String	It is the tool used for the assessment.
id	Yes	Enumerate(ro-crate-metaF-UJI	somef
total_passed_tests	Yes	Number	It is the number of tests passed in this principle. Note that for each principle, there is a battery of tests to analyze different aspects of the principle.

total_test_run	Yes	Number	It is the number of tests run.

explanations	Yes	String	It is a collection of messages with the results of each test.
assessment	Yes	Enumerate (pass	fail)
score	Yes	Number	It is the score of the assessment. It is the maximum between the scores given by the tools used.
total_score	Yes	Number	Is the maximum value of the score. Note that you can pass the test even if you don't have the maximum score (same happens with the F-UJI tool).

You can see in this example that we use two tools for the assessment. This is because the tools we use analyze the resource using the identifier, but not analyze the information provided by the ro-crate-metadata file, that is the file used by our service. For that, we have the internal tool ro-crate-metadata to analyze the information of this file, and complement the analysis done by the external tool. As you can see in the example, this tool is executed when the test of the tool fails.

Score¶

Each component has a summary of the scores obtained by category. You can find an example here:

"score": {

  "Findable": {

  "tests\_passed": 4,

  "total\_tests": 5,

  "score": 3.5,

  "total\_score": 7

  },

  "Accessible": {

  "tests\_passed": 2,

  "total\_tests": 3,

  "score": 1.5,

  "total\_score": 3

  },

  "Interoperable": {

  "tests\_passed": 3,

  "total\_tests": 3,

  "score": 3.0,

  "total\_score": 4

  },

  "Reusable": {

  "tests\_passed": 3,

  "total\_tests": 5,

  "score": 3,

  "total\_score": 10
  }

}

Note that when we talk about tests, we don't refer to the simple tests run in each principle, but to the test of the principle as overall (status field). Remember that we consider that each principle has one test, composed of multiple small tests.

You can find the complete JSON output in the following link.

Principles analyzed by resource - Research Object metadata¶

principle_id	F1
category_id	Findable
title	(Meta)data are assigned a persistent identifier
description	This check verifies if the RO has a persistent identifier ['w3id.org', 'doi.org', 'purl.org', 'www.w3.org']
total_passed_tests

total_tests_run

status	[pass
explanation	The identifier (https://google.es) of the root data entity is not persistent. The identifier should be store in any of this [w3id.org, doi.org, purl.org, www.w3.org]

principle_id	F2
category_id	Findable
title	Data are described with rich metadata
description	This check verifies if the the following minimum metadata ['author', 'license', 'description'] are present in the ro-crate
total_passed_tests

total_tests_run

status	[pass
explanation	Missing the following metadata: [license

principle_id	F3
category_id	Findable
title	Metadata clearly and explicitly include the identifier of the data they describe.
description	This check verifies that the hasPart elements exists and are describe in the ro
total_passed_tests

total_tests_run

status	[pass
explanation	This IRI could not be found: XXXXXXXX is described but has not been found locally

principle_id	R1.1
category_id	Reusable
title	(Meta)data are released with a clear and accessible data usage license.
description	This check verifies whether the RO has a licence. It also checks that there is a licence in RO parts.
total_passed_tests

total_tests_run

status	[pass
explanation	These entities have no licence: [XXXX,YYYY;ZZZZ]

principle_id	R1.2
category_id	Reusable
title	(Meta)data are associated with detailed provenance.
description	This check verifies that all elements of the RO have the following fields: ['author', 'datePublished', 'citation'].
total_passed_tests

total_tests_run

status	[pass
explanation	[XXXX does not have author, datePublished, citation] (list of elements)

DATA

principle_id	F1.1
category_id	Findable
title	Data is assigned a globally unique identifier.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Identifier is resolvable and follows a defined unique identifier syntax (IRI, URL),- Identifier is not resolvable but follows an UUID or HASH type syntax"

principle_id	F1.2
category_id	Findable
title	Data is assigned a globally unique identifier.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Identifier follows a defined persistent identifier syntax- Persistent identifier is resolvable

principle_id	F2
category_id	Findable
title	Data are described with rich metadata
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Missing the following metadata: XXXX,YYYY

principle_id	F2.1
category_id	Findable
title	Metadata includes descriptive core elements (creator, title, data identifier, publisher, publication date, summary and keywords) to support data findability.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Metadata has been made available via common web methods,- Metadata is embedded in the landing page XHTML/HTML code,- Metadata is accessible through content negotiation,- Metadata is accessible via typed links,- Metadata is accessible via signposting links,- Core data citation metadata is available,- Core descriptive metadata is available

principle_id	F3.2
category_id	Findable
title	Metadata includes the identifier of the data it describes.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Metadata contains data content related information (file name, size, type),- Metadata contains a PID or URL which indicates the location of the downloadable data content

principle_id	F3.2
category_id	Findable
title	Metadata is offered in such a way that it can be retrieved programmatically.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Metadata is registered in major research data registries (DataCite),- Metadata is given in a way major search engines can ingest it for their catalogues (JSON-LD, Dublin Core, RDFa),

principle_id	A1.1
category_id	Accesible
title	Metadata contains access level and access conditions of the data.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Information about access restrictions or rights can be identified in metadata,- fail: Data access information is machine readable,- Data access information is indicated by (not machine readable) standard terms

principle_id	A1.2
category_id	Accesible
title	Metadata is accessible through a standardized communication protocol.
description	-
total_passed_tests

total_tests_run	1
status	[pass
explanation	- Landing page link is based on standardized web communication protocols.

principle_id	A1.3
category_id	Accesible
title	Data is accessible through a standardized communication protocol.
description	-
total_passed_tests

total_tests_run	1
status	[pass
explanation	- Metadata includes a resolvable link to data based on standardized web communication protocols.

principle_id	I1.1
category_id	Interoperable
title	Metadata is represented using a formal knowledge representation language.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Parsable, structured metadata (JSON-LD, RDFa) is embedded in the landing page XHTML/HTML code,- Parsable, graph data (RDF, JSON-LD) is accessible through content negotiation, typed links or sparql endpoint

principle_id	I1.2
category_id	Interoperable
title	Metadata uses semantic resources
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Vocabulary namespace URIs can be identified in metadata,- Namespaces of known semantic resources can be identified in metadata

principle_id	I1.3
category_id	Interoperable
title	Metadata includes links between the data and its related entities.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Related resources are explicitly mentioned in metadata,- Related resources are indicated by machine readable links or identifiers

principle_id	R1.1
category_id	Reusable
title	Metadata specifies the content of the data.
description	-
total_passed_tests

total_tests_run	8
status	[pass
explanation	- Minimal information about available data content is given in metadata",- Resource type (e.g. dataset) is given in metadata",- Information about data content (e.g. links) is given in metadata",- Verifiable data descriptors (file info, measured variables or observation types) are specified in metadata",- File size and type information are specified in metadata",- Measured variables or observation types are specified in metadata",- Data content matches file type and size specified in metadata",- Data content matches measured variables or observation types specified in metadata

principle_id	R1.1.1
category_id	Reusable
title	Metadata includes license information under which data can be reused.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- Licence information is given in an appropriate metadata element,- Recognized licence is valid and registered at SPDX

principle_id	R1.2.1
category_id	Reusable
title	Metadata includes provenance information about data creation or generation.
description	-
total_passed_tests

total_tests_run	2
status	[pass
explanation	- - Metadata contains elements which hold provenance information and can be mapped to PROV- Metadata contains provenance information using formal provenance ontologies (PROV-O)

principle_id	R1.3.1
category_id	Reusable
title	Metadata follows a standard recommended by the target research community of the data.
description	-
total_passed_tests

total_tests_run	3
status	[pass
explanation	- Community specific metadata standard is detected using namespaces or schemas found in provided metadata or metadata services outputs,- Community specific metadata standard is listed in the re3data record of the responsible repository,- Multidisciplinary but community endorsed metadata (RDA Metadata Standards Catalog) standard is listed in the re3data record or detected by namespace"

principle_id	R1.3.2
category_id	Reusable
title	Data is available in a file format recommended by the target research community.
description	-
total_passed_tests

total_tests_run	4
status	[pass
explanation	- The format of a data file given in the metadata is listed in the long term file formats, open file formats or scientific file formats controlled list,- The format of the data file is an open format",- The format of the data file is a long term format",- The format of the data file is a scientific format"

ONTOLOGY

principle_id	F1
category_id	Findable
title	Persistent URL
description	This check verifies if the ontology has a persistent URL (w3id, purl, DOI, or a W3C URL).
total_passed_tests

total_tests_run	1
status	[pass
explanation	Ontology URI is persistent

principle_id	F1
category_id	Findable
title	Ontology URI is resolvable
description	Ontology URL is resolvable in application/rdf+xml.
total_passed_tests

total_tests_run	1
status	[pass
explanation	This check verifies if the ontology URI found within the ontology document is resolvable.

principle_id	F1
category_id	Findable
title	Version IRI.
description	This check verifies if there is an id for this ontology version, and whether the id is unique (i.e., different from the ontology URI).
total_passed_tests

total_tests_run	2
status	[pass
explanation	Version IRI defined, IRI is different from ontology URI.

principle_id	F1
category_id	Findable
title	Version IRI resolves.
description	This check verifies if the version IRI resolves.
total_passed_tests

total_tests_run	1
status	[pass
explanation	Version IRI resolves.

principle_id	F1
category_id	Findable
title	Consistent ontology IDs.
description	This check verifies if the ontology URI is equal to the ontology ID.
total_passed_tests

total_tests_run	1
status	[pass
explanation	Ontology URI is equal to ontology id.

principle_id	F2
category_id	Findable
title	Data are described with rich metadata.
description	This check verifies if the the following minimum metadata ['author', 'license', 'description'] are present in the research object.
total_passed_tests

total_tests_run	1
status	[pass
explanation	Missing the following metadata: author, license.

principle_id	F3
category_id	Findable
title	Ontology prefix
description	This check verifies if an ontology prefix is available.
total_passed_tests

total_tests_run	1
status	[pass
explanation	Prefix declaration found in the ontology: exo.

principle_id	F4
category_id	Findable
title	Prefix is in registry.
description	This check verifies if the ontology prefix can be found in prefix.cc or LOV registries. This check also verifies if the prefix resolves to the same namespaceprefix found in the ontology.
total_passed_tests

total_tests_run	2
status	[pass
explanation	Prefix declaration found with correct namespace (in prefic.cc).

principle_id	F4
category_id	Findable
title	Ontology in metadata registry.
description	Ontology not found in a public registry.
total_passed_tests

total_tests_run	1
status	[pass
explanation	This check verifies if the ontology can be found in a public registry (LOV).

principle_id	A1
category_id	Accesible
title	Open protocol.
description	The ontology uses an open protocol.
total_passed_tests

total_tests_run	2
status	[pass
explanation	This check verifies if the ontology uses an open protocol (HTTP or HTTPS).

principle_id	A1
category_id	Accesible
title	Content negotiation for RDF and HTML.
description	This check verifies of the ontology URI is published following the right content negotiation for RDF and HTML.
total_passed_tests

total_tests_run	2
status	[pass
explanation	Ontology available in: HTML, RDF.

principle_id	A1.1
category_id	Accesible
title	Open protocol.
description	This check verifies if the ontology uses an open protocol (HTTP or HTTPS).
total_passed_tests

total_tests_run	1
status	[pass
explanation	The ontology uses an open protocol.

principle_id	A2
category_id	Accesible
title	Metadata are accessible, even when ontology is not.
description	Metadata are accessible even when the ontology is no longer available. Since the metadata is usually included in the ontology, this check verifies whether the ontology is registered in a public metadata registry (LOV).
total_passed_tests

total_tests_run	1
status	[pass
explanation	Ontology not found in a public registry.

principle_id	I2
category_id	Interoperable
title	Vocabulary reuse (metadata).
description	Ontology reuses existing vocabularies for declaring metadata..
total_passed_tests

total_tests_run	1
status	[pass
explanation	This check verifies if the ontology reuses other vocabularies for declaring metadata terms.

principle_id	I2
category_id	Interoperable
title	Vocabulary reuse.
description	This check verifies if the ontology imports/extends other vocabularies (besides RDF, OWL and RDFS).
total_passed_tests

total_tests_run	1
status	[pass
explanation	Could not find any imported/reused vocabularies.

principle_id	R1
category_id	Reusable
title	Documentation: labels.
description	This check verifies the extent to which all ontology terms have labels (rdfs:label in OWL vocabularies, skos:prefLabel in SKOS vocabularies).
total_passed_tests

total_tests_run	7
status	[pass
explanation	Labels found for all ontology terms.

principle_id	R1
category_id	Reusable
title	Documentation: definitions.
description	This check verifies whether all ontology terms have descriptions (rdfs:comment in OWL vocabularies, skos:definition in SKOS vocabularies).
total_passed_tests

total_tests_run	7
status	[pass
explanation	Descriptions found for all ontology terms.

principle_id	R1
category_id	Reusable
title	HTML availability.
description	This check verifies if the ontology has an HTML documentation.
total_passed_tests

total_tests_run	5
status	[pass
explanation	Ontology available in HTML.

principle_id	R1
category_id	Reusable
title	Optional metadata.
description	This check verifies if the following optional metadata [doi, previous version,publisher, logo, backwards compatibility, status, modified, source, issued date] are present in the ontology.
total_passed_tests

total_tests_run	9
status	[pass
explanation	The following metadata was not found: doi, publisher, logo, status, modified, source, issued.

principle_id	R1.1
category_id	Reusable
title	License availability.
description	A license was found http://creativecommons.org/licenses/by/2.0/.
total_passed_tests

total_tests_run	1
status	[pass
explanation	This check verifies if a license associated with the ontology.

principle_id	R1.1
category_id	Reusable
title	License is resolvable.
description	This check verifies if the ontology license is resolvable.
total_passed_tests

total_tests_run	1
status	[pass
explanation	License could be resolved.

principle_id	R1.1
category_id	Reusable
title	(Meta)data are released with a clear and accessible data usage license.
description	This check verifies whether the element in the research object has a licence.
total_passed_tests

total_tests_run	1
status	[pass
explanation	This element has no licence.

principle_id	R1.2
category_id	Reusable
title	Basic provenance metadata.
description	This check verifies if basic provenance is available for the ontology: [author, creation date].
total_passed_tests

total_tests_run	2
status	[pass
explanation	All basic provenance metadata found!Warning: We could not find the following optional provenance metadata: contributor. Please consider adding them if appropriate.

principle_id	R1.2
category_id	Reusable
title	(Meta)data are associated with detailed provenance.
description	This check verifies that the element in the reserch object has the following fields: ['author', 'datePublished', 'citation'].
total_passed_tests

total_tests_run	1
status	[pass
explanation	This element has no :author, datePublished, citation.

principle_id	R1.2
category_id	Reusable
title	Detailed provenance metadata.
description	This check verifies if detailed provenance information is available for the ontology: [issued date, publisher].
total_passed_tests

total_tests_run	2
status	[pass
explanation	The following provenance information was not found: issued, publisher. Warning: We could not find the following provenance metadata: contributor Please consider adding them (if appropriate).

SOFTWARE

principle_id	F1
category_id	Findable
title	Software is assigned a globally unique and persistent identifier.
description	This test checks if there is at least one DOI for this Software.
total_passed_tests

total_tests_run	1
status	[pass
explanation	A Digital Object Identifier (DOI) is assigned to the Software.

principle_id	F1.1
category_id	Findable
title	Different versions of the same software must be assigned distinct identifiers.
description	This test checks if every version has different and uniques identifiers.
total_passed_tests

total_tests_run	1
status	[pass
explanation	Each version has a different identifier.

principle_id	F2
category_id	Findable
title	Software is described with rich metadata.
description	This check verifies if the following minimum metadata [title, description, license, installation instructions, requirements, creator, creationDate] are present.
total_passed_tests

total_tests_run	1
status	[pass
explanation	The software contains the following metadata [title, description, license, installation instructions, requirements, creator, creationDate].

principle_id	R1.1
category_id	Reusable
title	(Meta)data are released with a clear and accessible data usage license
description	This test verifies whether the element in the research object has a license
total_passed_tests

total_tests_run	1
status	[pass
explanation	This element have no licence

principle_id	R1.2
category_id	Reusable
title	(Meta)data are associated with detailed provenance
description	This test verifies that the element in the research object has the following fields: ['author', 'datePublished', 'citation']
total_passed_tests

total_tests_run	1
status	[pass
explanation	This element has no :author, datePublished or citation