The Lenses of Data Lens allow for functions to be included within the mappings. These functions allow for raw data to be transformed or filtered on its way to being translated to RDF. A wide array of functions are currently supported, including the majority of the GREL string functions - a full list of the supported functions can be found here. If however, when designing your mapping file for your Lens, you require a function to be executed that doesn’t cannot perform the your required operation simply by using the built-in functions, it is possible to create and use your own.

Steps

Table of Contents

style	none
exclude	Steps	style	none

1. Create your functions in Java

The logic for your function is carried out using Java code. First, create yourself a Java Class, this Class can contain as many functions as you wish; a function exists as a method with both input values and an output value. For example, let’s create a simple function to append a greeting to the input value.

Code Block

language	java

public class CustomFunctions {
    public static String appendGreeting(String value) {
        return value + " Hello World";
    }
}

2. Create the functions Turtle mapping

Next, you must create the mapping to your Java functions, this is done in Turtle (ttl). Below is the mapping required for our Greeting Function, along with a line by line description. The filename is irrelevant as this will be copied into the body of the API request.

Code Block

@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix doap:    <http://usefulinc.com/ns/doap#> .
@prefix fno:     <https://w3id.org/function/ontology#> .
@prefix fnoi:    <https://w3id.org/function/vocabulary/implementation#> .
@prefix fnom:    <https://w3id.org/function/vocabulary/mapping#> .
@prefix grelm:   <http://fno.io/grel/rmlmapping#> .
@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#> .
@prefix ex:      <http://example.com/function/> .

grelm:javaString a fnoi:JavaClass ;
    doap:download-page "./CustomFunctions.jar" ;
    fnoi:class-name    "CustomFunctions" .

ex:appendGreeting a fno:Function ;
  fno:name "Append greeting" ;
  rdfs:label "Append greeting" ;
  dcterms:description "Appends greeting" ;
  fno:expects ( ex:valueParam ) ;
  fno:returns ( ex:stringOut ) .

grelm:appendGreetingMapping a fnoi:Mapping ;
    fno:function         ex:appendGreeting ;
    fno:implementation   grelm:javaString ;
    fno:methodMapping    [ a                fnom:StringMethodMapping ;
                           fnom:method-name "appendGreeting" ] .

ex:valueParam a fno:Parameter ;
  fno:name "input value" ;
  rdfs:label "input value" ;
  fno:type xsd:string ;
  fno:predicate ex:valueParameter .

ex:stringOut a fno:Output ;
  fno:name "output string" ;
  rdfs:label "output string" ;
  fno:type xsd:string ;
  fno:predicate ex:stringOutput .

Code Block

# First we define our prefixes. These first seven prefixes are all required.
|
|
|
|
|
|
# Define your own custom prefix for your functions.

# Defining the link to your Java code.
# The link to your Jar.
# The link to your Java Class.

# Defining your function, the name to be referenced in your final mappings.
# The friendly name.
# The rdfs label.
# The function's description.
# The expected inputs to the function, multiple inputs are space separated, these must be defined as seen from line 27.
# The expected return value, this must be defined as seen from line 33.

# Defining the Java to Turlte link for your function. 
# The function name as specified in line 14.
# The Java Class as specified in line 10.
# The link to the Java method

 
# Defining the input value parameter.





# Defining the output value.

3. Build your Jar

The next step is to build your Java functions Class file into an executable Jar. To do this via the command line, simply execute the following two commands:

Code Block
javac CustomFunctions.java

Followed by:

Code Block
jar -cvf CustomFunctions.jar CustomFunctions.class

4. Push your files to your running Lens

Now we have our Turtle and Jar, we can push these files to a running Lens. These are both performed using an API call via an exposed endpoint on the Lens. This is a PUT request made to http://<lens-ip-address>:8080/functions.

The turtle file is sent by providing the file’s contents as the request’s body, along with a header of Content-Type: text/turtle. And the Jar file is sent simultaneously via the same endpoint, by providing a URL link to the location of the file as a path parameter with the key equal to path, for example, http://<lens-ip-address>:8080/functions?path=file:///var/local/CustomFunctions.jar. If using a local file location (i.e. the file:// scheme) to push to your running docker image, ensure your local storage is mounted using the -v flag. Alternatively, s3:// or http(s):// locations operate as expected.

Please note that the Lens’s UPLOAD_CUSTOM_FUNCTIONS configuration option must be set to true to expose this endpoint.

5. Creating your mapping file using your new function

Once all the previous steps have been completed, you are now able to use your new function in your mapping files. For an in-depth look at creating a mapping file from scratch, see our guide.

Strings passed into functions may be sourced from an rml:reference (The entire value of a field in the source document), an rr:template (A string that includes values of fields in the source document, as well as manually specified strings), as well as outputs from other Functions (functions may therefore be nested). Note that, when a function is used and its output is a NULL value, a triple will not be generated.

To see the full breakdown, refer to the functions section in our guide. However, using our previously created example function and a simple CSV input file with an id and name column, the mapping file will look like the following:

Code Block

@prefix rr:  <http://www.w3.org/ns/r2rml#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ql:  <http://semweb.mmlab.be/ns/ql#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix fnml: <http://semweb.mmlab.be/ns/fnml#> .
@prefix fno: <https://w3id.org/function/ontology#> .
@prefix ex:  <http://example.com/function/> .
@prefix exp:  <http://example.com/predicate/> .

@base <http://example.com/base/> .

<TriplesMap1>
  a rr:TriplesMap;

rml:logicalSource [
  rml:source "inputSourceFile.csv";
  rml:referenceFormulation ql:CSV
];

rr:subjectMap [
  rr:template "http://example.com/{id}"
];

rr:predicateObjectMap [
  rr:predicate exp:name ;
  rr:objectMap [
    fnml:functionValue [
      rr:predicateObjectMap [
        rr:predicate fno:executes ;
        rr:objectMap [ rr:constant ex:appendGreeting ]
      ] ;
      rr:predicateObjectMap [
        rr:predicate ex:valueParameter ;
        rr:objectMap [ rml:reference "name" ]
      ]
    ]
  ]
] .

Code Block






# The same prefix as defined in your functions mapping



















# A function is defined using `fnml:functionValue`
# It contains a number of `rr:predicateObjectMap`, this depends on the function being used - our example uses two
# In the first predicateObjectMap, we must specify the `rr:predicate` to equal `fno:executes`
# This is where we declare the function we wish to use, this is done by specifying the function IRI as an `rr:constant` in an `rr:objectMap`

# The subsequent predicateObjectMaps are for the function's input arguments
# Each input argument will have a predicate associated with it, this is function dependant so check the turtle mapping for you function
# This delcares the input argument by referencing the 'name' value from the source data, we use `rml:reference` as it defaults to a literal term type
# Some functions may require multiple predicateObjectMaps to be defined if there are multiple arguments

6. Using this example in your Lens

All example files created in this document can be downloaded from our Bitbucket. The easiest way to test custom functions on your Lens is to first use our example files. Steps 1, 2, 3, and 5, have already been completed for you, so all you need to do is trigger the API endpoints of you Lens as seen in step 4.

PUT request to http://<lens-ip-address>:8080/functions?path=https://bitbucket.org/data-lens/datalens-examples/raw/5be1841fda5dcdfd8ded84e12a3877a688ab4840/custom-functions/CustomFunctions.jar with header Content-Type: text/turtle, and body as ttl example.

Now you can test the new functions in your Lens by triggering the Process endpoint using our example as input:

2. GET request to http://<lens-ip-address>:8080/process?inputFileURL=https://bitbucket.org/data-lens/datalens-examples/raw/5be1841fda5dcdfd8ded84e12a3877a688ab4840/custom-functions/input-file.csv

Your output RDF should be similar to:

Code Block

<http://example.com/10001> <http://example.com/predicate/name> "Alice Hello World" <http://www.data-lens.co.uk/10f71463-5753-4452-9c80-76964af0f2f2>
<http://example.com/10002> <http://example.com/predicate/name> "Bob Hello World" <http://www.data-lens.co.uk/10f71463-5753-4452-9c80-76964af0f2f2>
<http://example.com/10003> <http://example.com/predicate/name> "Claire Hello World" <http://www.data-lens.co.uk/10f71463-5753-4452-9c80-76964af0f2f2>

Versions Compared

Old Version 5

New Version 6

Key

Steps

1. Create your functions in Java

2. Create the functions Turtle mapping

3. Build your Jar

4. Push your files to your running Lens

5. Creating your mapping file using your new function

6. Using this example in your Lens

Page Comparison

Versions Compared

Old Version 5

New Version 6

Key

Steps

1. Create your functions in Java

2. Create the functions Turtle mapping

3. Build your Jar

4. Push your files to your running Lens

5. Creating your mapping file using your new function

6. Using this example in your Lens