Structured File Lens - User Release Notes


  • Added support for adding Custom Functions at Lens startup using the config variables CUSTOM_FUNCTION_JAR_URL and CUSTOM_FUNCTION_TTL_URL


  • Added the ability to specify a specific mapping file to be used when processing an input file

    • For example: http://localhost:8080/process?inputFileURLs=file:///tmp/sflens/input-123.json&logicalSources=input.json&mappingFileURL=file://tmp/sflens/mapping/mapping.ttl


  • Support for ingesting Change Data Capture (CDC) Debezium data streams

  • Support for XSLX and ODS spreadsheet data formats

  • Support for multiple source and mapping file processing


  • Added support for the AWS Default Credentials Provider Chain. In addition to the environment variables, AWS Credentials can now be set in more ways.

  • Improved stability and bug fixes


  • Added support for Property Graphs by generating nodes and edges files

  • Lens data directories are created on startup minimising required config

  • Ability to save and restore configuration as well as editing on a running Lens

  • Added to the suite of built in functions

  • Added fine grain logging control


  • Added support for different output file types (NQuads, NTriples, JSON-LD, Turtle, Trig, and Trix)

  • Removed NamedGraphs being attached to triples when provenance is switched off


  • Returns an exception message when processing fails via endpoint triggering


  • AWS Marketplace compatibility

  • The Lens can now accept new Custom Functions via an endpoint


  • SFLENS-126: Fix Docker image security vulnerabilities as seen in ECR

  • SFLENS-118: Fixed persistent Kafka logging


  • SFLENS-104: Enhancements have been made to CSV transformation in order to more elegantly handle CSV files containing errors.

  • SFLENS-107: It is now possible to choose which provenance activities are generated for your transformed data.

  • SFLENS-112: Dead Letter Queue messages are now much richer, and include additional information in JSON format, to more easily facilitate consumption by an external process.

  • SFLENS-113: Update the POM to use our latest custom RMLMapper which prevents erroneous IRI encoding


  • SFLENS-102: Added support for the following new mapping functions:

    • retrieveValueFromDelimitedListTests

    • extractCharactersFromStartTest

    • extractCharactersFromEndTest

  • SFLENS-106: Fixed a problem which could cause large CSV files to be iterated over incorrectly.


This is the first official release of the Structured File Lens.

It is a highly configurable pipeline component, which will transform structured content into a Knowledge Graph compatible output.


  • Highly configurable transformation from structured content, to RDF.

  • Supplied as a Docker Container, available from DockerHub.

  • Supports the following input file formats:

    • XML

    • JSON

    • CSV

  • Supports the following output file formats:

    • NQuads

    • JSON-LD

  • Capable of iteratively transforming input files of any file size.

  • May be triggered via Message bus, or directly via RESTful endpoint.

  • Input / Output files may be accessed from on-premise, or cloud locations.

  • Provenance as standard.