Structured File Lens - User Release Notes

Owned by Russ Waterson (Unlicensed)
Last updated: Oct 28, 2022 by Russell Waterson
2 min read

2.0.15.286

2022-10-28

  • Added support for adding Custom Functions at Lens startup using the config variables CUSTOM_FUNCTION_JAR_URL and CUSTOM_FUNCTION_TTL_URL

 

2.0.14.283

2022-10-18

  • Added the ability to specify a specific mapping file to be used when processing an input file

    • For example: http://localhost:8080/process?inputFileURLs=file:///tmp/sflens/input-123.json&logicalSources=input.json&mappingFileURL=file://tmp/sflens/mapping/mapping.ttl

 

 

2.0.12.278

2022-11-11

  • Support for ingesting Change Data Capture (CDC) Debezium data streams

  • Support for XSLX and ODS spreadsheet data formats

  • Support for multiple source and mapping file processing

 

 

2.0.5.254

2022-06-20

  • Added support for the AWS Default Credentials Provider Chain. In addition to the environment variables, AWS Credentials can now be set in more ways.

  • Improved stability and bug fixes

 

 

2.0.4.250

2022-05-12

  • Added support for Property Graphs by generating nodes and edges files

  • Lens data directories are created on startup minimising required config

  • Ability to save and restore configuration as well as editing on a running Lens

  • Added to the suite of built in functions

  • Added fine grain logging control



1.5.2.222

2021-06-10

  • Added support for different output file types (NQuads, NTriples, JSON-LD, Turtle, Trig, and Trix)

  • Removed NamedGraphs being attached to triples when provenance is switched off



1.5.1.214

2020-10-02

  • Returns an exception message when processing fails via endpoint triggering



1.4.3.201

2020-08-28

  • AWS Marketplace compatibility

  • The Lens can now accept new Custom Functions via an endpoint

 

1.3.3.153

2020-05-15

  • SFLENS-126: Fix Docker image security vulnerabilities as seen in ECR

  • SFLENS-118: Fixed persistent Kafka logging

 

1.2.2.137

2020-03-17

  • SFLENS-104: Enhancements have been made to CSV transformation in order to more elegantly handle CSV files containing errors.

  • SFLENS-107: It is now possible to choose which provenance activities are generated for your transformed data.

  • SFLENS-112: Dead Letter Queue messages are now much richer, and include additional information in JSON format, to more easily facilitate consumption by an external process.

  • SFLENS-113: Update the POM to use our latest custom RMLMapper which prevents erroneous IRI encoding

 

1.1.0.102

2020/03/03

  • SFLENS-102: Added support for the following new mapping functions:

    • retrieveValueFromDelimitedListTests

    • extractCharactersFromStartTest

    • extractCharactersFromEndTest

  • SFLENS-106: Fixed a problem which could cause large CSV files to be iterated over incorrectly.

 

1.0.0.93

2020/02/03

This is the first official release of the Structured File Lens.

It is a highly configurable pipeline component, which will transform structured content into a Knowledge Graph compatible output.

 

  • Highly configurable transformation from structured content, to RDF.

  • Supplied as a Docker Container, available from DockerHub.

  • Supports the following input file formats:

    • XML

    • JSON

    • CSV

  • Supports the following output file formats:

    • NQuads

    • JSON-LD

  • Capable of iteratively transforming input files of any file size.

  • May be triggered via Message bus, or directly via RESTful endpoint.

  • Input / Output files may be accessed from on-premise, or cloud locations.

  • Provenance as standard.