...
This is the full User Guide for the Lens Writer, it contains an in-depth set of instructions to fully set up, configure, and run the Writer so you can start writing data as part of an end-to-end system. For a guide to get the Writer up and running in the quickest and simplest possible way, see the Quick Start Guide. Once deployed, you can utilise any of our ready-made sample output NQaud files to test your Writer. For a list of what has changed since the last release, visit the User Release Notes.
...
Table of Contents
Table of Contents |
---|
...
An Amazon Web Services Elastic Container Service (ECS) cluster, hosting a single EC2 instance. Running the following containers:
Apache Kafka
Data Lens: Lens Writer
An S3 bucket
Info |
---|
For more information on the Architecture and Deployment of an Enterprise System, see our guide. |
...
To use a local URL for directories and files, both the format of
file:///var/local/data-lens-output/
and/var/local/data-lens-output/
are supported.To use a remote http(s) URL for files,
https://example.com/input-rdf-file.nq
is supported.To use a remote AWS S3 URL for directories and files,
s3://example/folder/
is supported where the format iss3://<bucket-name>/<directory>/<file-name>
. If you are using an S3 bucket for any directory, you must specify an AWS access key and secret key.
Also included in the Writer, is the ability to delete your source NQuad input files after they have been ingested into your Triple Store. This is done by setting the DELETE_SOURCE
config value to true
. Enabling this means that your S3 Bucket or local file store, will not continuously fill up with RDF NQuad data generated from your Lenses.
Endpoint
First, the easiest way to ingest an RDF file into the Lens Writer is to use the built-in APIs. Using the process
GET endpoint, you can specify the URL of an RDF file to ingest, and in return, you will be provided with the success status of the operation.
...
For more information on how the provenance is laid out, as well as how to query in from you Triple Store, see the Provenance Guide.
...
REST API Endpoints
In addition to the Process Endpoint designed for ingesting data into the Writer, there is a selection of built-in exposed endpoints for you to call.
...