Below is a table containing all of the configurable options within the Lens Writer. To see how to set config variables, see the Quick Start Guide or the full User Guide.

With version 2.0 and beyond, no configuration is required at startup as config can be updated on a running Writer. To do this, simply call the endpoint /updateConfig?configEntry=<entry>&configValue=<value> where the entry the config item as seen below, and value is the new value you wish to set. Any configuration changed while the Lens is running can also be backed up and restored.

Lens Writer Configuration

Environment Variable	Entry	Default Value	Description	Version

Environment Variable	Entry	Default Value	Description	Version
FRIENDLY_NAME	friendlyName	Lens-Writer	The name you wish to set your Writer up with.	v1.3+
LICENSE	license		The License key provided required for running the Writer. Only required when running a non AWS Marketplace version of the Writer.	v1.3+
GRAPH_DATABASE_ENDPOINT	graphDatabaseEndpoint		The endpoint for your Knowledge Graph you wish to upload your data to.	v2.0+
GRAPH_DATABASE_TYPE	graphDatabaseType	sparql	The Knowledge Graph type, some Semantic Graphs will support the default `sparql` type (e.g. AllegroGraph), however certain graphs require specific type declaration, these include `graphdb`, `stardog`, `blazegraph`, `neptune-sparql` and `rdfox`. If you are using a Property Graph, you can set a specific Graph provider, including `neo4j`, `neptune-cypher`, `neptune-gremlin`, or the traversal language `cypher` or `gremlin`.	v2.0+
GRAPH_DATABASE_REASONING	graphDatabaseReasoning	false	Whether you want reasoning enabled or disabled. This only applies to Semantic Graphs.	v2.0+
GRAPH_DATABASE_USERNAME	graphDatabaseUsername		The username of your Knowledge Graph. Leave blank if your Knowledge Graph does not require any authentication.	v2.0+
GRAPH_DATABASE_PASSWORD	graphDatabasePassword		The password of your Knowledge Graph. Leave blank if your Knowledge Graph does not require any authentication.	v2.0+
CONFIG_BACKUP	configBackup	file:///var/local/config-backup/	The URL directory where the config will be backed up to when calling the upload config endpoint	v2.0+
DELETE_SOURCE	deleteSourceFile	false	Whether you wish to delete the source data file after it has been written to the Knowledge Graph	v1.3+
LENS_RUN_STANDALONE	runStandalone	true	The Lens Writer is designed to run as part of a larger end to end system with the Lens providing the Writer with RDF or CSV files to write to a Knowledge Graph. As part of this process, Kafka is used to communicate between services. This is turned off by default, however if you want to enable the running of the Lens Writer with connected services, set this property to false.	v1.3+
INGESTION_MODE	ingestionMode	insert	How to process the ingested data. `'insert'`: the new data is ingested in full and do not replace existing data. The new dataset adds new values to already existing subject-predicates. `'update'`: the new data is used for updating the existing data. The new dataset replaces value in existing subject-predicate. Please note this only applies to Semantic Graphs, in Property Graphs the Writer defaults to an upsert pattern, updating the properties for a given node or edge.	v1.4+

Kafka Configuration

Environment Variable	Entry	Default Value	Description	Version

Environment Variable	Entry	Default Value	Description	Version
KAFKA_BROKERS	kafkaBrokers	localhost:9092	The Kafka Broker is what tells the Writer where to look for your Kafka Cluster. Set with the following structure `<kafka-ip>:<kafka-port>`. The recommended port is `9092`.	v1.3+
KAFKA_TOPIC_NAME_SOURCE	topicNameSource	success_queue	The topic used for the Consumer to read messages from containing the URLs of the source data files to ingest.	v1.3+
KAFKA_TOPIC_NAME_DLQ	topicNameDLQ	dead_letter_queue	The topic used to push messages containing reasons for failure within the Writer. These messages are represented as a JSON.	v1.3+
KAFKA_GROUP_ID_CONFIG	groupIdConfig	consumerGroup1	The identifier of the group this consumer belongs to.	v1.3+
KAFKA_AUTO_OFFSET_RESET_CONFIG	autoOffsetResetConfig	earliest	What to do when there is no initial offset in Kafka or if an offset is out of range. `earliest`: automatically reset the offset to the earliest offset `latest`: automatically reset the offset to the latest offset	v1.3+
KAFKA_MAX_POLL_RECORDS	maxPollRecords	100	The maximum number of records returned in a single call to poll.	v1.3+
KAFKA_TIMEOUT	timeout	1000	Kafka consumer polling time out.	v1.3+

Provenance Configuration

Environment Variable	Entry	Default Value	Description	Version

Environment Variable	Entry	Default Value	Description	Version
RECORD_PROVO	recordProvo	false	Currently, the Lens Writer does not generate its own provenance meta-data and so this option is set to false	v1.3+

Logging Configuration

Environment Variable	Default Value	Description

Environment Variable	Default Value	Description
LOG_LEVEL_DATALENS	INFO	Log level for Data Lens loggers - change to DEBUG to see more in depth logs, or to WARN or ERROR to quiet the logging.
LOG_LEVEL_DROPWIZARD	INFO	Log level for Dropwizard loggers

Additional Logging Configuration

Environment Variable	Default Value	Description	Version

Environment Variable	Default Value	Description	Version
LOGGING_LEVEL	WARN	Global log level	v1.3+
LOGGING_APPENDERS_CONSOLE_TIMEZONE	UTC	Timezone for console logging	v1.3+
LOGGING_APPENDERS_TXT_FILE_THRESHOLD	ALL	Threashold for text logging	v1.3+
Log Format (not overridable)	%-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg %n	Pattern for logging messages	v1.3+
Current Log Filename (not overridable)	/var/log/datalens/text/current/application_${applicationName}_${timeStamp}.txt.log	Pattern for log file name	v1.3+
LOGGING_APPENDERS_TXT_FILE_ARCHIVE	true	Archive log text files	v1.3+
Archived Log Filename Pattern (not overridable)	/var/log/datalens/text/archive/application_${applicationName}_${timeStamp}_to_%d{yyyy-MM-dd}.txt.log	Log file rollover frequency depends on pattern in following property. For example %d{yyyy-MM-ww} declares rollover weekly	v1.3+
LOGGING_APPENDERS_TXT_FILE_ARCHIVED_TXT_FILE_COUNT	7	Max number of archived text files	v1.3+
LOGGING_APPENDERS_TXT_FILE_TIMEZONE	UTC	Timezone for text file logging	v1.3+
LOGGING_APPENDERS_JSON_FILE_THRESHOLD	ALL	Threashold for text logging	v1.3+
Log Format (not overridable)	%-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg %n	Pattern for logging messages	v1.3+
Current Log Filename (not overridable)	/var/log/datalens/json/current/application_${applicationName}_${timeStamp}.json.log	Pattern for log file name	v1.3+
LOGGING_APPENDERS_JSON_FILE_ARCHIVE	true	Archive log text files	v1.3+
Archived Log Filename Pattern (not overridable)	/var/log/datalens/json/archive/application_${applicationName}_${timeStamp}_to_%d{yyyy-MM-dd}.json.log	Log file rollover frequency depends on pattern in following property. For example %d{yyyy-MM-ww} declares rollover weekly	v1.3+
LOGGING_APPENDERS_JSON_FILE_ARCHIVED_FILE_COUNT	7	Max number of archived text files	v1.3+
LOGGING_APPENDERS_JSON_FILE_TIMEZONE	UTC	Timezone for text file logging	v1.3+
LOGGING_APPENDERS_JSON_FILE_LAYOUT_TYPE	json	The layout type for the json logger	v1.3+

Data Lens: Documentation

Configurable Options - Lens Writer v2.0

Lens Writer Configuration

Kafka Configuration

Provenance Configuration

Logging Configuration

Additional Logging Configuration

Related content