Config JSON

This is an automatically created documentation. It is not 100% accurate since the generator does not handle every edge case.

Instead of a list ConQuery also always accepts a single element.

The config.json is required for every type of execution. Its root element is a ConqueryConfig object.

Base AuthorizationConfig

An AuthorizationConfig defines the initial users that are created on application startup and other permission related options.

Different types of AuthorizationConfig can be used by setting type to one of the following values:

DEFAULT^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.auth.DefaultAuthorizationConfig

Supported Fields:

	Field	Type	Default
✎	initialRoles	list of `ProtoRole`
✎	initialUsers	list of `ProtoUser`	`[]`
✎	overviewScope	list of `String`	`["datasets","admin","*"]`

DEVELOPMENT^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.auth.DevelopmentAuthorizationConfig

No fields can be set for this type.

Base AuthenticationRealmFactory

An AuthenticationConfig is used to define how specific realms for authentication are configured.

Different types of AuthenticationRealmFactory can be used by setting type to one of the following values:

DEVELOPMENT^{_^✎}

Default configuration for the auth system. Sets up all other default components. This configuration causes that every request is handled as invoked by the super user.

Details

Java Type: com.bakdata.conquery.models.auth.develop.DevAuthConfig

No fields can be set for this type.

JWT_PKCE_REALM^{_^✎}

A realm that verifies oauth tokens using PKCE.

Since the adminEnd-UI mainly works with direct links that do not support setting of the Authorization-header to transport an access token and it also does not support the oauth code flow, this realm proxies the flow. 1. The user/client is redirected to the IDP for authentication and redirected back to the adminEnd 2. The this realm then picks up the code the transported authorization code from the query and redeems it for an access token and refresh token. 3. Both are converted to cookies, which are saved on the client upon redirection to the page the user initially wanted to visit. 4. With the redirection the client sends back the cookies from which the adminEnd extracts the access token and verifies it. (5.) If the previous step failed. The refresh token is extracted and exchanged for a fresh access token and refresh token and continues with step 3.

Details

Java Type: com.bakdata.conquery.models.config.auth.JwtPkceVerifyingRealmFactory

Supported Fields:

	Field	Type	Default	Description
✎	additionalTokenChecks	list of `String`	`[]`
✎	alternativeIdClaims	list of `String`	`[]`	Which claims hold alternative Ids of the user in case the user name does not match a user. Pay attention, that the user must not be able to alter the value of any of these claims.
✎	client	`String`	`null`	The client id is also used as the expected audience in the validated token. Ensure that the IDP is configured accordingly.
✎	idpConfiguration	`IdpConfiguration`	`null`	See wellKnownEndpoint.
✎	tokenLeeway	`@jakarta.validation.constraints.Min(0) int`	`60`	A leeway for token's expiration in seconds, this should be a short time. One Minute is the default.
✎	wellKnownEndpoint	`URI`	`null`	Either the wellKnownEndpoint from which an idpConfiguration can be obtained or the idpConfiguration must be supplied. If the idpConfiguration is given, the wellKnownEndpoint is ignored.

LOCAL_AUTHENTICATION^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.auth.LocalAuthenticationConfig

Supported Fields:

	Field	Type	Default	Description
✎	caffeineSpec	`String`	`""`
✎	directory	`File`	`"./storage"`
✎	jwtDuration	`@MinDuration(value=1, unit=TimeUnit.MINUTES) Duration`	`"12 hours"`
✎	passwordStoreConfig	XodusConfig		Configuration for the password store. An encryption for the store itself might be set here.
✎	storeName	`String`	`"authenticationStore"`	The name of the folder the store lives in.

OIDC_AUTHORIZATION_CODE_FLOW^{_^✎}

Factory for a simple realm that just forwards tokens to the IDP for verification.

Details

Java Type: com.bakdata.conquery.models.config.auth.OIDCAuthorizationCodeFlowRealmFactory

Supported Fields:

	Field	Type	Default	Example	Description
✎	client	`IntrospectionDelegatingRealmFactory`	`null`

OIDC_RESOURCE_OWNER_PASSWORD_CREDENTIAL_AUTHENTICATION^{_^✎}

Realm that supports the Open ID Connect Resource-Owner-Password-Credential-Flow with a Keycloak IdP.

Details

Java Type: com.bakdata.conquery.models.config.auth.OIDCResourceOwnerPasswordCredentialRealmFactory

Supported Fields:

	Field	Type	Default	Example	Description
✎	client	`IntrospectionDelegatingRealmFactory`	`null`

Base PluginConfig

A PluginConfig is used to define settings for Conquery plugins.

Different types of PluginConfig can be used by setting type to one of the following values:

FORM_BACKEND^{_^✎}

{@link PluginConfig} for an external form backend. The external form backend must implement the OpenAPI spec for external form backend.

Details

Java Type: com.bakdata.conquery.models.config.FormBackendConfig

Supported Fields:

	Field	Type	Default
✎	authentication	`@Valid AuthenticationClientFilterProvider`	`null`
✎	baseURI	`@NotNull URI`	`null`
✎	cancelTaskPath	`String`
✎	conqueryApiUrl	`@NotNull URL`	`null`
✎	formConfigPath	`String`	`"form-config"`
✎	healthCheckPath	`String`	`"health"`
✎	id	`String`	`null`
✎	postFormPath	`String`	`"task"`
✎	statusTemplatePath	`String`
✎	versionPath	`String`	`"version"`

MANUAL^{_^✎}

Configuration of manual links for {@link org.checkerframework.checker.signature.qual.InternalForm} and overriding of other {@link com.bakdata.conquery.apiv1.forms.Form}s.

Details

Java Type: com.bakdata.conquery.models.config.ManualConfig

Supported Fields:

	Field	Type	Default	Example	Description
✎	forms	map from `String` to `@ManualURI URI`			Maps a {@link CPSType} of a {@link com.bakdata.conquery.apiv1.forms.Form} to a manual URL. E.g.: EXPORT_FORM -> https://example.org/export-form FULL_EXPORT_FORM -> ./full-export-form

Other Types

Type APIConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.APIConfig

Supported Fields:

	Field	Type	Default	Example	Description
✎	allowCORSRequests	`boolean`	`false`
✎	caching	`boolean`	`true`

Type CSVConfig^{_^✎}

Holds the necessary information to configure CSV parsers and writers.

Details

Java Type: com.bakdata.conquery.models.config.CSVConfig

Supported Fields:

	Field	Type	Default
✎	comment	`char`	`"\u0000"`
✎	delimeter	`char`	`","`
✎	encoding	`@NotNull Charset`	`"UTF-8"`
✎	escape	`char`	`"\\"`
✎	lineSeparator	`String`	`"\n"`
✎	maxColumns	`int`	`1000000`
✎	parseHeaders	`boolean`	`true`
✎	quote	`char`	`"\""`
✎	skipHeader	`boolean`	`false`

Type ClusterConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.ClusterConfig

Supported Fields:

	Field	Type	Default	Description
✎	columnValuesPerChunk	`int`	`1000`
✎	connectRetryTimeout	`Duration`	`"30 seconds"`
✎	heartbeatTimeout	`Duration`	`"1 minute"`
✎	idleTimeOut	`Duration`	`"5 minutes"`
✎	initialIoBufferSize	`@DataSizeRange(min=64, max=2147483643) DataSize`	`"64 bytes"`	Defines the starting buffer allocation size. Larger can reduce reallocations, but can cause a greater memory demand. May only touch this for testing purposes.
✎	managerURL	`@Valid @NotNull InetAddress`	`"localhost"`
✎	mina	`@Valid @NotNull DefaultSocketSessionConfig`
✎	networkSessionMaxQueueLength	`int`	`5`
✎	port	`@io.dropwizard.validation.PortRange int`	`16170`

Type ConqueryConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.ConqueryConfig

Supported Fields:

	Field	Type	Default	Description
✎	api	APIConfig
✎	authentication	`@Valid @NotNull AuthenticationConfig`
✎	authenticationRealms	list of AuthenticationRealmFactory
✎	authorizationRealms	@Valid @NotNull AuthorizationConfig
✎	cluster	ClusterConfig
✎	csv	CSVConfig
✎	debugMode	`boolean` or `null`	`null`	null means here that we try to deduce from an attached agent
✎	failOnError	`boolean`	`false`
✎	frontend	FrontendConfig
✎	idColumns	`@Valid @NotNull IdColumnConfig`
✎	index	`@NotNull @Valid IndexConfig`
✎	jerseyClient	`@Valid @NotNull JerseyClientConfiguration`
✎	locale	LocaleConfig
✎	metricsConfig	`ConqueryMetricsConfig`
✎	plugins	list of PluginConfig	`[]`
✎	preprocessor	PreprocessingConfig
✎	queries	QueryConfig
✎	resultProviders	list of `ResultRendererProvider`		The order of this list determines the order of the generated result urls in a query status.
✎	search	`@NotNull @Valid SearchConfig`
✎	sqlConnectorConfig	SqlConnectorConfig
✎	standalone	StandaloneConfig
✎	storage	`@Valid @NotNull StoreFactory`

Type CurrencyConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.FrontendConfig$CurrencyConfig

Supported Fields:

	Field	Type	Default
✎	decimalScale	`int`	`2`
✎	decimalSeparator	`String`	`","`
✎	thousandSeparator	`String`	`"."`
✎	unit	`String`	`"€"`

Type DatabaseConnectionConfig^{_^✎}

Connection properties for a SQL database.

Currently supported are HANA and Prostgres databases, see {@link DatabaseConnectionConfig#dialect}.

Details

Java Type: com.bakdata.conquery.models.config.DatabaseConnectionConfig

Supported Fields:

	Field	Type	Default	Description
✎	connectivityTimeout	`Duration`	`null`
✎	databasePassword	`String`	`null`	Password used to connect to the database.
✎	databaseUsername	`String`	`null`	Username used to connect to the database.
✎	dialect	Dialect	`null`	SQL vendor specific dialect used to transform queries to SQL
✎	jdbcConnectionUrl	`String`	`null`	Connections url in JDBC notation.
✎	matchingStatsRetries	`@jakarta.validation.constraints.Min(1) int`	`3`	Retries for matching stats. This is a bit of a workaround because some DBMS seem to struggle to properly communicate workload to HikariCP.
✎	matchingStatsWorkers	`@jakarta.validation.constraints.Min(1) int`	`5`	Maximum workers to schedule for matching stats. This depends on your HikariCP configuration as well.
✎	primaryColumn	`String`	`"pid"`	Name of the column which is shared among the table and all aggregations are grouped by.
✎	propertiesFile	`String`	`null`

Type Dialect^{_^✎}

The dialect sets SQL vendor specific query transformation rules.

There is no fallback dialect, so the dialect must fit the targeted database. Values:

CLICKHOUSE: No description available
HANA: No description available

Details

Java Type: com.bakdata.conquery.models.config.Dialect

No fields can be set for this type.

Type FrontendConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.FrontendConfig

Supported Fields:

	Field	Type	Default	Description
✎	alwaysAllowCreateValue	`boolean`	`false`	If true, users are always allowed to add custom values into SelectFilter input fields.
✎	contactEmail	`String`	`null`
✎	currency	CurrencyConfig
✎	manualUrl	`URL`	`null`	The url that points a manual. This is also used by the {@link FormScanner} as the base url for forms that specify a relative url. Internally {@link URI#resolve(URI)} is used to concatenate this base url and the manual url from the form. An final slash ('{@code /}') on the base url has the following effect: No slash: {@code baseUrl = http://example.org/manual/welcome} {@code formUrl = ./form} → {@code http://example.org/manual/form} Slash: {@code baseUrl = http://example.org/manual/} {@code formUrl = ./form} → {@code http://example.org/manual/form}
✎	observationPeriodYears	`@jakarta.validation.constraints.Min(0) int`	`6`	Years to include in entity preview.
✎	visualisationPercentiles	`Range<Integer>`
✎	visualisationsHistogramLimit	`@jakarta.validation.constraints.Min(0) int`	`10`	Limit to number of histogram entries. Note, that zero and out of bounds values are tracked in separate bins, so you can have three additional bins.

Type LocaleConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.LocaleConfig

Supported Fields:

	Field	Type	Default	Description
✎	dateFormatMapping	map from `Locale` to `String`		Mappings from user provided locale to date format which is used in the generation of result tables. The formats are also available for parsing dates using the {@link DateReader}. However, the locale is neglected there and the formats are tried until one that fits is found.
✎	frontend	`@NotNull Locale`	`"und"`
✎	listFormats	list of `ListFormat`		List formats that are available for parsing inputs and (the first one) for rendering result tables. Spaces at the ends of the separator are only relevant for the output of results. For the input (parsing) the separator string can be surrounded by an arbitrary number of spaces.
✎	localeRangeStartEndSeparators	map from `Locale` to `String`		Mappings from user provided locale to date range format which is used in the generation of result tables. The formats are also available for parsing dates ranges using the {@link DateReader}. However, the locale is neglected there and the formats are tried until one that fits is found.
✎	parsingDateFormats	`@NotNull Set<String>`	`["yyyyMMdd"]`	Additional date formats that are available only for parsing.
✎	parsingRangeStartEndSeparators	`Set<String>`	`["/"]`	Additional date range formats that are available only for parsing.

Type MinaConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.MinaConfig

Supported Fields:

	Field	Type	Default	Description
✎	bothIdleTime	`int`	`300`	The delay in seconds before we notify a session that it has been idle on read and write. Default to infinite
✎	maxReadBufferSize	`int`	`500000000`	The maximum size of the buffer used to read incoming data
✎	minReadBufferSize	`int`	`64`	The minimum size of the buffer used to read incoming data
✎	readBufferSize	`int`	`8192`	The default size of the buffer used to read incoming data
✎	readerIdleTime	`int`	`300`	The delay in seconds before we notify a session that it has been idle on read. Default to infinite
✎	throughputCalculationInterval	`int`	`3`
✎	useReadOperation	`boolean`	`false`	A flag set to true when we allow the application to do a session.read(). Default to false
✎	writeTimeout	`int`	`0`	The delay to wait for a write operation to complete before bailing out
✎	writerIdleTime	`int`	`300`	The delay in seconds before we notify a session that it has been idle on write. Default to infinite

Type PreprocessingConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.PreprocessingConfig

Supported Fields:

	Field	Type	Default
✎	faultyLineThreshold	`@jakarta.validation.constraints.Min(0) @jakarta.validation.constraints.Max(1) double`	`0.01`
✎	maximumPrintedErrors	`@jakarta.validation.constraints.Min(0) int`	`10`
✎	nThreads	`@jakarta.validation.constraints.Min(1) int`	␀
✎	parsers	`ParserConfig`

Type QueryConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.QueryConfig

Supported Fields:

	Field	Type	Default	Description
✎	L1CacheSpec	`String`	␀	See {@link com.bakdata.conquery.models.query.ExecutionManager#executionInfosL1} for an explanation
✎	L2CacheSpec	`String`	␀	See {@link com.bakdata.conquery.models.query.ExecutionManager#executionInfosL1} for an explanation
✎	executionPool	`ThreadPoolDefinition`
✎	oldQueriesTime	`Duration`	`"30 days"`
✎	secondaryIdSubPlanRetention	`int`	`15`	Limits how many subQuery-Plans should be cached between executions: This number limits how many sub-plans are cached per core so that outliers do not cause massive memory overhead. TODO Implement global limit of active secondaryId sub plans

Type SqlConnectorConfig^{_^✎}

Configuration for SQL databases to send dataset queries to.

Multiple databases can be configured for different datasets.

Details

Java Type: com.bakdata.conquery.models.config.SqlConnectorConfig

Supported Fields:

	Field	Type	Default	Description
✎	databaseConfigs	map from `String` to DatabaseConnectionConfig	`null`	Keys must match {@link Dataset#getDataSource()}.
✎	enabled	`boolean`	`false`
✎	withPrettyPrinting	`boolean`	`false`	Determines if generated SQL should be formatted.

Type StandaloneConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.StandaloneConfig

Supported Fields:

	Field	Type	Default	Example	Description
✎	numberOfShardNodes	`int`	`2`

Type XodusConfig^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.XodusConfig

Supported Fields:

	Field	Type	Default
✎	cipherBasicIV	`long` or `null`	`null`
✎	cipherId	`String`	`null`
✎	cipherKey	`String`	`null`
✎	envCloseForcedly	`boolean` or `null`	`null`
✎	envGatherStatistics	`boolean` or `null`	`null`
✎	envIsReadonly	`boolean` or `null`	`null`
✎	envMaxParallelReadonlyTxns	`int` or `null`	`null`
✎	envMaxParallelTxns	`int` or `null`	`null`
✎	envMonitorTxnsCheckFreq	`int` or `null`	`null`
✎	envMonitorTxnsTimeout	`@NotNull @Valid Duration`	`"10 minutes"`
✎	envReadonlyEmptyStores	`boolean` or `null`	`null`
✎	envStoreGetCacheSize	`int` or `null`	`null`
✎	envTxnDowngradeAfterFlush	`boolean` or `null`	`null`
✎	envTxnReplayMaxCount	`int` or `null`	`null`
✎	envTxnReplayTimeout	`long` or `null`	`null`
✎	fullFileReadonly	`boolean` or `null`	`null`
✎	gcEnabled	`boolean` or `null`	`null`
✎	gcFileMinAge	`int` or `null`	`null`
✎	gcFilesDeletionDelay	`int` or `null`	`null`
✎	gcFilesInterval	`int` or `null`	`null`
✎	gcMinUtilization	`int` or `null`	`null`
✎	gcRenameFiles	`boolean` or `null`	`null`
✎	gcRunPeriod	`int` or `null`	`null`
✎	gcStartIn	`int` or `null`	`null`
✎	gcTransactionAcquireTimeout	`int` or `null`	`null`
✎	gcTransactionTimeout	`int` or `null`	`null`
✎	gcUseExclusiveTransaction	`boolean` or `null`	`null`
✎	gcUtilizationFromFile	`String`	`null`
✎	gcUtilizationFromScratch	`boolean` or `null`	`null`
✎	logCacheFreePhysicalMemoryThreshold	`long` or `null`	`null`
✎	logCacheNonBlocking	`boolean` or `null`	`null`
✎	logCacheOpenFilesCount	`int` or `null`	`null`
✎	logCachePageSize	`DataSize`	`null`
✎	logCacheShared	`boolean` or `null`	`null`
✎	logCacheUseNio	`boolean` or `null`	`null`
✎	logCleanDirectoryExpected	`boolean` or `null`	`null`
✎	logClearInvalid	`boolean` or `null`	`null`
✎	logDurableWrite	`boolean` or `null`	`null`
✎	logFileSize	`@MaxDataSize(value=1, unit=DataSizeUnit.GIGABYTES) DataSize`	`"400 megabytes"`
✎	logLockId	`String`	`null`
✎	logLockTimeout	`Duration`	`"1 second"`
✎	logSyncPeriod	`long` or `null`	`null`
✎	managementEnabled	`boolean` or `null`	`null`
✎	managementOperationsRestricted	`boolean` or `null`	`null`
✎	memoryUsage	`DataSize`	`null`
✎	memoryUsagePercentage	`int` or `null`	`null`
✎	treeMaxPageSize	`int` or `null`	`null`

Type XodusStoreFactory^{_^✎}

Details

Java Type: com.bakdata.conquery.models.config.XodusStoreFactory

Supported Fields:

	Field	Type	Default	Description
✎	bufferPerWorker	`@jakarta.validation.constraints.Min(1) int`	`20`	How many slots of buffering to use before the IO thread is put to sleep.
✎	cacheBinaryStore	map from one of AUTH_GROUP, AUTH_ROLE, AUTH_USER, BUCKETS, CONCEPTS, C_BLOCKS, DATASET, ENTITY_PREVIEW, ENTITY_TO_BUCKET, EXECUTIONS, FORM_CONFIG, ID_MAPPING, IMPORTS, INTERN_TO_EXTERN, NAMESPACES, SEARCH_INDEX, SECONDARY_IDS, STRUCTURE, TABLES, WORKER, WORKER_TO_BUCKETS to `boolean` or `null`		Map of flags for each {@link StoreMappings}, whether its binary store should be fully cached. This allows for faster store access as the {@link SerializingStore} can deserialize values in parallel. For example: cacheBinaryStore = Map.of( BUCKETS, true, C_BLOCKS, true );
✎	caffeineSpec	`String`	`"softValues"`	Cache spec for deserialized values. Conquery depends currently on `softValues` to avoid data race conditions. So a specification must include this option. See CaffeineSpec
✎	directory	`Path`	`"file://./storage"`
✎	loadEnvironmentWithMissingStores	`boolean`	`false`	If set, an environment will not be loaded if it misses a required store. If not set, the environment is loaded and the application needs to create the store. This is useful if a new version introduces a new store, but will also alter the environment upon reading.
✎	loadStoresOnStart	`boolean`	`false`
✎	readerWorkers	`@jakarta.validation.constraints.Min(1) int`	`4`	Number of threads reading from XoduStore.
✎	removeUnreadableFromStore	`boolean`	`false`	Flag for the {@link SerializingStore} whether to delete values from the underlying store, that cannot be mapped to an object anymore.
✎	unreadableDataDumpDirectory	`File`	`null`	When set, all values that could not be deserialized from the persistent store, are dump into individual files.
✎	validateOnWrite	`boolean`	`false`
✎	xodus	XodusConfig

Config JSON

Config JSON

Base AuthorizationConfig

DEFAULT ✎

DEVELOPMENT ✎

Base AuthenticationRealmFactory

DEVELOPMENT ✎

JWT_PKCE_REALM ✎

LOCAL_AUTHENTICATION ✎

OIDC_AUTHORIZATION_CODE_FLOW ✎

OIDC_RESOURCE_OWNER_PASSWORD_CREDENTIAL_AUTHENTICATION ✎

Base PluginConfig

FORM_BACKEND ✎

MANUAL ✎

Other Types

Type APIConfig ✎

Type CSVConfig ✎

Type ClusterConfig ✎

Type ConqueryConfig ✎

Type CurrencyConfig ✎

Type DatabaseConnectionConfig ✎

Type Dialect ✎

Type FrontendConfig ✎

Type LocaleConfig ✎

Type MinaConfig ✎

Type PreprocessingConfig ✎

Type QueryConfig ✎

Type SqlConnectorConfig ✎

Type StandaloneConfig ✎

Type XodusConfig ✎

Type XodusStoreFactory ✎

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

DEFAULT^{_^✎}

DEVELOPMENT^{_^✎}

DEVELOPMENT^{_^✎}

JWT_PKCE_REALM^{_^✎}

LOCAL_AUTHENTICATION^{_^✎}

OIDC_AUTHORIZATION_CODE_FLOW^{_^✎}

OIDC_RESOURCE_OWNER_PASSWORD_CREDENTIAL_AUTHENTICATION^{_^✎}

FORM_BACKEND^{_^✎}

MANUAL^{_^✎}

Type APIConfig^{_^✎}

Type CSVConfig^{_^✎}

Type ClusterConfig^{_^✎}

Type ConqueryConfig^{_^✎}

Type CurrencyConfig^{_^✎}

Type DatabaseConnectionConfig^{_^✎}

Type Dialect^{_^✎}

Type FrontendConfig^{_^✎}

Type LocaleConfig^{_^✎}

Type MinaConfig^{_^✎}

Type PreprocessingConfig^{_^✎}

Type QueryConfig^{_^✎}

Type SqlConnectorConfig^{_^✎}

Type StandaloneConfig^{_^✎}

Type XodusConfig^{_^✎}

Type XodusStoreFactory^{_^✎}