REST API

Katalogue features a REST API service to provide programmatic access to most of the content.

The REST API uses the Nodejs “oidc-provider” module for authentication. This module provides an OAuth 2.0 (RFC 6749) Authorization Server with support for OpenID Connect (OIDC) and many other additional features and standards. Currently, the client credentials flow is the only supported authentication flow in Katalogue.

Architecturally, the REST API service and the frontend service (spa) is routed to the same underlying endpoints/resources in the backend service. The only difference lies in the authentication and authorization mechanisms, and there are more endpoints/resources available for the frontend service than for the REST API service.

API Reference

A live documentation of all endpoints and required scopes can be found here:

https://demo-api.katalogue.se/docs.

The same API documentation is also bundled with your Katalogue instance. It will be available by default at API_ENDPOINT_BASE_URL/docs when the REST API service is enabled. The documentation path can be customized by changing the REST_API_DOCS_PATH config parameter.

Using the Katalogue REST API

1. Login as an admin and go to Settings -> REST API
2. Enable the service (it is disabled by default).
3. Add a client in the Clients table. Set the following properties:
   • Name this is only used to identify the client in the GUI. The Client Id used for authentication is created in the next step.
   • Description optional description to further describe purpose of the client
   • Scopes to assign permissions to access Katalogue resources (see more in the Scopes section below
4. When the new client is saved, a Client Id and Client Secret is generated and shown in a dialog in Katalogue. Save the Client Secret, it cannot be retrieved again once the dialog box is closed.
5. Use the Client Id and Client Secret to get an access token at the API_ENDPOINT_BASE_URL/oidc/token endpoint. It is also necessary to specify the OAuth scope when connecting, and it must match or be a subset of the scopes granted to the client. See a complete list of valid scopes below.

curl -u <CLIENT_ID>:<CLIENT_SECRET> -d "grant_type=client_credentials&scope=system.read task.execute" http://localhost:8080/oidc/token

// The reponse is a JSON object like so:

{
  "access_token":"<ACCESS_TOKEN>",
  "expires_in":300,                     // Number of seconds until the access token expires
  "token_type":"Bearer",
  "scope": "system.read task.execute"   // Granted scopes for the access token
}

3. Provide the access token as a bearer token in the authorization header when connecting to an endpoint. Example:

curl -X GET -H "Authorization: Bearer <ACCESS_TOKEN>" http://localhost:8080/api/system/all

# The reponse is a JSON object that lists all systems

Query Parameters

Read endpoints can accept query parameters that narrow the result, so a client asks for the rows it wants instead of fetching everything and sifting through it. The syntax follows the OData conventions, though Katalogue implements a deliberate subset rather than the full protocol.

Two parameters are available today:

Parameter	Purpose
$filter	Return only the rows matching a filter expression.
$top	Return at most this many rows.

They are accepted by almost all GET /<resource>/all endpoints. The API Reference remains authoritative — each parameter an endpoint accepts is listed under that endpoint.

Not the same as a datasource filter

Katalogue has two filter syntaxes, for two different jobs. $filter here narrows what you read out of Katalogue through the API. A datasource filter narrows what is imported into Katalogue from a source system, is written as a JSON object on a Connection, and uses different operator names — equal and ilike where $filter uses eq and contains.

Filter Expressions

A filter expression compares a property of the resource to a value. Properties are the ones the endpoint returns, and anything else is rejected — so a filter can never reach data the endpoint would not have returned anyway.

# Fields belonging to one dataset group
curl -X GET -H "Authorization: Bearer <ACCESS_TOKEN>" \
  --get --data-urlencode '$filter=dataset_group_id eq 51' \
  http://localhost:8080/api/field/all

Expressions can be combined with and and or, and grouped with parentheses. and binds tighter than or, so a eq 1 or b eq 2 and c eq 3 means a eq 1 or (b eq 2 and c eq 3).

# Fields in one dataset group whose name contains "customer", at most 200 of them
curl -X GET -H "Authorization: Bearer <ACCESS_TOKEN>" \
  --get \
  --data-urlencode '$filter=dataset_group_id eq 51 and contains(tolower(field_name),'"'"'customer'"'"')' \
  --data-urlencode '$top=200' \
  http://localhost:8080/api/field/all

Valid Operators And Functions

Operator	Meaning
eq	equal to
ne	not equal to
gt	greater than
ge	greater than or equal to
lt	less than
le	less than or equal to
in	is one of a list, e.g. dataset_id in (1,2,3)

Function	Meaning
contains(field,'text')	the property contains this substring
tolower(field) / toupper(field)	fold case before comparing

Values may be a quoted string, a number, true, false or null. Escape a quote inside a string by doubling it: 'it''s'. Use eq null and ne null for null checks — eq '' compares against an empty string, which is a different thing.

Case Sensitivity

String comparison is case-sensitive, as OData specifies. Wrap the property in tolower() to compare without regard to case:

contains(field_name,'ID')             # case-sensitive
contains(tolower(field_name),'id')    # case-insensitive

The difference is not academic: on one dataset group in a test instance the first matched 43 fields and the second 723.

Result Size

$top has no default. An endpoint that returns every row today still does, so adding a filter to an existing integration never silently truncates it. An explicit $top may not exceed the API_QUERY_MAX_TOP setting.

Wildcards

contains() matches a substring, so no wildcard character is needed. % and _ are matched literally — searching for 100% finds the text “100%” rather than everything beginning with “100”.

Errors

An unusable query parameter is rejected with 400 and a message naming the problem, rather than being ignored. That includes an unknown property or operator, a malformed expression, and a $top above the configured maximum. A parameter that is absent is simply not applied.

Scopes

Endpoints in the Katalogue API are protected by scopes. In order to get access to a specific endpoint, the required scope for the endpoint must be granted to the client and specified when fetching an access token from the /token endpoint. Scopes in Katalogue is on the format <RESOURCE>.<ACTION>, and multiple scopes can be combined as a whitespace-separated string like so: system.read task.execute.

The “resource” specifies the group of endpoints the scope gives access to, and the “action” roughly defines what methods are allowed. The rule of thumb is that the “read” action gives access to all GET methods, and “write” also gives access to PUT, POST and DELETE methods.

Scope Reference

The following scopes can be assigned to a client:

Scope	Description
system.read	Read permission for systems, datasources, dataset groups, datasets, fields and attributes
system.write	Read and write permission for systems, datasources, dataset groups, datasets, fields and attributes
datasource.read	Read permission for datasources, dataset groups, datasets, fields and attributes
datasource.write	Read and write permission for datasources, dataset groups, datasets, fields and attributes
dataset_group.read	Read permission for dataset groups, datasets, fields and attributes
dataset_group.write	Read and write permission for dataset groups, datasets, fields and attributes
dataset.read	Read permission for datasets, fields and attributes
dataset.write	Read and write permission for datasets, fields and attributes
field.read	Read permission for fields and attributes
field.write	Read and write permission for fields and attributes
attribute.read	Read permission for attributes
attribute.write	Read and write permission for attributes
glossary.read	Read permission for glossaries, entities and attributes
glossary.write	Read and write permission for glossaries, entities and attributes
entity.read	Read permission for entities and attributes
entity.write	Read and write permission for entities and attributes
custom_property.read	Read permission for custom properties
custom_property.write	Read and write permission for custom properties
connection.read	Read permission for connections
connection.write	Read and write permission for connections
user.read	Read permission for users and user groups
user.write	Read and write permission for users and user groups
settings.read	Read permission for application settings
settings.write	Read and write permission for application settings
task.read	Read permission for tasks and jobs
task.execute	Read and execute permission for tasks and jobs. Note that task.write is required to alter jobs.
task.write	Read and write permission for tasks and jobs. Note that task.execute is required to execute jobs.
changelog.read	Read permission for the changelog
search.read	Read permission for the search index. Required to perform searches.

Scope Resource Hierarchy

Some resources implicitly give access to other resources. E.g. the system.read scope will also give access to endpoints that require the datasource.read, dataset_group.read scopes and so on.

Here is the resource hierarchy list:

RESOURCE_HIERARCHY = {
  "system":         ["datasource", "dataset_group", "dataset", "field", "attribute"],
  "datasource":     ["dataset_group", "dataset", "field", "attribute"],
  "dataset_group":  ["dataset", "field", "attribute"],
  "dataset":        ["field", "attribute"],
  "field":          ["attribute"],
  "glossary":       ["entity", "attribute"],
  "entity":         ["attribute"],
}

Scope Action Hierarchy

Some actions implicitly give access to other actions. E.g. the system.read scope gives access to all GET methods in the “system” resource, while system.write in addition to GET methods also give access to PUT, POST and DELETE methods.

Here is the action hierarchy list:

ACTION_HIERARCHY = {
  "write":    ["read"],
  "execute":  ["read"],
  "read":     []
}

Advanced Configuration

Signing Key

By default, an internal private key is used to sign access tokens issued by Katalogue. This key is automatically generated in the background every time the REST API service is changed from “disabled” to “enabled”.

It is possible to use an external signing key instead of the internal key automatically generated by Katalogue. Set the config parameter REST_API_OIDC_SIGNING_KEY (either as an environment variable or in the config file) to use an external key.

The /services/api/utils/initialize.js utility tool can be used to generate external signing keys manually.

Signing Key Rotation

The signing key can be rotated manually with the “Rotate Signing Key” button in Settings -> REST API.

Note

Rotating the signing key will instantly invalidate any previously issued access tokens.

Note

Rotation of an external key must be managed externally, outside of Katalogue. If an external key is used, the Rotate Signing Key button in Settings -> REST API is disabled.

Endpoint Paths

See the config variables API_ENDPOINT_PATH, REST_API_OIDC_PATH and REST_API_DOCS_PATH to customize endpoint paths.

Access Token and Client Secret Expiration Times

See the config variables REST_API_ACCESS_TOKEN_TTL_IN_MINUTES and REST_API_CLIENT_SECRET_TTL_IN_DAYS to customize Time To Live / expiration times for access tokens and client secrets.

Rate Limiting

All API requests are subject to rate limiting. It is enabled by default and can be toggled with the RATE_LIMIT_ENABLED config parameter. Rate limiting uses a two-tier architecture:

Tier 1 (IP floor) runs globally before authentication and is keyed on the client IP address. It is intended as a DoS flood barrier — limits are set generously to accommodate multiple legitimate users sharing a corporate NAT or VPN.

Tier 2 (per-client) runs after the access token is verified and is keyed on the client_id from the access token. This is the primary enforcement layer for individual client quotas and is configured via the RATE_LIMIT_PER_USER_* parameters.

Read requests use separate limiters at each tier depending on the client type:

• Bearer token clients (REST API integrations) are subject to a higher-ceiling read limiter at both tiers (RATE_LIMIT_READ_API_POINTS / RATE_LIMIT_PER_USER_READ_API_POINTS).
• GUI users and unauthenticated requests use a lower read limit (RATE_LIMIT_READ_POINTS / RATE_LIMIT_PER_USER_READ_POINTS).

The higher ceiling for Bearer clients is intentional — integrations legitimately issue far more concurrent reads than a human user does.

Write requests (POST/PUT/DELETE) use a shared write limiter at each tier (RATE_LIMIT_WRITE_POINTS / RATE_LIMIT_PER_USER_WRITE_POINTS) regardless of client type.

Token requests (POST <oidc-path>/token) are subject to a dedicated login limiter with exponential backoff: the first block lasts RATE_LIMIT_LOGIN_DURATION_IN_SECONDS (default 60 s) and doubles on each subsequent violation burst, up to a 1-hour cap. This protects against credential-stuffing attacks on the token endpoint without permanently blocking legitimate clients.

When a limit is exceeded, the API responds with HTTP 429, a Retry-After header indicating how many seconds to wait before retrying and an error message like this:

{ "ok": false, "message": "Too many requests. Try again in 30s." }

Standard rate-limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) are included on every response by default. They can be suppressed by setting RATE_LIMIT_HEADERS_ENABLED to false. The Retry-After header on HTTP 429 responses is always present regardless of this setting.

All rate limits can be configured, see the Config Parameter Reference for all available tuning options.