REST API

Katalogue features a REST API service to provide programmatic access to most of the content.

The REST API uses the Nodejs “oidc-provider” module for authentication. This module provides an OAuth 2.0 (RFC 6749) Authorization Server with support for OpenID Connect (OIDC) and many other additional features and standards. Currently, the client credentials flow is the only supported authentication flow in Katalogue.

Architecturally, the REST API service and the frontend service (spa) is routed to the same underlying endpoints/resources in the backend service. The only difference lies in the authentication and authorization mechanisms, and there are more endpoints/resources available for the frontend service than for the REST API service.

Using the Katalogue REST API

1. Login as an admin and go to Settings -> REST API
2. Enable the service (it is disabled by default).
3. Add a client in the Clients table. Set the following properties:
   • Name this is only used to identify the client in the GUI. The Client Id used for authentication is created in the next step.
   • Description optional description to further describe purpose of the client
   • Scopes to assign permissions to access Katalogue resources (see more in the Scopes section below
4. When the new client is saved, a Client Id and Client Secret is generated and shown in a dialog in Katalogue. Save the Client Secret, it cannot be retrieved again once the dialog box is closed.
5. Use the Client Id and Client Secret to get an access token at the API_ENDPOINT_BASE_URL/oidc/token endpoint. It is also necessary to specify the OAuth scope when connecting, and it must match the scope granted to the client. See a complete list of valid scopes below.

curl -u <CLIENT_ID>:<CLIENT_SECRET> -d "grant_type=client_credentials&scope=system.read task.execute" http://localhost:8080/oidc/token

// The reponse is a JSON object like so:

{
  "access_token":"<ACCESS_TOKEN>",
  "expires_in":300,                     // Number of seconds until the access token expires
  "token_type":"Bearer",
  "scope": "system.read task.execute"   // Granted scopes for the access token
}

3. Provide the access token as a bearer token in the authorization header when connecting to an endpoint. Example:

curl -X GET -H "Authorization: Bearer <ACCESS_TOKEN>" http://localhost:8080/api/system/all

# The reponse is a JSON object that lists all systems

API Reference

A documentation of all endpoints and required scopes is available through a Swagger GUI as part of your Katalogue instance. It will be available at API_ENDPOINT_BASE_URL/api/docs when the REST API service is enabled.

Scopes

Endpoints in the Katalogue API are protected by scopes. In order to get access to a specific endpoint, the required scope for the endpoint must be granted to the client and specified when fetching an access token from the /token endpoint. Scopes in Katalogue is on the format <RESOURCE>.<ACTION>, and multiple scopes can be combined as a whitespace-separated string like so: system.read task.execute.

The “resource” specifies the group of endpoints the scope gives access to, and the “action” roughly defines what methods are allowed. The rule of thumb is that the “read” action gives access to all GET methods, and “write” also gives access to PUT, POST and DELETE methods.

The following scopes can be assigned to a client:

Scope	Description
system.read	Read permission for systems, datasources, dataset groups, datasets, fields and field descriptions
system.write	Read and write permission for systems, datasources, dataset groups, datasets, fields and field descriptions
datasource.read	Read permission for datasources, dataset groups, datasets, fields and field descriptions
datasource.write	Read and write permission for datasources, dataset groups, datasets, fields and field descriptions
dataset_group.read	Read permission for dataset groups, datasets, fields and field descriptions
dataset_group.write	Read and write permission for dataset groups, datasets, fields and field descriptions
dataset.read	Read permission for datasets, fields and field descriptions
dataset.write	Read and write permission for datasets, fields and field descriptions
field.read	Read permission for fields and field descriptions
field.write	Read and write permission for fields and field descriptions
field_description.read	Read permission for field descriptions
field_description.write	Read and write permission for field descriptions
glossary.read	Read permission for glossaries, business terms and field descriptions
glossary.write	Read and write permission for glossaries, business terms and field descriptions
business_term.read	Read permission for business terms and field descriptions
business_term.write	Read and write permission for business terms and field descriptions
custom_attribute.read	Read permission for custom attributes
custom_attribute.write	Read and write permission for custom attributes
connection.read	Read permission for connections
connection.write	Read and write permission for connections
user.read	Read permission for users and user groups
user.write	Read and write permission for users and user groups
settings.read	Read permission for application settings
settings.write	Read and write permission for application settings
task.read	Read permission for tasks and jobs
task.execute	Read and execute permission for tasks and jobs. Note that task.write is required to alter jobs.
task.write	Read and write permission for tasks and jobs. Note that task.execute is required to execute jobs.
changelog.read	Read permission for the changelog
search.read	Read permission for the search index. Required to perform searches.

Scope Resource Hierarchy

Some resources implicitly give access to other resources. E.g. the system.read scope will also give access to endpoints that require the datasource.read, dataset_group.read scopes and so on.

Here is the resource hierarchy list:

RESOURCE_HIERARCHY = {
  "system":         ["datasource", "dataset_group", "dataset", "field", "field_description"],
  "datasource":     ["dataset_group", "dataset", "field", "field_description"],
  "dataset_group":  ["dataset", "field", "field_description"],
  "dataset":        ["field", "field_description"],
  "field":          ["field_description"],
  "glossary":       ["business_term", "field_description"],
  "business_term":  ["field_description"],
}

Scope Action Hierarchy

Some actions implicitly give access to other actions. E.g. the system.read scope gives access to all GET methods in the “system” resource, while system.write in addition to GET methods also give access to PUT, POST and DELETE methods.

Here is the action hierarchy list:

ACTION_HIERARCHY = {
  "write":    ["read"],
  "execute":  ["read"],
  "read":     []
}

Advanced Configuration

Signing Key

By default, an internal private key is used to sign access tokens issued by Katalogue. This key is automatically generated in the background every time the REST API service is changed from “disabled” to “enabled”.

It is possible to use an external signing key instead of the internal key automatically generated by Katalogue. Set the config parameter REST_API_OIDC_SIGNING_KEY (either as an environment variable or in the config file) to use an external key.

The /services/api/utils/initialize.js utility tool can be used to generate external signing keys manually.

Signing Key Rotation

The signing key can be rotated manually with the “Rotate Signing Key” button in Settings -> REST API.

Note

Rotating the signing key will instantly invalidate any previously issued access tokens.

Note

Rotation of an external key must be managed externally, outside of Katalogue. If an external key is used, the Rotate Signing Key button in Settings -> REST API is disabled.

Endpoint Paths

See the config variables API_ENDPOINT_PATH, REST_API_OIDC_PATH and REST_API_DOCS_PATH to customize endpoint paths.

Access Token and Client Secret Expiration Times

See the config variables REST_API_ACCESS_TOKEN_TTL_IN_MINUTES and REST_API_CLIENT_SECRET_TTL_IN_DAYS to customize Time To Live / expiration times for access tokens and client secrets.