Data Sources

Manage external data source connections such as databases, cloud storage, and data warehouses. Register, test, and share data source connections across your organization.

All endpoints require a valid API key with the appropriate datasources:* scope.

Base URL: https://<your-instance>/api/v1

DataSource Object

{
  "id": "ds_abc123",
  "name": "production-postgres",
  "label": "Production PostgreSQL",
  "type": "postgresql",
  "category": "database",
  "status": "connected",
  "description": "Main production database for user records",
  "organizationId": "org_xyz789",
  "owner": {
    "id": "user_001",
    "email": "dev@example.com"
  },
  "permissions": {
    "allowAllUsers": false,
    "allowedUsers": ["user_002", "user_003"]
  },
  "metadata": {
    "tables": ["users", "orders", "products"],
    "schema": "public"
  },
  "createdAt": "2025-01-10T08:00:00Z",
  "updatedAt": "2025-01-18T16:30:00Z"
}

Note: The credentials field is never included in standard responses. Use the dedicated credentials endpoint to retrieve connection credentials.

`GET` /api/v1/datasources

List all data sources accessible to the authenticated user.

Scope: datasources:read

Query Parameters

Parameter	Type	Required	Default	Description
`search`	string	No	—	Search by name, label, or description
`type`	string	No	—	Filter by type: `postgresql`, `mysql`, `mongodb`, `s3`, `snowflake`, `bigquery`, `redis`, `mssql`
`category`	string	No	—	Filter by category: `database`, `storage`, `warehouse`, `cache`
`status`	string	No	—	Filter by status: `connected`, `disconnected`, `error`
`limit`	integer	No	`20`	Maximum number of results to return (1-100)
`offset`	integer	No	`0`	Number of results to skip for pagination
`sort`	string	No	`createdAt:desc`	Sort field and direction, e.g. `name:asc`, `type:asc`

Response `200 OK`

{
  "data": [
    {
      "id": "ds_abc123",
      "name": "production-postgres",
      "label": "Production PostgreSQL",
      "type": "postgresql",
      "category": "database",
      "status": "connected",
      "description": "Main production database for user records",
      "organizationId": "org_xyz789",
      "owner": {
        "id": "user_001",
        "email": "dev@example.com"
      },
      "permissions": {
        "allowAllUsers": false,
        "allowedUsers": ["user_002"]
      },
      "metadata": {},
      "createdAt": "2025-01-10T08:00:00Z",
      "updatedAt": "2025-01-18T16:30:00Z"
    }
  ],
  "pagination": {
    "total": 15,
    "limit": 20,
    "offset": 0,
    "hasMore": false
  }
}

`POST` /api/v1/datasources

Scope: datasources:write

Request Body

Field	Type	Required	Description
`name`	string	Yes	Unique identifier name (lowercase, hyphens allowed)
`label`	string	Yes	Human-readable display name
`type`	string	Yes	Data source type: `postgresql`, `mysql`, `mongodb`, `s3`, `snowflake`, `bigquery`, `redis`, `mssql`
`credentials`	object	Yes	Connection credentials (type-specific, see examples below)
`description`	string	No	Description of the data source
`category`	string	No	Category: `database`, `storage`, `warehouse`, `cache`
`metadata`	object	No	Additional metadata such as schema, tables, or region

Database credentials example:

{
  "name": "analytics-mysql",
  "label": "Analytics MySQL",
  "type": "mysql",
  "credentials": {
    "host": "db.example.com",
    "port": 3306,
    "username": "analytics_reader",
    "password": "secure-password-here",
    "database": "analytics"
  },
  "description": "Read-only MySQL connection for analytics queries",
  "category": "database",
  "metadata": {
    "schema": "public"
  }
}

S3 credentials example:

{
  "name": "data-lake-s3",
  "label": "Data Lake S3",
  "type": "s3",
  "credentials": {
    "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
    "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "region": "us-east-1",
    "bucket": "my-data-lake"
  },
  "description": "S3 bucket for raw data ingestion",
  "category": "storage"
}

Snowflake credentials example:

{
  "name": "warehouse-snowflake",
  "label": "Snowflake Warehouse",
  "type": "snowflake",
  "credentials": {
    "account": "xy12345.us-east-1",
    "username": "ETL_USER",
    "password": "secure-password-here",
    "warehouse": "COMPUTE_WH",
    "database": "ANALYTICS",
    "schema": "PUBLIC"
  },
  "description": "Snowflake data warehouse for reporting",
  "category": "warehouse"
}

Response `201 Created`

{
  "dataSourceId": "ds_ghi789"
}

`GET` /api/v1/datasources/:id

Retrieve a single data source by its ID. Credentials are excluded from the response.

Scope: datasources:read

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Response `200 OK`

{
  "id": "ds_abc123",
  "name": "production-postgres",
  "label": "Production PostgreSQL",
  "type": "postgresql",
  "category": "database",
  "status": "connected",
  "description": "Main production database for user records",
  "organizationId": "org_xyz789",
  "owner": {
    "id": "user_001",
    "email": "dev@example.com"
  },
  "permissions": {
    "allowAllUsers": false,
    "allowedUsers": ["user_002", "user_003"]
  },
  "metadata": {
    "tables": ["users", "orders", "products"],
    "schema": "public"
  },
  "createdAt": "2025-01-10T08:00:00Z",
  "updatedAt": "2025-01-18T16:30:00Z"
}

`PUT` /api/v1/datasources/:id

Update an existing data source. Only provided fields are updated. The type field cannot be changed after creation.

Scope: datasources:write

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Request Body

Field	Type	Required	Description
`name`	string	No	Updated identifier name
`label`	string	No	Updated display name
`credentials`	object	No	Updated connection credentials
`description`	string	No	Updated description
`category`	string	No	Updated category
`metadata`	object	No	Updated metadata

{
  "label": "Production PostgreSQL (Primary)",
  "credentials": {
    "host": "db-primary.example.com",
    "port": 5432,
    "username": "app_user",
    "password": "new-secure-password",
    "database": "app_db"
  },
  "description": "Updated primary production database connection"
}

Response `200 OK`

Returns the updated DataSource object (credentials excluded).

`DELETE` /api/v1/datasources/:id

Permanently delete a data source registration. This does not affect the external data source itself, only the connection record on the platform.

Scope: datasources:write

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Response `204 No Content`

No response body.

`POST` /api/v1/datasources/:id/test

Test the connection to the external data source using the stored credentials. Returns the connectivity status and any error details.

Scope: datasources:read

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Response `200 OK` (success)

{
  "success": true,
  "message": "Connection successful",
  "latencyMs": 45,
  "details": {
    "serverVersion": "PostgreSQL 15.4",
    "database": "app_db"
  }
}

Response `200 OK` (failure)

{
  "success": false,
  "message": "Connection failed",
  "error": "Connection refused: host db.example.com port 5432",
  "details": {}
}

`GET` /api/v1/datasources/:id/metadata

Retrieve the schema and structural metadata of the connected data source. For databases, this returns tables, columns, and types. For storage sources, this returns bucket contents or directory structure.

Scope: datasources:read

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Response `200 OK` (database example)

{
  "type": "postgresql",
  "database": "app_db",
  "schemas": [
    {
      "name": "public",
      "tables": [
        {
          "name": "users",
          "columns": [
            { "name": "id", "type": "uuid", "nullable": false, "primaryKey": true },
            { "name": "email", "type": "varchar(255)", "nullable": false, "primaryKey": false },
            { "name": "name", "type": "varchar(100)", "nullable": true, "primaryKey": false },
            { "name": "created_at", "type": "timestamp", "nullable": false, "primaryKey": false }
          ],
          "rowCount": 15420
        },
        {
          "name": "orders",
          "columns": [
            { "name": "id", "type": "uuid", "nullable": false, "primaryKey": true },
            { "name": "user_id", "type": "uuid", "nullable": false, "primaryKey": false },
            { "name": "total", "type": "decimal(10,2)", "nullable": false, "primaryKey": false },
            { "name": "status", "type": "varchar(50)", "nullable": false, "primaryKey": false }
          ],
          "rowCount": 84210
        }
      ]
    }
  ]
}

Response `200 OK` (S3 example)

{
  "type": "s3",
  "bucket": "my-data-lake",
  "region": "us-east-1",
  "prefixes": [
    "raw/",
    "processed/",
    "archive/"
  ],
  "objectCount": 12450,
  "totalSizeBytes": 5368709120
}

`GET` /api/v1/datasources/:id/credentials

Retrieve the stored connection credentials for a data source. Handle the response securely and avoid logging credential values.

Scope: datasources:read

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Response `200 OK` (database)

{
  "host": "db.example.com",
  "port": 5432,
  "username": "app_user",
  "password": "secure-password-here",
  "database": "app_db",
  "connectionString": "postgresql://app_user:secure-password-here@db.example.com:5432/app_db"
}

Response `200 OK` (S3)

{
  "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
  "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "region": "us-east-1",
  "bucket": "my-data-lake"
}

`PUT` /api/v1/datasources/:id/permissions

Update access permissions for the data source within the organization.

Scope: datasources:write

Path Parameters

Parameter	Type	Required	Description
`id`	string	Yes	The data source ID

Request Body

Field	Type	Required	Description
`allowAllUsers`	boolean	No	If `true`, all organization members can access this data source
`allowedUsers`	array	No	List of user IDs granted access (ignored if `allowAllUsers` is `true`)

{
  "allowAllUsers": false,
  "allowedUsers": ["user_002", "user_003", "user_005"]
}

Response `200 OK`

{
  "dataSourceId": "ds_abc123",
  "permissions": {
    "allowAllUsers": false,
    "allowedUsers": ["user_002", "user_003", "user_005"]
  }
}

Error Responses

All endpoints may return the following error responses:

Status	Description
`400 Bad Request`	Invalid request body or parameters
`401 Unauthorized`	Missing or invalid API key
`403 Forbidden`	Insufficient scope for the requested operation
`404 Not Found`	Data source not found or not accessible
`409 Conflict`	Data source name already exists in the organization
`422 Unprocessable Entity`	Validation error on request body
`429 Too Many Requests`	Rate limit exceeded
`500 Internal Server Error`	Unexpected server error

{
  "error": {
    "code": "validation_error",
    "message": "Field 'credentials.host' is required for type 'postgresql'",
    "status": 422
  }
}

DataSource Object​

GET /api/v1/datasources​

Query Parameters​

Response 200 OK​

POST /api/v1/datasources​

Request Body​

Response 201 Created​

GET /api/v1/datasources/:id​

Path Parameters​

Response 200 OK​

PUT /api/v1/datasources/:id​

Path Parameters​

Request Body​

Response 200 OK​

DELETE /api/v1/datasources/:id​

Path Parameters​

Response 204 No Content​

POST /api/v1/datasources/:id/test​

Path Parameters​

Response 200 OK (success)​

Response 200 OK (failure)​

GET /api/v1/datasources/:id/metadata​

Path Parameters​

Response 200 OK (database example)​

Response 200 OK (S3 example)​

GET /api/v1/datasources/:id/credentials​

Path Parameters​

Response 200 OK (database)​

Response 200 OK (S3)​

PUT /api/v1/datasources/:id/permissions​

Path Parameters​

Request Body​

Response 200 OK​

Error Responses​

DataSource Object

`GET` /api/v1/datasources

Query Parameters

Response `200 OK`

`POST` /api/v1/datasources

Request Body

Response `201 Created`

`GET` /api/v1/datasources/:id

Path Parameters

Response `200 OK`

`PUT` /api/v1/datasources/:id

Path Parameters

Request Body

Response `200 OK`

`DELETE` /api/v1/datasources/:id

Path Parameters

Response `204 No Content`

`POST` /api/v1/datasources/:id/test

Path Parameters

Response `200 OK` (success)

Response `200 OK` (failure)

`GET` /api/v1/datasources/:id/metadata

Path Parameters

Response `200 OK` (database example)

Response `200 OK` (S3 example)

`GET` /api/v1/datasources/:id/credentials

Path Parameters

Response `200 OK` (database)

Response `200 OK` (S3)

`PUT` /api/v1/datasources/:id/permissions

Path Parameters

Request Body

Response `200 OK`

Error Responses