Pipeline Templates API

A pipeline template is a blueprint for building a pipeline. You can predefine all components, such as input and output connectors, the pipeline name, and the target data model, or only some. Templates provide full flexibility.

Components of a pipeline template:

• Template name
• Pipeline name
• Input connector
• Output connector
• Target data model
• Developer mode
• Schedule
• Error threshold

Use this base URL and add the corresponding endpoint respectively:

Base URL

api-gateway.getnuvo.com/dp/api/v1/

Create

Endpoint

POST /pipeline-template/

Attributes

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

• HOURLY
• DAILY
• WEEKLY
• MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.

header_config

Defines how the header row is determined

type

Specifies whether nuvo's header detection is applied or if the set row_index is used to determine the header row:

• SMART: nuvo's header detection is used to define the header row
• STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.

Payload

{ "name": string, "pipeline_name": string, "input_connectors": [ string ], "output_connectors": [ string ], "tdm": string, "error_config": { "error_threshold": number }, "schedule_config": { "frequency": "HOURLY", "interval": number, "starts_on": "2024-09-02T15:38:58.450Z", "ends_on": "2024-09-02T15:38:58.450Z" }, "header_config": { "type": "SMART", "row_index": number }, "developer_mode": boolean (default: false) }

Response

Attributes

id

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

• HOURLY
• DAILY
• WEEKLY
• MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). The date and time cannot be in the past

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time

header_config

Defines how the header row is determined

type

Specifies whether nuvo's header detection is applied or if the set row_index is used to determine the header row:

• SMART: nuvo's header detection is used to define the header row
• STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

id

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

id

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

Response

{ "data": { "id": string, "name": string, "pipeline_name": string, "input_connectors": [ string ], "output_connectors": [ string ], "tdm": string, "error_config": { "error_threshold": number }, "schedule_config": { "frequency": "HOURLY", "interval": number, "starts_on": "2024-09-02T15:44:41.375Z", "ends_on": "2024-09-02T15:44:41.375Z" }, "header_config": { "type": "SMART", "row_index": number }, "developer_mode": boolean (default: false), "created_at": "2022-03-07 12:48:28.653", "created_by": { "id": string, "name": string, "identifier": string, "type": "USER" }, "updated_at": "2022-03-07 12:48:28.653", "update_by": { "id": string, "name": string, "identifier": string, "type": "USER" } } }

Update

Endpoint

PUT /pipeline-template/{id}

Attributes

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

• HOURLY
• DAILY
• WEEKLY
• MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.

header_config

Defines how the header row is determined

type

Specifies whether nuvo's header detection is applied or if the set row_index is used to determine the header row:

• SMART: nuvo's header detection is used to define the header row
• STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.

Payload

{ "name": string, "pipeline_name": string, "input_connectors": [ string ], "output_connectors": [ string ], "tdm": string, "error_config": { "error_threshold": number }, "schedule_config": { "frequency": "HOURLY", "interval": number, "starts_on": "2024-09-02T15:38:58.450Z", "ends_on": "2024-09-02T15:38:58.450Z" }, "header_config": { "type": "SMART", "row_index": number }, "developer_mode": boolean (default: false) }

Response

Attributes

id

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

• HOURLY
• DAILY
• WEEKLY
• MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.

header_config

Defines how the header row is determined

type

Specifies whether nuvo's header detection is applied or if the set row_index is used to determine the header row:

• SMART: nuvo's header detection is used to define the header row
• STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

id

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

id

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

Response

{ "data": { "id": string, "name": string, "pipeline_name": string, "input_connectors": [ string ], "output_connectors": [ string ], "tdm": string, "error_config": { "error_threshold": number }, "schedule_config": { "frequency": "HOURLY", "interval": number, "starts_on": "2024-09-02T15:44:41.375Z", "ends_on": "2024-09-02T15:44:41.375Z" }, "header_config": { "type": "SMART", "row_index": number }, "developer_mode": boolean (default: false), "created_at": "2022-03-07 12:48:28.653", "created_by": { "id": string, "name": string, "identifier": string, "type": "USER", "organization": string }, "updated_at": "2022-03-07 12:48:28.653", "update_by": { "id": string, "name": string, "identifier": string, "type": "USER", "organization": string } } }

Read by ID

Endpoint

GET /pipeline-template/{id}

Response

Attributes

id

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

• HOURLY
• DAILY
• WEEKLY
• MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). The date and time cannot be in the past

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time

header_config

Defines how the header row is determined

type

Specifies whether nuvo's header detection is applied or if the set row_index is used to determine the header row:

• SMART: nuvo's header detection is used to define the header row
• STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

id

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

id

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

Response

{ "data": { "id": string, "name": string, "pipeline_name": string, "input_connectors": [ string ], "output_connectors": [ string ], "tdm": string, "error_config": { "error_threshold": number }, "schedule_config": { "frequency": "HOURLY", "interval": number, "starts_on": "2024-09-02T15:44:41.375Z", "ends_on": "2024-09-02T15:44:41.375Z" }, "header_config": { "type": "SMART", "row_index": number }, "developer_mode": boolean (default: false), "created_at": "2022-03-07 12:48:28.653", "created_by": { "id": string, "name": string, "identifier": string, "type": "USER" }, "updated_at": "2022-03-07 12:48:28.653", "update_by": { "id": string, "name": string, "identifier": string, "type": "USER" } } }

Read all

To further refine the response you can use query parameters like sort, filters, pagination and options. Look at a more detailed explanation here.

Endpoint

GET /pipeline-template/

Response

Attributes

id

The ID of the pipeline template

name

The name of the pipeline template

pipeline_name

The name of the pipeline created using the template

input_connectors

The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here

output_connectors

The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here

tdm

The ID of the set target data model

error_config

Defines how the pipeline created using the template should handle errors that might occur during pipeline execution

error_threshold

A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail

schedule_config

Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed

frequency

Sets how often the pipeline is executed. It is intertwined with interval. For example, if frequency is set to HOURLY and interval is set to 2, the pipeline is executed every 2 hours:

• HOURLY
• DAILY
• WEEKLY
• MONTHLY

interval

Sets the interval based on the frequency at which the pipeline is executed. For example, if interval is set to 2 and frequency is set to HOURLY, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.

starts_on

The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g 2024-09-02T13:26:13.642Z). The date and time cannot be in the past

ends_on

The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time

header_config

Defines how the header row is determined

type

Specifies whether nuvo's header detection is applied or if the set row_index is used to determine the header row:

• SMART: nuvo's header detection is used to define the header row
• STATIC: The row at the specified row_index is used as the header row

row_index

The index of the row that should be used as the header row if type is set to STATIC

developer_mode

Defines if the pipeline created using the template is executed in developer mode (true) or not (false). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows

created_at

The date and time when the pipeline template was first created

created_by

Information about whom created the pipeline template

id

The ID of the user or sub-organization who created the pipeline template

name

The name of the user or sub-organization who created the pipeline template

identifier

The identifier of the user or sub-organization who created the pipeline template

type

Defines the type of user who created the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

updated_at

The date and time when the pipeline templatewas last updated

updated_by

Information about whom last updated the pipeline template

id

The ID of the user or sub-organization who last updated the pipeline template

name

The name of the user or sub-organization who last updated the pipeline template

identifier

The identifier of the user or sub-organization who last updated the pipeline template

type

Defines the type of user who last updated the pipeline template:

• USER: A user of your organization
• SUB_ORG: A sub-organization that is part of your organization

pagination

An object containing metadata about the result

total

The number of entries in the data array

offset

The offset set in the request parameters

limit

The limit set in the request parameters

Response

{ "data": [ { "id": string, "name": string, "pipeline_name": string, "developer_mode": boolean (default: false), "input_connectors": [ string ], "output_connectors": [ string ], "error_config": { "error_threshold": number }, "schedule_config": { "frequency": "HOURLY", "interval": 1, "starts_on": "2024-08-11T06:57:06.672Z", "ends_on": "2024-08-21T06:57:06.672Z" }, "header_config": { "type": "SMART", "row_index": number }, "created_at": "2022-03-07 12:48:28.653", "created_by": { "id": string, "name": string, "identifier": string, "type": "USER" }, "updated_at": "2022-03-07 12:48:28.653", "upadted_by": { "id": string, "name": string, "identifier": string, "type": "USER" } } ], "pagination": { "total": 0, "offset": 0, "limit": 0 } }

Delete

Endpoint

DELETE /pipeline-template/{id}

Response

Attributes

message

Message confirming the deletion of the pipeline template or providing an error message

Response

{ "data": { "message": string } }