Pipeline Templates API
A pipeline template is a blueprint for building a pipeline. You can predefine all components, such as input and output connectors, the pipeline name, and the target data model, or only some. Templates provide full flexibility.
Components of a pipeline template:
- Template name
- Pipeline name
- Input connector
- Output connector
- Target data model
- Developer mode
- Schedule
- Error threshold
Use this base URL and add the corresponding endpoint respectively:
Base URL
api-gateway.getnuvo.com/dp/api/v1/
Create
Endpoint
POST /pipeline-template/
Attributes
name
The name of the pipeline template
pipeline_name
The name of the pipeline created using the template
input_connectors
The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.
output_connectors
The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.
tdm
The ID of the set target data model
error_config
Defines how the pipeline created using the template should handle errors that might occur during pipeline execution
error_threshold
A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.
schedule_config
Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed
frequency
Sets how often the pipeline is executed. It is intertwined with interval
. For example, if frequency
is set to HOURLY
and interval
is set to 2, the pipeline is executed every 2 hours:
- HOURLY
- DAILY
- WEEKLY
- MONTHLY
interval
Sets the interval based on the frequency at which the pipeline is executed. For example, if interval
is set to 2 and frequency
is set to HOURLY
, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.
starts_on
The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.
ends_on
The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.
header_config
Defines how the header row is determined
type
Specifies whether nuvo's header detection is applied or if the set row_index
is used to determine the header row:
SMART
: nuvo's header detection is used to define the header rowSTATIC
: The row at the specifiedrow_index
is used as the header row
row_index
The index of the row that should be used as the header row if type
is set to STATIC
developer_mode
Defines if the pipeline created using the template is executed in developer mode (true
) or not (false
). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.
Payload
{
"name": "string",
"pipeline_name": "string",
"input_connectors": [
"string"
],
"output_connectors": [
"string"
],
"tdm": "string",
"error_config": {
"error_threshold": 0
},
"schedule_config": {
"frequency": "HOURLY",
"interval": 0,
"starts_on": "2024-09-02T15:38:58.450Z",
"ends_on": "2024-09-02T15:38:58.450Z"
},
"header_config": {
"type": "SMART",
"row_index": 0
},
"developer_mode": false
}
Response
Attributes
id
The ID of the pipeline template
name
The name of the pipeline template
pipeline_name
The name of the pipeline created using the template
input_connectors
The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.
output_connectors
The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.
tdm
The ID of the set target data model
error_config
Defines how the pipeline created using the template should handle errors that might occur during pipeline execution
error_threshold
A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail
schedule_config
Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed
frequency
Sets how often the pipeline is executed. It is intertwined with interval
. For example, if frequency
is set to HOURLY
and interval
is set to 2, the pipeline is executed every 2 hours:
- HOURLY
- DAILY
- WEEKLY
- MONTHLY
interval
Sets the interval based on the frequency at which the pipeline is executed. For example, if interval
is set to 2 and frequency
is set to HOURLY
, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time
starts_on
The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). The date and time cannot be in the past
ends_on
The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time
header_config
Defines how the header row is determined
type
Specifies whether nuvo's header detection is applied or if the set row_index
is used to determine the header row:
SMART
: nuvo's header detection is used to define the header rowSTATIC
: The row at the specifiedrow_index
is used as the header row
row_index
The index of the row that should be used as the header row if type
is set to STATIC
developer_mode
Defines if the pipeline created using the template is executed in developer mode (true
) or not (false
). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.
created_at
The date and time when the pipeline template was first created
created_by
Information about whom created the pipeline template
id
The ID of the user or sub-organization who created the pipeline template
name
The name of the user or sub-organization who created the pipeline template
identifier
The identifier of the user or sub-organization who created the pipeline template
type
Defines the type of user who created the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
updated_at
The date and time when the pipeline templatewas last updated
updated_by
Information about whom last updated the pipeline template
id
The ID of the user or sub-organization who last updated the pipeline template
name
The name of the user or sub-organization who last updated the pipeline template
identifier
The identifier of the user or sub-organization who last updated the pipeline template
type
Defines the type of user who last updated the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
Response
{
"data": {
"id": "string",
"name": "string",
"pipeline_name": "string",
"input_connectors": [
"string"
],
"output_connectors": [
"string"
],
"tdm": "string",
"error_config": {
"error_threshold": 0
},
"schedule_config": {
"frequency": "HOURLY",
"interval": 0,
"starts_on": "2024-09-02T15:44:41.375Z",
"ends_on": "2024-09-02T15:44:41.375Z"
},
"header_config": {
"type": "SMART",
"row_index": 0
},
"developer_mode": false,
"created_at": "2022-03-07 12:48:28.653",
"created_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER"
},
"updated_at": "2022-03-07 12:48:28.653",
"update_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER"
}
}
}
Update
Endpoint
PUT / pipeline - template / { id };
Attributes
name
The name of the pipeline template
pipeline_name
The name of the pipeline created using the template
input_connectors
The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.
output_connectors
The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.
tdm
The ID of the set target data model
error_config
Defines how the pipeline created using the template should handle errors that might occur during pipeline execution
error_threshold
A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.
schedule_config
Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed
frequency
Sets how often the pipeline is executed. It is intertwined with interval
. For example, if frequency
is set to HOURLY
and interval
is set to 2, the pipeline is executed every 2 hours:
- HOURLY
- DAILY
- WEEKLY
- MONTHLY
interval
Sets the interval based on the frequency at which the pipeline is executed. For example, if interval
is set to 2 and frequency
is set to HOURLY
, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.
starts_on
The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.
ends_on
The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.
header_config
Defines how the header row is determined
type
Specifies whether nuvo's header detection is applied or if the set row_index
is used to determine the header row:
SMART
: nuvo's header detection is used to define the header rowSTATIC
: The row at the specifiedrow_index
is used as the header row
row_index
The index of the row that should be used as the header row if type
is set to STATIC
developer_mode
Defines if the pipeline created using the template is executed in developer mode (true
) or not (false
). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows.
Payload
{
"name": "string",
"pipeline_name": "string",
"input_connectors": [
"string"
],
"output_connectors": [
"string"
],
"tdm": "string",
"error_config": {
"error_threshold": 0
},
"schedule_config": {
"frequency": "HOURLY",
"interval": 0,
"starts_on": "2024-09-02T15:38:58.450Z",
"ends_on": "2024-09-02T15:38:58.450Z"
},
"header_config": {
"type": "SMART",
"row_index": 0
},
"developer_mode": false
}
Response
Attributes
id
The ID of the pipeline template
name
The name of the pipeline template
pipeline_name
The name of the pipeline created using the template
input_connectors
The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.
output_connectors
The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.
tdm
The ID of the set target data model
error_config
Defines how the pipeline created using the template should handle errors that might occur during pipeline execution
error_threshold
A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.
schedule_config
Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed
frequency
Sets how often the pipeline is executed. It is intertwined with interval
. For example, if frequency
is set to HOURLY
and interval
is set to 2, the pipeline is executed every 2 hours:
- HOURLY
- DAILY
- WEEKLY
- MONTHLY
interval
Sets the interval based on the frequency at which the pipeline is executed. For example, if interval
is set to 2 and frequency
is set to HOURLY
, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.
starts_on
The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). The date and time cannot be in the past.
ends_on
The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g., 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time.
header_config
Defines how the header row is determined
type
Specifies whether nuvo's header detection is applied or if the set row_index
is used to determine the header row:
SMART
: nuvo's header detection is used to define the header rowSTATIC
: The row at the specifiedrow_index
is used as the header row
row_index
The index of the row that should be used as the header row if type
is set to STATIC
developer_mode
Defines if the pipeline created using the template is executed in developer mode (true
) or not (false
). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows
created_at
The date and time when the pipeline template was first created
created_by
Information about whom created the pipeline template
id
The ID of the user or sub-organization who created the pipeline template
name
The name of the user or sub-organization who created the pipeline template
identifier
The identifier of the user or sub-organization who created the pipeline template
type
Defines the type of user who created the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
updated_at
The date and time when the pipeline templatewas last updated
updated_by
Information about whom last updated the pipeline template
id
The ID of the user or sub-organization who last updated the pipeline template
name
The name of the user or sub-organization who last updated the pipeline template
identifier
The identifier of the user or sub-organization who last updated the pipeline template
type
Defines the type of user who last updated the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
Response
{
"data": {
"id": "string",
"name": "string",
"pipeline_name": "string",
"input_connectors": [
"string"
],
"output_connectors": [
"string"
],
"tdm": "string",
"error_config": {
"error_threshold": 0
},
"schedule_config": {
"frequency": "HOURLY",
"interval": 0,
"starts_on": "2024-09-02T15:44:41.375Z",
"ends_on": "2024-09-02T15:44:41.375Z"
},
"header_config": {
"type": "SMART",
"row_index": 0
},
"developer_mode": false,
"created_at": "2022-03-07 12:48:28.653",
"created_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER",
"organization": "string"
},
"updated_at": "2022-03-07 12:48:28.653",
"update_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER",
"organization": "string"
}
}
}
Read by ID
Endpoint
GET /pipeline-template/{id}
Response
Attributes
id
The ID of the pipeline template
name
The name of the pipeline template
pipeline_name
The name of the pipeline created using the template
input_connectors
The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here.
output_connectors
The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here.
tdm
The ID of the set target data model
error_config
Defines how the pipeline created using the template should handle errors that might occur during pipeline execution
error_threshold
A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail.
schedule_config
Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed
frequency
Sets how often the pipeline is executed. It is intertwined with interval
. For example, if frequency
is set to HOURLY
and interval
is set to 2, the pipeline is executed every 2 hours:
- HOURLY
- DAILY
- WEEKLY
- MONTHLY
interval
Sets the interval based on the frequency at which the pipeline is executed. For example, if interval
is set to 2 and frequency
is set to HOURLY
, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time
starts_on
The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). The date and time cannot be in the past
ends_on
The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time
header_config
Defines how the header row is determined
type
Specifies whether nuvo's header detection is applied or if the set row_index
is used to determine the header row:
SMART
: nuvo's header detection is used to define the header rowSTATIC
: The row at the specifiedrow_index
is used as the header row
row_index
The index of the row that should be used as the header row if type
is set to STATIC
developer_mode
Defines if the pipeline created using the template is executed in developer mode (true
) or not (false
). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows
created_at
The date and time when the pipeline template was first created
created_by
Information about whom created the pipeline template
id
The ID of the user or sub-organization who created the pipeline template
name
The name of the user or sub-organization who created the pipeline template
identifier
The identifier of the user or sub-organization who created the pipeline template
type
Defines the type of user who created the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
updated_at
The date and time when the pipeline templatewas last updated
updated_by
Information about whom last updated the pipeline template
id
The ID of the user or sub-organization who last updated the pipeline template
name
The name of the user or sub-organization who last updated the pipeline template
identifier
The identifier of the user or sub-organization who last updated the pipeline template
type
Defines the type of user who last updated the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
Response
{
"data": {
"id": "string",
"name": "string",
"pipeline_name": "string",
"input_connectors": [
"string"
],
"output_connectors": [
"string"
],
"tdm": "string",
"error_config": {
"error_threshold": 0
},
"schedule_config": {
"frequency": "HOURLY",
"interval": 0,
"starts_on": "2024-09-02T15:44:41.375Z",
"ends_on": "2024-09-02T15:44:41.375Z"
},
"header_config": {
"type": "SMART",
"row_index": 0
},
"developer_mode": false,
"created_at": "2022-03-07 12:48:28.653",
"created_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER"
},
"updated_at": "2022-03-07 12:48:28.653",
"update_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER"
}
}
}
Read all
To further refine the response you can use query parameters like sort
, filters
, pagination
and options
. Look at a more detailed explanation here.
Endpoint
GET /pipeline-template/
Response
Attributes
id
The ID of the pipeline template
name
The name of the pipeline template
pipeline_name
The name of the pipeline created using the template
input_connectors
The list of all input connectors used for the pipeline created using the template. Currently, we only support one input connector per pipeline. Find out more about connectors here
output_connectors
The list of all output connectors used for the pipeline created using the template. Currently, we only support one output connector per pipeline. Find out more about connectors here
tdm
The ID of the set target data model
error_config
Defines how the pipeline created using the template should handle errors that might occur during pipeline execution
error_threshold
A number between 0 and 100, representing the allowed percentage of erroneous cells during a pipeline execution. For example, if it is set to 10, it means that pipeline executions with less than 10% erroneous cells will be considered successful and will not fail
schedule_config
Defines when the pipeline created using the template is executed for the first and last time, as well as the interval at which it is executed
frequency
Sets how often the pipeline is executed. It is intertwined with interval
. For example, if frequency
is set to HOURLY
and interval
is set to 2, the pipeline is executed every 2 hours:
- HOURLY
- DAILY
- WEEKLY
- MONTHLY
interval
Sets the interval based on the frequency at which the pipeline is executed. For example, if interval
is set to 2 and frequency
is set to HOURLY
, the pipeline is executed every 2 hours. The next execution cannot be scheduled further into the future than 1 year from the set start date and time.
starts_on
The date and time when the pipeline is first executed, provided as a timestamp in UTC (e.g 2024-09-02T13:26:13.642Z). The date and time cannot be in the past
ends_on
The date and time when the pipeline is last executed, provided as a timestamp in UTC (e.g. 2024-09-02T13:26:13.642Z). This date and time cannot be earlier than the start date and time
header_config
Defines how the header row is determined
type
Specifies whether nuvo's header detection is applied or if the set row_index
is used to determine the header row:
SMART
: nuvo's header detection is used to define the header rowSTATIC
: The row at the specifiedrow_index
is used as the header row
row_index
The index of the row that should be used as the header row if type
is set to STATIC
developer_mode
Defines if the pipeline created using the template is executed in developer mode (true
) or not (false
). Use the developer mode to test pipelines in your testing environment. Pipeline executions in developer mode are free of charge. Deactivate it for production use. Please note that pipelines executed in developer mode will only output 100 rows
created_at
The date and time when the pipeline template was first created
created_by
Information about whom created the pipeline template
id
The ID of the user or sub-organization who created the pipeline template
name
The name of the user or sub-organization who created the pipeline template
identifier
The identifier of the user or sub-organization who created the pipeline template
type
Defines the type of user who created the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
updated_at
The date and time when the pipeline templatewas last updated
updated_by
Information about whom last updated the pipeline template
id
The ID of the user or sub-organization who last updated the pipeline template
name
The name of the user or sub-organization who last updated the pipeline template
identifier
The identifier of the user or sub-organization who last updated the pipeline template
type
Defines the type of user who last updated the pipeline template:
USER
: A user of your organizationSUB_ORG
: A sub-organization that is part of your organization
pagination
An object containing metadata about the result
total
The number of entries in the data array
offset
The offset set in the request parameters
limit
The limit set in the request parameters
Response
{
"data": [
{
"id": "string",
"name": "string",
"pipeline_name": "string",
"developer_mode": false,
"input_connectors": [
"string"
],
"output_connectors": [
"string"
],
"error_config": {
"error_threshold": 0
},
"schedule_config": {
"frequency": "HOURLY",
"interval": 1,
"starts_on": "2024-08-11T06:57:06.672Z",
"ends_on": "2024-08-21T06:57:06.672Z"
},
"header_config": {
"type": "SMART",
"row_index": 0
},
"created_at": "2022-03-07 12:48:28.653",
"created_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER"
},
"updated_at": "2022-03-07 12:48:28.653",
"upadted_by": {
"id": "string",
"name": "string",
"identifier": "string",
"type": "USER"
}
}
],
"pagination": {
"total": 0,
"offset": 0,
"limit": 0
}
}
Delete
Endpoint
DELETE /pipeline-template/{id}
Response
Attributes
message
Message confirming the deletion of the pipeline template or providing an error message
Response
{
"data": {
"message": "string"
}
}