Target Data Models (TDMs)

Target data models define the structure of the output data in all nuvo pipelines. They ensure that the mapped and transformed data is organized according to the desired format and schema required by the destination system.

You can create, read (all/by id), update and delete target data models using:

the target data model API
the target data model embeddables (coming soon)
the nuvo user platform (coming soon)

Within a target data model, you can define the name of each column at both a UI-based level (label) and a technical level (key). Additionally, you can apply required, unique, and regex validations, as well as specify the column type (e.g., float, int, date, currency, etc.)

This ensures that the pipeline’s output not only has the correct structure but also that the values are in the correct format.

For example, if you want the output to contain objects with the keys customer_name, domain_name, region, deal_size, address, and done, you would define the target data model as follows:

Example

[
  {
    label: "Customer Name",
    key: "customer_name",
    columnType: "string"
  },
  {
    label: "Domain Name",
    key: "domain_name",
    columnType: "url"
  },
  {
    label: "Region",
    key: "region",
    columnType: "string"
  },
  {
    label: "Deal Size",
    key: "deal_size",
    columnType: "float"
  },
  {
    label: "Address",
    key: "address",
    columnType: "string"
  },
  {
    label: "Done",
    key: "done",
    columnType: "boolean"
  }
]

How to create a TDM with the nuvo pipeline API?

The nuvo pipeline API provides a convenient way to create a target data model (TDM) for importing data via a pipeline.

Learn how to authenticate yourself with our authentication API and then use the target data model API to create your TDMs.

After setting up the general structure of your target data model, we recommend you to add more validation rules so that you can ensure that the user-submitted data is not only in the correct schema but also the values are in the preferred format.

The column class includes several properties. In the following, all properties of the column class are displayed, including its default value, data type and description.

label (required)


Type	`string`
Description	`label` is one of two required properties of a column object. The value is displayed within the UI to the user, who goes through the importing workflow. This value and the `key` value is used for the column matching.

key (required)


Type	`string`
Description	The value of `key` defines how your application calls a column. This value is not displayed to your users, but like the label, it is used for column matching.

alternativeMatches

Type [ string, ... ]

Description nuvo utilizes machine learning with a logic-based component to provide an intelligent and advanced mapping functionality. To be able to force a more precise matching, we offer an additional matching layer with the alternativeMatches property. This should help match the target and imported columns more accurately because every item of the alternativeMatches array is considered for calculating the similarity between imported and target model columns. We recommend adding abbreviations and synonyms here.


Type	`[ string, ... ]`
Description	nuvo utilizes machine learning with a logic-based component to provide an intelligent and advanced mapping functionality. To be able to force a more precise matching, we offer an additional matching layer with the `alternativeMatches` property. This should help match the target and imported columns more accurately because every item of the `alternativeMatches` array is considered for calculating the similarity between imported and target model columns. We recommend adding abbreviations and synonyms here.

dataValidations


Type	`[ DataValidation, ... ]`
Description	With the data validations option, you can define the preferred format for the values in one column as well as whether the values are required or have to be unqiue. You can also add customized error messages to guide your users on how to correct their input. Each object in the `dataValidations` array can contain two keys:

logic
errorMessage

logic


Type	`string`
Description	With this option, you can write complex, conditional validation rules with logical operators and our helper functions (`unique`, `regex`, `includes`). This property offers you the possibility to ensure the highest-possible data quality (see Validation rules below for more details)

errorMessage


Type	`string`
Description	With this option, you can add an fully customized error message to each validation rule. This notification text overwrittes the pre-built error message.

Validation rules

Data Validations

Data validations are a powerful tool for ensuring data integrity within your columns. They allow you to specify criteria that a value must meet to be considered valid, and you can add multiple validations to a each column. You can define conditions using logical operators like AND (&&) and OR (||) to create complex validation rules with nested conditions. To create a validation rule, write an expression representing the valid case. If the logic returns false, the defined error message will be displayed.

We also provide helper functions that you can use in conjunction with your validation expressions to create more sophisticated rules:

unique(['<column_key>', ...])
regex(row.<column_key>, { expression: '' })
includes(['<value>', ...], <column_key>)

Here are some examples of data validations with different levels of complexity and how to add them into your column definitions:

Each entry in column_a requires a value

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "row.column_a !== ''",
        errorMessage: "Column A is required."
      }
    ],
    ...
  }
]

Each entry in column_a requires a value if the entry in column_b is empty

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "row.column_a !== '' || row.column_b !== ''",
        errorMessage: "Column A is required when Column B is empty."
      }
    ],
    ...
  }
]

Either customer_id, company_id, or employee_id have to have a value

[
  {
    key: "customer_id",
    label: "Customer ID",
    columnType: "string",
    dataValidations: [
      {
        logic: "row.customerID !== '' || row.companyID !== '' || row.employeeID !== ''",
        errorMessage: "At least one of Customer ID, Company ID, or Employee ID must be provided."
      }
    ],
    ...
  }
]

Each entry in column_a requires a value if the entry in column_b is empty AND the entry in column_c has a value

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "row.column_b !== '' || row.column_c === '' || row.column_a !== ''",
        errorMessage: "Column A is required when Column B is empty and Column C has a value."
      }
    ],
    ...
  }
]

Each entry in column_a requires a number between 100 and 99999

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "regex(row.column_a, { expression: '^\\d{3,5}$' })",
        errorMessage: "Column A must be a number with 3 to 5 digits."
      }
    ],
    ...
  }
]

All entries in column_a have to be unique

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "unique(['column_a'])",
        errorMessage: "Column A must be unique across all entries."
      }
    ],
    ...
  }
]

All entries across column_a, column_b, and column_c have to be unique

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "unique(['column_a', 'column_b', 'column_c'])",
        errorMessage: "The combination of Column A, Column B, and Column C must be unique."
      }
    ],
    ...
  }
]

column_a requires a value if column_b is "Germany" and column_c is "Hamburg"

[
  {
    key: "column_a",
    label: "Column A",
    columnType: "string",
    dataValidations: [
      {
        logic: "!(includes(['Germany', 'Canada'], row.column_b) || row.column_a !== '')",
        errorMessage: "Column A is required when Column B is Germany and Column C is Hamburg."
      }
    ],
    ...
  }
]

columnType


Type	`"int"`, `"float"`, `"string"`, and more
Optional
Description	This option allows you to define the type of the column. You can either choose if the column should contain values which are an int, a float, a string or many more.

info

You can find a full list of column types with pre-build data validation rules in our column types documentation.

outputFormat


Type	`string`
Optional & usable if	`columnType: "date"`
Description	With this key, you can support all your preferred date and timestamp formats such as `MM/DD/YYYY`, `DD.MM.YYYY`, `YYYY-MM-DD`, etc.

Use the following date variables to create your desired format:

Type	Syntax	Output
Month	M	1, 2, 3, ..., 12
Month	Mo	1st, 2nd, 3rd, ..., 12th
Month	MM	01, 02, 03, ..., 12
Month	MMM	Jan, Feb, Mar, ..., Dec
Month	MMMM	January, February, March, ..., December
Day	D	1, 2, 3, ..., 31
Day	Do	1st, 2nd, 3rd, ..., 31st
Day	DD	01, 02, 03, ..., 31
Day	DDD	1, 2, 3, ..., 365
Day	DDDD	001, 002, ..., 365
Year	Y	1970, 1971, 1972, ..., +10000
Year	YY	70, 71, 72, ..., 27
Year	YYYY	1970, 1971, 1972, ...., 2027
Year	YYYYYY	-001970, -001971, -001972, ..., +001907
Hour	H	0, 1, 2, ..., 23
Hour	HH	00, 01, 02, ..., 23
Hour	h	1, 2, 3, ..., 12
Hour	hh	01, 02, 03, ..., 12
Hour	k	1, 2, 3, ..., 24
Hour	kk	01, 02, 03, ..., 24
Minute	m	0, 1, 2, ..., 59
Minute	mm	00, 01, 02, ..., 59
Second	s	0, 1, 2, ..., 59
Second	ss	00, 01, 02, ..., 59
Time zone	Z	-07:00, -06:00, -05:00, ..., +07:00
Time zone	ZZ	-0700, -0600, -0500, ..., +0700
Unix timestamp	X	855599530642
AM/PM	A	AM, PM
AM/PM	a	am, pm
Quarter	Q	1, 2, 3, 4
Quarter	Qo	1st, 2nd, 3rd, 4th

info

This table is based on the original table from the open source library Moment.js. You can find the original table and its documentation here. Please note that the table has been adjusted. You can use all variables given in the original table apart from the Day of Week ones.

In the following, you can find an example of how to implement a date column with the format MM/DD/YYYY and a timestamp column with the format YYYY-MM-DDTHH:mm:ss:

Example

[
  {
    label: "Date",
    key: "date",
    columnType: "date",
    outputFormat: "MM/DD/YYYY",
  },
  {
    label: "Timestamp",
    key: "timestamp",
    columnType: "date",
    outputFormat: "YYYY-MM-DDTHH:mm:ss",
  },
]

numberFormat


Type	`"eu"` \| `"us"`
Default	`"eu"`
Description	This property is applicable for `int`, `float`, `currency_eur`, `currency_usd`, and `percentage` column types. It affects how the numbers will be displayed at the review step. If the value is `"eu"`, then a comma will used as a decimal delimiter, and dots will be used as thousands delimiters. If the value is `"us"`, then a dot will used as a decimal delimiter, and commas will be used as thousands delimiters.

How to create a TDM with the nuvo pipeline API?​

label (required)​

key (required)​

alternativeMatches​

dataValidations​

logic​

errorMessage​

Validation rules​

Data Validations​

columnType​

outputFormat​

numberFormat​