Skip to main content

Cleaning Functions

Concept

Our cleaning functions are callback functions which run after a particular event inside the importing workflow. With their usage, you can provide feedback to the user and automatically transform the imported data to your required format to help the user submit its data faster and in the preferred quality without interacting with the user directly.

The cleaning functions are a perfect addition to our validation rules, which you can define inside the validations property. You can nearly cover every use case by transforming data and displaying infos, warnings and/or errors with customized messages to the user within the "Review Entries"-Step. For instance, you can do external server calls, compare the fetched data with the imported data, and display an error at every entry, which is a duplicate. You can also enrich entries, build column-cross dependencies, merge or split multiple columns, call third-party APIs for verification purposes and many more.

info

To help you getting started with your data transformations, we provide a cleaning function library within our knowledge base. You find pre-written functions for popular data transformation use cases, that you just have to adapt to your target data model.

In the following, you can see the difference between an importing process without smart reformatting and enrichment functions and a nuvo importer with a simple cleaning function to reformat the data into the dd.mm.yyyy format and enrich the entries with data already existing inside your database:

Without cleaning functionsWith cleaning functions

Implementation & Overview

Within our cleaning functions, you can define everything what JavaScript/TypeScript offers you. You can define variables, safe data to the local storage, do asynchronise server calls and much more.

Every cleaning function can return an array object with the keys info and value for each cell. With the value key you can transform the imported data and display the result inside the "Review Entries"-Step. Using the info array with the two keys level and message enable you to display notifications for every cell to the user during the import process. You can define if you either want to show a blue info (level: "info"), a yellow warning (level: "warning") or a red error message (level: "error"). Please note that only errors affect the importing behavior. The entries with at least one error are not submitted and appended to the result array object by default (You can change that behaviour by adjusting completeImportAction). Moreover, you can customize the displayed text of the notification by setting a value for the message key.

info

Please do not return either the info array or the value if you do not want to either display a notification message within the UI or modify the imported value.

For the implementation, you need to understand which cleaning functions are available and how you can use them. In the following, you can find a list containing an overview about our three cleaning functions:

columnHooks

DescriptionInside this function, you can access all the data of one particular column. You can apply data transformations and provide feedback to the UI such as an info, a warning and/or an error with a customized message.
ParametercolumnName: Callback function that must have the same name as the column to which it is to be applied

values: Parameter of the callback with which you can access all values of the accessed column
Run EventThe columnHook function is executed before the user enters the "Review Entries" Step.
ExampleIn our example, the columnHooks Cleaning Function is used to reformat the phone number, by removing the zeros at the beginning of the phone number and adding a plus sign instead.
<NuvoImporter
licenseKey="Your License Key"
settings={{
developerMode: true,
identifier: "customer_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
},
{
label: "Customer Name",
key: "customer_name",
},
{
label: "Phone Number",
key: "phone_number",
},
]
}}
columnHooks={{ phoneNumber: (values) => {
values.map(([item, index]) => {
let phoneNumber = item
if (/^[0]{2}/.test(phoneNumber)) {
phoneNumber = `+${item.slice(2, item.length)}`
}
return [
{
value: phoneNumber,
info: [
{
message: "We have automatically transformed your input into the correct format by converting \"00\" to \"+\".",
level: "info",
},
],
},
index,
]
};
}
}}
>
Import Customers
</NuvoImporter>

onEntryInit

DescriptionThis function iterates through every imported entry/row. In each iteration, you have access to each value of the entry. With this cleaning function, you can perfectly define column-cross dependencies between two or more columns, such as merge, split or value-based regex checks.
Parameterrow: Object which contains each data point of the selected entry
  rowIndex: Index of the current iteration
Run EventThis function is directly executed after the columnHooks callback and whenever a row is created manually within the "Review Entries"-Step.
ExampleIn our example, the current stock will be calculated with the uploaded data. The stock last month and the items sold last month are subtracted to get the stock this month.
<NuvoImporter
licenseKey="Your License Key"
settings={{
developerMode: true,
identifier: "product_data",
columns: [
{
label: "Product ID",
key: "product_id",
},
{
label: "Product Name",
key: "product_name",
},
{
label: "Items Sold Last Month",
key: "items_sold_last_month",
columnType: "int",
},
{
label: "Stock Last Month",
key: "stock_last_month",
columnType: "int",
},
{
label: "Stock This Month",
key: "stock_this_month",
columnType: "int",
},
],
}}
onEntryInit={(row, rowIndex) => {
if (Number(row?.items_sold_last_month ?? 0) >= Number(row?.stock_last_month ?? 0)) {
return {
stock_this_month: {
value: 0,
info: [
{
message: "We have automatically estimated this value.",
level: "info",
},
]}
};
}
const stockThisMonth = Number(row?.stock_last_month ?? 0) - Number(row?.items_sold_last_month ?? 0)
return {
stock_this_month: {
value: stockThisMonth > 0 ? stockThisMonth : 0,
info: [
{
message: "We have automatically estimated this value.",
level: "info",
},
]
},
};
}}

>
Import Product Data
</NuvoImporter>

onEntryChange

DescriptionThis cleaning function is very similar to the onEntryInit callback function. It does not iterate through every entry because it is only applied to one entry at a time, but within the function, you can access every entry value. The only difference is the run event.
Parameterrow: Object which contains each data point of the selected entry
rowIndex: Index of the current iteration
Run EventThis function runs every time a user changes a value of an entry/row on the particular entry/row. If the user changes the values of multiple entries at once, the callback function runs on every single entry.
ExampleIn our example, the current stock will be updated when the user changes either the stock last month, or the items sold last month. Those two values are then used to calculate the stock this month.
<NuvoImporter
licenseKey="Your License Key"
settings={{
developerMode: true,
identifier: "product_data",
columns: [
{
label: "Product ID",
key: "product_id",
},
{
label: "Product Name",
key: "product_name",
},
{
label: "Items Sold Last Month",
key: "items_sold_last_month",
columnType: "int",
},
{
label: "Stock Last Month",
key: "stock_last_month",
columnType: "int",
},
{
label: "Stock This Month",
key: "stock_this_month",
columnType: "int",
},
],
}}

onEntryChange={(row, rowIndex) => {
if (Number(row?.items_sold_last_month ?? 0) >= Number(row?.stock_last_month ?? 0)) {
return {
stock_this_month: {
value: 0,
info: [
{
message: "We have automatically estimated this value.",
level: "info",
},
]}
};
}
const stockThisMonth = Number(row?.stock_last_month ?? 0) - Number(row?.items_sold_last_month ?? 0)
return {
stock_this_month: {
value: stockThisMonth > 0 ? stockThisMonth : 0,
info: [
{
message: "We have automatically estimated this value.",
level: "info",
},
]
},
};
}}
>
Import Product Data
</NuvoImporter>