Cleaning Functions
Concept
Our cleaning functions serve as callbacks triggered after specific import events, allowing you to provide feedback to the user and transform the imported data. This improves the user efficiency by enabling faster data cleaning at the desired quality without interacting directly with the user.
These functions complement the pre-built validation rules defined in the validations
property. With these callbacks, you can implement advanced cleaning and validation logic to address all your use cases. It ranges from data transformation, displaying info, warnings, or customized error messages to users to performing external server requests to retrieve data, comparing this data with what's imported, and flagging duplicates with error messages. Additionally, you can enhance entries, establish dependencies between different columns, and utilize third-party APIs for verification, among other possibilities.
In the following sections, we highlight the contrast between an import process conducted without cleaning functions and an import empowered with a cleaning function. This callback not only reshapes the data but also has the capability to enrich entries using pre-existing data from your data source.
Without cleaning functions | With cleaning functions |
---|---|
To help you get started with the cleaning functions, we provide a comprehensive library within our knowledge base. This library contains pre-written callbacks tailored to frequent data transformation requirements. These functions can be readily adapted to suit your specific use case.
Implementation
In our cleaning functions, you can utilize JavaScript/TypeScript to its fullest. This includes variables, safe data storage, async server calls, and more.
Each cleaning function can return an array with info
and value
for each cell. value
transforms data for "Review Entries." Using info
with level
and message
, you can display user notifications during import. Notifications can be informational (blue), warning (yellow), or error (red). Errors affect the importing behavior. The entries with at least one error are not included in the result
array object of the onResults
function by default (You can change that behavior by adjusting completeImportAction
). Moreover, you can customize the displayed text of the notification by setting a value for the message
key.
For the implementation, it's essential to grasp the available cleaning functions and comprehend their utilization. Below, you'll discover a list providing an overview of our cleaning functions:
columnHooks
Description | Within this function, you have the ability to retrieve all the data from a specific column. You can implement data modifications and present customized UI notifications, including info messages, warnings, and/or errors |
Parameter | columnName: Callback function that must have the identical name as the column to which it will be employedvalues: The callback function's parameter consists of a 2d array. Each inner array contains a value at position 1 and an index at position 2 |
Run Event | The columnHooks function is executed before the user enters the "Review Entries" step |
In the given example, the columnHooks cleaning function is used to restructure phone numbers by eliminating leading zeros and replacing them with a plus sign:
- React
- Angular
- Vue
- JavaScript
<NuvoImporter
licenseKey="Your License Key"
settings={{
developerMode: true,
identifier: "customer_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
columnType: "string",
},
{
label: "Phone Number",
key: "phone_number",
columnType: "string",
},
],
}}
columnHooks={{
phoneNumber: (values) => {
return values.map(([item, index]) => {
let phoneNumber = item;
if (/^[0]{2}/.test(phoneNumber)) {
phoneNumber = `+${item.slice(2, item.length)}`;
}
return [
{
value: phoneNumber,
info: [
{
message: 'We have automatically transformed your input into the correct format by converting "00" to "+".',
level: "info",
},
],
},
index,
];
});
},
}}
/>
- Add the nuvo component in the HTML file.
<nuvo-importer [licenseKey]="licenseKey" [settings]="settings" [columnHooks]="columnHooks" />
- Define the property in the component file.
licenseKey!: string
settings!: SettingsAPI;
ngOnInit(): void {
this.licenseKey = "Your License Key"
this.settings = {
developerMode: true,
identifier: "customer_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
columnType: "string"
},
{
label: "Phone Number",
key: "phone_number",
columnType: "string"
},
],
}
}
columnHooks={
phone_number: (values) => {
return values.map(([item, index]) => {
let phoneNumber = item
if (/^[0]{2}/.test(phoneNumber)) {
phoneNumber = `+${item.slice(2, item.length)}`
}
return [
{
value: phoneNumber,
info: [
{
message: 'We have automatically transformed your input into the correct format by converting "00" to "+".',
level: "info",
},
],
},
index,
]
});
},
}
We're phasing out support for Vue 2. If you're still using Vue 2, you can use "nuvo-vuejs" v2.9 or lower.
- Add the nuvo component inside the
<template>
tag in the fileApp.vue
.
<template>
<div id="app">
<NuvoImporter :settings="settings" :licenseKey="licenseKey" :columnHooks="columnHooks" />
</div>
</template>
- Define the property inside the
<script>
tag in the component file.
export default {
name: "App",
components: {
NuvoImporter,
},
setup() {
const settings = {
developerMode: true,
identifier: "customer_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
columnType: "string",
},
{
label: "Phone Number",
key: "phone_number",
columnType: "string",
},
],
};
return { settings };
},
data: () => {
return {
licenseKey: "Your License Key",
columnHooks: {
phone_number: (values) => {
return values.map(([item, index]) => {
let phoneNumber = item;
if (/^[0]{2}/.test(phoneNumber)) {
phoneNumber = `+${item.slice(2, item.length)}`;
}
return [
{
value: phoneNumber,
info: [
{
message: 'We have automatically transformed your input into the correct format by converting "00" to "+".',
level: "info",
},
],
},
index,
];
});
},
},
};
},
};
Our vanilla JS syntax has changed since v2.9. If you use v2.8 or lower, please migrate to the latest version by following our migration guide.
<div class="nuvo-container" />
<script type="module">
import { launchNuvoImporter } from "nuvo-vanilla-js";
launchNuvoImporter(".nuvo-container", {
licenseKey: "Your License Key",
settings: {
developerMode: true,
identifier: "product_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
columnType: "string",
},
{
label: "Phone Number",
key: "phone_number",
columnType: "string",
},
],
},
columnHooks: {
phone_number: (values) => {
return values.map(([item, index]) => {
let phoneNumber = item;
if (/^[0]{2}/.test(phoneNumber)) {
phoneNumber = `+${item.slice(2, item.length)}`;
}
return [
{
value: phoneNumber,
info: [
{
message: 'We have automatically transformed your input into the correct format by converting "00" to "+".',
level: "info",
},
],
},
index,
];
});
},
},
onResults: (result, errors, complete, logs) => {
console.log("Result:", result);
console.log("Error rows:", errors);
console.log("Logs:", logs);
complete();
},
});
</script>
If you’re interested in exploring additional use cases, kindly navigate to our knowledge base available on the user platform.
onEntryInit
Description | This function iterates through every imported entry. In each iteration, you have access to each row value. With this cleaning function, you can perfectly define column-cross dependencies between two or more columns |
Parameter | row: This refers to an object that includes column keys paired with their corresponding values for a given row |
rowIndex: Index of the current iteration | |
Run Event | This function is executed immediately once after the columnHooks callback |
In the following example, we check the Country
field of each row after the mapping step. If the value is Germany
, we add DE
to its Country Code
field. Similarly, if the value is Italy
, we add IT
to the Country Code
field of that row:
- React
- Angular
- Vue
- JavaScript
<NuvoImporter
licenseKey="Your License Key"
settings={{
developerMode: true,
identifier: "data",
columns: [
{
label: "Country",
key: "country",
columnType: "string",
},
{
label: "Country Code",
key: "country_code",
columnType: "string",
},
],
}}
onEntryInit={(row, rowIndex) => {
if (row.country === "Germany") {
return {
country_code: {
value: "DE",
},
};
} else if (row.country === "Italy") {
return {
country_code: {
value: "IT",
},
};
}
}}
/>
- Add the nuvo component in the HTML file.
<nuvo-importer
[licenseKey]="licenseKey"
[settings]="settings"
[onEntryInit]="onEntryInit.bind(this)"
[onEntryChange]="onEntryChange.bind(this)"
/>
- Define the property in the component file.
licenseKey!: string
settings!: SettingsAPI;
ngOnInit(): void {
this.licenseKey = "Your License Key"
this.settings = {
developerMode: true,
identifier: "data",
columns: [
{
label: "Country",
key: "country",
columnType: "string"
},
{
label: "Country Code",
key: "country_code",
columnType: "string"
}
]
}
}
onEntryInit=(row, rowIndex) => {
if (row.country === "Germany") {
return {
country_code: {
value: "DE"
}
};
} else if (row.country === "Italy") {
return {
country_code: {
value: "IT"
}
};
}
}
We're phasing out support for Vue 2. If you're still using Vue 2, you can use "nuvo-vuejs" v2.9 or lower.
- Add the nuvo component inside the
<template>
tag in the fileApp.vue
.
<template>
<div id="app">
<NuvoImporter
:settings="settings"
:licenseKey="licenseKey"
:onEntryInit="onEntryInit"
:onEntryChange="onEntryChange"
/>
</div>
</template>
- Define the property inside the
<script>
tag in the component file.
export default {
name: "App",
components: {
NuvoImporter,
},
setup() {
const settings = {
developerMode: true,
identifier: "data",
columns: [
{
label: "Country",
key: "country",
columnType: "string",
},
{
label: "Country Code",
key: "country_code",
columnType: "string",
},
],
};
return { settings };
},
data: () => {
return {
licenseKey: "Your License Key",
};
},
methods: {
onEntryInit: (row, rowIndex) => {
if (row.country === "Germany") {
return {
country_code: {
value: "DE",
},
};
} else if (row.country === "Italy") {
return {
country_code: {
value: "IT",
},
};
}
},
},
};
Our vanilla JS syntax has changed since v2.9. If you use v2.8 or lower, please migrate to the latest version by following our migration guide.
<div class="nuvo-container" />
<script type="module">
import { launchNuvoImporter } from "nuvo-vanilla-js";
launchNuvoImporter(".nuvo-container", {
licenseKey: "Your License Key",
settings: {
developerMode: true,
identifier: "product_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
columnType: "string",
},
{
label: "Phone Number",
key: "phone_number",
columnType: "string",
},
],
},
onEntryInit: (row) => {
if (row.country === "Germany") {
return {
country_code: {
value: "DE",
},
};
} else if (row.country === "Italy") {
return {
country_code: {
value: "IT",
},
};
}
},
onResults: (result, errors, complete, logs) => {
console.log("Result:", result);
console.log("Error rows:", errors);
console.log("Logs:", logs);
complete();
},
});
</script>
For additional use cases, check out our knowledge base available on the user platform.
onEntryChange
Description | This cleaning function is applied after every action involving changes, creations, or deletions of at least one row. The data for each affected row is available within this function, allowing it to handle information from multiple rows after a single interaction, such as "Replace all", dragging a value via the drag functionality, or copy-pasting. |
Parameter | rows: An array of row objects containing all rows that either have been changed, deleted or createdEach row object contains:
logs: The logs contain details about custom-added options and columns during the mapping and the review step:
|
Run Event | This function is executed each time a user creates or deletes one or multiple rows as well as each time a user changes one or multiple entries within the "Review Entries" step. Changing an entry can be done through a manual change, the drag functionality, copy & paste, find & replace and through our cleaning assistant. If the user changes the values of multiple entries simultaneously, the callback function runs only once |
If the user changes the age value within the review step in the example given, it is validated again, and an error is displayed accordingly:
- React
- Angular
- Vue
- JavaScript
<NuvoImporter
licenseKey="Your License Key"
settings={{
developerMode: true,
identifier: "data",
columns: [
{
label: "ID",
key: "user_id",
columnType: "string",
},
{
label: "Name",
key: "user_name",
columnType: "string",
},
{
label: "Age",
key: "age",
columnType: "int",
},
],
}}
onEntryChange={(rows, logs) => {
return rows
.filter((row) => Number(row.data["age"]) > 50)
.map((row) => {
return {
rowIndex: row.rowIndex,
data: {
age: {
value: row.data.age,
info: [
{
message: "Age of the user should not be greater than 50",
level: "error",
},
],
},
},
};
});
}}
/>
Here is an example of rows
if you use copy and paste, where one row is changed, and one row is updated, and here is an example of logs
if you add one column and add an additional dropdown option to the column deal_stage
during the import process:
- rows
- logs
[
{
"actionType": "edit",
"changeLog": {
"user_id": null,
"full_name": null,
"age": null
},
"info": {
"user_id": [
{
"level": "error",
"message": "This user already exists in our database."
}
]
},
"data": {
"user_id": "0118",
"full_name": "Jane Doe",
"age": "55"
},
"rowIndex": 119
},
{
"actionType": "create",
"changeLog": null,
"info": {
"age": [
{
"level": "warning",
"message": "This age is suspiciously high. Please recheck this value."
}
]
},
"data": {
"user_id": "0119",
"full_name": "Jack Smith",
"age": "45"
},
"rowIndex": 120
}
]
{
"columns": {
"addedColumns": [
{
"label": "Revenue",
"key": "revenue",
"columnType": "currency_eur"
}
],
"addedOptions": [
{
"columnKey": "deal_stage",
"dropdownOptions": [
{
"label": "Done",
"value": "done",
"type": "string"
}
]
}
]
}
}
- Add the nuvo component in the HTML file.
<nuvo-importer
[licenseKey]="licenseKey"
[settings]="settings"
[onEntryInit]="onEntryInit.bind(this)"
[onEntryChange]="onEntryChange.bind(this)"
/>
- Define the property in the component file.
licenseKey!: string
settings!: SettingsAPI;
ngOnInit(): void {
this.licenseKey = "Your License Key"
this.settings = {
developerMode: true,
identifier: "data",
columns: [
{
label: "ID",
key: "user_id",
columnType: "string"
},
{
label: "Name",
key: "user_name",
columnType: "string"
},
{
label: "Age",
key: "age",
columnType: "int"
},
]
}
}
onEntryChange={(rows, logs) => {
return rows
.filter((row) => Number(row.data['age']) > 50)
.map((row) => {
return {
rowIndex: row.rowIndex,
data: {
age: {
value: row.data.age,
info: [
{
message:
'Age of the user should not be greater than 50',
level: 'error',
},
],
},
},
};
});
}}
We're phasing out support for Vue 2. If you're still using Vue 2, you can use "nuvo-vuejs" v2.9 or lower.
- Add the nuvo component inside the
<template>
tag in the fileApp.vue
.
<template>
<div id="app">
<NuvoImporter
:settings="settings"
:licenseKey="licenseKey"
:onEntryInit="onEntryInit"
:onEntryChange="onEntryChange"
/>
</div>
</template>
- Define the property inside the
<script>
tag in the component file.
export default {
name: "App",
components: {
NuvoImporter,
},
setup() {
const settings = {
developerMode: true,
identifier: "data",
columns: [
{
label: "ID",
key: "user_id",
columnType: "string",
},
{
label: "Name",
key: "user_name",
columnType: "string",
},
{
label: "Age",
key: "age",
columnType: "int",
},
],
};
return { settings };
},
data: () => {
return {
licenseKey: "Your License Key",
};
},
methods: {
onEntryChange: (rows, logs) => {
return rows
.filter((row) => Number(row.data["age"]) > 50)
.map((row) => {
return {
rowIndex: row.rowIndex,
data: {
age: {
value: row.data.age,
info: [
{
message: "Age of the user should not be greater than 50",
level: "error",
},
],
},
},
};
});
},
},
};
Our vanilla JS syntax has changed since v2.9. If you use v2.8 or lower, please migrate to the latest version by following our migration guide.
<div class="nuvo-container" />
<script type="module">
import { launchNuvoImporter } from "nuvo-vanilla-js";
launchNuvoImporter(".nuvo-container", {
licenseKey: "Your License Key",
settings: {
developerMode: true,
identifier: "product_data",
columns: [
{
label: "Customer Code",
key: "customer_code",
columnType: "string",
},
{
label: "Phone Number",
key: "phone_number",
columnType: "string",
},
],
},
onEntryChange: (rows, logs) => {
return rows
.filter((row) => Number(row.data["age"]) > 50)
.map((row) => {
return {
rowIndex: row.rowIndex,
data: {
age: {
value: row.data.age,
info: [
{
message: "Age of the user should not be greater than 50",
level: "error",
},
],
},
},
};
});
},
onResults: (result, errors, complete, logs) => {
console.log("Result:", result);
console.log("Error rows:", errors);
console.log("Logs:", logs);
complete();
},
});
</script>
You can also manipulate the values of other rows that are not part of the onEntryChange
function. This can be a quite powerful tool to serve your use case. To see an example, check out our sandbox.
If you want to discover other use cases, please navigate to our knowledge base, which is available on the user platform.