Skip to main content

Cleaning Functions

Concept​

Our cleaning functions serve as callbacks triggered after specific import events, allowing you to provide feedback to the user and transform the imported data. This improves the user efficiency by enabling faster data cleaning at the desired quality without interacting directly with the user.

These functions complement the pre-built validation rules defined in the validations property. With these callbacks, you can implement advanced cleaning and validation logic to address all your use cases. It ranges from data transformation, displaying info, warnings, or customized error messages to users to performing external server requests to retrieve data, comparing this data with what's imported, and flagging duplicates with error messages. Additionally, you can enhance entries, establish dependencies between different columns, and utilize third-party APIs for verification, among other possibilities.

In the following sections, we highlight the contrast between an import process conducted without cleaning functions and an import empowered with a cleaning function. This callback not only reshapes the data but also has the capability to enrich entries using pre-existing data from your data source.

Without cleaning functionsWith cleaning functions
info

To help you get started with the cleaning functions, we provide a comprehensive library within our knowledge base. This library contains pre-written callbacks tailored to frequent data transformation requirements. These functions can be readily adapted to suit your specific use case.

Implementation​

In our cleaning functions, you can utilize JavaScript/TypeScript to its fullest. This includes variables, safe data storage, async server calls, and more.

Each cleaning function can return an array with info and value for each cell. value transforms data for "Review Entries." Using info with level and message, you can display user notifications during import. Notifications can be informational (blue), warning (yellow), or error (red). Errors affect the importing behavior. The entries with at least one error are not included in the result array object of the onResults function by default (You can change that behavior by adjusting completeImportAction). Moreover, you can customize the displayed text of the notification by setting a value for the message key.

For the implementation, it's essential to grasp the available cleaning functions and comprehend their utilization. Below, you'll discover a list providing an overview of our cleaning functions:

columnHooks​

DescriptionWithin this function, you have the ability to retrieve all the data from a specific column. You can implement data modifications and present customized UI notifications, including info messages, warnings, and/or errors
ParametercolumnName: Callback function that must have the identical name as the column to which it will be employed

values: The callback function's parameter consists of a 2d array. Each inner array contains a value at position 1 and an index at position 2
Run EventThe columnHooks function is executed before the user enters the "Review Entries"-step

In the given example, the columnHooks cleaning function is used to restructure phone numbers by eliminating leading zeros and replacing them with a plus sign:

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "customer_data",
    columns: [
      {
        label: "Customer Code",
        key: "customer_code",
        columnType: "string",
      },
      {
        label: "Phone Number",
        key: "phone_number",
        columnType: "string",
      },
    ],
  }}
  columnHooks={{
    phoneNumber: (values) => {
      values.map(([item, index]) => {
        let phoneNumber = item;
        if (/^[0]{2}/.test(phoneNumber)) {
          phoneNumber = `+${item.slice(2, item.length)}`;
        }
        return [
          {
            value: phoneNumber,
            info: [
              {
                message: 'We have automatically transformed your input into the correct format by converting "00" to "+".',
                level: "info",
              },
            ],
          },
          index,
        ];
      });
    },
  }}
>
  Import data
</NuvoImporter>

If you’re interested in exploring additional use cases, kindly navigate to our knowledge base available on the user platform.

onEntryInit​

DescriptionThis function iterates through every imported entry. In each iteration, you have access to each row value. With this cleaning function, you can perfectly define column-cross dependencies between two or more columns
Parameterrow: This refers to an object that includes column keys paired with their corresponding values for a given row
  rowIndex: Index of the current iteration
Run EventThis function is executed immediately after the columnHooks callback and also whenever a row is manually created within the "Review Entries"-step

In the following example, we check the Country field of each row after the mapping step. If the value is Germany, we add DE to its Country Code field. Similarly, if the value is Italy, we add IT to the Country Code field of that row:

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "data",
    columns: [
      {
        label: "Country",
        key: "country",
        columnType: "string",
      },
      {
        label: "Country Code",
        key: "country_code",
        columnType: "string",
      },
    ],
  }}
  onEntryInit={(row, rowIndex) => {
    if (row.country === "Germany") {
      return {
        country_code: {
          value: "DE",
        },
      };
    } else if (row.country === "Italy") {
      return {
        country_code: {
          value: "IT",
        },
      };
    }
  }}
>
  Import data
</NuvoImporter>

For additional use cases, check out our knowledge base available on the user platform.

onEntryChange​

DescriptionThis cleaning function is very similar to the onEntryInit callback function. It does not iterate through every entry because it is only applied to one entry at a time, but within the function, you can access every value of it. The only difference is the run event.
Parameterrow: This refers to an object that includes column keys paired with their corresponding values for a row that the user has edited
  rowIndex: Index of the row that the user has edited
  options: An object with parameters of the change action. It includes a field called actionType which specifies the type of action that was performed, with valid values being "edit" or "delete".
Run EventThis function is executed each time a user modifies the value of an entry/row within the "Review Entries"-step. If the user alters the values of multiple entries simultaneously, the callback function runs multiple times, once for each individual entry

If the user changes the age value within the review step in the example given, it is validated again, and an error is displayed accordingly:

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "data",
    columns: [
      {
        label: "ID",
        key: "user_id",
        columnType: "string",
      },
      {
        label: "Name",
        key: "user_name",
        columnType: "string",
      },
      {
        label: "Age",
        key: "age",
        columnType: "int",
      },
    ],
  }}
  onEntryChange={(row, rowIndex, options) => {
    if (row.age > 50) {
      return {
        age: {
          value: row.age,
          info: [
            {
              message: "Age of the user should not be greater than 50",
              level: "error",
            },
          ],
        },
      };
    }
  }}
>
  Import data
</NuvoImporter>

This callback is closely similar to onEntryInit. If you want to discover other use cases, please navigate to our knowledge base, which is available on the user platform.