Skip to main content

Cleaning Functions

Concept​

Our cleaning functions serve as callbacks triggered after specific import events, allowing you to provide feedback to the user and transform the imported data. This improves the user efficiency by enabling faster data cleaning at the desired quality without interacting directly with the user.

These functions complement the pre-built validation rules defined in the validations property. With these callbacks, you can implement advanced cleaning and validation logic to address all your use cases. It ranges from data transformation, displaying info, warnings, or customized error messages to users to performing external server requests to retrieve data, comparing this data with what's imported, and flagging duplicates with error messages. Additionally, you can enhance entries, establish dependencies between different columns, and utilize third-party APIs for verification, among other possibilities.

In the following sections, we highlight the contrast between an import process conducted without cleaning functions and an import empowered with a cleaning function. This callback not only reshapes the data but also has the capability to enrich entries using pre-existing data from your data source.

Without cleaning functionsWith cleaning functions
info

To help you get started with the cleaning functions, we provide a comprehensive library within our knowledge base. This library contains pre-written callbacks tailored to frequent data transformation requirements. These functions can be readily adapted to suit your specific use case.

Implementation​

In our cleaning functions, you can utilize JavaScript/TypeScript to its fullest. This includes variables, safe data storage, async server calls, and more.

Each cleaning function can return an array with info and value for each cell. value transforms data for "Review Entries." Using info with level and message, you can display user notifications during import. Notifications can be informational (blue), warning (yellow), or error (red). Errors affect the importing behavior. The entries with at least one error are not included in the result array object of the onResults function by default (You can change that behavior by adjusting completeImportAction). Moreover, you can customize the displayed text of the notification by setting a value for the message key.

For the implementation, it's essential to grasp the available cleaning functions and comprehend their utilization. Below, you'll discover a list providing an overview of our cleaning functions:

columnHooks​

DescriptionWithin this function, you have the ability to retrieve all the data from a specific column. You can implement data modifications and present customized UI notifications, including info messages, warnings, and/or errors
ParametercolumnName: Callback function that must have the identical name as the column to which it will be employed

values: The callback function's parameter consists of a 2d array. Each inner array contains a value at position 1 and an index at position 2
Run EventThe columnHooks function is executed before the user enters the "Review Entries"-step

In the given example, the columnHooks cleaning function is used to restructure phone numbers by eliminating leading zeros and replacing them with a plus sign:

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "customer_data",
    columns: [
      {
        label: "Customer Code",
        key: "customer_code",
        columnType: "string",
      },
      {
        label: "Phone Number",
        key: "phone_number",
        columnType: "string",
      },
    ],
  }}
  columnHooks={{
    phoneNumber: (values) => {
      return values.map(([item, index]) => {
        let phoneNumber = item;
        if (/^[0]{2}/.test(phoneNumber)) {
          phoneNumber = `+${item.slice(2, item.length)}`;
        }
        return [
          {
            value: phoneNumber,
            info: [
              {
                message: 'We have automatically transformed your input into the correct format by converting "00" to "+".',
                level: "info",
              },
            ],
          },
          index,
        ];
      });
    },
  }}
>
  Import data
</NuvoImporter>

If you’re interested in exploring additional use cases, kindly navigate to our knowledge base available on the user platform.

onEntryInit​

DescriptionThis function iterates through every imported entry. In each iteration, you have access to each row value. With this cleaning function, you can perfectly define column-cross dependencies between two or more columns
Parameterrow: This refers to an object that includes column keys paired with their corresponding values for a given row
  rowIndex: Index of the current iteration
Run EventThis function is executed immediately once after the columnHooks callback

In the following example, we check the Country field of each row after the mapping step. If the value is Germany, we add DE to its Country Code field. Similarly, if the value is Italy, we add IT to the Country Code field of that row:

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "data",
    columns: [
      {
        label: "Country",
        key: "country",
        columnType: "string",
      },
      {
        label: "Country Code",
        key: "country_code",
        columnType: "string",
      },
    ],
  }}
  onEntryInit={(row, rowIndex) => {
    if (row.country === "Germany") {
      return {
        country_code: {
          value: "DE",
        },
      };
    } else if (row.country === "Italy") {
      return {
        country_code: {
          value: "IT",
        },
      };
    }
  }}
>
  Import data
</NuvoImporter>

For additional use cases, check out our knowledge base available on the user platform.

onEntryChange​

DescriptionThis cleaning function is applied after every single row change but only once when multiple rows were changed with one user interaction like e.g. "Replace all", dragging an entry value or copy and paste
Parameterrows: An array of row objects containing all rows that changed
Each row object contains:
  • rowIndex: Index of the row
  • data: Object of all column-keys with their current values
  • changeLog: Object of column-keys with their old values if the value was updated
  • actionType: The type of change that was performed:
    • "edit": Manual change of an entry
    • "delete": Delete a row
    • "create": Add a new row
    • "replace": Replacing a value with "Find & Replace"
Run EventThis function is executed each time a user creates or deletes a row as well as each time a user changes an entry within the "Review Entries"-step. Changing an entry can be done through a manual change, the drag functionality, copy & paste and find & replace. If the user changes the values of multiple entries simultaneously, the callback function runs only once

If the user changes the age value within the review step in the example given, it is validated again, and an error is displayed accordingly:

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "data",
    columns: [
      {
        label: "ID",
        key: "user_id",
        columnType: "string",
      },
      {
        label: "Name",
        key: "user_name",
        columnType: "string",
      },
      {
        label: "Age",
        key: "age",
        columnType: "int",
      },
    ],
  }}
  onEntryChange={(rows) => {
    return rows
      .filter((row) => Number(row.data["age"]) > 50)
      .map((row) => {
        return {
          rowIndex: row.rowIndex,
          data: {
            age: {
              value: row.data.age,
              info: [
                {
                  message: "Age of the user should not be greater than 50",
                  level: "error",
                },
              ],
            },
          },
        };
      });
  }}
>
  Import data
</NuvoImporter>

Here an example of the rows object, if you use copy and paste where one row is changed and one row is updated:

[
    {
        "actionType": "edit",
        "changeLog": {
            "user_id": null,
            "full_name": null,
            "age": null
        },
        "data": {
            "user_id": "0118",
            "full_name": "Jane Doe",
            "age": "55",
        },
        "rowIndex": 119
    },
    {
        "actionType": "create",
        "changeLog": null,
        "data": {
            "user_id": "0119",
            "full_name": "Jack Smith",
            "age": "45",
        },
        "rowIndex": 120
    }
]

This callback is closely similar to onEntryInit. If you want to discover other use cases, please navigate to our knowledge base, which is available on the user platform.