Skip to main content

Overview

dataHandler​

Use our dataHandler feature on top of our cleaning functions to solve complex and complicated import scenarios. Take advantage of the advanced data access directly after the upload or after the mapping to customize your importer to suit your requirements.

DescriptionThe dataHandler is a powerful tool that can be executed before the “Header Selection”- and/or “Review Entries”-step, providing access to the original data and its metadata as well as enabling you to add/remove columns and rows. It can be configured to run based on your needs.
With the dataHandler function, you can solve complex data manipulation scenarios, such as transposing data, merging and splitting columns, joining sheets, de-nesting data, and more. Unlike other cleaning functions that iterate through every entry, the dataHandler function works on the entire data at once, giving you complete control over input data.
In addition, with the dataHandler function, you can add, delete, and modify columns and rows in the data, providing even greater flexibility in data manipulation.
Whether you need to transform a single column or an entire dataset, the dataHandler function provides the flexibility and power required to do the job effectively and efficiently.
Run EventThe execution of this function can occur before the “Header Selection”- or before the “Review Entries”-step, or even both, depending on the configuration.
Parametermodifier: The modifier parameter is a function that enables modifying the data by adding or deleting columns and rows. This function allows customization of the data per your requirements. The modifier function includes the following sub-functions:
  • addRow() - Adds a new row to the data.
  • deleteRow()- Deletes an existing row from the data.
  • addColumn() - Adds a new column to the data.
  • deleteColumn() - Deletes an existing column from the data.
By utilizing these sub-functions, you can manipulate the data in a granular manner to meet your specific use case.
data: The data parameter is a data structure that provides detailed information about the imported file(s). It includes attributes such as the file size, file name, and file extension, as well as the original data of the input.
  • headerStep: data is an array of objects with the following properties:
    • fileSize - The size of the file in MB.
    • fileExtension - The extension of the file.
    • fileName - The name of the file.
    • data - Original data of the file.
    • sheetName - The name of the corresponding sheet.
  • reviewStep: data is an array of objects where each object represents a row.
logs: After the mapping step is completed as well as after the import process concludes, you have access to logs containing details about mappings and columns used during the import.
  • mappings - This comprises an array of objects, with each object containing a sourceColumn and a targetColumn. It provides a summary of the mappings executed during the import process.
  • columns - an object with addedColumns and addedOptions keys. If allowCustomColumns is set to true and new columns are added to the schema, the addedColumns field will be present in the logs and will display an array of added columns. If allowCustomOptions is set to true and new options are added to the columns, the addedOptions field will be present in the logs and will display an array of added options with the corresponding column.

Modifier​

headerStep:index: Specifies the position in the data where the new row will be inserted. If you pass an integer value for the index, the new row will be added to that specific index. If no index is passed or the index is null, a new row will be created at the end of the data.
data: Comprises an array of arrays, where each inner array represents a new row of values to be added.
reviewStep:index: Specifies the position in the data where the new row will be inserted. If you pass an integer value for the index, the new row will be added to that specific index. If no index is passed or the index is null, a new row will be created at the end of the data.
data: The data for the review step can have two formats based on your use case:
  • Without errors - an array of objects where each key represents a column key, and the corresponding value denotes the data to be added to the new row.
  • With errors - an array of objects where each key signifies a column, and the corresponding value represents an object containing the cell value and its associated error information.

It should be noted that the addRow() function allows multiple rows to be added to the record at once.

dataHandler={{
  headerStep: async (modifier, data) => {
    // Adding a row as a 2D array
    modifier.addRow({
      index: 0,
      data: [["Max", "Jordan"]],
    });
  },
  reviewStep: async (modifier, data, logs) => {
    // Adding a row without an error
    modifier.addRow({
      index: 0,
      data: [{ column_key: "Hans" }],
    });
    // Adding a row with an error
    modifier.addRow({
      index: 1,
      data: [
        {
          column_key: {
            value: "Charlie",
            info: [
              {
                message: "This cell was automatically added.",
                level: "info",
              },
            ],
          },
        },
      ],
    });
  },
}}

Returning data​

The dataHandler enables you to return the complete data set after making modifications. It offers the flexibility to choose whether or not to return the modified data. If you decide to return the data, the modifier operations will be applied to the returned data. Alternatively, if you opt not to return the data, the modifier operations will be directly applied to the source data.

Moreover, the dataHandler allows you to obtain the data in two distinct formats for each stage. This feature empowers you to select the format that aligns best with your specific use case. You can also return a single or multiple sheets within the headerStep function.

headerStep:You can return the data in one of two formats, single or multiple sheets:
  • Single sheet - you must return the data as a valid 2D array or an array of objects where each inner object/array represents a row.
  • Multiple sheets - you must return the data as an array of objects where each object represents a sheet containing the following information.
    • data - 2D array or an array of objects where each inner object/array represents a row.
    • fileName - name of the file.
    • sheetName - name of the sheet.
reviewStep: You can provide the data in one of two formats, with errors or without errors:
  • Without errors - you must return data as a JSON array where each object represents a row. Every row object must consist of keys representing columns, with corresponding values representing the cell value.
  • With errors - you must return data as an array of objects where each key signifies a column, and the corresponding value represents an object containing the cell value and its associated error information.
It is important to ensure that the keys of your JSON data should match the keys of the target data model (TDM).

Returning a single sheet:

dataHandler = {
  headerStep: async (modifier, data) => {
    // Your custom logic

    // Option 1: Return data as a 2D array
    return [
      ["id", "name"],
      ["12345", "Jason"],
      ["67890", "Max"],
    ];

    // Option 2: Return data as an array of objects
    return [
      { id: 12345, name: "Jason" },
      { id: 67890, name: "Max" },
    ];
  },
};

Returning multiple sheets:

dataHandler = {
  headerStep: async (modifier, data) => {
    // Your custom logic

    // NOTE: data can be in different formats (2D array & array of objects)
    return [
      {
        fileName: "Customers",
        sheetName: "Internal",
        data: [
          ["id", "name"],
          ["12345", "Jason"],
        ],
      },
      {
        fileName: "Products",
        sheetName: "Best sellers",
        data: [
          { id: 12, quantity: 24 },
          { id: 13, quantity: 88 },
        ],
      },
    ];
  },
};

Logs example​

{
  "mappings": [
    {
      "sourceColumn": "Company Identification Number",
      "targetColumn": "company_code"
    },
    {
      "sourceColumn": "Organisation",
      "targetColumn": "company_name"
    },
    {
      "sourceColumn": "Website",
      "targetColumn": "domain_name"
    },
    {
      "sourceColumn": "Address",
      "targetColumn": "adress"
    },
    {
      "sourceColumn": "Area",
      "targetColumn": "region"
    },
    {
      "sourceColumn": "Deal Value",
      "targetColumn": "deal_size"
    },
    {
      "sourceColumn": "Status",
      "targetColumn": "deal_status"
    },
    {
      "sourceColumn": "Pipeline",
      "targetColumn": "pipeline"
    },
    {
      "sourceColumn": "Expenditures",
      "targetColumn": "costs"
    },
    {
      "sourceColumn": "Active",
      "targetColumn": "deal_ongoing"
    },
    {
      "sourceColumn": "",
      "targetColumn": "full_name"
    }
  ],
  "columns": {
    "addedColumns": [
      {
        "label": "Revenue",
        "key": "revenue",
        "columnType": "currency_eur"
      }
    ],
    "addedOptions": [
      {
        "columnKey": "deal_stage",
        "dropdownOptions": [
          {
            "label": "Done",
            "value": "done",
            "type": "string"
          }
        ]
      }
    ]
  }
}

Implementation example​

In the given example, we display how to iterate through the data after the mapping with the reviewStep function. We implemented a logic that fills the addresscolumn by merging thestreet, city, and countrycolumns if these three columns contain data. We then remove thestreet, city, and country columns from the output schema.

<NuvoImporter
  licenseKey="Your License Key"
  settings={{
    developerMode: true,
    identifier: "product_data",
    columns: [
      {
        key: "full_name",
        label: "Full name",
        columnType: "string",
      },
      {
        key: "street",
        label: "Street",
        columnType: "string",
      },
      {
        key: "city",
        label: "City",
        columnType: "string",
      },
      {
        key: "country",
        label: "Country",
        columnType: "string",
      },
      {
        key: "Address",
        label: "Address",
        columnType: "string",
      },
    ],
  }}
  dataHandler={{
    reviewStep: async (modifier, data, logs) => {
      const dataLength = data.length;
      const newData = data;
      for (let i = 0; i < dataLength; i++) {
        const element = data[i];

        if (!element.Address && element.street && element.city && element.country) {
          newData[i].Address = `${element.street}, ${element.city}, ${element.country}`;
        }
      }

      modifier.removeColumn("street");
      modifier.removeColumn("city");
      modifier.removeColumn("country");
      return newData;
    },
  }}
/>

If you’re interested in exploring additional use cases, kindly navigate to our knowledge base available on the user platform.