Configuration

You can configure the Data Importer by passing in configuration props. An overview of the available props are:

<DataImporter
    pipelineId={"<YOUR_PIPELINE_ID>"}
    steps={{
        fileUpload: {...fileUploadConfig},
        schemaMapping: {...schemaMappingConfig},
        categorization: {...categorizationConfig}
    }}
    onCompletion={{
        webhookUrl: "https://some.web.hook.url",
        returnWhenComplete: false
    }}
    onError={{
        webhookUrl: "https://some.web.hook.url.error"
    }}
    customTheme={{
        color: "#f1a400",
        shouldInheritFont: false
    }}
>
    🚀 Custom Data Importer Button Text
</DataImporter>

Required Props

Prop	Description
`pipelineId`	The id of the Pipeline that will be used to run the job/upload. Check out the Pipeline Documentation for more info.

Optional Props

Prop	Default	Description
`steps`	`{ fileUpload: {}, schemaMapping: {} }`	`steps` specifies which steps that should be displayed on the modal, as well as the specific configurations for each step. Learn more about available step configurations.
`onCompletion`	`{}`	`{ webhookUrl: "some.web.hook.url", returnWhenComplete: true, callbacks: Callbacks }` `webhookUrl` - receives the a metadata payload once the job has run Webhook Metadata Payload Example `{ "connectorId": "ZtPOQ6uCUTNloAYQ9teFEqtanpifuH7FN5bzjtYtgG06", "destinationFileName": "joe.csv", "destinationFilePath": "s3://segnatestbucket/TEST_ARYAN/joe.csv", "finalDataLength": 27, "jobId": "V2G2vrTrnEUSB93McyBnkv4SSxEnglPh38FQQe49mzU5", "jobName": "testtting", "jobStatus": "done", "originalDataLength": 33, "pipelineId": "8HKyeqrQPWOXs20samuemmnwcR9UdCjGgen99KTJqTk1", "pipelineName": "test for loops", "uploadedFileName": "onboarding_sample.csv", "uploadedFileType": "csv" }` `returnWhenComplete` - a boolean that specifies whether the data importer modal should finish/close after the job has been fully processed. By default, the modal will process the file asynchronously and return immediately after the request has been received. `callbacks` - check out the callbacks section for more info
`onError`	`{}`	`{ webhookUrl: "some.web.hook.url", }` `webhookUrl` - receives a error payload whenever the user runs into any error Webhook Error Payload Example `{ "connectorId": "ZtPOQ6uCUTNloAYQ9teFEqtanpifuH7FN5bzjtYtgG06", "destinationFileName": "joe.csv", "destinationFilePath": "s3://segnatestbucket/TEST_ARYAN/joe.csv", "finalDataLength": null, "jobId": "V2G2vrTrnEUSB93McyBnkv4SSxEnglPh38FQQe49mzU5", "jobName": "testtting", "jobStatus": "error", "originalDataLength": 33, "pipelineId": "8HKyeqrQPWOXs20samuemmnwcR9UdCjGgen99KTJqTk1", "pipelineName": "test for loops", "uploadedFileName": "onboarding_sample.csv", "uploadedFileType": "csv" }`
`customTheme`	`{ color: "#f1a400", shouldInheritFont: false }`	Customize your data importer to fit your product. `color`- pass in a hex color code to specify the accent color to be used throughout the modal. `shouldInheritFont` - if true, the modal will inherit the parent font-family. Simply wrap the rendered the Data Importer with a component styled with your desired font.
`children`	`"Import"`	The text to be displayed on the Data Importer Button. Simply pass in the text as a child of `DataImporter`.

Steps

The Data Importer will guide users uploading their files through a series of steps. By default, the two steps are:

Step	Step Key	Functionality of Step
Upload	`fileUpload`	Required - Users can upload a file or connect their Google Sheet, and see a preview of the data. If the uploaded file is an Excel Workbook with multiple sheets, also allows user to select their sheet of choice
Column Matching	`schemaMapping`	Users can map/rename their input columns to match the Pipeline's schema. Segna will take the best guess for the mapping, and prompt the user to confirm or edit.
Categorization	`categorization`	Users can confirm, change, and remove category mapping suggestions made by Segna based on the specified category values on the pipeline, and the incoming data.

Each step has its own configuration object, which can be used to change what functionality is shown on the step.
As the Data Importer is built entirely on top of Segna APIs, some configuration for these API are also exposed out in the step configuration when applicable.

📘
You can configure which steps are shown or not shown using the steps prop. Passing in the step's key (such as schemaMapping) will include the step in the modal, and removing it will skip it.

`fileUpload`

fileUpload: {
    jobName: "some job name",
    destinationFileName: "folder/filename",
    webhookUrl: "some.web.hook.url",
    useFullData: false,
    scripts: [
        "scriptId1",
        "scriptId2"
    ],
    outputFileType: "csv",
    callbacks: Callbacks
}

📘
The parameters are a subset of startJob API call.

Key	Default	Description
`jobName`	`"jobId"`	Name that appears on your Segna Platform dashboard
`destinationFileName`	`"{pipelineId}`-`{jobId}"`	Name of cleaned file. Available only if a data bucket is your output destination.
`webhookUrl`		Webhook URL that gets sent metadata extracted from the file uploaded. Example of the payload can be found here.
`useFullData`	`false`	Whether to use the full dataset when going through the data importer flow. You should only specify `true` if you need to query metadata and require it to be representative of the entire file, however the upload process can be significantly slower with bigger files. The full file will always be processed regardless when outputting to your destination source.
`scripts`	`[]`	Array of `scriptIds` of scripts you have added via the Segna Platform. When the file is uploaded, the scripts will run in the order provided. The input of the script is Python dataframe, of the data after some preliminary cleaning, but before the field remapping happens. Check out scripts for more details.
`outputFileType`	`"csv"`	File type of cleaned file. Options include `csv`, `excel`. Available only if a data bucket is your output destination.
`callbacks`	`{}`	See Callbacks

`schemaMapping`

The schemaMapping page allows a user to confirm whether Segna has correctly mapped the input columns of the uploaded data to match the output columns of the desired schema on the Pipeline. Segna will guess most of the mapping, and only require the user to action low-confidence mappings.

schemaMapping: {
    callbacks: Callbacks,
}

Key	Default	Description
`callbacks`	`{}`	See Callbacks

`categorization`

The categorization page allows a user to confirm whether Segna has correctly remapped the values from the uploaded data to the category values set. The allowed category values of a category data type field are set on the pipeline.
Users will only be prompted to confirm or correct the remapping on the unique input values - therefore streamlining the categorization process.

categorization: {
    callbacks: Callbacks,
}

Key	Default	Description
`callbacks`	`{}`	See Callbacks

Callbacks

Each step's configuration object, as well as the runJobOnCompletion object, can be passed in a callbacks key:

{
  ...
  callbacks: {
    beforeCallback: ({ jobId }) => { someAction() }
    afterCallback: ({ jobId }) => { someAction() }
  };
};

beforeCallback and afterCallback trigger before and after 'Next' or 'Finish' is pressed on a modal step respectively.
Each callback is passed in the jobId for ease of reference.