Export data to SFTP hosts (preview)

  • Article

[This article is prerelease documentation and is subject to change.]

Use your customer data in third-party applications by exporting them to a Secure File Transfer Protocol (SFTP) location.

Tip

Azure Synapse Link for Dataverse is a generally available service that offers seamless, continuous replication of data from Microsoft Dataverse to Azure Data Lake. We recommend that you use Azure Synapse Link for Dataverse instead of Customer Insights data-out exports for further data processing of the tables generated by Customer Insights like:

  • Unified customer profiles (msdynci_customerprofile)
  • Unified activities (msdynci_unifiedactivity)
  • Segment membership (msdynci_segmentmembership)
  • Customer attribute measures (msdynci_customermeasure)
  • Enrichment data (msdynci_enrichment)
  • Prediction model output (msdynci_prediction)

For more information on table schemas, see Tables in Customer Insights - Data

Prerequisites

  • Availability of an SFTP host and corresponding credentials.

Known limitations

  • SFTP destinations behind firewalls are currently not supported.
  • The runtime of an export depends on your system performance. We recommend two CPU cores and 1 Gb of memory as minimal configuration of your server.
  • This export works only for CSV formatted files.
  • Up to 100 million customer profiles, which can take 90 minutes when using the recommended minimal configuration of two CPU cores and 1 Gb of memory.
  • If you use an SSH key for authentication, make sure you create your private key as PEM or SSH.COM format. If you are using Putty, convert your private key by exporting is as Open SSH. The following private key formats are supported:
    • RSA in OpenSSL PEM and ssh.com format
    • DSA in OpenSSL PEM and ssh.com format
    • ECDSA 256/384/521 in OpenSSL PEM format
    • ED25519 and RSA in OpenSSH key format

Set up connection to SFTP

You must be an administrator in Customer Insights - Data to add a connection.

  1. Go to Settings > Connections.

  2. Select Add connection and choose SFTP.

  3. Give your connection a recognizable name in the Display name field. The name and the type of the connection describe this connection. We recommend choosing a name that explains the purpose and target of the connection.

  4. Choose who can use this connection. By default, it's only administrators. For more information, see Allow contributors to use a connection for exports.

  5. Choose whether you want to authenticate through SSH or username/password for your connection and provide the necessary details. If you use an SSH key for authentication, make sure you create your private key as PEM or SSH.COM format. If you are using Putty, convert your private key by exporting is as Open SSH. The following private key formats are supported:

    • RSA in OpenSSL PEM and ssh.com format
    • DSA in OpenSSL PEM and ssh.com format
    • ECDSA 256/384/521 in OpenSSL PEM format
    • ED25519 and RSA in OpenSSH key format

  6. Enter the Hostname and specify the Export folder on the FTP server. The export folder must be lowercase. The system is not case-sensitive and will create folder names in lowercase even if you use capital letters in the folder name.

  7. Review the data privacy and compliance and select I agree.

  8. Select Save to complete the connection.

Configure an export

To configure this export, you must have permission for this connection type.

  1. Go to Data > Exports.

  2. Select Add export.

  3. In the Connection for export field, choose a connection from the SFTP section. Contact an administrator if no connection is available.

  4. Enter a name for the export.

  5. Choose if you want to export your data Gzipped or Unzipped and the field delimiter for the exported files.

  6. Select the tables, for example segments, that you want to export.

  7. Choose Next and determine if you want to send all fields in the selected tables. By default, all fields in your selected tables are exported. Clear the checkox next to the fields you don't want to export.

    Note

    Each selected table is split into a maximum of five output files when exported.

  8. Select Save.

Saving an export doesn't run the export immediately. The export runs with every scheduled refresh. You can also export data on demand.

Tip

Export of tables that contain a large amount of data can lead to multiple CSV files in the same folder for each export. Splitting exports happens for performance reasons to minimize the time it takes for an export to complete.