how to resolve this error while getting data from databricks from power bi.Not able to load table in power bi

Pratibha Khare 20 Reputation points
2023-10-23T18:45:35.15+00:00

DataSource.Error: ODBC: ERROR [HY000] [Microsoft][DSI] (20039) Cannot store ""."".""."REMARKS" value in temporary table without truncation. (Column metadata implied a maximum of 512 bytes, while provided value is 559 bytes

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,045 questions
{count} votes

4 answers

Sort by: Most helpful
  1. Sadiq Rahimansa 5 Reputation points
    2023-12-21T07:54:25.25+00:00

    This is because the comments in the Databricks table metadata in contains text more than 512 characters, which is quite normal. Power BI Enhanced metadata reader internally refers the "comments" columns as "REMARKS" and it seems to be have a size limitation of 512. If possible, if you could reduce the size of the comments, that will resolve it. For me, I can't do that because we use it for our data catalog and I am stuck now. Waiting for response from MicroSoft.

    1 person found this answer helpful.

  2. BhargavaGunnam-MSFT 28,616 Reputation points Microsoft Employee
    2023-10-23T21:52:33.23+00:00

    Hello Pratibha Khare,

    Welcome to the Microsoft Q&A forum.

    from the error message, the column “REMARKS” has a maximum length of 512 bytes, but the provided value is 559 bytes

    Can you please review your "REMARKS" column in the Databricks? If there are any values that exceed 512 bytes,

    you may need to truncate or otherwise modify these values before attempting to load the data into Power BI.

    Use Power Query: If you’re using Power BI Desktop, you can use Power Query to connect to your Databricks data. In Power Query, you can transform it before loading it into Power BI

    I hope this helps.

    Please let me know if you have any further questions

    0 comments No comments

  3. Sadiq Rahimansa 5 Reputation points
    2024-01-18T09:07:07.7233333+00:00

    Yes. Response from Msft below:

    The idea here is to modify the parameter MaxCommentLen to accommodate the required size you need for the column. This can just be done through the Power BI Desktop, since we ‘rent able to modify ODBC this parameter on PBI Service. We can achieve it following the next steps: 1. Create/edit a file inside your PowerBI Desktop installation folder, located at … Microsoft Power BI Desktop\bin\ODBC Drivers\Simba Spark ODBC Driver**microsoft.sparkodbc.ini1. In the file you can add/modify the below config for the value:[Driver]**MaxCommentLen=2048 1. Save and restart PBI Desktop.

    However, we found that this folder is quite restricted and no one has permissions to modify it. We are working with our security team to gain access. So, haven't tested it yet.

    0 comments No comments

  4. Mike 0 Reputation points
    2024-05-28T15:31:06.68+00:00

    Updating the ini file didn't work for us. We ended up adding the property to the registry entry. While this resolved the "Remarks" error we were getting, the metadata ("Remarks") isn't populating in Power BI ("Description").

    0 comments No comments