Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,409 questions
1 answer
Data labeling and classification solution in ADLS gen2
Hi, I need to classify and the label with the classification the data stored in Data Lake Storage e.g. assign Sensitive label on some folder/file and then have a way to present in some report, provide retention based on the classification etc. One…
asked 2020-08-25T14:33:30.657+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 57.00000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Karol Kolodziejczyk
1
Reputation point
commented 2020-09-08T18:00:44.07+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,376
Reputation points Microsoft Employee
1 answer
USQL Datalake Question
Is there anyway to backup the partition files of usql table? Or the only way is to import the data to csv file.
asked 2020-08-05T06:13:06.693+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 68%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
DSM
1
Reputation point
commented 2020-09-01T20:33:26.677+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
MartinJaffer-MSFT
26,051
Reputation points
2 answers
Import incremental data from Azure Data Lake
Good afternoon, I have the following scenario !! I have a table in SQL that always receives changes daily, and I need to send this data to Azure data lake So, today I take this data and send it in the datalake file: Historicos.csv …
asked 2020-08-20T17:58:50.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 25%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELS%3C/text%3E%3C/svg%3E)
Lucas Silva
1
Reputation point
commented 2020-08-31T18:20:50.803+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT
46,437
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Unable to load data into Azure Synapse from ADLS Gen 2 .csv file
Hi @ChiragMishraMSFT-1092 Upon trying to load the data from ADLS(csv files) into Azure DW, am encountering a error in loading the data in DW. Here is the error message: "message": "Failure happened on 'Sink' side.…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 96%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
Express route and Azure Data Lake gen 2
Hi, I am doing some testing around accessing Azure Data Lake Gen 2 from an On Premise hadhoop I need to access the service via Express Route What is the recommended setup if anyone has been through a configuration? thanks
asked 2020-06-02T06:10:02.287+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
accepted 2020-08-27T14:58:38.99+00:00
Anonymous
1 answer
One of the answers was accepted by the question author.
How do I create an exploratory zone on top of my Azure Data Lake
Hi, My sincere apologies if this question would have been already answered. However, I searched hard but couldn't find the relevant information. Here lies my question: I have a Data Lake setup in Azure already. Further on, I have a requirement…
asked 2020-08-26T00:09:45.337+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 44%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGD%3C/text%3E%3C/svg%3E)
Ghosh, Debabrata (External)
21
Reputation points
commented 2020-08-26T13:10:06.217+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 9%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HarithaMaddi-MSFT
10,136
Reputation points
1 answer
Write json document to azure table
Hi, I am using below code to write json document in a Azure Data Lake Gen2 container into a SQL Server table. Code: df =…
asked 2020-08-21T05:07:09.597+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 20%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERD%3C/text%3E%3C/svg%3E)
Raj D
586
Reputation points
commented 2020-08-26T10:55:25.62+00:00
PRADEEPCHEEKATLA-MSFT
84,381
Reputation points Microsoft Employee
0 answers
ADL Gen2 rename using Path REST API failure with SAS token
Hi, I am currently trying to rename a file within a container using the https://video2.skills-academy.com/en-us/rest/api/storageservices/datalakestoragegen2/path/create API and an account-level SAS token. The SAS token was generated from the Azure portal…
asked 2020-08-03T09:31:07.993+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 15%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anil
1
Reputation point
commented 2020-08-25T23:53:13.76+00:00
MartinJaffer-MSFT
26,051
Reputation points
1 answer
What is the recommended approach if we had to load data from an Azure Data Lake Gen 2 to Oracle Exadata?
What is the recommended approach if we had to load data from an Azure Data Lake Gen 2 to Oracle Exadata?
asked 2020-08-17T18:06:50.523+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 72%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAT%3C/text%3E%3C/svg%3E)
Anil TexasCowboy
1
Reputation point
commented 2020-08-21T15:38:33.69+00:00
HarithaMaddi-MSFT
10,136
Reputation points
1 answer
How to Transform files in subfolders with one script in databricks
i have a adls gen2 folder with sub folders with parquet files in each folder. My requirement is to transform all parquet files in sub folders and load into another folder in adls gen 2 with same folder structure with one script. is it possible to do or…
asked 2020-08-14T16:55:40.797+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 37%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
reddy
41
Reputation points
commented 2020-08-21T15:37:42.363+00:00
HarithaMaddi-MSFT
10,136
Reputation points
0 answers
ADLS Gen2 Upload issue within IBM Java 8 in AIX machine
Hi Team, We have deployed a web application in AIX machine with IBM Java 8. In this web application, we are invoking ADLS Gen2 Rest API to upload and download the files. We are getting the following error, when we try to upload the files to ADLS Gen2…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 7.000000000000001%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
1 answer
Ad integration pass through for HDInsight
HI, I know HDInsight with ESP feature enables AD integration while connecting to cluster. Also teh access to underlying Hive tables can be controlled using Apache Ranger. But i would like to know if the access permission on storage account or datalake…
asked 2020-07-20T10:50:21.813+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 61%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHG%3C/text%3E%3C/svg%3E)
Hari GS
21
Reputation points
commented 2020-08-17T22:01:24.007+00:00
HimanshuSinha-msft
19,376
Reputation points Microsoft Employee
3 answers
How to set ACL to folder and all its children recursively?
I'm using the following code to set a ACL to a folder and its children in Azure Storage Account StorageV2. We have more than 2 million files and this takes too long. I could't find any recursive parameter or a best aproach. Is this the best option to…
asked 2020-07-07T13:36:44.663+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 22%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDA%3C/text%3E%3C/svg%3E)
Daniel Argüelles
1
Reputation point
commented 2020-08-17T20:50:52.05+00:00
MartinJaffer-MSFT
26,051
Reputation points
2 answers
One of the answers was accepted by the question author.
import json payload from a rest api and save as json documents in adls gen2
Hi, I am trying to import json payload from a REST api GET method and save json documents into ADLS Gen2 using azure databricks. GET: https://myapi.com/api/v1/city GET method Output: [ {"id":2643743, …
1 answer
One of the answers was accepted by the question author.
How to improve throughput of COPY activity
Hello, Please advise me to improve the throughput of data copy from SAP. ![15331-image.png][1] [1]: /api/attachments/15331-image.png?platform=QnA The same number of rows copy from another SAP server(different SHIR), but that has high throughput. Your…
asked 2020-08-03T21:02:14.39+00:00
Anonymous
accepted 2020-08-08T11:29:55.133+00:00
Anonymous
1 answer
One of the answers was accepted by the question author.
Access Azure DataLake G2 from Azure subscription
Hello I 'm deploying an ETL scenario to move data from SQL server to Azure DataLake G2. When starting the process, I can see that specifying Azure DataLake G2 as target is defining Azure Blob Storage as target. Is this the right procedure ? …
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 34%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EC%3C/text%3E%3C/svg%3E)
1 answer
Upgrade Access Tier via Azure CLI or Python Code for Stoarge Account Gen2
We Want to update the access tiers in the ADLS Gen2 for Multiple paths and want to use Azure CLI or Python Code as per our requirement. According to Microsoft Documentation, We see only Portal and Power shell code to do it. Can anyone let us know if…
asked 2020-07-30T04:31:17.177+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 44%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Kurmindla, Anusha
1
Reputation point
commented 2020-08-06T11:03:10.203+00:00
Sumarigo-MSFT
44,906
Reputation points Microsoft Employee
1 answer
FileNotFoundException when using abfss to list files in Azure Databricks!
Hi team, I am trying to connect to ADLS2 using hadoop configurations: But when I am trying to use FS commands to list all the files on the path, i am getting File not found exception: import org.apache.hadoop.fs.{FileSystem, Path} …
asked 2020-08-04T18:22:18.177+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 59%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGA%3C/text%3E%3C/svg%3E)
Goel, Akanksha
66
Reputation points
commented 2020-08-06T06:44:29.74+00:00
PRADEEPCHEEKATLA-MSFT
84,381
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
import sql query data as a json document to azure data lake storage
Hi, I am new to Azure Data Lake Storage. I am trying to import data from different vendor applications where data is hosted on a SQL Database in Azure virtual machines, trying to convert table data to json document and store it in Azure Data Lake…
2 answers
Synapse updating the data lake json file?
is it possible to update the json file from synapse? After reading, i need to update the json rows. When i tried to update/delete the json through external table, it's error out with, won't support the DML operation. Kindly advise how to achieve…
asked 2020-07-16T20:01:09.163+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 55.00000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EE%3C/text%3E%3C/svg%3E)
ever
1
Reputation point
answered 2020-07-29T15:16:33.61+00:00
Girijesh Pandey
1
Reputation point