Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,601 questions
0 answers
Azure Synaps serverless scan file that should be filtered
Hey, We have a large amount of files stored in Storage Account in Parquet format with the following hierarchy. /m_id=%M_ID%/quarter=%QUARTER%/month=%MONTH%/partition_version=last/part-0.parquet'. examples for quarter values: 2024Q1, 2024Q2, 2024Q3…
asked 2024-07-01T08:48:06.6533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 11%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESL%3C/text%3E%3C/svg%3E)
Shlomi Lanton
56
Reputation points
asked 2024-07-01T08:48:06.6533333+00:00
Shlomi Lanton
56
Reputation points
1 answer
One of the answers was accepted by the question author.
App could not could not connect to Azure Synapse when created via ARM template - SSL connection could not be established
Hello. I need your help with smth that we find a difficult to resolve. Here's the current setup: I have created an Azure Synapse workspace using ARM templates. All is working well apart from when an App which has been build on Angular could not connect…
asked 2024-06-21T07:56:17.4333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 88%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXM%3C/text%3E%3C/svg%3E)
Xhevahir Mehalla
140
Reputation points
accepted 2024-07-01T08:30:07.06+00:00
Xhevahir Mehalla
140
Reputation points
0 answers
How can I enable Synapse Link on a source azure serverless sql db to connect to dedicated sql pool..
Context Our team is planning to regularly refresh our external valuations dedicated pool using staging data from our serverless sql database. To implement this I have created the dedicated sql pool and the azure sql db is ready with plenty of data. The…
asked 2024-07-01T06:11:32.8433333+00:00
Michael Minto
20
Reputation points
edited the question 2024-07-01T07:59:27.4366667+00:00
PRADEEPCHEEKATLA-MSFT
83,891
Reputation points Microsoft Employee
0 answers
After some time, viewing pipeline log links lead to wrong page
This is something I consistently experience if I use the same Synapse Analytics Workspace for a decent amount of time: My workflow is: Monitor Pipeline Runs Click on a run of interest This shows me the pipeline that ran and all the individual steps…
asked 2024-06-27T14:13:06.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 95%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Andrew Chung
20
Reputation points Microsoft Employee
edited a comment 2024-07-01T06:47:18.5666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,825
Reputation points Microsoft Vendor
1 answer
Synapse how to read excel with formula
Hello, in Synapse spark pool we need to read excel file that contains formula inside of cells (sometimes it is simple sum, sometimes complicated formula, sometimes it is reference to another spread sheet and cell). Unfortunately, we observe that some…
asked 2024-06-27T08:57:56.29+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 64%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETT%3C/text%3E%3C/svg%3E)
Tamashevich, Tatsiana
40
Reputation points
commented 2024-07-01T06:28:16.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
5,420
Reputation points Microsoft Vendor
1 answer
How to send data into displayHTML function in Azure Synapse Analytics
Hi, As mentioned in this link https://video2.skills-academy.com/en-us/azure/synapse-analytics/spark/apache-spark-data-visualization I want to use D3.js to visualize my data. How do I send data from my pyspark dataframe into displayHTML() to visualize it?
asked 2024-06-28T06:28:54.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 45%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Raghul Kannan
126
Reputation points
commented 2024-07-01T06:20:07.9666667+00:00
Raghul Kannan
126
Reputation points
1 answer
Unable to run an SQL query in Azure Synapse
Hi, I am unbale to run a simple sql query. It keeps throwing this following error. File 'https://storagecontainer/Financial Sample.xlsx' cannot be opened because it does not exist or it is used by another process. I assigned myself with Storage Blob Data…
asked 2024-06-30T14:25:48.4233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 17%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
shailaja Azure
0
Reputation points
answered 2024-07-01T05:46:47.4933333+00:00
PRADEEPCHEEKATLA-MSFT
83,891
Reputation points Microsoft Employee
0 answers
Custom logging in Azure Synapse Datapipeline to to log analytics Workspace - Help Requested
Requirement:- Want to add custom logging in azure synapse pyspark job using python logging module and spark config Getting error while implementing the same in the pyspark job ModuleNotFoundError: No module named 'pyspark.SparkConf pyspark script for…
asked 2024-06-26T23:18:27.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 21%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJY%3C/text%3E%3C/svg%3E)
Johny Yagappan
20
Reputation points
commented 2024-07-01T05:26:24.7633333+00:00
PRADEEPCHEEKATLA-MSFT
83,891
Reputation points Microsoft Employee
1 answer
Unable to connect On-Premises SQL DB through Spark Notebook in Synapse workspace
HI Team, We are unable to connect On-Premises SQL DB through Spark Notebook in Synapse workspace. We have been trying to connect with private endpoints from Synapse workspace to SQL DB as we have instructions not to use public networks. Followed below…
asked 2024-06-24T17:18:53.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 71%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sai Kishore Amara
20
Reputation points
commented 2024-07-01T05:19:18.0366667+00:00
phemanth
7,825
Reputation points Microsoft Vendor
2 answers
Issue with Copy Dataverse data into Azure SQL using Synapse Link Template
I am experiencing issues using ADF I am trying to use the template "Copy Dataverse data into Azure SQL using Synapse Link" to move f&o data from datalake to Azure…
asked 2024-05-30T07:08:10.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 20%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDO%3C/text%3E%3C/svg%3E)
Dipo Onibile
0
Reputation points
commented 2024-07-01T04:55:32.2433333+00:00
phemanth
7,825
Reputation points Microsoft Vendor
1 answer
Install Packages on Azure Synapse DEP-enabled workspaces
Hi I have a Synapse workspace with DEP enabled. Since PyPI libraries cannot be installed directly in the Spark pool, could you please advise on how to install the azure-mgmt-kusto, azure-kusto-data, and azure-keyvault-secrets packages on the pool? I…
asked 2024-06-26T02:17:08.77+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Abhiram Duvvuru
191
Reputation points Microsoft Employee
commented 2024-07-01T04:53:41+00:00
phemanth
7,825
Reputation points Microsoft Vendor
0 answers
Azure synapse link creates model.json with no partitions
Hi, I am following this link below to copy dataverse data to Azure SQL DB using azure data factory pipelines. The model.json file created by my synapse link for dataverse is correctly creating the schema and attribute names for all the tables/entities.…
asked 2024-06-25T19:12:57.98+00:00
Dipo Onibile
0
Reputation points
commented 2024-07-01T04:38:40.5766667+00:00
phemanth
7,825
Reputation points Microsoft Vendor
1 answer
Servicenow linked service used connectivity vendor
I am interested in using Servicenow Linked Service to extract data from Servicenow. There are two linked services available; the legacy version and version 2 that uses table API. The documentation mentions a built-in driver for enabling connectivity. Can…
asked 2024-06-27T10:50:37.5233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 99%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Amir Bagheripour
0
Reputation points
commented 2024-07-01T03:59:02.4966667+00:00
Harishga
5,420
Reputation points Microsoft Vendor
1 answer
ALTER TABLE ADD COLUMN changes made to Synapse Delta tables in a Notebook do not reflect in Lake Database
Running the following spark.sql("ALTER TABLE example_table ADD COLUMN example_column STRING") Successfully adds the column to the Delta table and this can be verified by running %%sql SELECT example_column FROM example_table However when…
asked 2024-06-30T20:51:36.6366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 75%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFL%3C/text%3E%3C/svg%3E)
Francesca Lightbown
0
Reputation points
answered 2024-06-30T23:00:54.5066667+00:00
hossein jalilian
4,690
Reputation points
1 answer
Is Synapse Serverless Down?
As of ~12:53 CET all of our queries on Synapse Serverless have been timing out with the message: [This query cannot be executed because we can't get enough capacity for optimal performance in time. This can be an intermittent issue. Please retry…
asked 2024-06-27T12:05:26.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 17%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Mark Mellink
35
Reputation points
commented 2024-06-30T16:34:29.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee
0 answers
azure synapse
hey i try to get sdk call that return me the info about the Azure Active Directory i cant find the sdk call for this
asked 2024-06-26T08:13:52.66+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 56.00000000000001%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETM%3C/text%3E%3C/svg%3E)
Tommy Mordehay
0
Reputation points
commented 2024-06-30T15:11:24.2333333+00:00
PRADEEPCHEEKATLA-MSFT
83,891
Reputation points Microsoft Employee
1 answer
How to install com.crealytics.spark.excel to Synapse Spark Pool
Hello, I have spark pool with following properties: According to the instruction mentioned under the Q&A https://video2.skills-academy.com/en-us/answers/questions/1185314/how-to-write-data-from-notebook-to-dedicated-sql-p?cid=kerryherger I have…
asked 2024-06-26T14:34:57.57+00:00
Tamashevich, Tatsiana
40
Reputation points
commented 2024-06-30T14:26:33.5733333+00:00
PRADEEPCHEEKATLA-MSFT
83,891
Reputation points Microsoft Employee
1 answer
Do Apache Spark Pools have a Service Tag?
Our company is rolling out more restrictive network security groups and I wanted to see if we could have our Apache Spark Pools' traffic be allowed via a service tag. When I tried to trace the IP address of the pools, the IPs varied widely so a static IP…
asked 2024-06-28T15:07:17.05+00:00
Andrew Chung
20
Reputation points Microsoft Employee
answered 2024-06-30T11:06:00+00:00
PRADEEPCHEEKATLA-MSFT
83,891
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Issue with running queries on Synapse SQL Endpoint - This query cannot be executed because we can't get enough capacity for optimal performance in time.
Dear Community, I am getting the error ""This query cannot be executed because we can't get enough capacity for optimal performance in time."" when running a query using a SQL Serverless endpoint in Synapse Analytics This error…
1 answer
Unable to run query on Synapse Serverless SQL Pool suddenly
I have a few of external table in an Azure Synapse Analytics Serverless SQL Pool which I need to query frequently. I suddenly got this error today. [This query cannot be executed because we can't get enough capacity for optimal performance in time. This…
asked 2024-06-27T06:30:27.38+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 12%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKW%3C/text%3E%3C/svg%3E)
Kenny Wong (HK)
40
Reputation points
answered 2024-06-28T21:12:33.17+00:00
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee