Function that works in python but sql doesn't pyspark
Function that works in python but sql doesn't pyspark I have successfully performed a function, and I have used it without problems, python, I have seen in videos and I have read that it can be implemented in both contexts, but I cannot do it myself Here…
Changes to file ingestion methodology Pyspark Azure Synapse from Data Lake.
Hi we have been experiencing some peculiarities in reading files from within the connected storage of Synapse Analytics over the last few days. These have been experienced while using pyspark in Notebooks. Our initial issue started using the pandas…
SQL connect error from Data bricks notebook
Hi , while connecting to sql server from databricks notebook with service principal as auth , im getting this error once every week .95% of the time , im not seeing this error but want to understand why im facing this and how i can resolve this ? …
Hierarchical Classification in ml.net
Is there way to achieve Hierarchical Classification in ml.net? Like Predict cat from animal category and then predict breed of that cat based on cat as a output? in multicast-classification?
Using SynapseML v0.9.5 in Spark pool
We aim to use the geocoding features of SynapseML library v0.9.5 over Synapse Spark pools. However, this library needs Spark 3.2+ while Spark pools just support Spark 3.1 & 2.7. So we can just install SynapseML v0.9.4 but it does not have the…
load data into SQL Server via Pyspark.
Hi everyone, I don't know what else to do. I tested it with several cluster versions. Need help. thanks. Error return in the description below: org.apache.spark.SparkException: Job aborted due to stage failure: Task 31 in stage 1.0 failed 4 times,…
Notebook execution is in Failed state in SynapseNotebook
When Synapse pipeline fails because of Notebook, this error can be seen in Monitor tab of Synapse Studio. However, in log analytics it doesn't show the error. Is there a way to get error details form Log analytics? LogAnalytics error: …
Microsoft.Data.Analysis DataFrame: Is there a best practice for normalizing the length of columns?
Hi, I'm building a DataFrame with DataFrameColumns that have various lengths. Is there a best practice/idiom to normalize the length of all columns before building the DataFrame? Chris
Using Sql Command in C# Visual Studio
Why does the SQL Command not Delete the Data from the Database, in the properties i have already set the sql command text on the Delete Comand. Is the sql Command used rightly? public Form1() { InitializeComponent(); sqlConnection1.Open(); } …
ML.net Model Builder onnx-Model input preprocessing
Hi. After I've trained an object detection model with Model Builder through Azure; is there a way to find out which preprocessing (like scaling the input image to 600x800) needs to be performed when the onnx-model is used for inference (like in a c++…
Microsoft.ML and Xamarin Forms (Android)
Hello! I'm trying to use Microsoft.ML on Xamarin Forms (Android) application. It compiles well and runs on x64 platform, but when I deploy it on a device (Arm64) and it tries to call Fit() of IEstimator<ITransformer> interface, it crashes with…
Hadoop and Big Data
I recently started studying Big Data in my university. But i am having troubles in understanding how is Hadoop and Big Data related? Anyone can explain this?
Compiler error "consume excess memory due to the indexing expression"
Hi, I am trying to construct a Bayesian network that contains 8 parents(which are iKorean, iOthrserv, iMay, dYrsserv, iSept80, iVietnam and iFeb55) pointing to 1 child(which is iRvetserv), and while compiling, I got the error like this: " [1] This…
Training Bayesian network with Infer.NET but doesn't know any prior and conditional probability
Hi, I am new to Infer.NET. I know this question might be asked a few years ago but some posts' links were broken so I just ask here again. Let assume that if I have the whole dataset for training and I would like to construct a Bayesian network in…
Give hints to train a category model in ML.NET?
I have quite good data for training a model to predict a category label based on many features. But it's it possible, or even necessary, to "hint" or give a list of categories for each record that I know for sure it can NOT be? Would it be…
Max of Column B based on Column A
I have a table with company names and their IDs. Some companies could have multiple IDs, so I want to create another table with the company names and maximum of the IDs (basically the latest created ID). I have been able to get the company names in a…
how i cane solve these error one report in porter appear these We couldn't connect to the Analysis Services server.
how i cane solve these error one report in porter appear these We couldn't connect to the Analysis Services server. Make sure you've entered the connection string correctly. Request IDab071746-2b75-80f4-4fb7-0177c5f4369b TimeTue Nov 30 2021…
ML.NET unsupervised learning for image classification
I want to know if it is possible to do image classification using ML.NET. If possible, I want to see an example. I am making a WindowsForm application in which you can specify the path to the folder with images. My task is to classify this set of…
How to get prediction interval from ML.NET model builder
I am trying to predict Candidate joining the company based on parameters like Location, Skills and Designation , I am using ML.NET model builder to build a model and predict number of days required by candidate to join the company , I want to get the…
Azure synapse Notebook error creating session Apache spark pool
I experiencing that error starting a session on notebook There is no running application on that pool. The day before all worked correctly. No changes has been performed on permission or spark pool I've also tried to remove and upload…