This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 41%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERH%3C/text%3E%3C/svg%3E)
I have a project where I have a large C(100,20) number of combinations with minor work being done for each combination set.
I am using Spark .NET with visual studio as my technology (see setup below): https://video2.skills-academy.com/en-us/dotnet/spark/tutorials/get-started
Spark .NET has a dataframe with SQL type commands. I am assuming I need to do a SQL type command to create the N choose K combinations with a user defined worker function to process the combinations.
The question is what does the code look like using Spark .NET with a DataFrame? If a DataFrame doesn't support an N choose K option, are there other options to keep the generation of the combinations distributed?
Hello @Robert Hogue ,
Welcome to the Microsoft Q&A platform.
Thanks for posting the question. I’m working with the product team and get back to you when I have more information.
Thanks for looking into this. I also have a feature request open on the GitHub site for Spark .NET
https://github.com/dotnet/spark/issues/627
I have tried to work with 4 microsoft gold partners on this without any success. Also, given the silence in the multiple forums I have posted too, I may need more of a direct response from the product team.
My proposal is to create a sample for N Choose K in the Spark .NET example area which I think would be helpful for documentation of more general permutation scenarios as well as for testing Spark .NET features.
My basic question was answered from the spark dotnet github area
https://github.com/dotnet/spark/issues/627
By using a cross join on two dataframes, I was able to create the combinations. This may not be the best way, and perhaps others will follow up with a better solution.
For N Choose K that would be K crossjoins using the N set.
Hello @Robert Hogue ,
Glad to know that your issue has resolved. And thanks for sharing the solution, which might be beneficial to other community members reading this thread.