VB.NET DataGridView to CSV Error: system.outofmemoryexception

jim brown 271 Reputation points
2020-12-14T16:53:31.49+00:00

I have the below code that works but if DGV is large I get a system.outofmemoryexception how would I change existing code to download in chunks or preferred method of doing this. any help is greatly appreciated.

If ComboBoxExport.Text = "Save As CSV" Then

            Dim sfd As SaveFileDialog = New SaveFileDialog()

            sfd.Title = "Save File Dialog"
            sfd.InitialDirectory = "C:\"
            sfd.Filter = "Text file (*.csv)|*.csv|Text file (*.txt)|*.txt|All files (*.*)|*.*"
            sfd.FilterIndex = 1
            'sfd.RestoreDirectory = True


            Try

                Dim headers = (From header As DataGridViewColumn In DataGridView1.Columns.Cast(Of DataGridViewColumn)()
                               Select header.HeaderText).ToArray
                Dim rows = From row As DataGridViewRow In DataGridView1.Rows.Cast(Of DataGridViewRow)()
                           Where Not row.IsNewRow
                           Select Array.ConvertAll(row.Cells.Cast(Of DataGridViewCell).ToArray, Function(c) If(c.Value IsNot Nothing, c.Value.ToString, ""))
                Using sw As New IO.StreamWriter(sfd.FileName)
                    sw.WriteLine(String.Join(",", headers))
                    For Each r In rows
                        'sw.WriteLine(String.Join(",", r))
                        sw.WriteLine("""" & String.Join(""",""", r) & """") ' Write DataGridView to csv handling commas in values
                    Next
                End Using

            Catch ex As Exception
                MsgBox("Save As CSV Error: " & ex.Message)
            End Try


        End If
VB
VB
An object-oriented programming language developed by Microsoft that is implemented on the .NET Framework. Previously known as Visual Basic .NET.
2,644 questions
0 comments No comments
{count} votes

Accepted answer
  1. jim brown 271 Reputation points
    2020-12-16T20:19:28.51+00:00

    Fixed by using DataTable rather then DataGridView. I was populating the DGV from DT.

     If ComboBoxExport.Text = "Save As CSV" Then
    
                Dim sfd As SaveFileDialog = New SaveFileDialog()
    
                sfd.Title = "Save File Dialog"
                sfd.InitialDirectory = "C:\"
                sfd.Filter = "Text file (*.csv)|*.csv|Text file (*.txt)|*.txt|All files (*.*)|*.*"
                sfd.FilterIndex = 1
                'sfd.RestoreDirectory = True
    
    
    
                Try
    
                    Dim headers = (From header As DataGridViewColumn In DataGridView1.Columns.Cast(Of DataGridViewColumn)()
                                   Select header.HeaderText).ToArray
    
    
                    Dim dt = DirectCast(DataGridView1.DataSource, DataTable)
    
                    Using sw As New IO.StreamWriter(sfd.FileName)
                        sw.WriteLine(String.Join(",", headers))
    
                        For ii = 0 To dt.Rows.Count - 1
    
                            For yy = 0 To dt.Columns.Count - 1
    
                                sw.Write(If(dt.Rows(ii).ItemArray(yy)?.ToString(), ""))
    
                                If yy < dt.Columns.Count - 1 Then
                                    sw.Write(",")
                                Else
                                    sw.WriteLine()
                                End If
    
                            Next
    
                        Next
    
                    End Using
    
    
    
                Catch ex As Exception
                    MsgBox("Save As CSV Error: " & ex.Message)
                End Try
    
    
            End If
    
    0 comments No comments

4 additional answers

Sort by: Most helpful
  1. Viorel 114.2K Reputation points
    2020-12-14T18:04:45.763+00:00

    Try something like this:

    Dim rows = From row As DataGridViewRow In DataGridView1.Rows.Cast(Of DataGridViewRow)()
                Where Not row.IsNewRow
    
    Using sw As New IO.StreamWriter(sfd.FileName)
        sw.WriteLine(String.Join(",", headers))
        For Each r In rows
            sw.WriteLine(String.Join(",", r.Cells.Cast(Of DataGridViewCell).Select(Function(c) """" & If(c.Value IsNot Nothing, c.Value.ToString.Replace("""", """"""), "") & """")))
        Next
    End Using
    

    Probably you should add '"' to headers too.


  2. Karen Payne MVP 35,366 Reputation points
    2020-12-14T22:26:46.047+00:00

    Hello,

    Try the following SomeDataGridView.ExportRows("TODO file name")

    Imports System.IO
    Public Module DataGridViewExtensions
        <Runtime.CompilerServices.Extension> _
        Public Sub ExportRows( sender As DataGridView,  pFileName As String, Optional  defaultNullValue As String = "(empty)")
            File.WriteAllLines(pFileName,(sender.Rows.Cast(Of DataGridViewRow)().
                                  Where(Function(row) Not row.IsNewRow).Select(Function(row) New With {
                                  Key row,
                                  Key .rowItem = String.Join(",", Array.ConvertAll(row.Cells.Cast(Of DataGridViewCell)().ToArray(), 
                                                                                   Function(c) (If(c.Value Is Nothing, defaultNullValue, c.Value.ToString()))))
                                                                                 }).Select(Function(row) row.rowItem)))
        End Sub
    End Module
    

  3. jim brown 271 Reputation points
    2020-12-16T14:03:40.457+00:00

    I would think it was common to have DGV with a million rows and twenty or so columns. my example has five hundred thousand rows and nineteen columns. here is the stacktrace it's just running out of memory in the loop. is there not a way to write out a DGV into chunks?

    Exception thrown: 'System.OutOfMemoryException' in System.Windows.Forms.dll
    at System.Windows.Forms.PropertyStore.SetObject(Int32 key, Object value)
    at System.Windows.Forms.DataGridViewBand.set_HeaderCellCore(DataGridViewHeaderCell value)
    at System.Windows.Forms.DataGridViewRow.Clone()
    at System.Windows.Forms.DataGridViewRowCollection.get_Item(Int32 index)
    at System.Windows.Forms.DataGridViewRowCollection.UnsharingRowEnumerator.System.Collections.IEnumerator.get_Current()
    at System.Linq.Enumerable.<CastIterator>d__aa1.MoveNext() at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()
    at _OracleClient.FormMain.ButtonExport_Click(Object sender, EventArgs e) in C:\DotNet-Projects\OracleClient\OracleClient\FormMain.vb:line 426

    48801-msdn1.jpg

    0 comments No comments

  4. Karen Payne MVP 35,366 Reputation points
    2020-12-16T17:46:36.247+00:00

    Hello @jim brown

    I would think it was common to have DGV with a million rows and twenty or so columns. my example has five hundred thousand rows and nineteen columns

    Actually this is uncommon, the rule of thumb is to load only what is needed. With this many rows the best option is to use a filter and/or a search feature and depending on the complexity of these features may require little or a good deal of code.

    One thought if possible is to load your data into a DataTable then use a BindingSource and set the DataSource of the BindingSource to the DataTable followed by DataGridView1.DataSource = bsCustomers or whatever name you want, bsCustomers relates to the code below.

    language Extension

    Public Module DataExtensions  
        <Runtime.CompilerServices.Extension>  
        Public Iterator Function Partition(dataTable As DataTable, partitionSize As Integer) As IEnumerable(Of IEnumerable(Of DataRow))  
            Dim numRows = Math.Ceiling(CDbl(dataTable.Rows.Count))  
            Dim index = 0  
            Do While index < numRows / partitionSize  
                Yield Partition(dataTable, index * partitionSize, index * partitionSize + partitionSize)  
                index += 1  
            Loop  
        End Function  
        Private Iterator Function Partition(dataTable As DataTable, index As Integer, endIndex As Integer) As IEnumerable(Of DataRow)  
            Dim indexer = index  
            Do While indexer < endIndex AndAlso indexer < dataTable.Rows.Count  
                Yield dataTable.Rows(indexer)  
                indexer += 1  
            Loop  
        End Function  
    End Module  
    

    Usage

    Replace the Console.WriteLine with write method to file and assert column values if needed.

    Dim dt = CType(bsCustomers.DataSource, DataTable)  
      
    Dim partitions As List(Of IEnumerable(Of DataRow)) = dt.Partition(10).ToList()  
      
    For Each enumerable As IEnumerable(Of DataRow) In partitions  
        For Each dataRow As DataRow In enumerable  
            Console.WriteLine(String.Join(",", dataRow.ItemArray))  
        Next  
    Next  
    

    Summary

    • As mentioned it's not common place to load that many rows
    • General rule is to load data into a container such as a DataTable or a List(Of T)
    0 comments No comments