ASP.NET - System.OutOfMemoryException - Asked By Thileep on 02-Aug-11 09:42 PM

I upload 400MB of file into sql server. during retreival of data it shows 'System.OutOfMemoryException' exception.
Any one got the solution?
Ravi S replied to Thileep on 02-Aug-11 09:50 PM
HI

Try to run a debug version to catch which code lines cause System.OutOfMemoryException.


For System.OutOfMemory.Exception, I'd like to give another suggestions for you reference.

  •   Try to check if this problem will go away after iisreset or terminating aspnet_wp.exe worker process in asp.net web server.
  •   The .net memory profiler is a good starting point coupled with a few well placed performance counters
  •    Do you have remote access serviced component in your web application? If yes, try to find if the memory problem is caused by the remote   serviced component.You can stop the remote serviced component to see if the memory exception goes away.
  •    Try to refer to the following links to find the memory exception
 #Watching Your Server Processes
http://forums.asp.net/247reference/a.aspx?u=http://msdn.microsoft.com/library/en-us/dnaspp/html/aspnet-watchserverproces
ses.asp?frame=true

#ASP.NET Performance Monitoring, and When to Alert Administrators
http://forums.asp.net/247reference/a.aspx?u=http://msdn.microsoft.com/library/en-us/dnaspp/html/monitor_perf.asp?frame=t
rue

#Production Debugging for .NET Framework Applications
http://forums.asp.net/247reference/a.aspx?u=http://msdn.microsoft.com/library/en-us/dnbda/html/DBGch02.asp?frame=true
Riley K replied to Thileep on 02-Aug-11 10:01 PM

First of all, you'll  get into trouble if you persist in keeping very large data structures in memory. There are some quite definite liimits both to virtual (in 32-bit systems) address space and physical memory.

To get the most out of what's there in 32-bit environments for ASP.NET, you should do two things:

1. Enable the /3GB switch in boot.ini.

2. For IIS 6: In the application pool properties, set the Maximum used memory (in megabytes) to 1800. (Without enabling the /3GB switch, the max value is approx. 800).

You do not need to have 3Gb or more physical memory to do this. If it's a dedicated web server and this is the only app, 2Gb is sufficient. It'll work with 1Gb as well, but slowly.

The above will almost get you the 2Gb you want. The next step to increase available memory is to go for an x64 version of the server os. The correct step is probably to redesign the app not to use memory in this way.

Try this and let me know

pete rainbow replied to Thileep on 02-Aug-11 11:20 PM
as stated in  the other post on this issue

do  not try and hold large blocks of data in memory, it's not good programming practice even in c++

this is why we have stream classes

also you shouldn't be trying to put large files into sql server a better model is to put them in a file system and store the path in sql server

having said that sql server does have the filestream thing to do this...

using (SqlConnection connection = new SqlConnection(
  connStringBuilder.ToString()))
{
  connection.Open();
 
  SqlCommand command = new SqlCommand("", connection);
  command.CommandText = "select Top(1) Photo.PathName(), "
  + "GET_FILESTREAM_TRANSACTION_CONTEXT () from employees";
 
  SqlTransaction tran = connection.BeginTransaction(
    System.Data.IsolationLevel.ReadCommitted);
  command.Transaction = tran;
 
  using (SqlDataReader reader = command.ExecuteReader())
  {
    while (reader.Read())
    {
      // Get the pointer for file
      string path = reader.GetString(0);
      byte[] transactionContext = reader.GetSqlBytes(1).Buffer;
 
      FileStream fileStream = new SqlFileStream(path,
        (byte[])reader.GetValue(1),
        FileAccess.ReadWrite,
        FileOptions.SequentialScan, 0);
 
      // Seek to the end of the file
      fs.Seek(0, SeekOrigin.End);
 
      // Append a single byte
      fileStream.WriteByte(0x01);
      fileStream.Close();
    }
  }
  tran.Commit();
}


see http://msdn.microsoft.com/en-us/library/cc716724.aspx#Y540
Jitendra Faye replied to Thileep on 03-Aug-11 12:00 AM
Problem

 
When doing a large upload in ASP.NET, the transfer fails with the error "System.OutOfMemoryException". The stacktrace contains the following:

[OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown.]



This exception is thrown by the .NET runtime and not our product. The error is caused by .NET reading in the entire contents of a POST into memory. As a result the OutOfMemoryException error message is thrown and the ASP.NET worker process will recycle itself.

Solution 
 

Reading in the entire contents of a request to memory is the default behavior of ASP.NET. This makes it impossible to upload very large files without causing problems. We solve this problem by providing an httpModule which can strip out the files from the upload and cache them to disk before ASP.NET has a chance to read in the request. We then reformat the request and pass it on with the binary data removed, into the .NET application where it is read and processed. The result is that the request .NET sees is only a few kb in size at most, and no memory problems will occur. 

 
For more information on configuring the httpModule, please check our documentation http://support.softartisans.com/fileupee/doc/install/install_dotnet.asp#httpmodule.

Jitendra Faye replied to Thileep on 03-Aug-11 12:01 AM

Problem

 
When doing a large upload in ASP.NET, the transfer fails with the error "System.OutOfMemoryException". The stacktrace contains the following:

[OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown.]
 
This exception is thrown by the .NET runtime and not our product. The error is caused by .NET reading in the entire contents of a POST into memory. As a result the OutOfMemoryException error message is thrown and the ASP.NET worker process will recycle itself.


Solution
 

Reading in the entire contents of a request to memory is the default behavior of ASP.NET. This makes it impossible to upload very large files without causing problems. We solve this problem by providing an httpModule which can strip out the files from the upload and cache them to disk before ASP.NET has a chance to read in the request. We then reformat the request and pass it on with the binary data removed, into the .NET application where it is read and processed. The result is that the request .NET sees is only a few kb in size at most, and no memory problems will occur. 

 
For more information on configuring the httpModule, please check our documentation here.