Microsoft XML - System.OutOfMemoryException

Author

System.OutOfMemoryException

Bala

2007-03-24, 4:18 am

I am trying to load a XML file into a dataset, but my file size is
around 420 MB. It is taking time to load but it is not the issue in my
case, Some time it is (only some time not every time) giving me a
System.OutOfMemoryException. Even tough the ram is 16 GB and virtual
memory is 8 GB total free space in the desk is around 12 GB. One of
the worst thing is we are getting this error in PRODUCTION and we
could not able to stimulate this error in other dev and test
environment.

Can you please help me out ..

Joe Fawcett

2007-03-24, 4:18 am

"Bala" <balamani.2u@XXXXXXXXXX> wrote in message
news:1174720797.030643.307680@l77g2000hsb.googlegroups.com...
>I am trying to load a XML file into a dataset, but my file size is
> around 420 MB. It is taking time to load but it is not the issue in my
> case, Some time it is (only some time not every time) giving me a
> System.OutOfMemoryException. Even tough the ram is 16 GB and virtual
> memory is 8 GB total free space in the desk is around 12 GB. One of
> the worst thing is we are getting this error in PRODUCTION and we
> could not able to stimulate this error in other dev and test
> environment.
>
> Can you please help me out ..
>
I would imagine that a 420 MB file needs about 1.5 - 2.0 GB of memory. This
could be more if there are includes etc. You might try processing another
way, XmlReader would be best although you haven't said what you do with the
file.
Does the problem occur when nothing else is running on the machine? What
does the task manager say is being used in virtual memory for that process
when it succeeds?

--

Joe Fawcett (MVP - XML)
http://joe.fawcett.name

Bjoern Hoehrmann

2007-03-25, 4:16 am

* Bala wrote in microsoft.public.xml:
>I am trying to load a XML file into a dataset, but my file size is
>around 420 MB. It is taking time to load but it is not the issue in my
>case, Some time it is (only some time not every time) giving me a
>System.OutOfMemoryException. Even tough the ram is 16 GB and virtual
>memory is 8 GB total free space in the desk is around 12 GB. One of
>the worst thing is we are getting this error in PRODUCTION and we
>could not able to stimulate this error in other dev and test
>environment.

In addition to what Joe said, there might be artificial limits imposed
on the software, for example, you might be running this as part of a web
server and the web server does not allow child processes to use more
than a certain amount of memory, while the development environment uses
other limits or other software. There may also be differences between
the machines running the application, for example, the production server
might be a 64bit system while the dev environment a 32bit system. With-
out us knowning the differences, there is not much we can say about it.
You might also consider posting in microsoft.public.dotnet.xml if you
are using .NET.
--
Björn Höhrmann · mailto:bjoern@hoehrmann.de · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

Rich-F

2007-04-05, 6:16 am

"Bala" wrote:

> I am trying to load a XML file into a dataset, but my file size is
> around 420 MB. It is taking time to load but it is not the issue in my
> case, Some time it is (only some time not every time) giving me a
> System.OutOfMemoryException. Even tough the ram is 16 GB and virtual
> memory is 8 GB total free space in the desk is around 12 GB. One of
> the worst thing is we are getting this error in PRODUCTION and we
> could not able to stimulate this error in other dev and test
> environment.
>
> Can you please help me out ..
>
You haven't given details of the OS you're using but check this list to see
what your server really can utilise:

http://msdn2.microsoft.com/en-us/library/aa366778.aspx