AW: AW: AW: [udig-devel] Heap space error adding/removing
graphicallayers
Jesse Eichar
jeichar at refractions.net
Fri Dec 8 09:31:07 PST 2006
Could you email me the command and I will try to break uDig as you have.
Jesse
On 8-Dec-06, at 12:38 AM, Schneider, Michael wrote:
> Thank you, Jesse, for your work and the informations. That might be
> a hint for me to have a closer look at.
>
> When I detected the problem first, I was not using uDIG as an RCP
> application but was using parts (plugins) of uDIG (seen from the
> user's point of view only the map editor and the layers view) in
> another RPC application.
> But I repeated what I did with the "whole uDIG" as an RPC
> application, and I could reproduce the error. Doing so I created a
> command which added the layers (before it removed the old ones).
> This was started from within Eclipse with no special settings set
> (except those being part of the uDIG SDK). A plugin with this
> command was the only enhancement to uDIG. I'm running on Windows XP
> Professional.
>
> Cheers,
> Michael
>
> Von: udig-devel-bounces at lists.refractions.net [mailto:udig-devel-
> bounces at lists.refractions.net] Im Auftrag von Jesse Eichar
> Gesendet: Donnerstag, 7. Dezember 2006 23:44
> An: User-friendly Desktop Internet GIS
> Betreff: Re: AW: AW: [udig-devel] Heap space error adding/removing
> graphicallayers
>
> I have spent a good portion of today profiling uDig and found
> something strange. As one adds and remove layers the Heap Size
> keeps growing and growing and growing. However if the garbage
> collector is ran the heap size goes right back down. It seems that
> it is the byte arrays in the BufferedImages that are taking a while
> to clear up. Perhaps if I pool them better rather than always
> recreating them it will help the heap problem. But it seems to me
> that the Garbage collector should kick in before a heap space error
> occurs. ( I haven't been able to reproduce the heap space error
> yet... I've added and removed 10 shapefile layers probably 30
> times without problems. And I have the heap space set to the
> default Java size.
>
> I have tested on both windows and OSX. I did the profiling on OSX
> but the performance seemed even better on Windows (shock!).
>
> What is your configuration? Are there any other variable that I
> can look at?
>
> Jesse
>
>
>
> On 7-Dec-06, at 6:05 AM, Jesse Eichar wrote:
>
>> Thanks Michael, I will look in to the problem and will get back
>> to you this evening on what I found.
>>
>> Jesse
>>
>> On 7-Dec-06, at 12:26 AM, Schneider, Michael wrote:
>>
>>> Jesse, thanks for the answers.
>>> Sure, it's better to use one large shapefile instead of several
>>> smaller ones. I tried that after detecting the heap space problem
>>> with the gif files.
>>>
>>> I think it is not the maximal amount of memory which causes the
>>> problem, but perhaps a memory leak or memory fragmentation: Even
>>> when adding and removing only one single layer I detect that
>>> problem after having added and removed a layer more than 30 times.
>>>
>>> Michael
>>>
>>> Von: udig-devel-bounces at lists.refractions.net [mailto:udig-devel-
>>> bounces at lists.refractions.net] Im Auftrag von Jesse Eichar
>>> Gesendet: Mittwoch, 6. Dezember 2006 18:13
>>> An: User-friendly Desktop Internet GIS
>>> Betreff: Re: AW: [udig-devel] Heap space error adding/removing
>>> graphical layers
>>>
>>> Hi,
>>>
>>> There is a limitation to the number of layers that can be
>>> realistically added because currently each layer has a buffer
>>> that it writes to that is the size of the screen and is in 32-bit
>>> colour. Obviously a good chunk of memory especially when the
>>> screen is large. So it is better to have large shapefiles than
>>> many shapefiles because the Shapefile renderer is highly
>>> optimized for large shapefiles and performance will be much
>>> better rendering a 1 GB shapefile than 100 100KB shapefiles.
>>>
>>> This decision was part of the design because it allows some fun
>>> optimizations (for example enabling and disabling layers) but as
>>> of 1.1.1 we will have an "intelligent" system that will set a
>>> maximum number of buffers depending on the memory size so that
>>> this issue will no longer be a problem.
>>>
>>> Jesse
>>>
>>> On 6-Dec-06, at 3:54 AM, Schneider, Michael wrote:
>>>
>>>> Is grid coverage also involved when the layers are created from
>>>> shapefiles?
>>>> Using a bunch of shapefile layers instead of the GIF layers I
>>>> also ran into the same heap space error after adding and
>>>> removing the layers several times. (Each shapefile only had one
>>>> single feature, a point in this case.)
>>>>
>>>> Michael
>>>>
>>>> Von: udig-devel-bounces at lists.refractions.net [mailto:udig-devel-
>>>> bounces at lists.refractions.net] Im Auftrag von Jesse Eichar
>>>> Gesendet: Dienstag, 5. Dezember 2006 18:47
>>>> An: User-friendly Desktop Internet GIS
>>>> Betreff: Re: [udig-devel] Heap space error adding/removing
>>>> graphical layers
>>>>
>>>> Hi
>>>>
>>>> There are 2 main possibilities that I can think of.
>>>>
>>>> 1. Make sure that you are passing in a heap space parameter to
>>>> your application or increase it if you already are. For example
>>>> -Xmx512M will assign 512 MB of memory to the heap space of the
>>>> JVM. That should help with the problem.
>>>> 2. GridCoverage support is less than perfect at the moment
>>>> (although Jody is playing with a new version that shows promise)
>>>> and takes much more memory than it should. So it is likely that
>>>> the World+Image is what is causing problems. I'm not totally
>>>> sure what can be done about this. But if it is causing problems
>>>> then perhaps some memory optimizations can be made.
>>>>
>>>> Jesse
>>>>
>>>>
>>>> On 5-Dec-06, at 3:33 AM, Schneider, Michael wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> using uDIG RC 5 I got a heap space error after adding several
>>>>> layers. Each layer was created from a very small graphic (png
>>>>> or gif file of less than 1 KB) together with a world file. The
>>>>> same was true when each layer was created from a small shapefile.
>>>>>
>>>>> The intension is as follows: The user creates a list of
>>>>> locations and these locations are shown on a map. When creating
>>>>> a new list of locations, the old locations are removed from the
>>>>> map and the new ones are displayed.
>>>>>
>>>>> I did it similar to the descriptions in the FAQ "How do I
>>>>> create a map programmatically" and "I want to add an image to
>>>>> the map, how do I do that?"
>>>>>
>>>>> I created a map using createCreateMapCommand
>>>>> I added about ten layers (see above what the layers looked
>>>>> like) with either createAddManyLayersCommand (all layers
>>>>> together) or createAddLayerCommand (one layer at a time)
>>>>> When a new list was created I removed the old layers using
>>>>> createDeleteLayer command, set the new bounding box with
>>>>> createSetViewportBBoxCommand and added the new layers to the map.
>>>>> After doing these several times I run into the heap space error.
>>>>> I also get this error when I only add and remove one layer
>>>>> (i.e. location lists with only one entry), but it takes longer
>>>>> to get the error.
>>>>>
>>>>> I also tried it "the old way" with MapFactory.processURLs -
>>>>> with a similar result: The error appeared as well, but it took
>>>>> longer to get it.
>>>>>
>>>>> Am I doing it the wrong way? Is there a better way?
>>>>>
>>>>> I also wanted to try RC 6a. Are there any changes related to
>>>>> the layers view and showing the map? In RC 5 after adding the
>>>>> layer to the map they were displayed at once. In RC 6a I can
>>>>> see them in the catalog and in the project window, but they do
>>>>> not appear in the layers view automatically and are not
>>>>> displayed. I'll have a closer look to that to see whether I get
>>>>> the heap space error in RC 6a as well.
>>>>>
>>>>> Michael
>>>>>
>>>>> _______________________________________________
>>>>> User-friendly Desktop Internet GIS (uDig)
>>>>> http://udig.refractions.net
>>>>> http://lists.refractions.net/mailman/listinfo/udig-devel
>>>>
>>>> _______________________________________________
>>>> User-friendly Desktop Internet GIS (uDig)
>>>> http://udig.refractions.net
>>>> http://lists.refractions.net/mailman/listinfo/udig-devel
>>>
>>> _______________________________________________
>>> User-friendly Desktop Internet GIS (uDig)
>>> http://udig.refractions.net
>>> http://lists.refractions.net/mailman/listinfo/udig-devel
>>
>> _______________________________________________
>> User-friendly Desktop Internet GIS (uDig)
>> http://udig.refractions.net
>> http://lists.refractions.net/mailman/listinfo/udig-devel
>
> _______________________________________________
> User-friendly Desktop Internet GIS (uDig)
> http://udig.refractions.net
> http://lists.refractions.net/mailman/listinfo/udig-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.refractions.net/pipermail/udig-devel/attachments/20061208/30d620ef/attachment-0001.html
More information about the udig-devel
mailing list