Heap works fine with segment

The large object heap on Windows systems

  • 8 minutes to read

The .NET Garbage Collector (GC) divides objects into the categories "small" and "large". When an object is large, some of its attributes are more important than with a small object. Compression, i.e. copying into the main memory at another point in the heap, can be time-consuming. Therefore, the garbage collector places large objects in the large object heap (LOH). This article explains what makes an object a large object, how to clean large objects, and how large objects affect performance.

Important

This article explains the large object heap in the .NET Framework and .NET Core, which can only run on Windows systems. The large object heap in .NET implementations on other platforms is not addressed.

How does an object get into the large object heap?

An object is considered a large object if it is at least 85,000 bytes in size. This number was determined by the performance optimization. If the allocation requirement for an object is over 85,000 bytes, the runtime allocates this to the large object heap.

The following section provides some basics about the garbage collector for a better understanding.

The garbage collector is a generation-based collector. It has three generations: Generation 0, Generation 1, and Generation 2. There are three generations because in a well-set up app, most objects become inactive in Generation 0. In a server app, the mappings for each request should become inactive after the request is completed. The pending assignment requests go to generation 1 and then become inactive. Basically, Generation 1 serves as a buffer between areas with new objects and areas with long-lived objects.

Small objects are always assigned to generation 0 and, depending on their lifespan, are promoted to either generation 1 or 2. Large objects are always assigned in generation 2.

Large objects belong to generation 2 because they are only cleaned up during a garbage collection for generation 2. When a generation is cleaned up, all younger generations are also cleaned up. For example, in a garbage collection for generation 1, generations 1 and 0 are cleaned up. If generation 2 is garbage collected, the entire heap is cleaned up. For this reason, generation 2 garbage collection is also called full garbage collection designated. This article uses “Generation 2 garbage collection” instead of “full garbage collection,” but the terms are synonymous.

The generations provide a logical view of the garbage collection heap. From a physical point of view, objects are stored in managed heap segments. A managed heap segment is a block of memory that the operating system's garbage collector reserves (by calling the VirtualAlloc function) for managed code. When loading the CLR, the garbage collector first allocates two heap segments: one for small objects (the small object heap) and one for large objects (the large object heap).

The allocation requirements are then met by placing managed objects in these managed heap segments. If the object is smaller than 85,000 bytes, it is placed in the segment for the small object heap, otherwise in the segment for a large object heap. Segments are adopted (in smaller blocks) when more and more objects are assigned to them. With the small object heap, objects that are not cleaned up during garbage collection are promoted to the next generation. Objects that are not cleaned up in a generation 0 garbage collection are now considered generation 1 objects, and so on. However, objects that outlast the oldest generation are still considered to be the oldest generation objects. So the objects that survive generation 2 are generation 2 objects, and the objects that survive the large object heap are objects of the large object heap (which are cleaned up in generation 2).

User code can only be assigned in generation 0 (small objects) or in the large object heap (large objects). Only the garbage collector can allocate generation 1 (by promoting retained objects from generation 0) and generation 2 (by promoting retained objects from generations 1 and 2).

When a garbage collection is triggered, the garbage collector goes through all active objects and compresses them. However, since the compression is time-consuming, adjusted garbage collection the large object heap. This creates a free list of inactive objects that can later be reused to meet allocation requirements for large objects. Adjacent inactive objects are converted into a single free object.

.NET Core and .NET Framework (starting with .NET Framework 4.5.1) include the GCSettings.LargeObjectHeapCompactionMode property, which users can use to specify that the large object heap should be compressed during the next full blocking garbage collection. In the future, the large object heaps in .NET may be compressed automatically. Therefore, if you are mapping large objects and want to make sure they do not move, you should peg these objects.

Figure 1 shows a scenario in which generation 1 is formed after the first garbage collection for generation 0 where and are inactive. Generation 2 is formed after the first garbage collection for generation 1 in which and are inactive. Note that this and the following images are for illustrative purposes only. They contain only a few objects to better represent what is happening on the heap. In practice, a lot more objects are usually involved in a garbage collection.


Figure 1: Garbage Collection for Generation 0 and 1.

Figure 2 shows that after a generation 2 garbage collection with and inactive, the garbage collection creates free memory that can be used by and. These are then used to fulfill an allocation requirement for. The space after the last object () to the end of the segment can also be used to meet allocation requirements.


Figure 2: After a garbage collection for generation 2

If there is not enough space to meet the allocation requirements for large objects, garbage collection tries first to request more segments from the operating system. If that fails, generation 2 garbage collection is triggered to free up space.

In the case of a garbage collection for generation 1 or 2, the garbage collector releases segments for the operating system in which there are no active objects by calling the VirtualFree function. The space from the last active object to the end of the segment is retained (except in the ephemeral segment where generation 0 and generation 1 are active and where the garbage collector keeps some space as it is immediately allocated to your application). The free memory blocks are still committed even though they are being reset. This means that the operating system does not have to write the data in them back to the data carrier.

Since the large object heap is only cleaned up during garbage collections for generation 2, a segment of the large object heap can only be released during such a garbage collection. Figure 3 illustrates a scenario in which the garbage collector frees up one segment (segment 2) for the operating system and reserves more space for the remaining segments. When the vacated space at the end of the segment needs to be used to meet allocation requirements for large objects, memory is committed again. (See the documentation for VirtualAlloc for an explanation of applying and canceling.)


Figure 3: The large object heap after a garbage collection for generation 2

When is a large object cleaned up?

Generally, garbage collection occurs when any of the following three conditions are true:

  • The allocation exceeds the threshold of generation 0 or the large object.

    The threshold is a property of a generation. A threshold is set for a generation when the garbage collector allocates objects to it. If the threshold is exceeded, garbage collection is triggered for that generation. Therefore, when allocating small or large objects, use the thresholds for generation 0 and the large object heap, respectively. When the garbage collector maps to generation 1 or 2, it uses their thresholds. These thresholds are dynamically adjusted during the execution of the program.

    This is the normal case. Most garbage collections are triggered because of allocations in the managed heap.

  • The GC.Collect method is called.

    When the parameterless GC.Collect () method is called or another overload is passed as an argument to GC.MaxGeneration, the large object heap is cleaned up along with the rest of the managed heap.

  • The system has little memory.

    This occurs when the garbage collector receives a high memory usage notification from the operating system. If the garbage collector considers a generation 2 garbage collection to be productive, it is triggered.

Impact of the LOH on performance

Allocations to the large object heap affect performance in the following ways:

  • Allocation costs:

    The CLR guarantees that the memory will be cleaned up for each new object that is released. This means that for a large object, the allocation cost is completely dominated by garbage collection (unless garbage collection is triggered). If it takes two cycles to clean one byte, it means that it takes 170,000 cycles to clean the smallest large object. It takes approximately 16 milliseconds to clean up the memory of a 16 MB object on a 2 GHz computer. This leads to high costs.

  • Cleanup costs:

    Because the large object heap and generation 2 are cleaned up together, garbage collection is triggered for generation 2 if either of the two thresholds is exceeded. If generation 2 garbage collection was triggered because of the large object heaps, generation 2 does not necessarily become significantly smaller after the garbage collection. Unless there is a lot of data in Generation 2, the impact will be minimal. However, if Generation 2 is large, performance issues can arise if many Garbage Collections are triggered for Generation 2. If many large objects are allocated temporarily and you have a large, small object heap, you may be spending too much time on garbage collections. In addition, the allocation costs can add up if you continue to allocate and release very large objects.

  • Array elements with reference types:

    Very large objects in the large object heap are typically arrays (it is extremely rare for a very large instance object to exist). If the elements of an array contain many references, there is a cost that does not apply to elements with few references. If the element does not contain any references, the garbage collector does not have to iterate over the array at all. For example, if you were using an array to store nodes in a binary tree, this could be implemented by referencing the right and left nodes of a node through the actual nodes:

    If is large, the garbage collector must go through at least two references per item. An alternative approach is to store the index of the right and left nodes:

    Do not refer to the data of the left node with, but with. The garbage collector then does not need to look at references for the left and right nodes.

Of these three factors, the first two are usually more relevant than the third. Therefore, it is recommended that you allocate a pool of large objects that you reuse rather than allocate temporary objects.

Collect performance data for the LOH

Before collecting performance data for any particular area, you should have done the following:

  1. Find evidence that you should review this area

  2. Investigate other known areas with no result that could explain the existing performance problem

For more information on memory and CPU basics, see the blog post Understand the problem before you try to find a solution.

You can use the following tools to collect data about the performance of the large object heaps:

.NET CLR storage performance counters

These memory performance counters are usually a good first step in investigating performance problems. However, it is recommended that you use ETW events. You can configure the system monitor as shown in Figure 4 by adding the indicators you want. The following are relevant for the large object heap:

  • Number of garbage collections for generation 2

    The number of generation 2 garbage collections that have been performed since the process started. The counter is incremented at the end of a generation 2 garbage collection (also known as full garbage collection). This indicator shows the last recorded value.

  • Size of the heap for large objects

    Displays the current size of the large object heaps in bytes (including free space). This counter is not updated with every allocation, but only at the end of a garbage collection.

A common way to view performance counters is to use Performance Monitor (perfmon.exe). Add the appropriate counter for processes that are important to you using Add Counters. You can save the performance counter data to a log file in Performance Monitor, as shown in Figure 4:

Figure 4: The large object heap after a garbage collection for generation 2

Performance counters can also be queried programmatically. Many users collect performance data this way as part of their day-to-day testing processes. When indicators with unusual values ​​are displayed, other methods can then be used to obtain more detailed data to facilitate investigation.

Note

It is recommended that you use ETW events rather than performance counters because ETW provides more information.

ETW events

The garbage collector has many ETW events that can help you understand what the heap is doing and why. The following blog posts demonstrate how to capture and understand GC events with ETW:

Examine the Garbage Collection Trigger Reason column to identify generation 2 excessive garbage collections caused by temporary allocations from large object heaps. If you want to do a simple test that only temporarily maps large objects, you can use the PerfView command line to collect information about ETW events:

The result is similar to the following:

Figure 5: ETW events viewed using PerfView

You will find that all generation 2 garbage collections have been completed and triggered by AllocLarge. This means that the allocation of a large object triggered this garbage collection. The mappings are known to be temporary because the LOH survival rate (Retention rate of the large object heaps) shows "1%".

You can capture additional ETW events that show who assigned these large properties. For example, use the following command line:

This captures an AllocationTick event that is raised approximately every 100,000 allocations. This means that an event is triggered whenever a large object is assigned. You can then look at one of the garbage collection heap mapping views, which shows the call stacks that have large objects allocated:

Figure 6: View of the heap allocation for a garbage collection

As you can see, this is a simple test that only maps large objects from its method.

Debugger

If you only have a memory dump and need to examine what objects are in the large object heap, you can use the .NET-provided SOS debug extension.

Note

The debugging commands mentioned in this section apply to the Windows debuggers.

The following is an example of the output for analyzing the large object heaps:

The size of the large object heaps is 49,738,016 bytes (16,754,224 + 16,699,288 + 16,284,504). Between addresses 023e1000 and 033db630, 8,008,736 bytes are occupied by an array of System.Object objects and 6,663,696 bytes are occupied by an array of System.Byte objects. 2,081,792 bytes are occupied by free space.

Sometimes the debugger indicates that the total size of the large object heaps is less than 85,000 bytes. This happens because the runtime uses the large object heap to allocate certain objects that are smaller than a large object.

Because the large object heap is not compressed, the large object heap is sometimes mistaken for the source of fragmentation. Fragmentation means:

  • The fragmentation of the managed heap, which is characterized by the amount of free space between managed objects. In the SOS debugger, the command shows the amount of free space between managed objects.

  • The fragmentation of the virtual memory address space, which is the memory marked as. You can get this using various debugger commands in WinDbg.

    The following example shows virtual memory fragmentation:

More often than not, virtual memory fragmentation is caused by temporary large objects that require the garbage collector to periodically retrieve new managed heap segments from the operating system and reclaim empty segments for the operating system.

You can determine if the large object heap is causing the virtual memory fragmentation by setting a breakpoint on VirtualAlloc and VirtualFree and determining who is calling these functions. For example, you can set a breakpoint as follows to show who has tried to allocate virtual memory blocks larger than 8 MB from the operating system:

This command breaks the debugger and only displays the call stack when VirtualAlloc is called with an allocation size greater than 8MB (0x800000).

In CLR 2.0 a feature called VM hoarding added, useful for scenarios where segments (including the large and small object heaps) are accessed and shared frequently. If you want to define the VM Hoarding feature, specify a start flag name via the hosting API. The CLR reserves the working memory for these segments and puts them on a standby list instead of freeing empty segments again for the operating system. (Note that the CLR cannot do this on segments that are too large.) The CLR uses these segments later to meet requests for new segments. The next time your app needs a new segment, the CLR will use one of this standby list if it can find one that is large enough.

The VM Hoarding feature is also useful for applications where you want to keep segments that have already been retrieved. These include, for example, server apps that are executed as the dominant apps in the system in order to avoid exceptions due to insufficient memory.

It is strongly recommended that you test your application carefully when using this feature and ensure that your application's memory usage is relatively stable.