Skip to content

Running out of memory kills GeoDMS silently: no error, no log line, no dump #1158

Description

@jipclaassens

Running out of memory kills GeoDMS silently: no error, no log line, no dump

Symptom

A config whose calculation does not fit in memory does not fail with a GeoDMS error. The process
grows until Windows terminates it. There is no exception, no WER report, no dump, and nothing
in the GeoDMS eventlog
. The only trace is in the Windows System log:

Microsoft-Windows-Resource-Exhaustion-Detector, event 2004, 2026-07-15 01:05:15
Windows successfully diagnosed a low virtual memory condition. The following programs consumed
the most virtual memory: GeoDmsGuiQt.exe (6908) consumed 639747825664 bytes, devenv.exe (22840)
consumed 3320340480 bytes, and msedgewebview2.exe (12172) consumed 1168519168 bytes.

639,747,825,664 bytes = 595.8 GiB, against a 616.2 GiB commit limit (128 GiB RAM + 488 GiB
pagefile). From the user's side the application simply disappeared overnight.

Observed on /MaakOntkoppeldeData/Write_FinalMutationTable in the Redevelopment config
(BAG_Selection_Area = 'Nederland'). Progression, sampled every 20-25 s:

00:26:51  Priv=107 GB  WS=89 GB   SysFree=24.8 GB
00:27:55  Priv=181 GB  WS=1.7 GB  SysFree=0.004 GB
00:34:57  Priv=361 GB  WS=0.2 GB  SysFree=0.003 GB
00:38:40  Priv=500 GB  WS=3.8 GB  Commit=521 GB (85% of limit)
00:45:58  Priv=512 GB  WS=15 GB   Commit=534 GB   <- 29 worker threads at ~3.5% CPU each: page-fault bound
01:05:15  killed by the OS at 595.8 GiB

Note the last phase: 512 GB committed against 15 GB resident means essentially every access is a
hard fault to the pagefile. It was never going to finish, and nothing said so.

Why there is nothing to stop it

The intended brake exists but is disabled. WaitForAvailableMemory (MemGuard.cpp:186) is a
pass-through — its entire wait/backoff body is commented out at :187-223:

void WaitForAvailableMemory(std::size_t requestedSize)
{
	ConsiderMakingFreeSpace(requestedSize);
/*
	memory_info info;
	if (info.SufficientFreeSpace(requestedSize))
		return;
	...  // exponential backoff, re-check, "Resuming anyway, hoping the System page file ..."
*/
}

It is called from FixedAlloc.cpp:144 and :638, i.e. on every allocator chunk — exactly the
right place to notice and refuse. With the body disabled, the only remaining brake is the coarse
IsLowOnFreeRAM() heuristic in OperationContext.cpp:1059 and ParallelTiles.h:149, which
limits concurrency but never refuses an allocation. So there is no path that produces an
out-of-memory error; the process just grows until the OS intervenes.

What triggered it here (for context, not the bug itself)

PrepBAG.dms:1257:

attribute<WP4> pand_type (MutTable) := ='merge(TyperingsJaar_idx, WP4, '+AsItemList('perJaar/'+classifications/Tijd/Jaren/name)+')';

where each perJaar/<year> (:1252) resolves to
BAG_Tabel/Per1Jan/<year>/select/uq_pand_nr/pand_type[rlookup(MutTable/pand_bag_nr, ...)].

merge takes every year as an argument, so all year branches must be live simultaneously, and
each one pulls a full Per1Jan/<year>/select plus AfleidingPandtype (a neighbour-geometry
analysis over ~10M panden). That the config asks for this much is arguably the user's problem.
That GeoDMS cannot say so is not.

Requested behaviour

Fail with an actionable, named error — "out of memory while calculating
/Analyse/Redev_obv_hele_bag/PrepBAG/PandTypering_Mutaties/pand_type" — instead of being
terminated by the OS. Re-enabling WaitForAvailableMemory's guard (with a hard refusal once the
commit limit is approached, rather than only waiting) would give a place to raise it.

Secondary: an interrupted write leaves an mmd that looks complete but is not

The dictionary was written at 00:28:43, 37 minutes before the process died. It declares
attributes whose data never arrived:

attribute</classifications/BAG/WP4> pand_type(.);      <- no pand_type data file on disk

unit<UInt32> ZonderCorrecties                          <- no Range, no data directory
{
	attribute<UInt64> vbo_bag_nr(.);
}

The result is an mmd that reads as valid until you touch pand_type or ZonderCorrecties, then
fails with "no primary data found in storage". It also silently replaced a good, complete mmd
written earlier the same day. Same family as #1130 / #1155 (dictionary emitted before the data is
certain), but the trigger here is process death mid-write rather than the range chicken-and-egg.

Found alongside #1156 and #1157, from the same user report.

Metadata

Metadata

Labels

crashTermination of programdiagnosticsinhoud eventlog: warnings, progress report, errors, Error messages

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions