You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Development (#10)
* Test for crash on exit
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
* Test for crash on exit
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
---------
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
* Initial latency metrics
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
---------
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
* Development (#10)
* Test for crash on exit
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
* Test for crash on exit
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
---------
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
* Deploy cla action
* Deploy cla action
---------
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
Co-authored-by: rdkcmf <github@code.rdkcentral.com>
Co-authored-by: Alan Ryan <20208488+Alan-Ryan@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR introduces a new cross-scope “latency sequence” measurement capability (recording ordered locations and reporting aggregated timings), backed by shared memory, and adds perf-test / unit-test hooks to exercise it.
Changes:
Added latency tracking infrastructure: shared-memory block, sequence/location model, and a circular buffer for timestamps.
Added a new public C API for latency (RDKLatency*) and a perf test scenario that exercises nested calls + fork.
Updated a few existing components (msgqueue name buffer sizing, logging env var handling, timer shutdown behavior, minor test tweaks).
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 36 comments.
The reason will be displayed to describe this comment to others. Learn more.
SharedMemoryBlock is managed as a process-wide singleton (_shared_memory_block), but the destructor does not reset _shared_memory_block to nullptr. If any code deletes the singleton (e.g., via a owning class destructor) and later calls get_instance(), it will return a dangling pointer. Consider either (a) making get_instance() return a non-owning pointer that is never deleted, or (b) adding explicit lifetime management + nulling the static on destruction.
The reason will be displayed to describe this comment to others. Learn more.
CircBufferObject::records is declared as records[sizeof(CircBufferRecord) * MAX_TIME_STAMPS], which makes the array length a byte-count multiple (e.g., 400 elements) instead of MAX_TIME_STAMPS elements. This inflates CircBufferObject (and therefore Sequence/Location) unexpectedly and breaks assumptions in code that treats maxRecords as the array capacity. It should be sized as records[MAX_TIME_STAMPS].
The reason will be displayed to describe this comment to others. Learn more.
CircBufferObject includes char* name, and Location/Sequence structs (and their circular buffers) are stored in shared memory. Persisting a process-local pointer in shared memory is unsafe across processes and will become invalid when another process maps the segment at a different address. Consider removing the pointer from the shared-memory layout (or storing a fixed-size name or an offset within the SHM region).
The reason will be displayed to describe this comment to others. Learn more.
Location stores Sequence* sequence, but Location/Sequence are placed in shared memory. A raw pointer to a SHM-mapped object is not stable across processes (different mapping base addresses), so sequence will be invalid when read from another process. Consider storing a sequence index/offset instead of a pointer, and resolve it relative to the mapped base in each process.
The reason will be displayed to describe this comment to others. Learn more.
CircBufferObject::records is declared as CircBufferRecord records[sizeof(CircBufferRecord) * MAX_TIME_STAMPS], which allocates (sizeof(record)*MAX) records rather than MAX records. This mismatches how the buffer is indexed (0..maxRecords-1) and inflates the shared-memory layout. It should likely be records[MAX_TIME_STAMPS] (or change the type to a byte array if you intended raw storage).
The reason will be displayed to describe this comment to others. Learn more.
When creating a new location, the circular buffer backing storage (timeStampCirBuffer) isn’t reset, and the CircularBuffer constructor currently does not clear head/tail/currentSize/records. This can leave new locations with garbage circular-buffer state. Ensure new buffers are zeroed and initialized (use CircularBuffer::initialize() for new allocations).
The reason will be displayed to describe this comment to others. Learn more.
GetLocationsDepth() does not guard against _current_sequence->location_offset == INVALID_OFFSET, so it can compute _locations + (-1) and dereference it. Return 0 early when location_offset is invalid.
The reason will be displayed to describe this comment to others. Learn more.
GetDataRecord() constructs PerfLocation from _current_sequence->location_offset without checking INVALID_OFFSET, which can dereference out-of-bounds memory for a sequence with no locations. Guard and return nullptr (or an empty record) when location_offset is invalid.
The reason will be displayed to describe this comment to others. Learn more.
RDKLatencyReport() prints a heap-allocated DataRecord chain returned by GetDataRecord() but never frees it (and advances pData so the original pointer is lost). This leaks on every report. Keep the root pointer and delete/free the chain after printing (or return an owning smart pointer/value type).
The reason will be displayed to describe this comment to others. Learn more.
Several log statements use [%s] with circBuffer->name (e.g., in the FULL-buffer log). If name hasn’t been set (or is an invalid SHM pointer), passing it to %s is undefined and can crash. Guard against null/invalid names (or avoid storing pointers in SHM).
The reason will be displayed to describe this comment to others. Learn more.
rootDiff is a uint64_t, but it’s logged with %lf (double) in LOG(eTrace, "Root timestamp ... root diff %lf"), which is undefined behavior. Use the correct integer format (or cast to double if that’s the intent).
The reason will be displayed to describe this comment to others. Learn more.
RemoveTimeStamp(count) calls _timeStamps.pop(...), but pop() removes the oldest (tail) entry and does not remove a specific key. This breaks the logic in AddTimeStamp() that tries to remove the matching parent timestamp for the same count. Add an erase-by-key API (or change storage) so the timestamp for the requested count is removed deterministically.
The reason will be displayed to describe this comment to others. Learn more.
These functions are declared static with constructor/destructor attributes, but the definitions are non-static. That’s a linkage mismatch and will typically fail to compile (“static declaration follows non-static” / multiple linkage). Make the definitions static to match the forward declarations (or drop static from both).
The reason will be displayed to describe this comment to others. Learn more.
The attached_intances decrement and == 0 decision are protected by the semaphore, but the actual cleanup (sem_destroy() / shm_unlink()) happens after unlocking. Another process can attach/detach in between, leading to premature destruction/unlink. Keep the lock held across decrement+check+final cleanup (or use a more robust refcounting strategy).
Suggested change
// Decrement the number of attached instances
lock();
_shared_block->attached_intances--;
if(_shared_block->attached_intances == 0) {
bDelete = true;
}
unlock();
if(bDelete) {
// Destroy the semaphore
sem_destroy(&_shared_block->semaphore);
// Last process in the shared memory segment
// Unlink the shared memory segment
shm_unlink(SHARED_MEMORY_NAME);
_shared_memory_block = nullptr;
// Decrement the number of attached instances and, if this is the
// last attached process, perform final cleanup while still holding
// the semaphore so no other process can attach in between.
lock();
_shared_block->attached_intances--;
if(_shared_block->attached_intances == 0) {
bDelete = true;
// Destroy the semaphore before unmapping the shared memory.
// Do not unlock after this, because the semaphore is no longer valid.
The reason will be displayed to describe this comment to others. Learn more.
The wait loop uses static int nWaitCount = 0; inside SharedMemoryBlock::initialize(). Because it’s static, a previous initialization attempt can leave nWaitCount high and cause subsequent attempts to fail immediately. Make this a normal local counter (and consider logging less aggressively inside the spin loop to avoid flooding logs).
The reason will be displayed to describe this comment to others. Learn more.
In the eThreshold and eExit cases, the code writes the name into msg.msg_data.entry.szName, but the active union members are threshold and exit. This will populate the wrong struct and can corrupt the message contents. Write into msg.msg_data.threshold.szName / msg.msg_data.exit.szName respectively.
The reason will be displayed to describe this comment to others. Learn more.
LogModuleInit() unconditionally prints [DEBUG] ... messages to stdout during library initialization. This is noisy for consumers and bypasses the project’s logging controls. Prefer using the existing LOG(...) macro (respecting verbosity) or guard these prints behind the env var / a compile-time debug flag.
The reason will be displayed to describe this comment to others. Learn more.
SharedMemoryBlock::get_instance() always returns the singleton even if SharedMemoryBlock::initialize() failed (constructor sets _initialized=false but callers aren’t prevented from using _shared_block==nullptr). This can lead to null dereferences in lock()/get_data() downstream. Consider making get_instance() return nullptr on initialization failure (and/or delete the partially-constructed singleton).
The reason will be displayed to describe this comment to others. Learn more.
In the eExit case the name is copied into msg.msg_data.entry.szName rather than msg.msg_data.exit.szName. That leaves exit.szName empty and overwrites unrelated union storage. Copy into the exit member for exit messages.
The reason will be displayed to describe this comment to others. Learn more.
GetChildName() treats child_offset == 0 as “no child”, but offsets use INVALID_OFFSET (-1) as the sentinel elsewhere. A valid first child at index 0 would be incorrectly hidden. Check for INVALID_OFFSET instead of 0 (and consider returning an empty string rather than a newline).
The reason will be displayed to describe this comment to others. Learn more.
GetChildName() computes the child pointer using byte arithmetic ((uint8_t*)_locations + child_offset), but child_offset is used as an index everywhere else (e.g., it’s assigned i in PerfSequence::AddLocation). This will point at the wrong address. Use _locations + child_offset (or multiply by sizeof(Location) if the offset is intended to be bytes).
The reason will be displayed to describe this comment to others. Learn more.
GetDataRecord() divides by _location->count when computing average and average_elapsed. When count == 0 this is a divide-by-zero (NaN/inf) and can propagate into reporting. Guard the division (e.g., return 0 averages when count is 0).
The reason will be displayed to describe this comment to others. Learn more.
PerfSequence::~PerfSequence() does not delete _current_location, which is allocated with new in SetCurrentLocation(...). This leaks per-instance memory (and may leak repeatedly if sequences are recreated). Ensure _current_location is deleted (or convert it to a value type / smart pointer).
Suggested change
// Clean up the shared memory block
if(_shared_memory_block != nullptr) {
// `_shared_memory_block` is obtained via SharedMemoryBlock::get_instance(),
// so PerfSequence does not own it and must not delete it here.
// Treat the shared memory block as process-lifetime from this file.
_shared_memory_block = nullptr;
if(_current_location != nullptr) {
delete _current_location;
_current_location = nullptr;
}
// Clean up the shared memory block
if(_shared_memory_block != nullptr) {
// `_shared_memory_block` is obtained via SharedMemoryBlock::get_instance(),
// so PerfSequence does not own it and must not delete it here.
// Treat the shared memory block as process-lifetime from this file.
The reason will be displayed to describe this comment to others. Learn more.
RDKLatencyReport() walks a DataRecord chain returned by PerfSequence::GetDataRecord(), but that chain is allocated with new (recursively) and is never freed. Repeated reporting will leak memory. Add an explicit cleanup/free routine for the DataRecord tree (or return an owning smart pointer / value type) after printing.
The reason will be displayed to describe this comment to others. Learn more.
The latency test loop runs 499 iterations and each iteration can fork() (via test_latency_2). This can make the test binary very slow and resource-intensive by default. Consider lowering the default iterations / removing the fork from the hot loop, and making the iteration count configurable via argv/env.
The reason will be displayed to describe this comment to others. Learn more.
CircularBuffer::push/pop/peek/find emit eWarning logs for normal control-flow states like full/empty/not-found (e.g., buffer full in push() will happen routinely once more than MAX_TIME_STAMPS are recorded, especially for root timestamps that are never popped). Since warnings are not gated by s_VerboseLog, this can flood logs and significantly slow down recording. Consider downgrading these to eTrace or gating them behind the extended-logging flag.
The reason will be displayed to describe this comment to others. Learn more.
PerfSequence logs the shared-memory addresses using LOG(eError, ...) during normal construction. eError messages are always emitted and may alarm users / spam logs even in healthy runs. Consider lowering these to eTrace/eWarning (or removing) unless this indicates an actual error condition.
Suggested change
LOG(eError, "Sequence array at %p\n", _sequences);
The reason will be displayed to describe this comment to others. Learn more.
When shm_open(..., O_CREAT | O_EXCL, ...) fails, the code logs ERR_LOG("shm already created") at error level before checking errno. For the expected EEXIST case this will produce an error log on every non-first attach. Consider only logging at eTrace/eWarning for EEXIST, and reserve eError for unexpected errno values.
Suggested change
// Create the shared memory segment exclusively. If it already exists, return an error
Reason for change: Fix race condition where process terminate happens
before the timer thread is started.
Test Procedure: Test using gst-inspect
Risks: None
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
Reason for change: Simplifed the timer loop and
added some additional tests.
Test Procedure: Test using make quick-exit-test
Risks: None
Signed-off-by: Douglas Adler <douglas.adler@yahoo.com>
This public header uses uint32_t in its declarations but does not include the header that defines it. Any consumer that includes rdk_mem_tracker.h without first including <stdint.h> will fail to compile; add the fixed-width integer include to this header itself.
Singleton teardown skips shared memory cleanup
src/rdk_perf_sequence.cpp:97
Dropping this pointer without destroying the singleton means SharedMemoryBlock::~SharedMemoryBlock() never runs. Consequently attached_intances is never decremented and the shared-memory object is never cleaned up by normal library teardown; later runs reuse stale shared state.
Public header uses uint32_t without including its definition
rdkperf/rdk_mem_tracker.h:21
This public header uses uint32_t in its declarations but does not include the header that defines it. Consumers that include rdk_mem_tracker.h directly can fail to compile depending on include order; make the header self-contained.
This public header uses uint32_t in both its C and C++ declarations but never includes <stdint.h>. Any consumer that includes this header without first including another header defining uint32_t will fail to compile. Include the type header directly here.
Shared memory block is never released
src/rdk_perf_sequence.cpp:98
This destructor only drops the local pointer; it never destroys/releases the SharedMemoryBlock. Consequently attached_intances is never decremented and the POSIX shared-memory object is never unlinked, so every process that loads the library leaves persistent shared state and the documented last-process cleanup cannot occur.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.