Release 1.10.0 - #412
Merged
Merged
Release 1.10.0#412
Conversation
#161) Description: Introduced marker type DataModelTable for dynamic tables in telemetry. Enhanced T2 agent to resolve and collect data from all instances of multi-instance object tables when dynamic markers are configured. Added logic to parse DataModelTable, filter markers based on telemetry profile configuration, and encode them in telemetry reports after collection, ensuring telemetry reports include per-instance data for configured dynamic markers. Added support for an "index" parameter in configuration to restrict data collection to specific indexes if required. Reason for change: Enable telemetry to capture dynamic, per-instance data from tables (e.g., Host table, Wi-Fi associated devices) where indexes are not fixed, ensuring richer and more flexible telemetry collection. Signed-off-by: onkar.panchare1 <onkar.panchare@telekom-digital.com> Co-authored-by: Shibu Kakkoth Vayalambron <shibu.kakkoth@gmail.com>
* Update build_inside_container.sh * Update t2common.c
* Fix memory safety issues blocking PR-161 L1 tests Fix double-free crash, NULL dereference, memory leaks, and buffer overflow risks identified in telemetry PR-161 code review. Key fixes: - Expose freeProfile() and call on error in processConfiguration() - Add NULL checks before Vector_Create() and Vector_Size() - Free allocated strings before error returns in reportgen.c - Add bounds checking for strcat/strcpy operations - Move debug logging before free() calls - Update test mocks for new function signature Fixes L1 test crash (exit code 134) and resolves all critical memory safety issues in PR-161. * Correct merge conflict errors * Fix astyle formatting compliance errors * Potential fix for pull request finding 'CodeQL / Potentially unsafe use of strcat' --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…ork connectivity (#367) * Xconf thread shpuld wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * Update telemetry2_0.service to start after systimemgr.service * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> * Disable NTP sync indicator compile flag in L2 container build path (#368) * Initial plan * Disable NTP sync indication flag in container build path Agent-Logs-Url: https://github.com/rdkcentral/telemetry/sessions/fe1b0563-e051-4df8-a022-0ae17a31725d Co-authored-by: shibu-kv <89052442+shibu-kv@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shibu-kv <89052442+shibu-kv@users.noreply.github.com> * RDKEMW-18410: Xconf thread should wait for the NTP indicator for network connectivity Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> --------- Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com> Co-authored-by: Yogeswaran K <yogeswaran_k@comcast.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: shibu-kv <89052442+shibu-kv@users.noreply.github.com>
…ator (#372) * RDKEMW-18410: Xconf thread should wait Indefinitely for the NTP indicator --------- Signed-off-by: Yogeswaran K <yogeswaran_k@comcast.com>
* Update L1-tests.yml * Update L2-tests.yml * Update datamodelTest.cpp * Update datamodelTest.cpp
Co-authored-by: Shibu Kakkoth Vayalambron <shibu.kakkoth@gmail.com>
…allback path * Fix zone NULL/uninitialized issues in timezone fallback reader - Validate ftell return: reject -1 (error) and 0 (empty file) - Add malloc NULL check to prevent NULL pointer dereference - Initialize zone buffer to empty string before fscanf - Reset zone after each iteration to catch stale data - Check fscanf return == 1 instead of != EOF to catch format errors and whitespace-only files - Remove impossible 'zone != NULL' check (zone is stack-local pointer to heap memory, never NULL inside the loop) - Don't reset zoneValue to NULL on empty reads (preserve any previously read valid value)
…loss when new log file grows past seek value (#392) * RDKEMW-21242: Detect log rotation via inode tracking to prevent data loss when new log file grows past seek value Signed-off-by: yogeswaransky <yogeswaransky@github.com> * RDKEMW-21242: Detect log rotation via inode tracking to prevent data loss when new log file grows past seek value Signed-off-by: yogeswaransky <yogeswaransky@github.com> * RDKEMW-21242: Detect log rotation via inode tracking to prevent data loss when new log file grows past seek value Signed-off-by: yogeswaransky <yogeswaransky@github.com> --------- Signed-off-by: yogeswaransky <yogeswaransky@github.com> Co-authored-by: yogeswaransky <yogeswaransky@github.com>
…ot Reason and Log Upload (#385) * Update dcautil.c Co-authored-by: Abhinav P V <Abhinav_Valappil@comcast.com> Co-authored-by: Aryan Y <aryan_duttay@comcast.com>
* RDKEMW-21242: Detect log rotation via inode tracking to prevent data loss when new log file grows past seek value Signed-off-by: Yogeswaran K <yogeswaransky@gmail.com> * RDKEMW-21242: Detect log rotation via inode tracking to prevent data loss when new log file grows past seek value Signed-off-by: Yogeswaran K <yogeswaransky@gmail.com> --------- Signed-off-by: Yogeswaran K <yogeswaransky@gmail.com> Co-authored-by: Shibu Kakkoth Vayalambron <shibu.kakkoth@gmail.com>
…rting (#398) * RDKBDEV-3443 : Add typed fields support for telemetry reporting Reason for change: Telemetry currently serializes TR-181 parameter values as strings regardless of their native data type. Add compile-time support to preserve native TR-181 types when encoding telemetry JSON reports. Test Procedure: 1. Build telemetry with SUPPORT_TYPING_FIELDS enabled. 2. Configure a telemetry profile containing string, boolean, integer, unsigned integer, float and double TR-181 parameters. 3. Generate a telemetry report and verify that boolean and numeric values are encoded using their native JSON types. 4. Build telemetry with SUPPORT_TYPING_FIELDS disabled and verify that the existing string serialization behavior remains unchanged. 5. Run the telemetry unit tests and verify that all tests pass. Risks: Low. The new behavior is protected by the SUPPORT_TYPING_FIELDS compile-time flag. When the flag is disabled, the existing telemetry serialization behavior is preserved. Signed-off-by: Abdelkarim El Hosni abdelkarim.elhosni@sfr.com * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Signed-off-by: Abdelkarim El Hosni abdelkarim.elhosni@sfr.com Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* RDKB-65730: Integrate Dynamic Table Support - Build Time Disabled --------- Signed-off-by: Yogeswaran K <yogeswaransky@gmail.com>
…tests (#404) * RDKB-65730: This PR extends the Telemetry 2.0 profile parser to support dynamic table (“dataModelTable”) Configuration in the MsgPack profile flow, aligning MsgPack behavior more closely with the existing JSON dynamic-table parsing path. Changes: Added MsgPack dynamic-table parser helper (parseDataModelTableParamsMsgpack) under ENABLE_DYNAMIC_TABLE_SUPPORT. Implemented MsgPack handling for type == "dataModelTable" in addParameterMsgpack_marker_config, including optional index expansion and dynamic-table structure parsing. Introduced a msgpack_add_param label to allow the no-index dynamic-table case to fall through into the existing parameter-add path. --------- Signed-off-by: Yogeswaran K <yogeswaransky@gmail.com>
* Fix the gaps in workflows to upload L1 test results to portal
* RDKEMW-23103: Fix the erros with deriving PR details
The upload scripts were not running within the source checkout directory
* RDKEMW-23103: Remove duplicate L1 test execution in every develop PR
Include posting coverage summary to every PR in develop
* Fix truncated coverage report summary to the PR
Contributor
There was a problem hiding this comment.
Pull request overview
Release 1.10.0 bundles multiple feature additions and infrastructure updates across telemetry runtime, schema, and CI/testing, notably adding dynamic TR-181 table traversal (dataModelTable), optional typed field reporting, and improvements around time/NTP gating and log handling, along with expanded L1/L2 test coverage and workflow upload behavior.
Changes:
- Add
dataModelTableparsing/encoding support (including MsgPack path) and corresponding L1/L2 tests and schemas. - Introduce optional typed-value emission (
SUPPORT_TYPING_FIELDS) and wire it through CCSP/RBUS → reportgen. - Update unit/integration test runners and CI workflows to generate/aggregate JSON reports and upload artifacts/results.
Reviewed changes
Copilot reviewed 61 out of 63 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| test/run_ut.sh | Runs expanded unit-test set, creates GTest report dir, filters noisy logs, aggregates JSON outputs. |
| test/run_l2.sh | Builds/starts mock RBUS provider and adds L2 test for dataModelTable. |
| test/README.md | Documents how to run L1/L2 tests via containers. |
| test/functional-tests/tests/test_datamodeltable.py | Adds L2 integration scenarios for dataModelTable. |
| test/functional-tests/tests/report_profiles.py | Adds report-profile blobs for dataModelTable L2 tests. |
| test/functional-tests/tests/mock_table_provider.c | Adds mock RBUS table provider for L2 dynamic-table testing. |
| telemetry2_0.service | Adds systimemgr ordering/dependency for time sync coordination. |
| source/xconf-client/xconfclient.c | Adds NTP-indicator gating and hardens timezone fallback file parsing. |
| source/utils/t2common.h | Adds dynamic-table structs and cleanup/matching helpers behind feature flag. |
| source/utils/t2common.c | Implements dynamic-table cleanup and wildcard matching helper. |
| source/test/xconf-client/xconfclientTest.cpp | Adds tests for getTimezone() timeZoneDST fallback behavior. |
| source/test/t2parser/Makefile.am | Adds t2parser_dynamictable_gtest.bin target and flags/sources. |
| source/test/t2parser/gtest_main.cpp | Makes GTest JSON output filename unique per binary. |
| source/test/scheduler/gtest_main.cpp | Ensures GTest output flag set before InitGoogleTest. |
| source/test/reportgen/reportgenTest.cpp | Updates tests for new encodeParamResultInJSON signature. |
| source/test/reportgen/Makefile.am | Adds reportgen_dynamictable_gtest.bin target and flags/sources. |
| source/test/reportgen/gtest_main.cpp | Makes GTest JSON output filename unique per binary. |
| source/test/mocks/SystemMock.h | Adds mocks for inotify/clock_gettime/select to support new gating logic. |
| source/test/mocks/SystemMock.cpp | Implements wrappers and selective interception for time/select APIs. |
| source/test/mocks/profileStub.c | Adds lightweight profile stubs for unit tests without full dependency graph. |
| source/test/mocks/FileioMock.h | Guards PATH_MAX definition for portability in test environment. |
| source/test/DYNAMICTABLE_TESTS_README.md | Documents dynamic-table and memory-safety test suites. |
| source/test/dcautils/Makefile.am | Adds sources/includes needed by updated tests (incl. profile stub). |
| source/test/dcautils/gtest_main.cpp | Ensures GTest output flag set before InitGoogleTest. |
| source/test/ccspinterface/gtest_main.cpp | Adds JSON output path configuration for ccspinterface gtests. |
| source/test/bulkdata/profilexconfTest.cpp | Updates mock expectations for encodeParamResultInJSON signature change. |
| source/test/bulkdata/profile_dynamictable_Test.cpp | Adds unit tests for profile NULL safety / dynamic-table cleanup paths. |
| source/test/bulkdata/Makefile.am | Adds profile dynamic-table gtest target and related flags/sources. |
| source/test/bulkdata/gtest_main.cpp | Makes GTest JSON output filename unique per binary. |
| source/test/bulkdata/datamodelTest.cpp | Updates test profile URL to mock endpoint. |
| source/t2parser/t2parser.h | Adds MAX_PATH_LENGTH constant for safer path building. |
| source/t2parser/t2parser.c | Adds dynamic-table parsing (JSON + MsgPack), memory-safety cleanups, and map/string comparisons hardening. |
| source/t2dm/cosa_apis.h | Formatting/bracing cleanup for typedef structs. |
| source/reportgen/reportgen.h | Extends encodeParamResultInJSON signature and adds isDataModelTable decl. |
| source/reportgen/reportgen.c | Adds dynamic-table JSON encoding, optional typed fields, and helpers for nested JSON building. |
| source/reportgen/Makefile.am | Wires SUPPORT_TYPING_FIELDS define via configure conditional. |
| source/protocol/http/multicurlinterface.c | Minor formatting/casting adjustments. |
| source/dcautil/legacyutils.h | Adds inode-aware seek tracking struct for rotation detection. |
| source/dcautil/dcautil.h | Adds backup/previous-log sentinel constants and wait helper declaration. |
| source/dcautil/dcautil.c | Adds inotify-based synchronization with backup_logs and writes completion sentinel. |
| source/dcautil/dca.c | Implements inode-aware rotation detection and suppresses inode checks for custom log paths. |
| source/ccspinterface/rbusInterface.c | Adds typed-field support and minor cast formatting. |
| source/ccspinterface/Makefile.am | Wires SUPPORT_TYPING_FIELDS define via configure conditional. |
| source/ccspinterface/ccspinterface.c | Adds typed-field support for CCSP values and minor cast formatting. |
| source/ccspinterface/busInterface.h | Extends TR-181 value struct with optional type enum. |
| source/bulkdata/t2eventreceiver.c | Minor cast formatting adjustment. |
| source/bulkdata/profilexconf.c | Updates encodeParamResultInJSON call signature. |
| source/bulkdata/profile.h | Exposes freeProfile and adds dynamic-table list to Profile (flagged). |
| source/bulkdata/profile.c | Exposes freeProfile, frees dynamic-table list, passes table list to reportgen, and adds async delete path. |
| schemas/t2_reportProfileSchema.schema.json | Adds dataModelTable and restricted dataModelSimple schema definitions. |
| README.md | Documents build-time dynamic-table feature flag and behavior. |
| docs/architecture/overview.md | Updates architecture overview to mention dynamic tables. |
| configure.ac | Adds configure flags/conditionals for typing fields and dynamic-table support. |
| CHANGELOG.md | Adds 1.10.0 release notes and links. |
| build_inside_container.sh | Enables dynamic-table support in container build script. |
| .github/workflows/L2-tests.yml | Makes result push URL configurable via secret and skips upload if absent. |
| .github/workflows/L1-tests.yml | Adds coverage summary comment, artifacts upload/download, and configurable result push URL. |
| .github/workflows/code-coverage.yml | Adjusts formatting and coverage summary capture behavior. |
| .github/skills/quality-checker/SKILL.md | Expands skill documentation to include L1/L2 test execution guidance. |
| .astylerc | Updates astyle option name. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+92
to
+100
| for report in `ls /tmp/Gtest_Report/*.json`; do | ||
| if [ -f "$report" ]; then | ||
| echo "Contents of $report:" | ||
| cat "$report" | ||
| echo "----------------------------------------" | ||
| else | ||
| echo "No JSON report files found in /tmp/Gtest_Report/" | ||
| fi | ||
| done |
Comment on lines
+160
to
+169
| AC_ARG_ENABLE([support-typing-fields], | ||
| AS_HELP_STRING([--enable-support-typing-fields],[enable SUPPORT_TYPING_FIELDS (default is no)]), | ||
| [ | ||
| case "${enableval}" in | ||
| yes) ENABLE_SUPPORT_TYPING_FIELDS=true;; | ||
| no) AC_MSG_ERROR([SUPPORT_TYPING_FIELDS is disabled]) ;; | ||
| *) AC_MSG_ERROR([bad value ${enableval} for --enable-support-typing-fields]) ;; | ||
| esac | ||
| ], | ||
| [echo "SUPPORT_TYPING_FIELDS is disabled"]) |
Comment on lines
+1096
to
+1108
| /* Parse inotify events for NTP sync indicator */ | ||
| ssize_t offset = 0; | ||
| while (offset < len) | ||
| { | ||
| struct inotify_event *event = (struct inotify_event *)(buf + offset); | ||
| if (event->len > 0 && strcmp(event->name, NTP_SYNC_FILENAME) == 0) | ||
| { | ||
| T2Info("NTP sync detected (%s created), proceeding with xconf fetch\n", NTP_SYNC_INDICATOR); | ||
| result = true; | ||
| break; | ||
| } | ||
| offset += sizeof(struct inotify_event) + event->len; | ||
| } |
Comment on lines
+591
to
+599
| char *parameterName = strdup(paramValues[valIndex]->parameterName); | ||
| char *parameterWild = strdup(paramValues[valIndex]->parameterName); | ||
| size_t basePathLength = strlen(basePattern); | ||
| cJSON *currentObject = arrayItem; | ||
| char concatenatedKey[256] = ""; | ||
| bool firstIndexHandled = false; | ||
| bool parameterConfigured = false; | ||
| char *token = strtok(parameterName + basePathLength, "."); | ||
|
|
Comment on lines
+643
to
+652
| if (parameterName) | ||
| { | ||
| free(parameterName); | ||
| } | ||
| if (parameterWild) | ||
| { | ||
| free(parameterWild); | ||
| } | ||
| cJSON_Delete(currentObject); | ||
| return T2ERROR_FAILURE; |
Code Coverage Summary |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.