-
Notifications
You must be signed in to change notification settings - Fork 25
Wip tmp tracability improvements #572
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
a5019c3
score_metamodel: autodiscover requirement types for metrics
FScholPer 3ba6ee0
docs: clarify requirement type autodiscovery override
FScholPer 2c638a3
lint fix
FScholPer 4123cfc
Merge branch 'main' into tracability-improvements
FScholPer d892ca6
Refine requirement type autodiscovery
FScholPer f967a55
Merge branch 'main' into tracability-improvements
FScholPer 192204c
Merge branch 'main' into tracability-improvements
FScholPer ac72772
WIP: Re-doing everything
MaximilianSoerenPollak 0089e33
WIP: Saving before del process reqs
MaximilianSoerenPollak a4cc1c9
WIP: Enable saving of metrics to global var
MaximilianSoerenPollak e12df0f
WIP: REfactor
MaximilianSoerenPollak 8afc248
Feat: Add testing & futher improvements
MaximilianSoerenPollak 88cd5f1
Fix: Formatting & Linting
MaximilianSoerenPollak 53ed1ac
Fix: Fixing PR comments
MaximilianSoerenPollak 00edcb2
Chore: Formatting
MaximilianSoerenPollak 55970c5
Fix: Fixing label call
MaximilianSoerenPollak 607c01a
Fix: Fix imports & linking to tool_req
MaximilianSoerenPollak 9ac3b89
Fix: Formatting & Linting
MaximilianSoerenPollak File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -40,20 +40,35 @@ Typical Setup | |
|
|
||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Docs has been generated by Copilot. Has not yet been updated / looked through for the new changes made. |
||
| For details, see :ref:`setup`. | ||
|
|
||
| Minimal Configuration Example | ||
| ----------------------------- | ||
| Configuration | ||
| ------------- | ||
|
|
||
| Default Behavior (No Configuration Needed) | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| By default, ``score_metamodel`` autodiscovers requirement types from the | ||
| repository needs present in the current build. Requirement types are identified | ||
| from ``needs_types`` entries tagged with ``requirement`` or | ||
| ``requirement_excl_process``. | ||
|
|
||
| This is the recommended setup for most repositories. | ||
|
|
||
| In ``docs/conf.py``: | ||
| Optional Override for Requirement Types | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| If a repository needs to force a specific set of requirement types, set an | ||
| explicit override in ``docs/conf.py``: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| score_metamodel_requirement_types = "feat_req,comp_req,aou_req" | ||
| score_metamodel_include_external_needs = False | ||
|
|
||
| When this setting is provided, the explicit list is used instead of | ||
| autodiscovery. | ||
|
|
||
| Use ``score_metamodel_include_external_needs = True`` only in repositories that | ||
| intentionally aggregate requirements across module dependencies, such as | ||
| integration repositories. Use ``False`` for module repositories to gate only on | ||
| local traceability. | ||
| integration repositories. | ||
|
|
||
| Building the Dashboard | ||
| ---------------------- | ||
|
|
@@ -87,7 +102,7 @@ There are two common modes: | |
|
|
||
| **Module repository** | ||
|
|
||
| - Set ``score_metamodel_include_external_needs = False``. | ||
| - No setting needed. Local-only scope is the default. | ||
| - Gate only on the needs owned by the repository itself. | ||
| - Use this for per-module implementation progress and traceability. | ||
|
|
||
|
|
@@ -136,6 +151,296 @@ For a new consumer repository: | |
| 4. Introduce modest thresholds in CI. | ||
| 5. Raise thresholds over time as the repository matures. | ||
|
|
||
| .. | ||
| ╓ ╖ | ||
| ║ .. The Following part has been generated by Copilot ║ | ||
| ╙ ╜ | ||
|
|
||
| Metrics Needpie functions | ||
| ========================= | ||
|
|
||
| Overview | ||
| -------- | ||
|
|
||
| These helpers read values from a nested dictionary named ``CALCULATED_METRICS``. | ||
| A metric is selected by a **colon-separated path**, for example: | ||
|
|
||
| - ``overall_metrics:total`` | ||
| - ``overall_metrics:with_test_link`` | ||
| - ``types:bug:total`` | ||
|
|
||
| All resolved values are converted to ``int`` and appended to a mutable | ||
| ``results`` list passed by the caller. | ||
|
|
||
| The ``needs`` parameter is accepted for integration compatibility, but it is not | ||
| used by the current implementations. | ||
|
|
||
| Path format | ||
| ----------- | ||
|
|
||
| Paths are split using ``:`` and then resolved step by step. | ||
|
|
||
| Example: | ||
|
|
||
| .. code-block:: text | ||
|
|
||
| path = "overall_metrics:total" | ||
|
|
||
| is resolved like: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| current = CALCULATED_METRICS | ||
| current = current["overall_metrics"] | ||
| current = current["total"] | ||
|
|
||
| Function reference | ||
| ------------------ | ||
|
|
||
| _get_key_values(results, argument_paths) | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Purpose | ||
| ^^^^^^^ | ||
|
|
||
| Internal helper that appends metric values for a list of paths. | ||
|
|
||
| Behavior | ||
| ^^^^^^^^ | ||
|
|
||
| 1. Iterate over each item in ``argument_paths``. | ||
| 2. Strip whitespace. | ||
| 3. Skip empty paths. | ||
| 4. Resolve the path inside ``CALCULATED_METRICS``. | ||
| 5. Convert to integer and append to ``results``. | ||
|
|
||
| Notes | ||
| ^^^^^ | ||
|
|
||
| - Modifies ``results`` in place. | ||
| - Does not return a new list. | ||
| - Raises ``KeyError`` if a path key does not exist. | ||
| - Raises ``ValueError`` if a resolved value cannot be converted to ``int``. | ||
|
|
||
| get_metrics_with_overall_total_considered(...) | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Purpose | ||
| ^^^^^^^ | ||
|
|
||
| Compute a remainder against global overall total. | ||
|
|
||
| Behavior | ||
| ^^^^^^^^ | ||
|
|
||
| 1. Append ``CALCULATED_METRICS["overall_metrics"]["total"]`` as first value. | ||
| 2. Append each metric referenced by ``kwargs.values()``. | ||
| 3. Replace the first value with: | ||
|
|
||
| ``overall_total - sum(all other appended values)`` | ||
|
|
||
| Typical use case | ||
| ^^^^^^^^^^^^^^^^ | ||
|
|
||
| Use this when your pie/chart needs an "Other" bucket based on the global total. | ||
|
|
||
| Example | ||
| ^^^^^^^ | ||
|
|
||
| Assume: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| CALCULATED_METRICS = { | ||
| "overall_metrics": { | ||
| "total": 100, | ||
| "with_test_link": 30, | ||
| "with_review": 20, | ||
| } | ||
| } | ||
|
|
||
| Call: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| results = [] | ||
| get_metrics_with_overall_total_considered( | ||
| needs=[], | ||
| results=results, | ||
| a="overall_metrics:with_test_link", | ||
| b="overall_metrics:with_review", | ||
| ) | ||
|
|
||
| Result: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| # Step 1 append total: [100] | ||
| # Step 2 append selected: [100, 30, 20] | ||
| # Step 3 remainder: [50, 30, 20] | ||
| results == [50, 30, 20] | ||
|
|
||
| get_metrics_with_custom_type_total_considered(...) | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Purpose | ||
| ^^^^^^^ | ||
|
|
||
| Support custom total path if provided as the last kwarg value. | ||
|
|
||
| Behavior | ||
| ^^^^^^^^ | ||
|
|
||
| If the **last** provided path ends with ``:total``: | ||
|
|
||
| 1. Append that last path as baseline total. | ||
| 2. Append all preceding paths as components. | ||
| 3. Replace the first value with: | ||
|
|
||
| ``baseline_total - sum(components)`` | ||
|
|
||
| Otherwise: | ||
|
|
||
| - Append all provided paths directly (no subtraction). | ||
|
|
||
| Important ordering rule | ||
| ^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| The special total path must be the **last** kwarg value, for example: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| get_metrics_with_custom_type_total_considered( | ||
| needs=[], | ||
| results=results, | ||
| part_a="types:feature:done", | ||
| part_b="types:feature:in_progress", | ||
| total="types:feature:total", # must be last | ||
| ) | ||
|
|
||
| Example with custom total | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Assume: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| CALCULATED_METRICS = { | ||
| "types": { | ||
| "feature": { | ||
| "total": 40, | ||
| "done": 10, | ||
| "in_progress": 5, | ||
| } | ||
| } | ||
| } | ||
|
|
||
| Call: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| results = [] | ||
| get_metrics_with_custom_type_total_considered( | ||
| needs=[], | ||
| results=results, | ||
| done="types:feature:done", | ||
| in_progress="types:feature:in_progress", | ||
| total="types:feature:total", | ||
| ) | ||
|
|
||
| Result: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| # baseline total: 40 | ||
| # components: 10, 5 | ||
| # remainder: 40 - (10 + 5) = 25 | ||
| results == [25, 10, 5] | ||
|
|
||
| Example without custom total | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| Call: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| results = [] | ||
| get_metrics_with_custom_type_total_considered( | ||
| needs=[], | ||
| results=results, | ||
| done="types:feature:done", | ||
| in_progress="types:feature:in_progress", | ||
| ) | ||
|
|
||
| Result: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| # no subtraction mode | ||
| results == [10, 5] | ||
|
|
||
| get_just_metrics(...) | ||
| ~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Purpose | ||
| ^^^^^^^ | ||
|
|
||
| Append only the selected metric values. | ||
|
|
||
| Behavior | ||
| ^^^^^^^^ | ||
|
|
||
| - Interprets each kwarg value as a path. | ||
| - Resolves and appends each value. | ||
| - No total baseline and no remainder calculation. | ||
|
|
||
| Example | ||
| ^^^^^^^ | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| results = [] | ||
| get_just_metrics( | ||
| needs=[], | ||
| results=results, | ||
| a="overall_metrics:with_test_link", | ||
| b="overall_metrics:with_review", | ||
| ) | ||
| # results might become [30, 20] | ||
|
|
||
| How to use these functions correctly | ||
| ------------------------------------ | ||
|
|
||
| 1. Pass a mutable list in ``results`` (usually ``[]`` initially). | ||
| 2. Provide metric paths through keyword argument values. | ||
| 3. Use colon-separated keys (``a:b:c``), not dot notation. | ||
| 4. For custom-total mode, ensure the total path is the last kwarg value. | ||
| 5. Expect in-place mutation of ``results``. | ||
|
|
||
| Common pitfalls | ||
| --------------- | ||
|
|
||
| - Empty kwargs in custom-total function can lead to index errors | ||
| (because ``values[-1]`` is accessed). | ||
| - Invalid paths raise ``KeyError``. | ||
| - Non-integer values raise ``ValueError`` during ``int(...)`` conversion. | ||
| - ``print(results)`` currently causes side effects during execution; remove if | ||
| quiet behavior is preferred in production. | ||
|
|
||
| Testing recommendations | ||
| ----------------------- | ||
|
|
||
| Add unit tests for: | ||
|
|
||
| - Basic path resolution with one and multiple paths. | ||
| - Whitespace and empty-path handling. | ||
| - Overall total remainder logic. | ||
| - Custom total behavior when last path ends with ``:total``. | ||
| - Behavior when no custom total is provided. | ||
| - Error handling for missing keys and non-integer values. | ||
| - Empty ``kwargs`` behavior in custom-total function. | ||
|
|
||
|
|
||
| Related Guides | ||
| -------------- | ||
|
|
||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably should have this in the extension so it's set for all things that import docs-as-code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AlexanderLanin thoughts on this?