Describe the bug
#1269 stopped the Query Planner from logging learner PII at INFO, and its fix (#1270) changed only Query Planner files. The same learner data is still logged at INFO by the services downstream and alongside it: GraphQL, the Query Cache, the Advisor, and semantic search. LOG_LEVEL defaults to INFO (components/lif/logging/core.py), so these lines run on the normal request path wherever the services are deployed.
In the case that matters most, #1269 removed the log line that recorded matched field values in the Query Planner, but GraphQL logs the Query Planner's entire response one hop later, on every query.
Where (verified on main at d8bc7cc)
| File:line |
Logged at INFO |
Learner data |
components/lif/openapi_to_graphql/type_factory.py:849 |
Response: {response_json}: the full Query Planner response, on every successful query |
field values |
components/lif/openapi_to_graphql/type_factory.py:842 |
Query: {query}: the query filter |
person identifier |
components/lif/openapi_to_graphql/type_factory.py:959 |
Update mutation payload: {payload}: filter plus the input being written |
identifier and field values |
components/lif/query_cache_service/core.py:278 |
===> CALL MADE TO ADD: {lif_record}: the whole LIFRecord, on every cache add |
identifiers and field values |
components/lif/langchain_agent/core.py:213 |
Response: {response}: the Advisor's final answer text, every turn |
field values (answers describe the learner's record) |
bases/lif/advisor_restapi/core.py:310 |
Summarization response for {username}: {response}: the conversation summary |
learner content, with the username |
components/lif/semantic_search_service/core.py:424 |
Generated GraphQL for query [...]: includes the filter literal |
person identifier |
semantic_search_service/core.py:399 also logs the user's natural-language question, which can contain a name. It's lower risk, but worth deciding alongside the rest.
How this was scoped, and its limits
A search of every logger.info/logger.warning call in bases/ and components/ that interpolates a body-shaped variable (response*, result*, record*, payload, query, data, body, fragments, person, learner, lif_record, input_dict, filter_dict): 15 hits, 7 carry learner data (above). The other 8 were checked and excluded: a query status response (query_planner_restapi/core.py:250), six MDR data-model metadata lines (ParentEntityId, transformation counts), and a single boolean (type_factory.py:608).
This is a pattern search, so it can miss a log line that uses a variable name outside that list. Treat the table as the known minimum, not a guarantee. The Identity Mapper's identifier logging is tracked separately in #1318.
Why it matters
#1269 treated identifiers and field values at INFO as urgent, and #341 records that a person identifier "must never be logged or persisted". Fixing the Query Planner while GraphQL logs the same payload one step later leaves the exposure in place, and it makes the #1269 fix look more complete than it is.
Acceptance criteria
- None of the seven lines above logs learner identifiers or field values at
INFO or WARNING. Keep the operational signal: counts, sizes, status codes, timings, route templates.
- The Advisor lines log metadata about the turn or summary (length, tokens, cost), not its text.
- A test per service that captures log output for a representative request and asserts a known identifier and field value do not appear. It must fail on current
main, so it cannot pass by never reaching the logging path.
- Before closing, a repo-wide sweep with a wider pattern than the one above, with its denominator stated in the PR (how many log calls examined, how many changed, why the rest are safe), so the fix isn't narrower than the problem again.
Related
Describe the bug
#1269 stopped the Query Planner from logging learner PII at
INFO, and its fix (#1270) changed only Query Planner files. The same learner data is still logged atINFOby the services downstream and alongside it: GraphQL, the Query Cache, the Advisor, and semantic search.LOG_LEVELdefaults toINFO(components/lif/logging/core.py), so these lines run on the normal request path wherever the services are deployed.In the case that matters most, #1269 removed the log line that recorded matched field values in the Query Planner, but GraphQL logs the Query Planner's entire response one hop later, on every query.
Where (verified on
mainatd8bc7cc)INFOcomponents/lif/openapi_to_graphql/type_factory.py:849Response: {response_json}: the full Query Planner response, on every successful querycomponents/lif/openapi_to_graphql/type_factory.py:842Query: {query}: the query filtercomponents/lif/openapi_to_graphql/type_factory.py:959Update mutation payload: {payload}: filter plus the input being writtencomponents/lif/query_cache_service/core.py:278===> CALL MADE TO ADD: {lif_record}: the wholeLIFRecord, on every cache addcomponents/lif/langchain_agent/core.py:213Response: {response}: the Advisor's final answer text, every turnbases/lif/advisor_restapi/core.py:310Summarization response for {username}: {response}: the conversation summarycomponents/lif/semantic_search_service/core.py:424Generated GraphQL for query [...]: includes the filter literalsemantic_search_service/core.py:399also logs the user's natural-language question, which can contain a name. It's lower risk, but worth deciding alongside the rest.How this was scoped, and its limits
A search of every
logger.info/logger.warningcall inbases/andcomponents/that interpolates a body-shaped variable (response*,result*,record*,payload,query,data,body,fragments,person,learner,lif_record,input_dict,filter_dict): 15 hits, 7 carry learner data (above). The other 8 were checked and excluded: a query status response (query_planner_restapi/core.py:250), six MDR data-model metadata lines (ParentEntityId, transformation counts), and a single boolean (type_factory.py:608).This is a pattern search, so it can miss a log line that uses a variable name outside that list. Treat the table as the known minimum, not a guarantee. The Identity Mapper's identifier logging is tracked separately in #1318.
Why it matters
#1269 treated identifiers and field values at
INFOas urgent, and #341 records that a person identifier "must never be logged or persisted". Fixing the Query Planner while GraphQL logs the same payload one step later leaves the exposure in place, and it makes the #1269 fix look more complete than it is.Acceptance criteria
INFOorWARNING. Keep the operational signal: counts, sizes, status codes, timings, route templates.main, so it cannot pass by never reaching the logging path.Related
type_factory.pynear line 959, but doesn't change its logging