Problem
Currently, each batch iteration in Read_step is recorded as a single row in the event model (and in the resulting Tiled DataFrame).
To normalize the number of suggestions per row, tiled_writer / event modeling currently relies on null/empty packing of the suggestion_ids array. This approach has proven fragile and makes downstream analysis more cumbersome than necessary.
Proposed Change
Split each of the trials in an iteration into independent rows, and subsequently into individual trigger and read calls. That removes the need for any padding.
The loss of explicit per-iteration granularity should be relatively low impact, since the iterations can still be reconstructed/grouped using uid when needed.
Problem
Currently, each batch iteration in Read_step is recorded as a single row in the event model (and in the resulting Tiled DataFrame).
To normalize the number of suggestions per row, tiled_writer / event modeling currently relies on null/empty packing of the suggestion_ids array. This approach has proven fragile and makes downstream analysis more cumbersome than necessary.
Proposed Change
Split each of the trials in an iteration into independent rows, and subsequently into individual trigger and read calls. That removes the need for any padding.
The loss of explicit per-iteration granularity should be relatively low impact, since the iterations can still be reconstructed/grouped using uid when needed.