-
Notifications
You must be signed in to change notification settings - Fork 74
refactor: apply complexipy refactor suggestions (evaluation demo, do not merge) #1115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -64,6 +64,47 @@ | |
| ) | ||
|
|
||
|
|
||
| def _build_sample_table( | ||
| dataset: InMemoryDataset, | ||
| *, | ||
| internal_cols: list[str], | ||
| col_limit: int, | ||
| ) -> Table: | ||
| """Build a rich table for one stream's sample records.""" | ||
| table = Table( | ||
| show_header=True, | ||
| show_lines=True, | ||
| ) | ||
| if len(dataset.column_names) > col_limit: | ||
| # We'll pivot the columns so each column is its own row | ||
| table.add_column("Column Name") | ||
| for _ in range(len(dataset)): | ||
| table.add_column(overflow="fold") | ||
| for col in dataset.column_names: | ||
| table.add_row( | ||
| Markdown(f"**`{col}`**"), | ||
| *[escape(str(record[col])) for record in dataset], | ||
| ) | ||
| else: | ||
| for col in dataset.column_names: | ||
| table.add_column( | ||
| Markdown(f"**`{col}`**"), | ||
| overflow="fold", | ||
| ) | ||
|
|
||
| for record in dataset: | ||
| table.add_row( | ||
| *[ | ||
| escape(str(val)) | ||
| for key, val in record.items() | ||
| # Exclude internal Airbyte columns. | ||
| if key not in internal_cols | ||
| ] | ||
| ) | ||
|
Comment on lines
+79
to
+103
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🙋 Human Input Needed: correct on both counts, but both predate this PR — the helper moved this code verbatim, so I'm deliberately not fixing them behind merge-blocking Confirmed against Your suggested shape is the right fix — derive |
||
|
|
||
| return table | ||
|
|
||
|
|
||
| class Source(ConnectorBase): # noqa: PLR0904 | ||
| """A class representing a source that can be called.""" | ||
|
|
||
|
|
@@ -661,6 +702,11 @@ def get_samples( | |
|
|
||
| return results | ||
|
|
||
| # TK: C003/extract_helper; complexity 20 -> 4; measured=False. | ||
| # TK: Suggested replacement: Extract lines 692-738 into a named helper function. | ||
| # TK: Applied a module-level table builder; print_samples still prints the same Table. | ||
| # TK: Reviewer: confirm rendered output remains byte-identical and note | ||
| # TK: the module total is unchanged. | ||
| def print_samples( | ||
| self, | ||
| streams: list[str] | Literal["*"] | None = None, | ||
|
|
@@ -698,43 +744,17 @@ def print_samples( | |
| ) | ||
| dataset = samples[stream] | ||
|
|
||
| table = Table( | ||
| show_header=True, | ||
| show_lines=True, | ||
| ) | ||
| if dataset is None: | ||
| console.print( | ||
| Markdown("**⚠️ `Error fetching sample records.` ⚠️**"), | ||
| ) | ||
| continue | ||
|
|
||
| if len(dataset.column_names) > col_limit: | ||
| # We'll pivot the columns so each column is its own row | ||
| table.add_column("Column Name") | ||
| for _ in range(len(dataset)): | ||
| table.add_column(overflow="fold") | ||
| for col in dataset.column_names: | ||
| table.add_row( | ||
| Markdown(f"**`{col}`**"), | ||
| *[escape(str(record[col])) for record in dataset], | ||
| ) | ||
| else: | ||
| for col in dataset.column_names: | ||
| table.add_column( | ||
| Markdown(f"**`{col}`**"), | ||
| overflow="fold", | ||
| ) | ||
|
|
||
| for record in dataset: | ||
| table.add_row( | ||
| *[ | ||
| escape(str(val)) | ||
| for key, val in record.items() | ||
| # Exclude internal Airbyte columns. | ||
| if key not in internal_cols | ||
| ] | ||
| ) | ||
|
|
||
| table = _build_sample_table( | ||
| dataset, | ||
| internal_cols=internal_cols, | ||
| col_limit=col_limit, | ||
| ) | ||
| console.print(table) | ||
|
|
||
| console.print(Markdown("--------------")) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Could you keep the internal-column filter consistent for headers and rows?
When
dataset.column_namescontains an entry frominternal_cols, lines 89-93 add a header for it, but lines 95-102 omit that value from each row. The rows and headers then describe different column sets. Could you derivevisible_columnsonce and use it for both loops, and add a regression case with an internal column? wdyt?Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🙋 Human Input Needed: the mismatch is real but preexisting on
main— I'd rather not fix behavior in this PR.Verified: the header loop iterates
dataset.column_nameswhile the row loop filtersinternal_cols, and that asymmetry is byte-for-byte whatprint_samplesalready does onmain— this PR only moved the block into_build_sample_table. Sinceget_recordsresults carry_airbyte_extracted_at/_airbyte_meta/_airbyte_raw_id, the non-pivoted branch does render shifted values today.Two reasons to leave it here: this PR is a deliberately unmergeable evaluation artifact for
complexipysuggestions, and fixing display behavior would put a real bug fix behindTKmarkers that block merge. AJ — want me to open a separate small PR againstmainfor thevisible_columnsfix (plus a regression test with an internal column)?Devin session