Skip to content

[SPARK-58480][PYTHON][TEST] Declare DataFrame golden test cases as methods of a test class #57691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dtenedor wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[SPARK-58480][PYTHON][TEST] Declare DataFrame golden test cases as methods of a test class #57691

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions dev/.rat-excludes
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,9 +145,3 @@ testCommitLog
spark-logo.svg
spark-logo-rev.svg
.nojekyll
# PySpark DataFrame golden test fixture scripts
# (python/pyspark/sql/tests/df_golden/scripts/<group>/). These are executable
# test fixtures, not source, so they are excluded like the SQL golden inputs
# (see .*\.sql above). Add one directory name per golden group below. RAT matches
# path components, so each name must be unique across the whole repository.
group_by
2 changes: 1 addition & 1 deletion dev/sparktestsupport/modules.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -639,8 +639,8 @@ def __hash__(self):
"pyspark.sql.tests.coercion.test_python_udf_input_type",
"pyspark.sql.tests.coercion.test_pandas_udf_return_type",
"pyspark.sql.tests.coercion.test_python_udf_return_type",
"pyspark.sql.tests.df_golden.test_df_golden",
"pyspark.sql.tests.df_golden.test_df_golden_framework",
"pyspark.sql.tests.df_golden.test_group_by",
],
)

Expand Down
3 changes: 0 additions & 3 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,6 @@ ignore = [
"python/pyspark/errors/error_classes.py" = ["E501"]
# Examples contain some unused variables.
"examples/src/main/python/sql/datasource.py" = ["F841"]
# DataFrame golden test scripts are executed by the framework with ``spark``
# injected into their namespace, so it is undefined at lint time.
"python/pyspark/sql/tests/df_golden/scripts/**/*.py" = ["F821"]

[dependency-groups]
_py4j = ["py4j>=0.10.9.9"]
Expand Down
551 changes: 393 additions & 158 deletions python/pyspark/sql/tests/df_golden/df_golden.py
View file
Open in desktop

Large diffs are not rendered by default.

Loading