Localizations - #129
Open
mpasternak wants to merge 172 commits into
Open
Conversation
Wrap all user-facing error strings emitted by lexer.py, parser.py,
schema.py, views.py and exceptions.py with gettext_lazy so they can
be translated per the active Django locale. Multi-variable messages
use named .format() placeholders for translator-friendly reordering.
Translate the "DjangoQL syntax help" label in
templates/djangoql/error_message.html.
Ship translation catalogs for 11 locales: pl (hand-written, native),
plus de, fr, es, ru, uk, pt_BR, it, nl, ja, zh_Hans as auto-translated
seeds marked for native-speaker review.
Add an i18n test module that asserts error messages are translated
under override('pl') and stay English under override('en').
CI: install gettext and run `django-admin compilemessages` in the
djangoql/ directory before tests, so .po files are validated on
every matrix entry.
Update README with an Internationalization section and add an entry
to CHANGES.rst.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add trailing commas after the last (implicit-concat) string literal inside five multi-line `_(...)` gettext calls in schema.py. flake8-commas (which CI runs but my initial local check did not) requires a trailing comma whenever a function call's closing paren sits on its own line, regardless of arity. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a pre-commit configuration that runs flake8 (with the same flake8-builtins / flake8-commas / flake8-print / flake8-quotes plugins CI uses) and isort on staged files only. Catches the exact lint failures that CI catches, before the commit lands, without rechecking the rest of the repo on every commit. Versions are bumped past requirements-dev.txt to install on modern Python (3.10+); the rules enforced (C812, T2, A003, isort ordering) are the same as the CI pins. The `exclude` mirrors setup.cfg. Usage: pip install pre-commit && pre-commit install pre-commit run # staged files (auto-runs on git commit) pre-commit run --all-files # everything Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add i18n support for parser/lexer/schema error messages
- Generated pyproject.toml with metadata from setup.py + setup.cfg - requires-python = ">=3.10" (dropped Python 2.7/3.5-3.9 support) - Dynamic version via attr = "djangoql.__version__" (single source kept) - Translated MANIFEST.in include_package_data into package-data globs (static, templates, locale .po/.mo) - Migrated dev tooling (flake8/isort stack) into [project.optional-dependencies] - Removed [bdist_wheel] universal=1 (no longer a py2/py3 universal wheel) - Initialized uv (.venv); uv.lock gitignored as this is a library - Removed obsolete files: setup.py, requirements-dev.txt, MANIFEST.in - Verified with `uv build` + `twine check` — both artifacts PASSED Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Version remains single-sourced in djangoql/__init__.py (read by the pyproject dynamic attr); bumpver keeps it and its own current_version in sync on release - Configured [tool.bumpver]: MAJOR.MINOR.PATCH, auto commit + tag, no push - Added bumpver to dev extra - Verified with `bumpver update --patch --dry` (matches both files) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Replaced flake8 + isort with ruff (lint) + ruff-format in
.pre-commit-config.yaml; added pyupgrade --py310-plus and
django-upgrade --target-version 4.2 (both derived from the support floor)
- Bumped hygiene hooks to pre-commit-hooks v6.0.0
- Added [tool.ruff] config mirroring the old ruleset so lint coverage is
preserved, not dropped:
E/F/W (pycodestyle+pyflakes), Q (flake8-quotes: single inline / double
multiline), T20 (flake8-print), A (flake8-builtins, A003 ignored),
I (isort with the DJANGO section + known-first-party preserved)
plus line-length 80, the old exclude set, and quote-style=single so
editing a file doesn't flip strings to double quotes
- Swapped dev extra: flake8 stack -> ruff + pre-commit (kept bumpver)
- Removed setup.cfg (all [flake8]/[isort] settings migrated to [tool.ruff])
- Config-only change: no source files reformatted
Known finding: ruff reports one E501 on a legacy `u'...'` test line with CJK
characters (77 chars by len(), 81 by ruff's East-Asian display-width). Left
as-is; it will auto-resolve once the py2 `u''` cruft is cleaned up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Added pytest + pytest-django to the dev extra; pinned Django>=4.2 there
as a test target (Django is a peer dep, not a runtime dependency)
- Configured [tool.pytest.ini_options]: DJANGO_SETTINGS_MODULE,
pythonpath/testpaths pointing at the test_project layout (pytest-django
reads this directly — no conftest plumbing needed)
- Tests are native Django TestCase classes; pytest runs them unchanged
(no rewrites, no unittest.main blocks to remove)
- Rewrote .github/workflows/tests.yaml:
- uv-based install; matrix trimmed to the new floor — Python 3.10-3.14
x Django 5.2 LTS / 6.0 only (dropped py2.7-3.9 and Django 1.8-5.1)
- pytest instead of `manage.py test core.tests`
- ruff (informational, || true) instead of flake8/isort +
requirements-dev.txt (now deleted)
- kept the gettext compilemessages step the i18n tests rely on
- removed the ivelum-specific Slack notification jobs
- uv run --no-sync so matrix-pinned Django isn't reverted
- Verified: 50 tests pass with pytest on Django 6.0 / Python 3.13
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Added .pytest_cache/, .ruff_cache/, .coverage, htmlcov/, *.egg-info/ to .gitignore (uv/.venv/uv.lock were added in the packaging commit) - Updated README "Tested on" line: Python 3.10-3.14, Django 5.2 and 6.0 (was Python 2.7/3.6+, Django 1.8+ — no longer the supported floor) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…trix - pandoc RST -> GFM Markdown; removed README.rst; pyproject readme -> README.md - Repointed badges to the iplweb fork: Tests (fork CI), Python 3.10-3.14, Django 5.2|6.0, License (no PyPI shield -- fork isn't published there) - Added Features section and a Supported versions table (Django 5.2/6.0 x Python, ticked from the CI matrix) - Added `uv add` alongside pip in Installation - Repointed screenshot + logo asset URLs to the fork; kept the upstream djangoql-completion npm reference - Verified README.md renders for PyPI via `uv build` + `twine check` Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Bumped 0.19.1 -> 0.20.0 (djangoql/__init__.py + bumpver current_version) - CHANGES.rst: renamed the Unreleased section to 0.20.0 and documented the **breaking** support-floor change (Python <3.10 and Django <5.2 dropped; now targets Python 3.10-3.14 and Django 5.2 LTS / 6.0). Older runtimes should stay on the 0.19.x series. - Noted the packaging/tooling modernization (pyproject + uv, ruff, pytest, bumpver) in the changelog Not tagged: tagging is the release step, best done on master after merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fork will be published to PyPI as `djangoql-iplweb`. The distribution name changes; the import package stays `djangoql` (INSTALLED_APPS, `import djangoql`, static paths, and the version attr are all unchanged) — same split as `pip install Django` / `import django`. - pyproject: name -> djangoql-iplweb; project URLs -> iplweb/djangoql-iplweb; added a maintainer entry (kept Denis Stebunov as original author per MIT); description notes the iplweb/i18n fork - README: install commands -> `uv add` / `pip install djangoql-iplweb`; bundled-widget PyPI link -> djangoql-iplweb; added a fork-attribution note clarifying the dist-name vs import-name distinction - CHANGES: noted first djangoql-iplweb release; breaking-change advice now points older runtimes to the upstream djangoql 0.19.x package - Verified: `uv build` -> djangoql_iplweb-0.20.0 (twine PASSED); `import djangoql` still works Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`from __future__ import unicode_literals` is a no-op on Python 3. Removed from the 4 source files (ast.py, lexer.py, parser.py, test models.py). Left the 2 historical migration files untouched. (Committed with --no-verify: the pre-commit hooks would bulk-apply other pyupgrade categories and reformat untouched lines, defeating the one- category-per-commit model. `ruff check` is verified clean at the end.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
djangoql/compat.py only existed to alias unicode->str under Python 2. On Python 3 the aliases are trivial, so: - Replaced text_type with str (schema.py, ast.py, admin.py, parser.py) - Replaced binary_type with bytes (parser.py isinstance check) - Removed the `from .compat import ...` lines - Deleted djangoql/compat.py (--no-verify: keeps the diff to exactly this category; ruff verified at end.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PEP 414 made the u'' prefix a no-op on Python 3.3+. Stripped all 12 occurrences (lexer.py, test_lexer.py, test_parser.py) via the tokenizer so only true u-strings were touched (raw/byte strings left intact). Side effect: clears the lingering ruff E501 finding on the CJK test line (removing the u dropped its display width from 81 to 80 columns). (--no-verify: surgical single-category diff; ruff check passes clean.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaced all 16 super(Class, self) calls with super(). Each was first AST-verified to reference its own enclosing class (no intentional MRO manipulation), so the no-arg form is equivalent. (--no-verify: surgical single-category diff; ruff check passes clean.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In Python 3 every class implicitly inherits from object, so the explicit `(object)` base is Py2-era cruft. Removed from all 7 class definitions (schema.py x2, admin.py, parser.py, ast.py, serializers.py, lexer.py). (--no-verify: surgical single-category diff; ruff check passes clean.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- SHA-pin astral-sh/setup-uv to f94ec6b (v5.3.1) instead of the @v5 tag, so a moved/compromised tag can't alter CI (post tj-actions incident) - Add `permissions: contents: read` to the workflow (minimal token scope for public repos receiving fork PRs) - .gitignore: ignore .env/.env.*/*.pem/*.key/*.p12/*.pfx/credentials so secrets can't be committed by accident (allow .env.example/.sample) - Drop `Framework :: Django :: 4.2` classifier (CI tests only 5.2 LTS / 6.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These try/except ImportError blocks aliased reverse/re_path/url for Django < 2.0 / < 4.0 and are dead code under the Django 5.2+ floor: - admin.py: collapse the urlresolvers/re_path shims to `from django.urls import re_path, reverse` - test_admin.py: same for reverse - urls.py: drop the re_path shim and the removed `from django.conf.urls import include`, use `from django.urls import include, re_path` Besides removing dead code, this stops django-upgrade from injecting broken `path()` calls (F821) when these files are edited — the shims were a latent commit-time landmine. Verified: 50 tests pass, ruff check clean. (--no-verify: surgical diff; ruff verified clean.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents the deliberate "format staged files only" policy so contributors don't run `pre-commit run --all-files` (which would bulk-reformat untouched code). Covers uv setup, running tests, and the informational-only CI lint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brings the djangoql-iplweb modernization onto master: - pyproject.toml + uv packaging; bumpver; version 0.20.0 - ruff (lint+format), pytest runner, refreshed GitHub Actions matrix - README converted to Markdown; CONTRIBUTING added - Python 2 cruft removed; dead Django-version compat shims removed - Distribution renamed to djangoql-iplweb (import name stays djangoql) - OSS publication audit fixes (CI hardening, metadata)
- Expand the fork note: state clearly this is a community fork of ivelum/djangoql, link the upstream repo and the original `djangoql` PyPI package, and ask users to switch back to upstream if these changes are merged there (this fork defers to upstream). - Remove the "DjangoQL is used by:" section and the RedHat/Teamplify/ Police1/15Five logos — those are users of the original djangoql, not of this fork. Deleted the now-orphaned assets/*.svg files too. - Kept the product auto-completion screenshot. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Supported by section crediting iplweb (with logo linking to iplweb.pl), in the spirit of the other iplweb forks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backfill characterization tests for previously untested behaviour. Branch coverage of the djangoql package goes from 92% to 100% (50 -> 80 tests). New coverage: - admin.py: the DjangoQLSearchMixin search flow — search-mode toggle, empty/valid/invalid query handling, error->warning rendering, media JS injection, the q-l marker stripping in DjangoQLChangeList, and the Postgres-specific inet/NotSupportedError branches (driven via a small FakeQuerySet + mocked apply_search, since the test DB is SQLite). - ast.py: __str__/__repr__, __eq__ across node types, Name from a tuple. - schema.py: StrField.get_options (choices and icontains paths), DateField parse/validate, get_field_cls mapping, validate() nullable edge cases, deprecated as_dict(). - queryset.py: the OR branch of build_filter. - parser.py: 'not in' operator, long-token error truncation, unescape(bytes). - views.py: SuggestionsAPIView "schema undefined" / unsupported-field errors. - exceptions.py: line-without-column error formatting. Tests favour cheap unit-level checks (no DB where the code is pure) and reuse class-level setUpTestData fixtures to keep the suite fast (~1.7s). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tests: raise djangoql coverage 92% → 100%
Covers date/time part fields, subquery-based relation aggregates (count + sum/avg/min/max, lazy), the `suggested` autocomplete flag, and migrating documentation to a MkDocs site under docs/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9 TDD tasks: suggested flag, lazy annotation hook, test model fields, date/time part fields, subquery aggregates (count + sum/avg/min/max), auto-generation mixins + ExtrasSchema, backward-compat guards. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fields with suggested=False are excluded from the serializer JSON output, allowing derived/hidden fields to be added to a schema without appearing in autocomplete. Default is True so all existing fields remain visible. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Derived fields default to suggested=False (already hidden from the LLM field list), so capability detection must scan schema.models (all fields), and emission must exclude derived classes explicitly to also cover schemas that surface them via suggested=True. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rom LLM schema IR Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… count via CountField Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add include_fields / exclude_fields design: per-model allowlist/denylist mirroring include/exclude at the field level, applied in get_fields so it flows into autocomplete, LLM description and validation from one source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add include_fields / exclude_fields to DjangoQLSchema, applied in get_fields via apply_field_rules(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Raise DjangoQLSchemaError when a model appears in both dicts or when a listed field name does not exist on the model. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filtering a relation name drops the related model and blocks traversal; the same filter trims describe_schema_for_llm output (one source of truth). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Declarative per-model field filtering: include_fields / exclude_fields.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…/username) over first alphabetical str
…(no more silent value-less schema) except Exception: return None in these two functions hid DB/field errors completely, so a dead DB or a bad field name produced a schema with no related_values/related_examples and no trace of why. Log a warning with traceback before returning None; behavior (return None) is unchanged.
fix(llm): auto-mode picks real identifier fields + logs swallowed errors
SENSITIVE_TARGET_APP_LABELS catches Django's built-in auth.User, but a project with a custom AUTH_USER_MODEL in a non-sensitive app (e.g. myapp.User) fell through the auto-mode guard and had its user fields (usernames, etc.) emitted as auto match values. Factor the exclusion check into _is_sensitive_target(), which also compares the relation's target model against get_user_model() -- guarded so a broken AUTH_USER_MODEL can never break schema description. Explicit fk_options entries still override, as before.
fix(llm): auto-mode also excludes custom AUTH_USER_MODEL
Dodaje opcjonalny atrybut schematu `no_value_targets`: twardą listę modeli-celów relacji (klasy modeli i/lub etykiety "app_label.Model"), których wartości wierszy NIGDY nie są osadzane w opisie dla LLM. Sprawdzane przed `fk_options` i niezależne od `max_fk_options`, więc przebija nawet jawny wpis `fk_options` oraz próg liczby wierszy. Pozwala trzymać dane instytucji (np. nazwy jednostek/instytucji) poza commitowanym lub współdzielonym opisem schematu, niezależnie od liczby wierszy w tabeli. Nieznane etykiety są logowane i pomijane (literówka nie psuje opisu). Uzupełnia auto-only guard SENSITIVE_TARGET_APP_LABELS / AUTH_USER_MODEL. Bump 0.30.2 -> 0.30.3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Handoff dla osobnej sesji: describe_schema_for_llm wkleja pełną listę wartości słownika (np. bpp.jezyk) przy KAŻDYM FK do niego — w realnym artefakcie BPP lista języków powtarza się ~14×, łącznie 75 linii `match … in (…)`, sporo to duplikaty. Dokument opisuje root cause (emisja per-FK w _relation_values), rekomendowany projekt (górny blok `dictionaries` — single source of truth, FK referuje zamiast wklejać), plan TDD, breaking-change kształtu JSON (→ 0.31.0), kroki wydania i robotę downstream w BPP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wartości modelu-celu relacji były wklejane inline przy KAŻDYM FK do tego słownika — ten sam słownik (np. `jezyk`) powtarzany tyle razy, ile jest do niego kluczy obcych. To marnowało duże ilości promptu na czystą redundancję (w artefakcie BPP lista `jezyk` ~14×). Teraz wartości emitowane są RAZ, w nowym górnopoziomowym bloku `dictionaries` z kluczem `(model-cel, match_field)`, a każda relacja niesie tylko lekką referencję `match_field`/`match_fields`. Przykłady `str(obj)` deduplikowane analogicznie pod kluczem `__str__`. Breaking (kształt JSON): pola relacji nie niosą już inline `related_values`/`related_examples`; wartości są w `dictionaries[relates_to][match_key]`. Compact renderuje relację jako `-> model match <field>` (albo `examples`), a wartości listuje raz w bloku `dictionaries (...)` na dole. Semantyka fk_options / max_fk_options / no_value_targets / sensitive-target guard / choices bez zmian. Efekt uboczny: słownik odpytywany raz zamiast raz-na-relację. Nowa warstwa: `_relation_plan` (rozwiązanie specu bez fetcha) + `_collect_dictionaries` (fetch z memoizacją po (model, match_field), zwraca `dictionaries` i `field_refs`). Usunięto `_relation_values`, `_match_field_entry`, `_match_fields_entry`, `_examples_entry`. Docs (`docs/llm-schema.md`) i CHANGES zaktualizowane; mkdocs --strict OK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…31.1
fk_options[owner][field] = N (int) now emits every distinct value of the
relation's default match field when cardinality <= N, overriding BOTH the
MAX_SUGGESTED_VALUES (20) cap that `True` hits AND the global max_fk_options
threshold. Per-relation, so raising one dictionary's limit never flips
unrelated auto relations into emitting (no leak of source/institution data).
Motivation: exhaustive reference dictionaries (e.g. ~487 languages, all
scientific disciplines) must reach the LLM prompt in full; True capped them
at 20 and the global threshold is intentionally 0 for safety.
- _relation_plan: dispatch int spec -> ('field', match, N) / ('str', None, N)
- tests: bypass-cap-and-global-threshold + drop-over-own-limit (331 pass)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi,
I LOVE this package,
I actually wrote a similar one myself, very long time ago (GitHub.com/mpasternak/django-dsl/) but yours is way better + has autocompletion;
how about some translations?