feat: add truncate argument to todb function

[ChrisJr404]

Closes #669.

This PR has the objective of making the truncate step in todb() optional, so you can load into an existing table without first deleting its rows.

As @juarezr pointed out on the issue, todb() and appenddb() already share the internal _todb() and only differ by the truncate flag, so this just surfaces that flag on todb() with a default of True to keep the current behavior unchanged. Passing truncate=False now appends instead, same as appenddb().

Changes

1. Added a truncate keyword argument to todb(), defaulting to True.
2. Passed it through to the internal _todb() call instead of the previously hard-coded True.
3. Updated the todb() docstring to describe the new argument and the append case.
4. Added unit tests covering the default, the truncate=False passthrough, and an sqlite round trip that keeps existing rows.
5. Documented the change in docs/changes.rst.

Testing

• .venv/bin/python -m pytest petl/test/io/test_db.py

Result:

• 12 passed, 2 warnings

The two warnings are the existing sqlite generator cleanup warnings from test_fromdb, they show up on master too.

Checklist

Use this checklist to ensure the quality of pull requests that include new code and/or make changes to existing code.

• Source Code guidelines:
  • Includes unit tests
  • New functions have docstrings with examples that can be run with doctest
  • New functions are included in API docs
  • Docstrings include notes for any changes to API or behavior
  • All changes are documented in docs/changes.rst
• Versioning and history tracking guidelines:
  • Using atomic commits whenever possible
  • Commits are reversible whenever possible
  • There are no incomplete changes in the pull request
  • There is no accidental garbage added to the source code
• Testing guidelines:
  • Tested locally using tox / pytest
  • Rebased to master branch and tested before sending the PR
  • Automated testing passes (see CI)
  • Unit test coverage has not decreased (see Coveralls)
• State of these changes is:
  • Just a proof of concept
  • Work in progress / Further changes needed
  • Ready to review
  • Ready to merge

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Add optional truncation control to todb()

✨ Enhancement 🧪 Tests 📝 Documentation 🕐 10-20 Minutes

AI Description

• Adds optional truncation control while preserving todb()'s existing default behavior.
• Allows callers to append rows through todb() without switching APIs.
• Documents and tests flag passthrough and SQLite row preservation.

Diagram

graph TD
  Caller["Caller"] --> API["todb API"] --> Choice{"truncate?"}
  Choice -->|true| Delete["Delete rows"] --> Loader["Shared loader"] --> Table["Database table"]
  Choice -->|false| Preserve["Keep rows"] --> Loader

Loading

High-Level Assessment

The current approach is optimal: it exposes an existing internal capability without duplicating database logic and preserves backward compatibility with truncate=True. Requiring callers to select appenddb() was considered but does not support runtime configuration through one API.

Files changed (3) +63 / -4

Enhancement (1) +9 / -4

db.pyExpose truncation control through 'todb()' +9/-4

Expose truncation control through 'todb()'

• Adds a backward-compatible 'truncate=True' keyword argument to 'todb()' and forwards it to '_todb()'. The docstring explains that disabling truncation preserves existing rows and appends new data.

petl/io/db.py

Tests (1) +52 / -0

test_db.pyCover default truncation and append behavior +52/-0

Cover default truncation and append behavior

• Adds passthrough tests for the default and 'truncate=False' values. An SQLite round-trip verifies that disabling truncation retains existing rows while inserting new ones.

petl/test/io/test_db.py

Documentation (1) +2 / -0

changes.rstRecord the new 'todb()' truncation option +2/-0

Record the new 'todb()' truncation option

• Adds the feature and issue reference to the Version 1.7.20 changelog.

docs/changes.rst

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 📜 Skill insights (0)

1. Temporary database file leaks 🐞 Bug ☼ Reliability

Description

The new SQLite test creates a NamedTemporaryFile(delete=False) but never closes its file object
explicitly or removes its path; closing only the SQLite connection leaves a database artifact after
every test run. Repeated or parallel runs therefore accumulate temporary files and rely on
nondeterministic cleanup of the original handle.

Code

petl/test/io/test_db.py[R130-131]

+    f = NamedTemporaryFile(delete=False)
+    conn = sqlite3.connect(f.name)

Evidence

The test opens the named temporary file and then a separate SQLite connection to its path, while its
only cleanup closes conn; no f.close() or path removal exists before the test ends. The database
loader uses the caller-supplied connection and only closes its own cursor, so it cannot clean up
this test-owned file.

petl/test/io/test_db.py[128-151]
petl/io/db.py[451-474]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new SQLite round-trip test creates a named temporary file with deletion disabled, but only closes the SQLite connection. The temporary database remains on disk, and the original temporary-file handle is not explicitly closed.

## Issue Context
Preserve the current assertions while making resource cleanup deterministic, including when setup, database operations, or assertions fail. A pytest temporary-path fixture or a `try`/`finally` that closes both handles and unlinks the file would address the leak.

## Fix Focus Areas
- petl/test/io/test_db.py[128-151]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context sources

✅ Tickets: 🎫 Feature Request: make truncate optional in io.db.to_db()

Tip of the day

💡 Did you know, you can switch off images and animations for a plain-text comment

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[qodo-free-for-open-source-projects]

1. Temporary database file leaks 🐞 Bug ☼ Reliability

The new SQLite test creates a NamedTemporaryFile(delete=False) but never closes its file object
explicitly or removes its path; closing only the SQLite connection leaves a database artifact after
every test run. Repeated or parallel runs therefore accumulate temporary files and rely on
nondeterministic cleanup of the original handle.

Agent Prompt

## Issue description
The new SQLite round-trip test creates a named temporary file with deletion disabled, but only closes the SQLite connection. The temporary database remains on disk, and the original temporary-file handle is not explicitly closed.

## Issue Context
Preserve the current assertions while making resource cleanup deterministic, including when setup, database operations, or assertions fail. A pytest temporary-path fixture or a `try`/`finally` that closes both handles and unlinks the file would address the leak.

## Fix Focus Areas
- petl/test/io/test_db.py[128-151]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[coveralls]

Coverage Report for CI Build 32783092771

Coverage increased (+0.01%) to 91.912%

Details

• Coverage increased (+0.01%) from the base build.
• Patch coverage: 27 of 27 lines across 1 file are fully covered (100%).
• No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	15368
Covered Lines:	14125
Line Coverage:	91.91%
Coverage Strength:	0.92 hits per line

---

💛 - Coveralls