Skip to content

NetBelt 1.3.4 (draft): fixes for the Codex review of 1.3.3 - #9

Draft
gomanizm wants to merge 21 commits into
mainfrom
fix/v1.3.4
Draft

gomanizm wants to merge 21 commits into
mainfrom
fix/v1.3.4

Conversation

@gomanizm

@gomanizm gomanizm commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Draft for 1.3.4: fixes for the external (Codex) review of v1.3.3 (R01–R06, R08–R11) and two display memory leaks. Not for merge yet; the user decides after review. The changes for users are in CHANGELOG [Unreleased].

Not in this branch:

  • R07 (MIB IMPORTS) is kept on a separate local branch until its compatibility policy is decided.
  • FTP: the test port collision and the non-exclusive control listener are pending a decision (proposals prepared, not applied).

Checked locally (Windows 11, Python 3.11.9, requirements.txt):

  • Full test suite, one process, on 43b2980: 4488 passed, 2 skipped. One earlier run (33b1b14) failed test_ftp_server_overwritten_queued_retr once; that is a pre-existing test port collision, reproduced on 9fee4af.
  • PyInstaller 6.22.3 --clean build, then tools/check_bundled_notices.py: exit 0 (on 33b1b14; later commits do not touch the build). The exe also survived a 20-second isolated start.

CI (windows-latest, Tests): 82f4165 passed (4454 passed, 1 skipped).

Not verified yet (separate from the 20-second start):

  • Running as administrator
  • Auto-update when installed under Program Files
  • Paths with Japanese characters or spaces
  • Antivirus false positives

🤖 Generated with Claude Code

gomanizm and others added 21 commits October 4, 2026 00:01
The README said the "allow through firewall" button adds a port rule,
and the five panel tooltips said UAC appears once. The button actually
does three things: it adds the port rule (no program, all profiles),
deletes every inbound rule that targets NetBelt.exe with name=all -
hand-made and block rules included, which is how it clears the block
rule Windows creates when the first-run prompt is cancelled - and then
allows NetBelt.exe on all ports and all profiles.

Counted with netsh and ShellExecuteW replaced by a simulated rule table
(no real firewall or UAC): the first press asks 2 times for TFTP, SFTP,
SNMP Trap and Syslog on one protocol, 3 times for FTP (control and
passive) and Syslog on UDP and TCP, and every later press still asks
once, because the self rule is deleted and re-added without a check.
Refusing the first prompt does not stop the next one.

The README section (and a short English counterpart) now lists the
three actions, the UAC counts, and that NetBelt never needs to run as
administrator. The tooltips name the deletion and the per-panel maximum.
Behaviour is unchanged; the new test ties the stated counts to the
simulated ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README limited group auto commands to SSH and Telnet in both
languages. _run_auto_commands skips only the COM ports detected
automatically and never looks at the protocol, so a registered device
receives them over a console (serial) line too. The group editor's help
text was corrected on 2026-09-20 (dec2225); the README was left behind,
and it ships as README.txt in the release package.

Both bullets now say a group's registered devices get the commands over
SSH, Telnet or console, and that the COM ports listed automatically
under the tree's console group belong to no group, so nothing is sent
to them. The new test takes the group label from device_tree.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a password looks like stored ciphertext ("DPAPI:cisco123"), the
device dialog asks before saving it. The question said the value would
be written to config.json in plain text and that "the password could
not be decrypted" would appear at every start. The second half has been
false since d5d4628 (v1.3.1): the startup count only includes real DPAPI
blobs, so such a literal password loads without any notice and still
connects. Saving one with the real DPAPI and reading it back twice gave
load_warning=None both times.

The question now only says the value is written unencrypted, in plain
text; the comment above it is corrected the same way. The check itself
and its default (No) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An ECONNRESET while receiving STOR/APPE/STOU data now replies
"426 Connection lost; transfer aborted." and closes the transfer as
interrupted (the reservation is released through
on_incomplete_file_received), regardless of how long the connection had
been silent. Windows discards unread receive data when the RST arrives,
so a reset right after the last data could store a 0-byte or truncated
file that was reported as 226 / complete. Normal EOF (FIN), downloads
and the existing keepalive / ETIMEDOUT path are unchanged.

Remove the silence-based heuristic (_last_recv_at,
_SILENCE_MARGIN_SECONDS, _data_connection_lost) that no longer has a
caller. Update the two existing tests that pinned the old 226 result
and the module docstring, as approved by the user for R03.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g path

ConfigManager reads "config.json" relative to the working directory, so
launching NetBelt from another folder picked up whatever config.json was
there. For another application's JSON the constructor added a Default
group and rewrote it without a backup (Infinity, dropped duplicate keys,
plaintext passwords turned into DPAPI ciphertext), and for a top-level
list the backup cleanup deleted that folder's config.json.backup_* files.

A JSON that is not NetBelt-shaped (top level is not a dict, or a dict
without "groups" that has keys NetBelt never writes) is now neither
loaded, written, backed up nor cleaned up, and later save_config calls
return False. load_warning tells the user its absolute path and how to
fix it. {}, hand-written configs with only NetBelt keys, configs with a
broken "groups" and unreadable JSON keep the existing self-repair.

The backup path is now absolute, and the version info dialog shows the
absolute path of the config file in use. The storage location itself
is unchanged (R08, first step).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The onefile bootloader before 6.22.1 trusts inherited _PYI_* variables,
so an elevated NetBelt.exe could be made to load python311.dll from a
directory chosen through its environment. 6.22.1 and 6.22.2 wrongly
reject launches through junctions, symlinks and ImDisk, so go straight
to 6.22.3.

pyinstaller-hooks-contrib moves to 2026.7 in the same change: 6.22.3
requires it, and pinning only one of the two makes pip stop with
ResolutionImpossible in both the release build and the test workflow.
The other pins already satisfy 6.22.3's requirements (pip check is clean).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
paramiko.config imports invoke inside a try block and only needs it for
SSHConfig "Match exec", which NetBelt never uses. PyInstaller still
followed the import and bundled 46 Invoke modules (with vendored yaml,
lexicon and fluidity), while THIRD-PARTY-NOTICES.txt carried neither
Invoke's entry nor its BSD-2-Clause text.

Excluding it changes the bundle by invoke and pty only (pty is imported
by invoke.runners alone); an isolated offscreen start of the rebuilt exe
stays up with a clean log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BUILD_ONLY claimed its members are never bundled, but NetBelt.exe has
always carried setuptools (179 modules with its vendored parts, imported
on every start by pyi_rth_setuptools) and packaging, and carried Invoke
until the previous commit. None of them appeared in the notices.

setuptools and packaging stay in the exe so start-up behaviour does not
change; they now get their entries and licence texts. setuptools ships
the dist-info of each vendored part under setuptools/_vendor/, so those
licence files are read from the installed package and printed under the
setuptools entry. Invoke moves to its own EXCLUDED_FROM_EXE set, tied to
the spec's excludes by a test.

THIRD-PARTY-NOTICES.txt is regenerated with the new generator in a
fresh environment from requirements.txt; the change only adds the
packaging and setuptools entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The notices are generated from the build environment and the exe's
contents are decided by PyInstaller's import graph, so the two can drift
apart unnoticed; 1.3.3 shipped invoke, setuptools and packaging without
listing them.

This adds the pure part of tools/check_bundled_notices.py: it sorts the
PYZ modules and CArchive entries of an exe, maps top-level names through
packages_distributions and bundled files through RECORD (binary-only
packages such as PyQt6-Qt6 have no top-level module of their own), and
reports distributions missing from the list, names it cannot map, and
vendored parts whose licence text is not in the parent's section.
Standard library, NetBelt's own modules, PyInstaller's bootstrap and
plain DLLs are left out. The command line and the CI step follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the command line to tools/check_bundled_notices.py: it reads
dist/NetBelt.exe with PyInstaller's CArchiveReader (same reader in
6.17.0 and 6.22.3), maps what it finds through the build environment's
metadata, and exits 1 when a bundled distribution is missing from
THIRD-PARTY-NOTICES.txt, 2 when it cannot check at all.

build-release.yml runs it right after "Build executable", against the
notices regenerated earlier in the same job, so a PyInstaller or hook
update, or a floating dependency such as urllib3, cannot change the
bundle silently again. The step sits after "Run tests", outside what
test_ci_tests_workflow compares with tests.yml.

Against the published v1.3.3 exe it reports invoke, packaging and
setuptools; against a fresh 6.22.3 build of this branch it passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What happened (measured on 9fee4af, 127.0.0.1): max_client_connections
(32) counts TCP connections only, so an authenticated client could open
any number of SFTP sessions inside one connection (40 and 200 sessions,
one server thread each). transport.accept() was called once, so every
later channel stayed referenced from transport.server_accepts after it
closed (1000 open/close cycles left 1000 entries until disconnect).
Repeating subsystem('sftp') on one channel also started one SFTP thread
per request (20 requests, 20 threads), bypassing any channel count.

Why this fix (decided: 10 channels per connection, the OpenSSH
MaxSessions default, released when a channel closes):
- Count session channels in check_channel_request and refuse the 11th
  with OPEN_FAILED_RESOURCE_SHORTAGE, so the client sees "Resource
  shortage"; bare session channels without a subsystem count too.
- The handler loop now takes every channel with accept(timeout=0.5) and
  holds it until it closes. paramiko keeps channels in a weak map, so a
  dropped reference gets the channel closed by GC; accept() stays the
  only thing that removes entries from server_accepts (removing them in
  the subsystem request can miss the first channel and cut the
  connection 20 s later). Closed channels the handler has already
  accepted are released at the next channel open, which the close
  always precedes, so immediate reopening still works; a closed channel
  not yet accepted keeps its slot until accept().
- Allow one subsystem start per channel (RFC 4254 6.5).
- Refusals go through _log_limited ("channel refused") and
  _emit_activity, like the other refusal notices.
The 20 s accept() deadline after key exchange is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What happened (measured on 9fee4af, 127.0.0.1): one authenticated SFTP
session opened 8189 read handles in 4.3 s (300 of 300 in 0.14 s in a
shorter run). Every handle holds a UCRT low-level file descriptor. The CRT allows
8192 of them per process (a limit of the CRT's file I/O, not of Windows
handles), so while they were held, Python's open() in the same process
(settings, logs, FTP/TFTP files) and getaddrinfo failed with EMFILE;
QFile and established sockets kept working (measured). The GUI and all
servers share that process. The channel cap does not bound this.

Why this fix (decided: a file limit separate from the channel limit,
recommended values 256 per session and 2048 server-wide):
- _OpenFiles counts open files per session and in total under one lock,
  shared across connections. An open over either limit is refused with
  SFTP_FAILURE (what OpenSSH's sftp-server sends when open() fails) and
  reported through _log_limited ("open limit") and _emit_activity.
- Handles give their slot back once on close (_CountedHandle, now the
  base of _WriteHandle); a failed open gives it back at once.
  session_ended, which finish_subsystem calls before closing leftover
  handles, returns whatever the session still holds, so a close that
  raises there cannot leak the server-wide count. It never raises.
- 256 matches MAX_FDS in Win32-OpenSSH, whose sftp-server runs one
  process per session; 2048 is a quarter of the CRT's 8192 and leaves
  the rest to settings, logs, FTP/TFTP files and name resolution
  (NetBelt itself uses 3). OpenSSH sftp and scp kept at most 1 file open per session in a
  20-file put -r / get -r. Directory handles hold no descriptor in
  paramiko and are not counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Directory moves (home, parent, double-click, context-menu open) went
through SFTPManager.change_directory, which called normalize (REALPATH)
synchronously on the GUI thread. A 1.0 s device delay froze the whole
event loop for 1.0 s (30 s on a silent device). Codex review R05.

The lock is still taken on the GUI thread with _acquire_for_gui, so a
move during a transfer is refused after 0.5 s as before. With the lock
held, normalize runs on a throwaway thread, which issues list_directory
for the normalized path before releasing the lock, so a refresh waiting
for the lock follows the move. If the session was folded while waiting,
the result is dropped without a listing or a second notice. Timeouts
still report "ディレクトリ変更エラー" and fold the session.

test_sftp_fail_survives_modal_deletion now reproduces the nested-loop
defense with create_directory, which still reports synchronously on
the GUI thread (approved by the user on the condition that both tests
still exercise the same defense; checked with sys.settrace and four
mutants). New tests cover late success and failure, closing the panel
or the detached window, disconnecting during the move, and a stale
result arriving after reconnecting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What happened (measured on 9fee4af): once the trap list reached max_traps
and started dropping its oldest rows, the process kept growing with every
trap shown, about 0.57 KB per trap with 4 VarBind rows and 3.3 KB with 28
(event loop running, panel offscreen). gc object counts, QStandardItem
wrapper counts and tracemalloc stayed flat; the growth was on the C++ heap.
A HeapWalk count of busy blocks rose by exactly 2 x (1 + child rows) per
trap.

Why: QStandardItemModel.insertRow and QStandardItem.appendRow were called
with a Python list. That argument is a QList<QStandardItem *> marked
/Transfer/, and PyQt6's mapped type returns sipGetState(sipTransferObj)
from %ConvertToTypeCode, so sip treats the QList it new'd for the call as
handed over and never deletes it. Each call leaves the QList and its
element array behind, even for an empty list. The single-item overloads,
setChild and setItem do not convert a list and leave nothing. Whether the
items were created in Python or by the model, and whether rows were removed
with removeRow or takeRow, made no difference.

Fix: place the items one at a time. The VarBind rows are built with
setChild on the first-column item before the row goes into the model (no
per-row model signals), then the row is inserted with insertRow(0, item)
and the other four columns are filled with setData on their indexes, so
the model creates the items without layout signals (setItem emits
layoutChanged per cell, which shrinks a stretched last column when the
columns overflow). The tree has the same rows, columns, children, order,
column widths and expansion behaviour, and adding a trap takes as long
as before. After the fix the busy block count and private bytes stay
flat past the limit.

The SFTP client file list (sftp_panel._update_file_list) used the same
list form; it is fixed in a separate commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trap list's max_traps limit only applies once the GUI has inserted a
trap, and the receiver thread emitted every decoded trap straight into the
Qt queue. While the GUI was blocked nothing was delivered and the queue grew
without bound (R06: 15,999 traps queued with 0 delivered; about 2.8 KB per
ordinary trap, 67-128 KB with a crafted 60 KB value, 5-7 MB/s up to about
50 MB/s), and everything poured in once it resumed. Syslog, FTP, TFTP and
SFTP already cap their delivery backlog; traps were the one receiver left.

The receiver now hands each trap to a per-run counter (_TrapBacklog)
before emitting it. Past the limit, newly received traps are dropped and
counted, as Syslog does, so the start of a storm is what survives. The
limit is max(1000, max_traps), so users who raised max_traps do not start
losing traps they would have kept.

SNMPTrapReceiver is a QThread rebuilt on every start, so the counter is
created by SNMPManager and travels with each queued trap (trap_queued carries
the counter and the trap), and the manager receives it in its own QObject
method rather than through .emit (47ecbde). After a stop and restart,
deliveries still queued from the old run release the old counter, not the
new one. A standalone receiver without a counter keeps emitting
trap_received for every trap as before.

The manager reports newly dropped traps of the current run through
trap_dropped(count, limit, last_time), and logs one line when a backlog
first overflows and one summary (count, first and last drop time) once it
drains. The docstrings that said the backlog stays at 3 or fewer, costs
2.3 KB per trap and is released afterwards are replaced with the measured
figures, and note that datagrams dropped by the OS receive buffer before
NetBelt reads them cannot be counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit drops and counts traps once the delivery backlog is
full, but the count only reached a signal and the log. The SNMP panel now
keeps a persistent line under the receiver status, for example
"取りこぼし: 200 件(配送待ちの上限 1000 件。最後 10:15:42)". It is shown
only while the count is above zero and is reset when reception starts and
when the list is cleared; stopping leaves it in place so it can still be
read. The line is not a row of the trap list, so the CSV / JSON / TXT
exports contain only traps from devices, and it does not use the modal
error_occurred path.

README (SNMP Trap): explain the cap and what is dropped, where the count
and the log lines appear, that NetBelt can only count its own backlog and
not datagrams Windows drops when the receive buffer overflows, and how to
check for those by comparing with the sender's trap counter. It also notes
that the "Receive Errors" line of netstat -s -p udp (「受信エラー」 on
Japanese Windows) did not count receive-buffer overflows in a test against
127.0.0.1 on Windows 11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The log lines for an overflowing trap backlog are developer log strings in
English, while the rest of the app speaks Japanese. Say which lines to look
for (they contain "Trap backlog") so the count in the log can be found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What happened (measured on 9fee4af, same Python and PyQt6 6.10.1 before and
after): every refresh of the SFTP client file list left heap blocks behind
that were never freed. With a 50-entry listing each refresh added exactly
100 busy heap blocks (about 4.4 KB of heap data, about 5.5 KB of private
bytes), so 500 refreshes left 50,000 blocks and about 2.2 MB. Removing the
rows, or closing and deleting the panel, did not give it back.

Why: _update_file_list added each row with self.model.appendRow([...]),
the same list form the trap list used (see "fix(snmp): stop the trap list
from leaking a QList per row"). PyQt6 treats the QList<QStandardItem *> it
builds for that /Transfer/ argument as handed over and never deletes it,
leaving the QList and its element array behind on every call.

Fix: place the items one at a time, as the trap list does. The name item
goes in with the single-item appendRow and the size, permission and time
columns are filled with setData on their indexes, so the model creates
those items without layout signals. After the fix the same 500 refreshes
leave 0 blocks, 0 bytes and no private byte growth, and closing and
deleting a panel returns the heap to its count before the panel was
made.

The rows, columns, texts, UserRole data, flags, roles, order, selection
behaviour and status text are the same as before (compared on 9fee4af
and pinned in tests/test_sftp_file_list_regression.py). A refresh takes
as long as before (2.2 ms for 50 entries and about 140 ms for 10,000,
offscreen with the view shown). No code in src listens to the model's
layout signals.

No other list-taking appendRow / insertRow / insertColumn call is left in
src (grep). QTreeWidgetItem(parent, [str]) in the device tree and
setHorizontalHeaderLabels take a QStringList, which is not /Transfer/, and
did not add blocks in the same kind of measurement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collect the user-facing entries of this branch under [Unreleased]:
PyInstaller 6.22.3, the FTP reset handling, the SFTP server caps, the
SNMP trap backlog cap and dropped-trap indicator, the foreign
config.json protection and config path display, the SFTP move off the
GUI thread, the trap list and SFTP file list memory fixes, the notices
and README corrections, and the known limitations that remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user asked (2026-10-04) to confirm, for the caps of 256 open files
per SFTP session and 2048 server-wide: that concurrent opens never go
over them, that every slot comes back on a failed open, on close and on
disconnect, and that settings, logs and the other servers keep working
while the server-wide cap is reached. tests/test_sftp_open_file_cap.py
only opened one file at a time, covered a missing file as the only
failed open, a channel close as the only disconnect, and never reached
2048 with the real values. No product change: no leak, double return or
race was found.

test_sftp_open_file_cap_concurrency.py
- _OpenFiles hit from 8-20 threads released by a Barrier, with a short
  switch interval: 320 takes on one session grant exactly 256; 10
  sessions x 256 grant exactly 2048 and refuse 512 as "total"; a
  take/give_back race on limits 2/10 stays within both and ends at 0.
- Real paramiko clients sending OPENs without waiting for replies: 300
  on one session (256 granted, 44 refused with SFTP_FAILURE), and 300
  each from 9 sessions on 3 connections at once (2048 granted, 652
  refused, none over 256 per session, sampled peak within both). Closing
  them all, also concurrently, brings the count back to 0.

test_sftp_open_file_cap_release.py
- Failed opens (missing folder, O_EXCL on an existing file, a folder, a
  read-only file, outside the root, after stop) give the slot back
  before the reply arrives.
- A closed handle sent CLOSE again, and the session ending after a
  close, do not give back twice; a handle closed twice gives back once.
- Dropping the connection without closing 256 files (SSH disconnect and
  an RST) and stopping the server with files open give everything back,
  and the next start counts from 0. Each is also run with the leftover
  close raising: paramiko then closes nothing more, so only
  session_ended can give the slots back. Without that variant, removing
  the return from session_ended went unnoticed (the leftover closes
  returned the slots one by one).

test_sftp_open_file_cap_full_server.py (2 connections x 4 sessions x 256)
- ConfigManager.save_config writes, and the config loads again.
- main._setup_logging (the frozen build's log) opens, and print and
  the SFTP server's own refusal line reach the file.
- TFTP upload and download, FTP STOR and RETR (passive port kept off
  the control port), Syslog over UDP and TCP, all on 127.0.0.1.
- Another SFTP client can list, stat, mkdir and rename, and only a new
  open is refused with SFTP_FAILURE; a held handle still reads.
- The CRT descriptors in use are counted (at least 2048, at most half of
  the CRT's 8192), and 1024 more files open with open(). Measured on
  this PC: 2051 in use while 2048 were held, and 6141 more os.open()
  calls succeeded before EMFILE.

Mutations of sftp_server.py, each caught by the new tests: no lock in
_OpenFiles, no give_back on close, give_back on every close, no return
in session_ended, no return on a failed open, no total check, no
per-session check, end() keeping the session record, give_back
subtracting the whole session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…drains

The summary of traps dropped at the delivery cap was written only once
the GUI had delivered every queued trap. Applying an update closes the
window and then calls QApplication.quit(), which ends the event loop
without delivering what is still queued, so a storm in progress left
only "reached its limit" in the log and the dropped count nowhere
(measured with the real MainWindow: 1000 queued, 500 dropped, no
summary line). Closing the window normally was not affected.

Write the unsummarised count when reception stops, and when it starts
again after the receiver thread ended on its own, without waiting for
the queue to drain. The queued traps still arrive afterwards and still
add to the panel's count, and the drained summary is not written twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant