{183728233}: Add Lua push write by emelialei88 · Pull Request #6027 · bloomberg/comdb2

emelialei88 · 2026-06-15T16:43:41Z

No description provided.

roborivers

Cbuild submission: Success ✓.
Regression testing: 4/679 tests failed ⚠.

The first 10 failing tests are:
queuedb_rollover_noroll1_generated **quarantined**
cdb2jdbc
consumer_non_atomic_default_consumer_generated **quarantined**
fdb_compat
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
fdb_compat
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**
sc_timepart [timeout] **quarantined**

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
sp_snapshot_generated
sc_parallel_logicalsc_generated
consumer_non_atomic_default_consumer_generated **quarantined**
fdb_push_lua
fdb_compat
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
fdb_push_lua
manual_partition
fdb_compat
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_redo [failed with core dumped]
sc_resume_logicalsc_generated **quarantined**
consumer_non_atomic_default_consumer_generated **quarantined**
fdb_compat
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Error ⚠.
Regression testing: 0/0 tests failed ⚠.

roborivers

Cbuild submission: Error ⚠.
Regression testing: 0/0 tests failed ⚠.

roborivers

Cbuild submission: Error ⚠.
Regression testing: 0/0 tests failed ⚠.

roborivers

Cbuild submission: Error ⚠.
Regression testing: 0/0 tests failed ⚠.

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
sqllogfill_reset_gen
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
sc_resume_logicalsc_generated **quarantined**
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
manual_partition
ssl_set_cmd
ssl_prefer
ssl_dbname
skipscan **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
commit_lsn_map
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
reco-ddlk-sql [timeout] **quarantined**

dorinhogea · 2026-07-23T12:10:29Z

+        fdb_push_connector_t *push = sp->clnt->fdb_push;
+        int nparams = sqlite3_bind_parameter_count(stmt);
+        if (nparams > 0) {
+            fdb_push_set_params(push, calloc(nparams, sizeof(struct param_data)));


check calloc return for NULL

dorinhogea

please see comments, thank you

emelialei88 · 2026-07-24T16:13:58Z

    if (!clnt->in_client_trans) {
        goto free;
    }


Follow-up (perf, non-blocking): SP push writes reconnect on every exec.

When the SP is not in an outer transaction (!clnt->in_client_trans), each push write closes its remote connection right after running (fdb_free_tran, db/fdb_push.c:820).

So this loop reconnects every time:

local stmt = db:prepare("INSERT INTO LOCAL_remdb.t VALUES (?, ?)") for i = 1, 10000 do stmt:bind(1, i); stmt:bind(2, 'row'..i); stmt:exec() -- reconnects each time end

Gotcha: db:begin() inside the SP does not help. It doesn't set in_client_trans. Only an outer client transaction reuses the handle:

begin exec procedure ... commit

Fix: keep the remote connection open across execs in one SP call. Free it once at SP teardown (reset_sp). Gate it so it never double-frees a dtran owned by an outer transaction. Semantics stay the same — each write is already auto-committed remotely. We just skip the reconnect.

We could consider this later; I would say, since remote info is cached (fdb and sockets), cost should be low.

emelialei88 · 2026-07-24T16:18:32Z

        if (p->nparams && p->params)
            dohsql_free_params(&p->nparams, &p->params,
                               p->nparams-1);


why did we pass p->nparams-1? If the number of params is 1 does it mean we don't free anything?

we decide to address this in a future PR

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
reco-ddlk-sql **quarantined**
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**

roborivers

Cbuild submission: Success ✓.
Regression testing: Success ✓.

The first 10 failing tests are:
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**

dorinhogea

looks good to me!

Signed-off-by: Emelia Lei <wlei29@bloomberg.net>

roborivers

Cbuild submission: Error ⚠.
Regression testing: Success ✓.

The first 10 failing tests are:
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**

emelialei88 force-pushed the feat/lua-push branch 2 times, most recently from 1fe3177 to d7dd67d Compare June 15, 2026 16:48

roborivers suggested changes Jun 15, 2026

View reviewed changes

dorinhogea reviewed Jun 16, 2026

View reviewed changes

Comment thread lua/sp.c Outdated

dorinhogea reviewed Jun 16, 2026

View reviewed changes

Comment thread db/fdb_push.c Outdated

dorinhogea reviewed Jun 16, 2026

View reviewed changes

Comment thread lua/sp.c

emelialei88 force-pushed the feat/lua-push branch from d7dd67d to 2372267 Compare June 16, 2026 20:44

This comment was marked as outdated.

Sign in to view

emelialei88 force-pushed the feat/lua-push branch from 2372267 to ea9dda9 Compare June 18, 2026 19:59

roborivers suggested changes Jun 18, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch from ea9dda9 to 5bcbc89 Compare June 24, 2026 18:37

roborivers suggested changes Jun 24, 2026

View reviewed changes

roborivers approved these changes Jun 25, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch 3 times, most recently from 28f01d5 to ec68fcd Compare June 25, 2026 19:57

roborivers suggested changes Jun 25, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch from ec68fcd to 6c09249 Compare June 25, 2026 20:47

roborivers suggested changes Jun 25, 2026

View reviewed changes

roborivers approved these changes Jun 26, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch from 6c09249 to 4c0f998 Compare June 29, 2026 22:03

roborivers suggested changes Jun 29, 2026

View reviewed changes

roborivers suggested changes Jun 30, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch 2 times, most recently from df2c0cb to f69789d Compare June 30, 2026 16:21

roborivers suggested changes Jun 30, 2026

View reviewed changes

roborivers approved these changes Jul 1, 2026

View reviewed changes

roborivers suggested changes Jul 1, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch from 5c0ffe8 to 8b6b1f1 Compare July 1, 2026 15:57

roborivers suggested changes Jul 1, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch from 8b6b1f1 to b7b83f0 Compare July 1, 2026 18:23

roborivers approved these changes Jul 1, 2026

View reviewed changes

emelialei88 changed the title ~~{183728233}: Add Lua push~~ {183728233}: Add Lua push write Jul 2, 2026

emelialei88 force-pushed the feat/lua-push branch from b7b83f0 to e9f9670 Compare July 7, 2026 16:44

roborivers approved these changes Jul 7, 2026

View reviewed changes

emelialei88 marked this pull request as ready for review July 20, 2026 14:24

dorinhogea reviewed Jul 23, 2026

View reviewed changes

Comment thread lua/sp.c Outdated

dorinhogea reviewed Jul 23, 2026

View reviewed changes

Comment thread db/fdb_push.c

dorinhogea reviewed Jul 23, 2026

View reviewed changes

Comment thread lua/sp.c Outdated

dorinhogea reviewed Jul 23, 2026

View reviewed changes

Comment thread lua/sp.c

dorinhogea reviewed Jul 23, 2026

View reviewed changes

Comment thread lua/sp.c

dorinhogea requested changes Jul 23, 2026

View reviewed changes

emelialei88 commented Jul 24, 2026

View reviewed changes

emelialei88 force-pushed the feat/lua-push branch 2 times, most recently from 1dc1952 to c537a5a Compare July 24, 2026 17:06

roborivers suggested changes Jul 24, 2026

View reviewed changes

roborivers approved these changes Jul 24, 2026

View reviewed changes

emelialei88 mentioned this pull request Jul 30, 2026

Fix fdb_push bound-param leak in fdb_push_free #6098

Open

emelialei88 force-pushed the feat/lua-push branch from c537a5a to 659d12e Compare July 30, 2026 18:46

dorinhogea approved these changes Jul 30, 2026

View reviewed changes

support fdb push write in lua

5d7db6f

Signed-off-by: Emelia Lei <wlei29@bloomberg.net>

emelialei88 force-pushed the feat/lua-push branch from 659d12e to 5d7db6f Compare July 30, 2026 20:05

emelialei88 merged commit cc97a7e into bloomberg:main Jul 30, 2026
3 of 4 checks passed

roborivers suggested changes Jul 30, 2026

View reviewed changes

Uh oh!

Conversation

emelialei88 commented Jun 15, 2026

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

roborivers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dorinhogea Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dorinhogea left a comment

Choose a reason for hiding this comment

Uh oh!

emelialei88 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!