Skip to content

PG/PGDump: Handle backslash in StringList - #15429

Merged
dbaston merged 1 commit into
OSGeo:masterfrom
wslaghekke:master
Oct 23, 2026
Merged

dbaston merged 1 commit into
OSGeo:masterfrom
wslaghekke:master

Conversation

@wslaghekke

Copy link
Copy Markdown
Contributor

What does this PR do?

Inside pg array literals both " and \ are special, but the function only escaped " this pr adds the backslash so its escaped properly in the output.

What are related issues/pull requests?

AI tool usage

  • AI (Y-a-t-il-un-Copilot-dans-l'avion, Chat-j'ai-pété, Jean-Claude Dusse or something similar) supported my development of this PR. See our policy about AI tool use. Use of AI tools must be indicated.
    Claude found this issue while working with ogr2ogr and concluded its the backslash. I added the code in ogrpgdumplayer.cpp myself

Tasklist

  • Make sure code is correctly formatted (cf pre-commit configuration)
  • Add test case(s)
  • Add documentation
  • Updated Python API documentation (swig/include/python/docs/)
  • Review
  • Adjust for comments
  • All CI builds and checks have passed

Environment

Provide environment details, if relevant:

  • OS:
  • Compiler:

Inside pg array literals both " and \ are special, but the function only escaped "
@jratike80

jratike80 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Can you add a simple test case that I could run with ogr2ogr?
EDIT
Never mind, I think I can read the test case that is in your commit.

@wslaghekke

Copy link
Copy Markdown
Contributor Author

Can you add a simple test case that I could run with ogr2ogr? EDIT Never mind, I think I can read the test case that is in your commit.

It needs PG to show it easiest

cat > t.geojson <<'J'
{"type":"FeatureCollection","features":[
{"type":"Feature","properties":{"l":["a\\b"]},"geometry":null},
{"type":"Feature","properties":{"l":["a\\"]},"geometry":null}]}
J
PG="PG:dbname=pdok host=localhost user=pdok password=pdok"

# 1. COPY (default), both features: fails on the trailing backslash
ogr2ogr -f PostgreSQL "$PG" t.geojson -nln t_copy -overwrite

# 2. COPY, only the first feature: no error, but the backslash disappears
ogr2ogr -f PostgreSQL "$PG" t.geojson -nln t_copy1 -overwrite -limit 1

# 3. INSERT: both correct
ogr2ogr -f PostgreSQL "$PG" t.geojson -nln t_insert -overwrite --config PG_USE_COPY NO

psql "postgresql://pdok:pdok@localhost/pdok" -c "select 'copy' as path, l from t_copy1 union all select 'insert', l from t_insert"

Or using pgdump

echo '{"type":"Feature","properties":{"ok":["a\"b"],"broken":["a\\b"]}}' |
  ogr2ogr -f PGDump /vsistdout/ /vsistdin/ -nlt NONE --config PG_USE_COPY YES | grep -A1 ^COPY

that prints
{"a\\"b"} {"a\\b"}
but the second one should have each backslash escaped so should be
{"a\\\\b"}

@jratike80

jratike80 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Sorry, I am not a programmer and the examples are a bit hard for me to follow.

I tried to make a comparison with native tools. I tried to insert an array of two strings which remind common Windows paths c:\data and c:\"Program Files". I suppose they should be inserted this way:

create table backslash (backslashtest text[]);
insert into backslash
values ('{"c:\\data","c:\\\"Program Files\""}');

Then

pg_dump.exe --file "c:\\data\\dumptest.sql" --host "localhost" --port "5432" --username "user" --no-password --format=p --large-objects --inserts --encoding "LATIN1" --verbose --table "public.backslash" "my_pg"

The dumptest.sql file contains now

INSERT INTO public.backslash VALUES ('{"c:\\data","c:\\\"Program Files\""}');

while output from ogr2ogr -f PGDUMP gdaldump.sql...yields

INSERT INTO "public"."backslash" ("backslashtest") VALUES (ARRAY['c:data','c:"Program Files"']);

Is my test valid? Does it match with your test?

I wonder if it would be good to open also a GitHub ticket with a compact description about what has been wrong. Then the the fix could be included into the release news. It may be that some users have created workarounds.
`

@jratike80 jratike80 closed this Oct 22, 2026
@jratike80 jratike80 reopened this Oct 22, 2026
@dbaston dbaston added the backport release/3.13 Backport to release/3.13 label Oct 22, 2026
@dbaston

dbaston commented Oct 22, 2026

Copy link
Copy Markdown
Member

@jratike80 I like your example data, and I think it demonstrates a second bug.

Suppose we have backslash.geojson:

{
"type": "FeatureCollection",
"name": "backslash",
"features": [
{"type":"Feature","properties":{"backslashtest":["c:\\data","c:\\\"Program Files\""]},"geometry":null}
]
}

on master, the following fails:

$ gdal vector convert backslash.geojson "PG:dbname=dan" --output-layer backslash --overwrite-layer
0...10...20...30...40...50...60...70...80...90...100 - done.
ERROR 1: COPY statement failed.
ERROR:  malformed array literal: "{"c:\data","c:\\"Program Files\""}"
DETAIL:  Unexpected array element.
CONTEXT:  COPY backslash, line 1, column backslashtest: "{"c:\data","c:\\"Program Files\""}"

with this PR, it succeeds and the table appears to have the correct contents

$ psql dan -c "table backslash"
 ogc_fid |            backslashtest             | wkb_geometry
---------+--------------------------------------+--------------
       1 | {"c:\\data","c:\\\"Program Files\""} |
(1 row)

However, if we try to convert from PG->PGDUMP, we get incorrect data:

$ gdal vector convert "PG:dbname=dan" --input-layer backslash --output-format PGDUMP /vsistdout/ | grep INSERT
INSERT INTO "public"."backslash" ("ogc_fid", "backslashtest") VALUES (1, ARRAY['c:data','c:"Program Files"']);

But we get the expected result when converting GeoJSON->PGUMP:

gdal vector convert backslash.geojson --output-format PGDUMP /vsistdout/ | grep INSERT
INSERT INTO "public"."backslash" ("backslashtest") VALUES (ARRAY['c:\data','c:\"Program Files"']);

So I think there is an issue when reading from Postgres, completely separate from the issue addressed by this PR. What do you think?

@dbaston

dbaston commented Oct 23, 2026

Copy link
Copy Markdown
Member

So I think there is an issue when reading from Postgres, completely separate from the issue addressed by this PR

That issue --> #15430

@wslaghekke

Copy link
Copy Markdown
Contributor Author

I created #15432 to describe the bug in both directions expanded on your example @dbaston hope thats ok.
Not sure how we want to link them, do we refer to the prs from the issue or the other way round

@dbaston
dbaston merged commit 739fcf0 into OSGeo:master Oct 23, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport release/3.13 Backport to release/3.13

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants