Postgres COPY command running manually but not through JAVA

Question

I am running this command via JAVA program builder

plpgsql = "\"path_to_psql_executable\psql.exe" -U myuser -w -h myhost -d mydb -a -f "some_path\copy_7133.sql" 2> "log_path\plsql_7133.log\"";

ProcessBuilder pb = new ProcessBuilder("C:\\Windows\\System32\\cmd.exe", "/c", plpgsql);
Process p = pb.start();
p.getOutputStream().close();
p.waitFor();

This is returning me the following error:

ERROR: invalid byte sequence for encoding "UTF8": 0xbd CONTEXT: COPY copy_7133, line 4892

The catch is if I the run the SQL command manually in cmd, then it is copying all of the data successfully giving me the number of rows inserted. Not able to figure out the reason

NOTE: The code is causing problem only for one particular file, for rest working fine.

EDIT: Copy command being run:

\copy s_m_asset_7140 FROM 'C:\ER\ETL\Unzip_7140\asset.csv' csv HEADER QUOTE '"' ENCODING 'UTF8';

The last error the command gave:

psql:C:/ER/ETL/Unzip_7140/copy_s_m_asset_7140.sql:1: ERROR: invalid byte sequence for encoding "UTF8": 0xa0 CONTEXT: COPY s_m_asset_7140, line 10282

But there doesn't seem to be any special character except a '-'. Not sure what it is not able read.

Few more details abt DB: show client_encoding; "UNICODE"

show server_encoding; "UTF8"

Why are you using a ProcessBuilder to run the copy command? copy is a SQL statement that can also be run directly through JDBC. — user330315
– user330315, Commented Jun 30, 2017 at 7:04
The error message suggests that psql is expecting an UTF-8 encoded file, but your input file isn't. Either change the client_encoding in your SQL script to match your input file or change the encoding of the input file. — user330315
– user330315, Commented Jun 30, 2017 at 7:07
@a_horse_with_no_name: My file is UTF-8 encoded .csv file. My concern is if there is any encoding issue, then why is the same command running successfully manually? — Aman Rathi
– Aman Rathi, Commented Jun 30, 2017 at 7:20

Aman Rathi · Accepted Answer · 2017-07-03 09:24:47Z

0

Worked. But still not understand why UTF8 did not work. I changed the encoding to LATIN1 and it worked

\copy s_m_asset_7140 FROM 'C:\ER\ETL\Unzip_7140\asset.csv' csv HEADER QUOTE '"' ENCODING 'LATIN1';

Can somebody pls explain why UTF8 did not work?

answered Jul 3, 2017 at 9:24

Aman Rathi

476 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Postgres COPY command running manually but not through JAVA

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related