Hello, can anyone help me with a SQL importing issue.
I have a table, all its fields are Text, and i want to import from csv to the table which have identical titles
I used to do the import from csv to database table as in the picture, now i am going on ec2 instance and downloaded the same csv and trying to import it with command line.
My command is as follows:
\copy "en.openfoodfacts.org.products"
FROM '/var/lib/postgresql/data/en.openfoodfacts.org.products.csv'
WITH (
FORMAT csv,
DELIMITER E'\t',
NULL '',
HEADER true,
QUOTE '"',
ESCAPE '"'
);
Through intellije i get errors but much less and the importation never stops. through docker postgres container i get an error and it stops the whole process and rolls back all changes.
The error as far as i can tell is because of quotes in the fields of the table, there are strings that contain ( " ' ` , ) which causes the issues i believe but i cannot understand how the intellije is working fine with small amount of errors but on docker its not and its catching errors much faster.
This is an error :
postgres=# \copy "en.openfoodfacts.org.products" FROM '/var/lib/postgresql/data/en.openfoodfacts.org.products.csv' WITH (FORMAT csv, DELIMITER E'\t', NULL '', HEADER true, QUOTE '"', ESCAPE '"')
ERROR: missing data for column "bicarbonate_100g"
CONTEXT: COPY en.openfoodfacts.org.products, line 15413: "0011110198532 http://world-en.openfoodfacts.org/product/0011110198532/inter-american-products-12-new..."