mercredi 22 février 2017

Comparing a csv to a table in postgres

As a part of my project where we are migrating data in csv to postgres tables, I need to build a script to compare csv data to postgres in order to validate the migration . It is not direct row to row comparison moreover the column order in source and target may not be same.

I have tried using Jmeter for the comparison. I read the csv line by line.pick the id column data from the line and search in id column of the data base. I then the compare the entire row in csv to the entire row in data base (string comparison).

This won't work if my column order is not matching. Also I have id column in csv and id column in database is not same.It is translated.

eg. If id in csv is abc. Then i need to pick corresponding database id from a intermediate translation table.

Translation table: csv_id database_id 1 a 2 b 3 c

csv:

id name age 1 john 22 2 peter 34 3 sam 26

posrgres data base(translated id):

id first_name age b peter 34 a john 22 c sam 26

Aucun commentaire:

Enregistrer un commentaire