How to do data comparison test for large number of records

In my project, I need to do lot of data feed testing where I need to compare two sets of data.

1. First set a.k.a. Old, will have the current version of data (as-is version of data)
2. Second set a.k.a. New, will have the newer version of data (with the necessary code changes)

The 'New' file can vary by the 'Old' file in terms of: record count, addition/removal of columns, change in column definition. The data in the file normally have about 50-60 columns and can go upto 200k records.

The data is normally pulled from 3 different databases and about 12-15 tables.

I need to essentially do two things:

1. Make sure the data is matching for the columns which are matching between 'old' and 'new' files
2. Run database queries to validate for the new columns in 'new' file

Currently, Im doing my validation using VLOOKUP, but I'm trying to see if there are any tool which will help me do this faster.

Thank you guys! Your responses are much appreciated.

Thanks, Ganesh