Another crazy question for the board.
I am trying to compare 25 huge SQL databases. They are both proprietary and in the same layout/format. Unfortunately the database programmer tied the export of the data to a system timestamp on all of the datasets making our comparison tools which look for any unique changes show every line as modified. (When I find him... death)
~10,000,000 lines of data in each 25 databases with 700 rows in each line means a ton of programming, if I can even get them to unlock the darn thing. Plus the database's are changed on a daily bases so......
Effectively if I can force each system to think it is midnight on the same day down to the second all the banks will match up and no more problems.
Anybody know how to do that to a windows xp system.
FYI changing the server time to hold steady is not a option we check for time differentials above one second and then server to server time system will comes in and manually changes the time to master.
These systems can be removed from the network when I am going this.
Lemme get this straight, you need to make this program think that the system time is not changing?
You can script the system time, so you could write the script to change the time to whatever you need whenever you need it. You'll need administrative rights to do so and it will impact domain membership and the ability to connect to shares if left that way too long.
GameboyRMH wrote:
Lemme get this straight, you need to make this program think that the system time is not changing?
I need the actual system time to to stay constant for a few minutes effectively. The export of the data takes ~25 seconds to generate the file so each line can get a different timestamp.
If I had access to the raw data then this could be solved with a simple find/replace. But due to the IP involved we are not allowed to see into the databanks, just query to see if they have changed.
This might be able to do what you want if it allows a fixed time (looks like it does):
http://www.nirsoft.net/utils/run_as_date.html
Export the data normally, then use awk to strip off the time stamp column (or sed to set them all identically) in the file, diff to compare the rest.
Giant Purple Snorklewacker wrote:
Export the data normally, then use awk to strip off the time stamp column (or sed to set them all identically) in the file, diff to compare the rest.
If this is an option, this is better.
Giant Purple Snorklewacker wrote:
Export the data normally, then use awk to strip off the time stamp column (or sed to set them all identically) in the file, diff to compare the rest.
That is the standard way to do it I understand but we kind of have our hands tied with the legal side of things.
wearymicrobe wrote:
Giant Purple Snorklewacker wrote:
Export the data normally, then use awk to strip off the time stamp column (or sed to set them all identically) in the file, diff to compare the rest.
That is the standard way to do it I understand but we kind of have our hands tied with the legal side of things.
What? How could that be a legal problem?
GameboyRMH wrote:
This might be able to do what you want if it allows a fixed time (looks like it does):
http://www.nirsoft.net/utils/run_as_date.html
Trying this right now, it simulated out correctly on our digital system.
GameboyRMH wrote:
wearymicrobe wrote:
Giant Purple Snorklewacker wrote:
Export the data normally, then use awk to strip off the time stamp column (or sed to set them all identically) in the file, diff to compare the rest.
That is the standard way to do it I understand but we kind of have our hands tied with the legal side of things.
What? How could that be a legal problem?
The company swears up and down that they have company secrets and IP on the linking of data in the file They are of course crazy. Part of me thinks the files are just full of crap and they don't want people to see it.
Even crazier, I don't see how running the databases through a Linux shellscript (possibly on Cygwin on the same computer) could violate such company secrets.
wearymicrobe wrote:
GameboyRMH wrote:
This might be able to do what you want if it allows a fixed time (looks like it does):
http://www.nirsoft.net/utils/run_as_date.html
Trying this right now, it simulated out correctly on our digital system.
Self Quoting, this will set the date like I want it to but time still keeps on ticking.
wearymicrobe wrote:
wearymicrobe wrote:
GameboyRMH wrote:
This might be able to do what you want if it allows a fixed time (looks like it does):
http://www.nirsoft.net/utils/run_as_date.html
Trying this right now, it simulated out correctly on our digital system.
Self Quoting, this will set the date like I want it to but time still keeps on ticking.
Run a loop script to run this every second?
Any way that you can truncate the system timestamp when you do the compares? In other words, export everything like normal, but import into another database to compare, and round them back to the nearest hour or something.
turboswede wrote:
wearymicrobe wrote:
wearymicrobe wrote:
GameboyRMH wrote:
This might be able to do what you want if it allows a fixed time (looks like it does):
http://www.nirsoft.net/utils/run_as_date.html
Trying this right now, it simulated out correctly on our digital system.
Self Quoting, this will set the date like I want it to but time still keeps on ticking.
Run a loop script to run this every second?
It would have to be every half-second or you'd have a race condition and the clock could tick over (assuming this database only uses second-level resolution, anything finer and you're SOL). In any case that app doesn't work like that, it doesn't change the system clock itself (which would screw a lot of things up), it launches another app with a spoofed system clock.
I think if there's any way to strip the time stamp out of it, it's going to be easier to do that than to make time stand still.
Is this proprietary database hosted in a real SQL database server such as Oracle, Sybase, SQL Server, MySQL, DB2Postgres... etc...where you or a DBA have administrative rights?
If so, this problem can be solved pretty easily by scripting up your own comparison routine. You can use the system tables to generate most of the SQL needed with a little scripting magic so you don't have to spend your life typing it all up per object. PM me if you need help with that sort of thing.
Giant Purple Snorklewacker wrote:
Is this proprietary database hosted in a real SQL database server such as Oracle, Sybase, SQL Server, MySQL, DB2Postgres... etc...where you or a DBA have administrative rights?
I
SQL sever but I don't have rights. I sent the programming company a strong email and asked for viewing rights, its just we will need a NDA among other things to get it done correctly which will add months to what should be a one day gig.
