My company has an outdated micromilling program that mimics the population and changes this population over the course of several years.
During each year, the program creates a binary file with a record for each person who has their own characteristics (for example, age, income status, income ... about 20 fields).
We currently have several utilities that read these files and create summary reports. The problem is that every time someone wants a new report, a new utility must be written.
Changing the program so that records are stored in the database instead of binary files is out of the question (I asked ... several times). I have written several programs that import binary files into a database and then run queries in the created tables. The problem here is that it always takes longer to import data and execute a query than to run a utility program written in C ++ that simply reads the records one by one and accumulates the necessary data. Often binary files contain more than 30 million records, and the import step is forever.
So here is my question. Is there anything that would allow me to specify the structure of my binary file and then run the SQL queries in the file? I think you can use ODBC to run queries in text files, but I have never seen anything like this for binaries.
If there is nothing available, what are the steps I need to take to create something that could trigger the request directly in my file? I understand that this will probably be much higher than my capabilities, but it can’t stop me from finding out where I need to start.
anon
source
share