Best Approach to SQL Data Processing

I have a data intensity problem that requires a lot of massing and data manipulation, and I put it there to find out if anyone has an idea on how to approach it.

In the simplest form. I have many tables that can be combined to give me a list of prices for dentists and how much each procedure fee is.

therefore, we have several tables that look like this.

Dentist | Procedure1 | Procedure2 | Procedure3 | .........| Procedure?
John    | 500        | 342        | 434        | .........| 843
Dave    | 343        | 434        | 322        | NULLs....|
Mary    | 500        | 342        | 434        | .........| 843
Linda   | 500        | 342        | Null       | .........| 843

Dentists may have a different number of procedures and different prices for each procedure. But there are many dentists who have the same number of procedures and the same indicators as with him. Inside, we create a unique identifier for each of these so-called payment lists.

like John will be 001, Dave will be 002, but Mary will pay 001, and Linda will be 003 It's not so bad if I have to deal with this data once, but these payment lists are presented in flat files (csvs), which I I basically have DTS up to SQL server to work with. and they come on a monthly basis. Pricing can vary from month to month for each dentist, who would then put them in another unique identifier from the inside.

Can someone shed light on how best to approach this problem so that it is most effective for processing on a monthly basis, without having to process tons of data manipulations?

  • What is the best approach to finding duplicate fee lists?
  • , ? 2, . ?
  • , , , zipcodes, - , ?
  • - .net, , SQL-.

, , .

+3
1

, , , :

Doctor: DoctorID, DoctorDetails...
FeeSchedule: DoctorID, ScheduleID, EffectiveDate, OtherDetailAtThisLevel...
FeeScheduleDetail: ScheduleID, ProcedureCode, Fee, OtherDetailAtThisLevel...

, , , .

SSIS , - , . , SSIS.

. , .

, (SSIS - ) SQL ( T-SQL UNPIVOT). , .

, , ? . . :

ZipCodeSchedule: ZipScheduleID, ZipCode, EffectiveDate
ZipCodeScheduleDetail: ZipScheduleID, ProcedureCode, Fee

(, - , UCR).

+1

Source: https://habr.com/ru/post/1751487/


All Articles