All geek questions in one place

Why is it (possibly) a bad idea to process database data with programming languages ​​like C / Python?

So, I am working on the task of calculating medians for every 100 records in a giant MySQL table, which seems to be a simple problem, but ends up with very complex SQL code. One of my friends who saw my work asked me: why don’t you load data into memory and process it using C or Python, wouldn’t it be easier? My intuition is that this is a bad idea. But can someone clarify why this is not suggested? Thanks!

+4
source share
2 answers

In general, yes, you have to do your heavy lifting in the database. If your data set is quite small, it does not matter if you are performing calculations on the database server or on the database client.

The main consideration, whether to do the calculations on the db server and on the db client, usually has performance. If you are doing heavy computing on a db client, you may need to transfer a lot of data through the db connection. When using large datasets, transferring the entire table to the client can lead to performance problems, and if your database server lives on a different machine than your application server (that is, not local), then the overhead of data transfer becomes even worse.

, , , . SQL , , , , .

, , , , .

, . , , , , , .

+2

, , MySQL db... - " , ".

, , , , MySQL, .

:

  • (, localhost).
  • : ( ).
  • ( , MySQL ).
  • MySQL ( SQL-)
+3

Source: https://habr.com/ru/post/1526254/

More articles:


All Articles