A very specific MySQL query that I want to improve

This is my scenario: I have a table containing events, each event has a field called "created" with the timestamp in which this event was created. Now I need to sort the events from the newest to the oldest, but I do not want MySQL to return all of them. I need only the last one for a given interval, for example, within 24 hours (EDIT: I would like to have a flexible solution not only within 24 hours, but maybe every few hours). And I need only the last 10 days. I have achieved this, but I am sure of the most inefficient features, that is, something like this:

$timestamp = time(); for($i = 0; $i < 10; $i++) { $query = "SELECT * FROM `eventos` WHERE ... AND `created` < '{$timestamp}' ORDER BY `created` DESC LIMIT 1"; $return = $database->query( $query ); if($database->num( $return ) > 0) { $event = $database->fetch( $return ); $events[] = $event; $timestamp = $timestamp - 86400; } } 

I hope I was clear enough. Thanks, Jesus.

+3
source share
6 answers

Assuming you want to get the latest event with the highest date created in the last 10 days.

to get the latest timestamp per day

 $today = date('Ym-d'); $tenDaysAgo = date('Ym-d', strtotime('-10 day')); $innerSql = "SELECT date_format(created, '%Y-%m-%d') day, MAX(created) max_created FROM eventos WHERE date_format(created, '%Y-%m-%d') BETWEEN '$today' and '$tenDaysAgo' GROUP BY date_format(created, '%Y-%m-%d')"; 

Then we can select all the events corresponding to those that were created.

 $outerSql = "SELECT * FROM eventos INNER JOIN ($innerSql) as A WHERE eventos.created = A.max_created"; 

I did not have the opportunity to verify this, but the principles should be good enough.

If you want to group some other arbitrary number of hours, you must change innerSql:

 $fromDate = '2012-07-06' // or if you want a specific time '2012-07-06 12:00:00' $intervalInHours = 5; $numberOfIntervals = 10; $innerSql = "SELECT FLOOR(TIMESTAMPDIFF(HOUR, created, '$fromDate') / $intervalInHours) as grouping, MAX(created) as max_created FROM eventos WHERE created BETWEEN DATE_SUB('$fromDate', INTERVAL ($intervalInHours * $numberOfIntervals) HOUR) AND '$fromDate' GROUP BY FLOOR(TIMESTAMPDIFF(HOUR, created, '$fromDate') / $intervalInHours)"; 
+1
source

If you have an index with created as the leading column, MySQL can do the reverse scan. If you have a 24-hour period that does not have any events, you can return a string that is NOT from this period. To make sure you get the row in this period, you really need to include the bottom border in the created column, something like this:

 SELECT * FROM `eventos` WHERE ... AND `created` < FROM_UNIXTIME( {$timestamp} ) AND `created` >= DATE_ADD(FROM_UNIXTIME( {$timestamp} ),INTERVAL -24 HOUR) ORDER BY `created` DESC LIMIT 1 

I think the big key to performance here is the index with created as the leading column, as well as all (or most) of the other columns referenced by the WHERE clause, and make sure this index is used by your query.

If you need a different time interval, up to the second, this approach can be easily generalized.

 SELECT * FROM `eventos` WHERE ... AND `created` < DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL 0*{$nsecs} SECOND) AND `created` >= DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL -1*{$nsecs} SECOND) ORDER BY `created` DESC LIMIT 1 

From your code, it looks like 24-hour periods are limited at any time ... if the time function returns, for example. 1341580800 ('2012-07-06 13:20'), then your ten periods will all be from 13:20 on a specific day to 13:20 the next day.

(NOTE: make sure that if your parameter is an integer timestamp unix, this is correctly interpreted by the database.)

It might be more efficient to print ten lines in a single query. If there is a guarantee that the "timestamp" is unique, then you can create such a request, but the request text will be much more complicated than yours now. We could come to terms with getting MAX (timestamp_) for each period, and then join this to get the row ... but it will be really dirty.

If I tried to pull out all ten lines, I would probably try using the UNION ALL approach, which is not very pretty, but at least it could be customized.

 SELECT p0.* FROM ( SELECT * FROM `eventos` WHERE ... AND `created` < DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL 0*24 HOUR) AND `created` >= DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL -1*24 HOUR) ORDER BY `created` DESC LIMIT 1 ) p0 UNION ALL SELECT p1.* FROM ( SELECT * FROM `eventos` WHERE ... AND `created` < DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL -1*24 HOUR) AND `created` >= DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL -2*24 HOUR) ORDER BY `created` DESC LIMIT 1 ) p1 UNION ALL SELECT p2.* FROM ( SELECT * FROM `eventos` WHERE ... AND `created` < DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL -2*24 HOUR) AND `created` >= DATE_ADD(FROM_UNIXTIME({$timestamp}),INTERVAL -3*24 HOUR) ORDER BY `created` DESC LIMIT 1 ) p2 UNION ALL SELECT p3.* FROM ... 

Again, this can be generalized to go through a few seconds as an argument. Replace HOUR with SECOND and replace β€œ24” with the bind parameter, which takes a few seconds.

It is quite long, but it should work fine.


Another really dirty and complicated way to return this to a single result set is to use the inline view to get the final timestamp for ten periods, something like this:

  SELECT p.period_end FROM (SELECT DATE_ADD(t.t_,INTERVAL -1 * i.i_* {$nsecs} SECOND) AS period_end FROM (SELECT FROM_UNIXTIME( {$timestamp} ) AS t_) t JOIN (SELECT 0 AS i_ UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 ) i ) p 

And then attach it to your table ...

  ON `created` < p.period_end AND `created` >= DATE_ADD(p.period_end,INTERVAL -1 * {$nsecs} SECOND) 

And pull MAX (created) for each GROUP BY p.period_end period, wrap it in an inline view.

And then attach this to your table to get each row.

But it is really very dirty, difficult to understand and unlikely to be faster (or more effective) than what you are already doing. The best thing you could do is the time it takes to complete your 9 requests.


+3
source

I would add another column, which is a date (not time), and then use MySQL "group by" to get the most recent for each date.

http://www.tizag.com/mysqlTutorial/mysqlgroupby.php/

This tutorial does just that, but by product type instead of date. This should help!

+1
source

Do you want all events within 10 days or only one event per day for a 10-day period?

In any case, consider MySQL date functions for help. This should help you get the date range you need.

0
source

Try the following:

  SELECT * FROM eventos WHERE created BETWEEN DATE_SUB(DATE(NOW()), INTERVAL 10 DAY) AND DATE_ADD(DATE(NOW()), INTERVAL 1 DAY) ORDER BY created DESC LIMIT 10 
0
source

Here you will receive the first event of the day in the last 10 days.

  SELECT * FROM eventos WHERE created BETWEEN DATE_SUB(DATE(NOW()), INTERVAL 10 DAY) AND DATE_ADD(DATE(NOW()), INTERVAL 1 DAY) GROUP BY DATE(created) ORDER BY MAX(created) DESC LIMIT 10 
0
source

Source: https://habr.com/ru/post/921894/


All Articles