Based on the chat discussions for 3 groups, and this provided SQLFiddle for the test data (there is not much data there).
Due to testing data with a sliding window where now() refers to this data, the following variable was used to freeze now() . Just to facilitate testing and verifying output.
So, let's end up changing 4 links in the code used (note that group 3 uses it twice).
Variable now() :
select @theNow:=now();
Request:
select id,sentNum,message,sentTime,startAtTime,sentByTime,msgType,theGrp from ( select id,sentNum,message,sentTime,startAtTime,sentByTime,msgType,theGrp, if( sentNum!=@lastSentNum ,greatest(@sentNumChg:=1,0),least(@sentNumChg:=0,1)) as dummy1, if( theGrp!=@lastGrp ,greatest(@grpChg:=1,0),least(@grpChg:=0,1)) as dummy2, if(@sentNumChg=1 or @grpChg=1,@seqNum:=1,@seqNum: =@seqNum +1) as seqNum, @lastSentNum:=sentNum as setLast01, @lastGrp:=theGrp as setLast02 from ( -- GROUP 1: sentByTime<=now(), INVITE select `id`, `sentNum`, `message`, `sentTime`, `startAtTime`, `sentByTime`, `msgType`, 1 as theGrp from SmsQueue where sentByTime< =@theNow and msgType='invite' UNION ALL -- GROUP 2 startAtTime<=now(), BROADCAST select `id`, `sentNum`, `message`, `sentTime`, `startAtTime`, `sentByTime`, `msgType`, 2 as theGrp from SmsQueue where startAtTime< =@theNow and msgType='broadcast' UNION ALL -- GROUP 3: sentByTime>now() && startAtTime<=now(), INVITE select `id`, `sentNum`, `message`, `sentTime`, `startAtTime`, `sentByTime`, `msgType`, 3 as theGrp from SmsQueue where sentByTime>@theNow and startAtTime< =@theNow and msgType='invite' ) d1 cross join (select @sentNumChg:=0,@grpChg:=0,@lastSentNum:='',@lastGrp:=0,@seqNum:=0) as xParams order by sentNum,theGrp,sentByTime,id -- id is the tie-break ) d2 where (theGrp=1 and seqNum<3) or (theGrp=2 and seqNum=1) or (theGrp=3 and seqNum=1) order by sentNum,theGrp;
Output (at the moment, my client tool is a text call): 
See my general comments at the top of this my answer for advanced use of variables.