You can use the tabIndex property and set it to -1 for all elements that you do not want to cycle through the tab.
<input type="text" id="no-tab-cycle" tabIndex="-1"/>
Of course, you will need to control this behavior with some smart selectors using jQuery or something like that, but it depends on how complex your forms are.
Maybe someone has a better answer.
jQuery , my-popup-dialog, , ,
$('input, textarea, select').each(function(index) {
$(this).attr('tabIndex', '-1');
});
$('#my-popup-dialog input, #my-popup-dialog textarea, #my-popup-dialog select').each(function(index) {
$(this).removeAttr('tabIndex');
});