Are you sure this is a good idea?
It is not at all clear that SDL uses significantly less resources than X-windows. Why do you believe that? Also note that X windows are much more commonly used for general software than SDL, so you are setting yourself up for all kinds of problems. I believe that you should reconsider your assumptions.
, Debian . , , QEmu.
Edit:
: SDL X11 Linux, Linux. Google , , , , -, . , !
, , QEMU X11, SDL. X11, , , .