Ffmpeg: get frame size or short time

Question

I want to make a program that automatically removes frames from mp4 when the volume for these frames is below a given threshold. How can I get the volume of each frame? Or, conversely, the volume for time t in the video? Or for timespan dt? (I'm not yet committed to FFmpeg, or even mp4, so feel free to suggest alternatives.)