That's right. Any application running in the background (within various parameters for when it is legal) will affect the availability of the CPU. Apple applications can work in many other situations than 3rdparty applications, and they will also compete with you. I especially had problems with Mail.app in the past.
Memory performance is a bit more complicated, but yes, other applications are stored in memory at the same time, and you can confidently generate memory warnings earlier than other applications in memory than otherwise. Basically, you should be able to get as much memory as you would without other applications, but this is not entirely true. In particular, do not forget that Apple apps do not always follow the same rules as 3rdparty apps, and if they eat a lot of memory, they may or may not be killed.
Another performance consideration is network bandwidth, and this is certainly the way background applications can compete with you. I don’t think Apple applies bandwidth limits for background applications, and downloading large files is the main background activity. (There is some discussion that the AppStore may reject applications that hit the network too much in the background, but I don’t know about the official position on this. In any case, of course, it is legal to use some bandwidth in the background, and this bandwidth is not available for the foreground application.)