It's hard to say without knowing more about your setup, but one simple gain is to make sure that Trac works in something like mod_pythonthat saves Python runtime in memory. Otherwise, each HTTP request will cause Python to start, import all the modules, and finally process the request. Using mod_python(or FastCGI, depending on what you prefer) will eliminate this download and skip straight to the good.
In addition, as your Trac database grows and the number of users using this site grows, you are likely to outgrow the default SQLite database. At this point, you should consider moving the database to PostgreSQL or MySQL, because they will be able to process concurrent queries much faster.