Peter, I think if permission checks are taking more than
about 50ms there is something wrong with your db.  On
my scalability box with 40k users and 350k objects it
takes 33ms.  Are you sure you have vacuumed/analyzed your
db?