I started by reading the documentation and some existing code and came to the conclusion that the preferred method within sql up to now has been:
select 1 from acs_object_party_privilege_map ppm where ppm.party_id=x and ppm.privilege='foo' and ppm.object_id=y;
I read a forum post in which it was noted that the PostgreSQL optimiser seems to be making different decisions in PostgreSQL 7.4.x from those made in earlier versions which have resulted in slower performance during permissions lookups.
For the purposes of this document I have performed all of the following queries on an x686 650Mhz laptop. The absolute times reported therefore do not reflect the real world performance of an OpenACS instance on an industrial grade server platform but rather provide an indication of relative speed. My intention here is to float some ideas which I hope will stimulate discussion and improve performance. I expect there to be flaws in some of my assumptions and I suspect that there may be some errors/bugs in my queries - please shoot me down in flames if I deserve it! ![]()
I began by running a benchmark:
TEST A-1)
explain analyse select 1 from acs_object_party_privilege_map ppm where ppm.party_id=448 and ppm.privilege='admin' and ppm.object_id=16631;
Total runtime: 459.076 ms
[The party_id 448 is site-wide-admin and so has many descendent privileges. This impacts performace because of the larger number of rows returned from the acs_object_party_privilege_map view.]
TEST B-1)
explain analyse select 1 from acs_object_party_privilege_map ppm where ppm.party_id=-1 and ppm.privilege='admin' and ppm.object_id=16631;
Total runtime: 173.728 ms
[The party_id -1 is 'the public' and has no aggregated permissions on the system. There are also no explicit permissions set for this party_id.]
As things stand it seems that the use of the acs_object_party_privilege_map view works best when there are simple cascading permissions. This is well suited to sites with a high volume of unregistered visitors but may be less than ideal in a tightly controlled corporate application where many explicit permissions are to be granted.
Based on this conclusion I became comcerned that if I proceed to implement permissions in the Project Manager at the level of projects and tasks, because of the potential number of explicitly granted permissions, I might create a significant performance issue.
I was interested in the design decision that led to the creation of acs_object_party_privilege_map as a view rather than as a table maintained by triggers. I have come to the conclusion that the reason that this works well is that the PostgreSQL optimiser is smart enough to only create enough of the total dataset to satisfy the requirements of the query so that the entire view does not ever have to be created or stored. However it is the behaviour of the optimiser that we do not have control over and this has led to frustrations in the past. I wondered whether we could change the structure of our request such that we guide the optimiser more firmly towards the execution plan that we believe is appropriate.
So I decided to attempt to replicate the functionality of the above without using the acs_object_party_privilege view at all. I decided that the following structure might be appropriate:
select 1 from acs_permissions
where object_id in (...)
and grantee_id in (...)
and privilege in (...);
The sub-selects would each return a small dataset (ideally by index scanning) to the main query which would then only need to join a modest number of rows. Gone would be the need for dynamically creating portions of the enormous flattened combination hierarchy.
Here is the full experimental query:
select 1 from acs_permissions
where privilege in (select descendant from acs_privilege_descendant_map where privilege='admin')
and object_id in (select ancestor_id from acs_object_context_index where object_id=16631)
and (grantee_id in (select group_id from group_member_index where member_id=448) or grantee_id=448);
[The 'or' clause is required because the group_member_index does not hold an object as an ancestor or itself as do the other flattened hierarchies.]
Here are the test results to solve the same problem as benchmarked above:
TEST A-2)
explain analyse select 1 from acs_permissions where privilege in (select descendant from acs_privilege_descendant_map where privilege='admin') and object_id in (select ancestor_id from acs_object_context_index where object_id=16631) and (grantee_id in (select group_id from group_member_index where member_id=448) or grantee_id=448);
Total runtime: 444.026 ms
TEST B-2)
explain analyse select 1 from acs_permissions where privilege in (select descendant from acs_privilege_descendant_map where privilege='admin') and object_id in (select ancestor_id from acs_object_context_index where object_id=16631) and (grantee_id in (select group_id from group_member_index where member_id=-1) or grantee_id=-1);
Total runtime: 104.635 ms
The result is a similar TEST A result but a significantly faster TESTB at a little over half the time.
I was disappointed by this but since it is the handling of views that seems to give rise to the variance between versions of PostgreSQL I was keen to find a way to improve things.
Since we only ever want a true or false result from a permissions request, it ocurred to me that we might make use of the 'LIMIT 1' construct. Perhaps the optimiser will be clever enough to do what Perl does when executing boolean statements and break of execution as soon as the condition has been fulfilled.
Here are the results using limit with my experimental query:
TEST A-3)
explain analyse select 1 from acs_permissions where privilege in (select descendant from acs_privilege_descendant_map where privilege='admin') and object_id in (select ancestor_id from acs_object_context_index where object_id=16631) and (grantee_id in (select group_id from group_member_index where member_id=448) or grantee_id=448) limit 1;
Total runtime: 11.749 ms
TEST B-3)
explain analyse select 1 from acs_permissions where privilege in (select descendant from acs_privilege_descendant_map where privilege='admin') and object_id in (select ancestor_id from acs_object_context_index where object_id=16631) and (grantee_id in (select group_id from group_member_index where member_id=-1) or grantee_id=-1) limit 1;
Total runtime: 104.808 ms
Aha! ![]()
No change to the time taken for a search when there are no permissions entries for obvious reasons but a very significant improvement for queries involving users with lots of aggregated permissions.
So now the results for using 'LIMIT 1' with the original query:
TEST A-4)
explain analyse select 1 from acs_object_party_privilege_map ppm where ppm.party_id=448 and ppm.privilege='admin' and ppm.object_id=16631 limit 1;
Total runtime: 0.891 ms
TEST B-4)
explain analyse select 1 from acs_object_party_privilege_map ppm where ppm.party_id=-1 and ppm.privilege='admin' and ppm.object_id=16631 limit 1;
Total runtime: 393.342 ms
Double WOW!
...the query for party_id 448 is another 10x faster, however the query for 'the public' is of course no quicker.
This seems to throw up the following thoughts which I have summarised below and would welcome comments on.
1) Perhaps we could employ these different query structures depending upon the anticipated situation. For example, in admin pages or in the project manager application (where documents are to be restricted to a small number of parties) we could use the query shown in example TEST 4-A, whilst for nodes more likely to be viewed publically and edited infrequently (such as a news item) it may be more appropriate to use the structure proposed in TEST B-2.
2) Analysis of the query execution plans reveals an opportunity for performance improvement by recreating the privilege hierarchy table to utilise integer keys. Because the acs_privilege_descendent_map table is currently in plain english, the query plan calls for a 'text filter' comparison on every row in the nested loop. In the documentation for PostgreSQL it states that Unicode support significantly impacts performance. Since the internationalisation support in recent versions of OpenACS requires Unicode to be compiled into PostgreSQL, it could be that there are gains to be made here by eliminating the need for unicode supporting text filters in the permissions infrastructure. The query execution result seems to support this conclusion.
3) At the moment there is no index on the privilege column of the acs_privilege_descendent_map table. Addition of an index on this field may provide an incremental improvement over 2) above.
If I think of anything else I will post it.
I'd be very grateful for input from others.
Regards
Richard
Request notifications