For each community a 'non member' portal_id is created (automatically) with different elements/portlets. You will see if you (select * from portals;).
Non-members will be shown a 'non member' portal. If your portlet is not associated with this portal_id, it will not be shown to 'non members'.