The objectClass shouldn't matter since in Test 3 above I'm filtering solely on memberOf. For completeness sake I tested it anyway, but same results as expected. Users in my directory all have the following objectClass attributes, so it doesn't really matter which one I use:
objectClass: top
objectClass: person
objectClass: organizationalperson
objectClass: inetorgperson
objectClass: inetuser
objectClass: posixaccount
objectClass: krbprincipalaux
objectClass: krbticketpolicyaux
objectClass: ipaobject
objectClass: ipasshuser
objectClass: ipaSshGroupOfPubKeys
objectClass: mepOriginEntry
I changed the filter as suggested with the following:
update ofproperty
set propvalue = '(&(objectClass=organizationalperson)(memberOf=cn=openfire-users,cn=groups,cn=a ccounts,dc=REDACTED,dc=com))'
where name = 'ldap.searchFilter';
Authentication attempts all fail as I described above with no verbose log output (just a simple authentication failure for admin user).
Using ldapsearch with the proposed filter works as expected, but again this is the same as all my tests. It's only Openfire that doesn't work with the filter.
ldapsearch -Y GSSAPI -b 'cn=users,cn=accounts,dc=REDACTED,dc=com' '(&(objectClass=organizationalperson)(memberOf=cn=openfire-users,cn=groups,cn=a ccounts,dc=REDACTED,dc=com))'
<trimmed output>
# search result
search: 4
result: 0 Success
# numResponses: 7
# numEntries: 6
Returns all 6 users that are in the openfire-users group.