Tweaking identity cache performance for the K2 Server

Configuration settings for optimizing Identity cache performance for the K2 Server.

The Identity Service is a new cache mechanism that has been introduced from K2 4.5 KB001370. This new cache mechanism is designed to work with any user provider or group provider which has been configured to work with K2, such as Active Directory users and groups, SharePoint Groups and K2 roles. Previously, one configured the Cache Timeout value via the Active Directory User Manager (ADUM) settings. Note that this will now be set to zero for future releases to disable the ADUM cache.

Backend database tables

To examine the internals of the Identity Framework, we will discuss the following backend tables used in the K2HostServer database:

Identity.CacheConfiguration
Identity.RoleItem
Identity.Identity
Identity.IdentityMember
Identity.IdentityUpdate

If any database tables are to be edited, it is recommended that a database backup be created beforehand.

Database table values which are not mentioned in this topic are not user-configurable.

The Identity.CacheConfiguration table

The [Identity].[CacheConfiguration] table sets the time parameters of the Identity Provider (K2, K2SQL, CUSTOM) cache settings and the Microsoft SQL command timeout setting. The default properties of these settings are as follows:

These settings may be defined in milliseconds (ms), seconds (s), minutes (m), hours (h) or days (d).

If a value of 0s is entered, the relevant setting will be turned off. Time values which are greater than default will increase the time between reloading and reduce the load on the server, but will increase information latency.

The three types of Identities that are cached within the K2 Server are the User, the Role, and the Group.

The CacheTimeout settings set when the cached properties of that identity will expire. After this configured time period has expired, the K2Server will resolve the identity against the provider.

In regard to the dynamicCacheTimeout, a normal user will be resolved on the thread according to the CacheTimeout configuration. Dynamic identities will be excluded from this timed resolving and will resolve on demand. When a worklist item is opened, all dynamic identities will be resolved before the server queries the current user membership (in case the user is included in the result). For example, a dynamic identity might be an online user whose identity needs to be checked every time the worklist is requested. This has a performance impact on the worklist, and should only be used for true dynamic cases. The dynamicCacheTimeout setting therefore configures the time to expiration of the cached dynamic identity.

The ContainersCacheTimeout setting sets the time until expiry of the K2 server cache that contains the external relational identity settings (i.e. all the groups that a particular user or group also belongs to).

The MembershipCacheTimeout setting sets the time until expiry of the K2 server cache that contains the internal relational identity settings (i.e. all the users within a role or group).

The resolvedExpiredIdentities setting sets the period of time in which the server checks for expired identities and resolves them.

The sqlCommandTimeout setting sets the maximum amount of time the calling application will wait for an identity to be cached before a timeout exception is raised.

The CacheThreadCount setting is not used at this time.

The ServerLockTimeout setting sets the maximum amount of time a particular server thread should wait for a lock to be released before throwing an error that it could not acquire a lock on a particular identity for update purposes. If this time is exceeded, it will retry once before the thread is aborted, and an error is thrown in the Host Server Log files.
The default ServerLockTimeout setting is five minutes, and will offer optimal server performance. It is not recommended that this timeout setting be set to less than 60 seconds. More than five minutes could negatively affect the server’s performance, depending on factors such as server workload and time taken for the server to update the cache from Active Directory or another security label. It is recommended that you contact your regional K2 support representative for timeout-period recommendations.

The Identity.RoleItem table

This table holds the configuration of the SmartObject/Groups/Users defined in a K2 role.

The table contains records that match up with the items as configured in the Management Console.

The Role Type column may contain the following values:

User
Group
SmartObject (The Data column contains the SmartObject connection details. It also includes the method, property and filter values to use).

Here is an example of the XML string contained in the Data column:

<smartobject name="TestUsers" guid="679a3cfd-eb29-4a8c-bce5-7e72857a851b" connectionString="Integrated=True;IsPrimaryLogin=True;Authenticate=True;EncryptedPassword=False;Host=blackpearl;Port=5555" methodName="GetList" isListMethod="True" propertyName="UserID" expectedType="1" xmlns="<a href="http://schemas.k2.com/roles/smartObjectdefinition"><inputs><input'>http://schemas.k2.com/roles/smartObjectdefinition"><inputs><input name="TestUsers" type="0">denallix\codi</input></inputs></smartobject>

From the Identity.RoleItem table, the IdentityID column links back to the Identity.Identity table ID column for the role's cache settings. The Identity.RoleItem table acts as the definition for the role, and the Identity.Identity table stores the role item and also the members. The linkage between the role and the members is contained in the Identity.IdentityMember table.

The Identity.Identity table

This table stores the cache expiry information for the User/Groups/Roles. You can force expire the relevant User/Group/Role cache by modifying the relevant expiry datetime value.

For every item, there are three different expiry timestamp fields (ExpireOn, MembersExpireOn and ContainersExpireOn). There are also three main methods on the Identity Service (GetIdentity, GetIdentityMembers, and GetIdentityContainers). These relate to the three expiry timestamp fields mentioned above.

ExpireOn will be updated if the GetIdentity method is called (e.g. a Get User Details call on UMUser SmartObject).
MembersExpireOn will be updated when GetIdentityMembers is called. (e.g. Get Group Users call on UMUser SmartObject).
ContainersExpireOn will be updated when GetIdentityContainers is called (e.g OpenWorklist call). This will return all the groups and roles for a user and recursively get roles and groups for each of the direct containers.

For a group, notice that the MembersExpireOn and ExpireOn fields are refreshed when a group is requested to be resolved (for example, when a new client event is hit which utilizes the group (refreshes the group membership and group properties respectively), or an E-mail event that sends an E-mail to the specific group). Other possibilities include the UMUser SmartObject when executing the GetGroupUsers method. On the other hand, when a user accesses his task list, the user’s ExpireOn and the ContainersExpireOn fields are refreshed. ExpireOn refreshes the identity Properties XML field and ContainersExpireOn refreshes the groups and roles that the user belongs to. The MembersExpireOn field does not change, and its value originates from the first time the Identity Service is used for the user.

If you want to refresh cache items, set all the three expiry dates to some time in the past for the relevant identity items.

Also note that there is a Dynamic flag for each record. Currently, this only applies to K2 roles. If it is set to True, it will ignore the CacheTimeout settings and instead use the dynamicCacheTimeout settings (30s by default). Queries will be queried fresh after 30 seconds from the last query. This is useful for keeping task items current when users are removed from or added to the K2 role. However, there will be a performance impact when identities are resolved, as all dynamic identities are refreshed before members or containers are selected.

Do not delete values from the Identity.Identity table. Other K2 modules rely on the ID’s in this table. In the future, other components will also rely on these ID’s in regard to users/groups/roles.

This is also where the K2 roles are saved. If the records are deleted, all roles will be deleted as well.

The Identity.IdentityMember table

This table holds the linkages between the Roles/Groups and the individual users (i.e. a many-to-many relationship). In the Identity.IdentityMember table, both the ContainerID and MemberID columns link back to the Identity.Identity table ID primary key column. If you are familiar with K2.net 2003 destination queues (which is the predecessor to K2 roles), you will note that this is similar to the relationship between the DestQueue and DestQueueUser tables in the K2Server database. The only difference is that it stores the ID of the user instead of the user name (for performance reasons).

The Identity.IdentityUpdate table

This table should be empty most of the time. It is used only when identities are updated. The results from the providers are moved to this table (Bulk Insert) and the Identity.Identity and Identity.IdentityMember table will be updated in transaction. This table is cleared once the update has completed. You will see that this table is used by the bulk container and member update stored procedure calls.