

Exposing JBossCache Management Attributes
=========================================


Author: Jerry Gauthier (jgauthier@novell.com)
Revision: $Id: ManagementAttributes.txt,v 1.4 2006/01/19 00:15:16 jerrygauth Exp $
JIRA issue: http://jira.jboss.com/jira/browse/JBCACHE-56


Background
----------
The intent of this issue is to provide common cache characteristics such as 
hit/miss ratio, read/write ratio, number of cached nodes and attributes, etc.
The issue's comments provide an extensive list of cache characteristics that 
are considered desirable to expose.  It's also proposed that this 
implementation expose cache notifications via JMX in addition to the current 
TreeCacheListener implementation.

Design Overview
---------------
The initial design focus is on how to incorporate this capability into 
JBossCache.  Once the design work and initial implementation is complete,
it should be possible to review the list of attributes to be monitored and 
incorporate them over time, expanding or contracting the list as appropriate.
Initially, the implementation will focus on simple characteristics such as 
number of nodes, number of attributes, number of hits, number of misses, 
and number of puts.

Design Alternatives
-------------------
The simplest approach would be to add code to TreeCache to capture the 
necessary statistics and then expose them through the TreeCacheMBean.  
This is already in place for *number of nodes* and *number of attributes.*  
The obvious disadvantages here are that TreeCache.java is already quite 
large and adding numerous new methods to the TreeCache MBean will further 
clutter it and make it difficult for users to readily locate relevant 
statistics.

An  alternative approach here is to provide a new MBean named 
TreeCacheManagement MBean that will be used to expose the  management 
statistics.  This MBean will be similar to the existing TreeCacheView MBean; 
it will be an optional MBean that's dependent on and associated with a 
configured TreeCache service.  The TreeCacheManagement MBean will have a 
reference to its associated TreeCache instance so it can easily access 
statistics that are exposed by the TreeCache such as Number of Nodes and 
Number of Attributes.

Providing a new MBean resolves the second disadvantage noted above; 
the current TreeCache MBean won't be cluttered with new methods.
However the first disadvantage remains as the new logic to capture cache 
statistics will still need to be located in the TreeCache implementation.  
It will also be necessary to add public methods to the TreeCache to make 
the statistics available to the new MBean.  This could probably be 
minimized by providing the statistics via a single method (e.g., by 
returning a collection or array)  rather than using individual methods per 
statistic.

In order to minimize changes to the TreeCache class, it's possible to 
introduce a new interceptor named CacheMgmtInterceptor.  This interceptor 
can be used to capture various statistics as it intercepts TreeCache methods.  
For example, any invocation of TreeCache.getKeyValueMethodLocal() would result 
in a hit or a miss, depending on whether the method returned a null value.  
In this case, it's transparent to the TreeCache implementation that hits and 
misses are being counted.  The interceptor accumulates the statistics and 
provides methods for callers to access them.  The new TreeCacheManagement 
MBean can obtain a reference to the interceptor via the interceptor chain 
exposed by the TreeCache MBean so the statistics will be accessible to the 
new MBean.

There are at least two potential problems with the interceptor approach.  
First, some statistics may not be accessible via interceptors.  For example, 
if a desired statistic isn't the input or output of a method, interception 
probably won't be able to capture it.  In this case, it will probably be 
necessary to introduce logic into TreeCache to expose the statistic.  It's 
not clear whether this applies to any of the statistics currently of 
interest; this will be determined as they're implemented in turn.

A second potential problem relates to modifying logic flow in TreeCache.  
For example, all lookups ultimately invoke the method identified by 
TreeCache.getKeyValueMethodLocal().  If this situation changes in the future, 
it may not be obvious to the developer modifying the code that this condition 
was a prerequisite for   logic in the interceptor that ascertained hits and 
misses.  If the logic were in-line in TreeCache, it would be much more obvious 
to someone modifying the implementation.

Users should have the ability to disable the statistics gathering 
implementation if they wish to do so.  Since deployment of the 
TreeCacheManagement MBean is optional, they can easily prevent the MBean 
from being deployed.  However the interceptor still needs to be excluded 
from the TreeCache interceptor chain.  The interceptor chain is established 
when the TreeCache service is created; at this point the TreeCacheManagement 
MBean hasn't been created as it's dependent on the TreeCache MBean.  
Consequently it's not possible for the interceptor factory to detect the 
presence or absence of the TreeCacheManagement MBean and act accordingly.  
One possibility here would be for the TreeCacheManagement MBean to 
dynamically add the interceptor but it appears that the current intention of 
the TreeCache interceptor stack is to not support dynamic changes.  A simpler 
alternative would be to add a boolean attribute to the TreeCache MBean 
configuration specifying whether statistics should be maintained.  The 
interceptor factory can then ascertain whether to incorporate the management 
interceptor or not.

Design Proposal
---------------
My proposal is to use a new TreeCacheManagment MBean to expose statistics, 
beginning with simple ones such as hits and misses.  I think it makes sense to 
introduce a new interceptor rather than add the logic directly to TreeCache.  
I would add a new configuration attribute to TreeCache so that the management 
interceptor can be enabled or disabled appropriately.

TreeCache 2.0
-------------
One observation is that JBossCache 2.0 will introduce a Cache interface that 
hides the underlying implementation.  It might be preferable to incorporate 
the new management attributes directly in Treecache and TreeCacheMBean for 
now.  During 2.0 development, this can be resisted as part of the overall 
restructuring.

Attribute Definitions
---------------------
We need to define various cache management statistics in the context of 
JBossCache.  Here are my current proposals for some basic measurements.

***Store Operations***
Stores are used to provide the read/write ratio for the cache. The read/write 
ratio is (hits + misses) / stores.

Putting an attribute is a store operation, reputting an attribute is a store 
operation, putting a map can be interpreted as one store operation per 
attribute in the map. What about putting a node? Under this proposal, only 
attributes puts are counted as store operations.

***Hits and Misses***
Hits and misses are used to provide the hit/miss ratio for the cache. This 
ratio is calculated as hits / (hits + misses).

A hit is a get attribute operation that ultimately results in an object being 
returned to the client. So a get operation that fails over to a custom cache 
loader is a hit if the cache loader returns the requested object. A miss is 
any get attribute operation that fails to return the requested object.

What about getting nodes rather than attributes? Under this proposal, only 
getting attributes are counted as hits or misses. This may be misleading if 
the user gets the data map for a node and then reads it. Any thoughts on 
how this should be treated?

Final Design
------------
The final design utilizes one mbean per interceptor.  Each interceptor can 
optionally expose its management attributes/statistics via a corresponding 
mbean.  The basic cache statistics (time running, hits, misses, ...) are 
provided by a new interceptor named CacheMgmtInterceptor.

A boolean flag named useInterceptorMbeans has been added to TreeCache to 
specify whether mbeans should be registered (and statistics maintained) for 
the cache.  

If the flag is enabled, each interceptor has its associated mbean 
registered dyanmically when the cache is started, occurring after after the 
interceptor stack has been initialized.  If an interceptor doesn't provide its 
own mbean, a base mbean is provided so that the interceptor will always be 
represented in the cache interceptor mbean hierarchy displayed by the mbean 
server.

If the cache is running in standalone mode, TreeCacheMBean will also be 
registered dynamically so that it will be included at the top of the cache's 
mbean hierarchy.

Each interceptor's mbean exposes a boolean flag that can be used to 
enable/disable statistics gathering for the interceptor.

By default, the useInterceptorMbeans flag is enabled so that interceptor mbeans 
will be registered unless explicitly configured otherwise.

See the document entitled InterceptorMBeans for futther details on the 
interceptor mbean registration process.

See the document entitled ManagementAttributesStatus for a list of attributes 
and operations exposed or planned for exposure via cache interceptor mbeans.
 

