Cacheonix in Public Cloud Environments without Multicast

Posted: July 2nd, 2016 | Author: | Filed under: Distributed Caching

Multicast is an amazing protocol when used wisely because it provides a medium for delivering messages in a group with minimal network overhead. On the ideal network a single packet maybe enough to deliver a message to thousands of hosts.

Multicast has its disadvantages. First, the packet delivery is not guaranteed, and Cacheonix addresses this by building a reliable multicast protocol on top of the standard unreliable multicast. Second, multicast has a potential for abuse becuase it’s easy to write a program that floods the network with UDP packets and even makes it impossible to communicate of the network. Unlike TCP/IP, there is nothing in standard multicast that prevents this. TCP/IP has a built-in collaborative features such as back-off in case of errors, and multicast being a lower-level protocol has none. It’s this second disadvantage that most likely forces some network administrators and all public cloud providers such as Amazon with its AWS and Google with its GAE to disable support multicast on their networks.

The good news is that Cacheonix takes in account the lack of support for multicast by public cloud providers and some LAN environments. To address this issue we developed a multicast-over-TCP/IP protocol. What’s cool about it that it allows to designate some hosts as “known addresses”, so whenever a new host joins a Cacheonix cluster or members of the cluster need to send a message to the group, the protocol will use those known-address hosts as broadcasters for the cluster. To ensure resiliency of the infrastructure it makes sense to designate more then one known-address host. This way if a broadcaster fails, other broadcasters can continue to support normal cluster operation. You can learn about configuring multicast-over-TCP/IP here.

See also:

  • Multicast Standard
    • (No) Comments: Post a response

Caching web requests on Tomcat, Jetty and JBoss

Posted: May 31st, 2016 | Author: | Filed under: Java web cache

We come to developing Cacheonix web cache after having to ask ops for adding caching to Apache/Ngnx and never getting it. It was time to take matters back to dev hands. We wanted a zero-effort, ideally zero-conf, cache for the content generated by web apps. That’s how Cacheonix web cache came to be. All you need to do is to add Cacheonix to your project:

<dependency>
    <groupId>org.cacheonix</groupId>
    <artifactId>cacheonix-core</artifactId>
    <version>2.2.2</version>
<dependency>

and then add Cacheonix filter to web.xml and you are all set:

<filter>
    <filter-name>CacheonixWebCache</filter-name>
    <filter-class>org.cacheonix.cache.web.RequestCacheFilter</filter-class>
</filter>

...

<filter-mapping>
       <filter-name>CacheonixWebCache</filter-name>
       <url-pattern>*.htm</url-pattern>
</filter-mapping>

Zero-conf set up caches the request, automatically generates client cache headers If-Modified-Since and uses gzip compression if the client supports gzip compression.

Check out the docs for in-depth background and advanced configuration: http://wiki.cacheonix.org/display/CCHNX20/Cacheonix+Web+Cache

Let us know what you think.

(No) Comments: Post a response

What is scalability, exactly?

Posted: May 11th, 2016 | Author: | Filed under: Scalability

Long story short, scalability is an ability of an application to process more requests by adding more computing resources. If your Java application can handle more work if you add more CPU, RAM or disk to your server, it’s called vertical scalability. In other words, your application scales vertically if it runs better if you give it a bigger, fatter server.

Horizontal scalability, on the other hand, is an ability to process more requests through adding more machines to your cluster. A cluster could be a simple bunch of independent servers or AWS or GAE machines behind a load balancer or an advanced infrastructure where servers communicate with each other to share work.

Note that scalability is not is performance. Performance is a number of requests per second. And while we are here, capacity is a maxim number of requests per second that your app can do without degradation of latency. And latency is the time between the client sends a request and the time it gets the response back.

(No) Comments: Post a response

Using Distributed ConcurrentHashMap

Posted: March 21st, 2016 | Author: | Filed under: Distributed Caching

Before, if you wanted to make atomic modifications to the distributed hash table provided by Cacheonix, you could use the distributed locks, such as in the example below:


      Cacheonix cacheonix = Cacheonix.getInstance();
      ReadWriteLock readWriteLock = cacheonix.getCluster().getReadWriteLock();
      Map map = cacheonix.getCache(mapName);
      Lock lock = readWriteLock.writeLock();
      lock.lock();
      try {
         if (!map.containsKey(key)) {
            return map.put(key, value);
         } else {
            return map.get(key);
         }
      } finally {
         lock.unlock();
      }

That approach worked but it required multiple network round trips to get things done. The latest Cacheonix release adds a distributed implementation of ConcurrentHashMap. Now you can modify distributed hash maps atomically, with minimal latency, by using the new JDK-compatible distributed ConcurrentMap API:


     Cacheonix cacheonix = Cacheonix.getInstance();
     java.util.concurrent.ConcurrentMap concurrentMap = cacheonix.getCache(mapName);
     return concurrentMap.putIfAbsent(key, value);


Distributed ConcurrentMap by Cacheonix supports all four methods of ConcurrentMap:

  • V putIfAbsent(K key, V value);
  • boolean remove(Object key, Object value);
  • boolean replace(K key, V oldValue, V newValue);
  • V replace(K key, V value);

Hope this helps.

Regards,

Slava Imeshev
Cacheonix Open Source Java Cache

(No) Comments: Post a response

Resolving Java Performance Problems in Production

Posted: February 20th, 2016 | Author: | Filed under: General

Performance problems in production can be tricky. This post describes a proven sequence for resolving urgent production performance problems for Java.

Sources of Performance Problems

The main sources of the performance problems in production are:

  • coding errors
  • internal and external bottlenecks
  • garbage collection (GC)

Coding errors are places in the code where the application acts in a way that leads to significant reduction of speed such as heavy repeated calculations inside a loop. Internal bottlenecks are improper, excessive synchronization, exhausted thread pools and connection pools. External bottlenecks are serially accessed data sources such as databases, file and socket I/O.

Manifestation of Performance Problems

Performance problems in production usually manifest themselves as users complaining about the application being ‘slow’. While ‘slow’ may be a matter of perception, generally it means that the application is abnormally slow. Performance problems may seriously damage your company business and must be resolved as soon as possible. The problem resolution sequence below worked well for me for many years.

Sequence for Resolving Performance Problems

1. Make sure JVM has correct heap settings. The maximum and minimum head should be set to the same value. Example: -Xmx1000k -Xms1000k.

2. Check memory consumption on the server. Make sure that the system is not swapping. Run top or ps to check if the server as a whole and the application itself is not swapping. Increase memory on the server if it doesn’t have enough memory or move some applications out.

3. Check server CPU usage. If the server is underloaded, this is usually an indication of an internal or external bottleneck. Thread dumps, heap dumps and profiling must help to identify the bottlenecks.

4. Take and analyze thread dumps. This method is great because it doesn’t require any instrumentation, takes a minute, and often is enough to isolate the problem. During the periods of slowness, take 5-6 thread dumps. Just send kill -3 if your application is running under *nix. Separate each thread dump by a 10 second interval. Check the thread dumps for places where threads remain within the same execution stack for extended periods of time. Most of the times the problem stack traces will contain application classes. Watch for thread pools all stuck in file or network I/O. Watch for exhausted thread pools. Watch for execution threads blocked on the same objects. Fix found problems.

5. Take and analyze heap dumps. Take 2-3 heap dumps during normal operation using JMap and during the system slowing down or JVM being completely unresponsive. Analyze the heap dumps using JProfiler for objects having large number of instances. Also, compare a normal heap dump with the one taken during the slowdown. Find objects counts that grew significantly since the period of normal operation. Such objects strongly indicate a memory leak. Find and fix the problem in the code.

6. Profile while running a load test. In the QA environment, run a load test the simulates the same sequence of user actions that production users are having a problem with. Use a profiler such as JProfiler to identify CPU and allocation hot spots. Fix found problems.

7. Profile in production. Profiling adds some overhead, so the application will run slower than normally. Use a single server to profile if running a cluster. To minimize overhead, configure JProfiler properly:

  • Use sampling instead of instrumentation
  • Add a narrow list of packages that are interesting from a user-perspective
  • Do not record allocations
  • Only record CPU when there are reports from users about performance issues

8. Tune GC. Enable GC in production by using command line option –XX:+PrintGCDetails. Watch for major GCs occurring abnormally often. The main causes of frequent major GCs are the application running out of heap due to memory leaks and excessive object creation and incorrect GC parameters. Profile the application for memory leaks and excessive object creation. Fix found problems. Adjust GC parameters for better performance. Tuning GC should be the last thing to do. Check our collection of useful resources on tuning Java GC.

9. Cache hard to get data. If the performance bottleneck is found and it is a database or other serially accessed data source or a hard-to-compute data source, cache that data. Here are a few articles on how to approach caching. Or drop a message in Cacheonix support forum; we’ll try to help.

Hope this helps

Slava Imeshev

(No) Comments: Post a response

Presenting on Scaling Java Applications at Code Camp 2012

Posted: September 21st, 2012 | Author: | Filed under: General

This is a great chance to meet developers of open source distributed cache Cacheonix in person. I will be presenting on scaling Java applications horizontally at two separate times on Sunday, October 7 at Silicon Valley Code Camp. Click on the links below to register:

  1. Architecture for scaling Java applications to multiple servers
  2. Developing scalable Java applications with open source clustered cache Cacheonix

Silicon Valley Code Camp is a new type of community event where developers learn from fellow developers. It is totally free and will occur on October 6 and 7.

See you at Code Camp 2012!


Regards,

Slava Imeshev
Main committer, Cacheonix

(No) Comments: Post a response

Cacheonix as Distributed DataNucleus L2 Cache

Posted: September 18th, 2011 | Author: | Filed under: Distributed Caching

DataNucleus can now use Cacheonix as a distributed L2 Cache.

DataNuclues access platform is an open source standards-based JPA API that provides persistence and query services for  a wide set of datastores. Andy Jefferson, the project lead at DataNucleus.org, has let me know today that DataNucleus Access Platform has added support for Cacheonix. Now DataNucleus applications can scale out with ease by using Cacheonix as a distributed L2 cache.

It’s great to see growing adoption of Cacheonix by persistence frameworks and we are looking forward to further collaboration with DataNucleus.

See Also:

Regards,

Slava Imeshev

(No) Comments: Post a response

How to Store a List in a Cache

Posted: September 3rd, 2011 | Author: | Filed under: General

Storing a list in a cache and getting it from the cache is easy. Here is how.

The simplest way is just to put the list to the cache. Note that if you want to use a strongly typed cache, you need to provide a concrete list type:


import java.util.ArrayList;
import org.cacheonix.Cacheonix;
import org.cacheonix.cache.Cache;

...

   /**
    * Puts a list to the cache.
    *
    * @param key  a key identifying the list.
    * @param list the list to cache.
    */
   public void putListToCache(String key, ArrayList<String> list) {

      final Cache<String, ArrayList<String>> listCache = Cacheonix.getInstance().getCache("listCache");
      listCache.put(key, list);
   }


   /**
    * Gets a list from the cache.
    *
    * @param key a key identifying the list.
    * @return the cached list or null if the list wasn't found in the cache or if the list is null.
    */
   public ArrayList<String> getListFromCache(String key) {

      final Cache<String, ArrayList<String>> listCache = Cacheonix.getInstance().getCache("listCache");
      final ArrayList<String> list = listCache.get(key);
      return list;
   }

While the above works, there is a better way, which is creating a first-class object for the cached list.

Usually lists don’t come alone and are accompanied by some metadata. For example, if you are caching a database result set, in future you might want to cache time it took to retrieve the result set in addition to the result set itself. Wrapping the list into a class provides an ability to maintain a stable signature of the cached object. If you need to add more fields in the future, you don’t have to change all places where a cached list is put to and retrieved from the cache.

A cached result set object that contains a list would then look like the following. Note that it declares a signature interface Immutable that signals the cache that this object can be passed by reference:


import java.io.Externalizable
import org.cacheonix.cache.Immutable;

...

  public class CachedResultSet implements Externalizable, Immutable {

    private List<String> list = null;


    public CachedResultSet(List<String> list) {
      this.list = new ArrayList<String>(list);
    }


    /**
     * Required by Externalizable.
     */
    public CachedResultSet() {
    }

    public List getList() {
      return new ArrayList(list);
    }

    public void writeExternal(ObjectOutput objectOutput) throws IOException {

      if (list == null) {

        objectOutput.writeBoolean(true);
      } else {

        objectOutput.writeInt(list.size());
        for (int i = 0; i < list.size(); i++) {

          objectOutput.writeUTF(list.get(i));
        }
      }
    }


    public void readExternal(ObjectInput objectInput) throws IOException, ClassNotFoundException {

      if (!objectInput.readBoolean()) {

        int size = objectInput.readInt();
        list = new ArrayList<String>(size);
        for (int i = 0; i < size; i++) {

          list.add(objectInput.readUTF());
        }
      }
    }
  }

Getting the cached result set would look like this:

   public CachedResultSet getListFromCache(String key) {

      final Cache<String, CachedResultSet> listCache = Cacheonix.getInstance().getCache("listCache");
      final CachedResultSet list = listCache.get(key);
      return list;
   }

Regards,

Slava Imeshev

(No) Comments: Post a response

Per-select Caching for MyBatis

Posted: August 21st, 2011 | Author: | Filed under: Distributed Caching

Good news: We’ve just added support for per-select caching for MyBatis to our open source Java cache Cacheonix that addresses major problems with default MyBatis caching.

Problems With Default MyBatis Caching

There are a few problems with the way caching works in MyBatis:

  • MyBatis keeps results of all selects in a single, namespace-wide, cache. This means any invalidation caused by inserts or updates will flush cached results of all selects, including those that are really not affected by the changes.
  • It is impossible to ignore invalidation completely and to have pure time-only expiration for some selects.
  • It is impossible to have different cache configurations for particular selects.

Benefits of Cacheonix Adapter for MyBatis

Cacheonix have addressed these concerns by adding the following features:

  1. Per-select MyBatis caches
  2. Ability to turn off invalidation
  3. Ability to turn off namespace cache
  4. Cache templates for select caches

For more information and examples check Cacheonix documentation: Configuring MyBatis Cache Adapter.

Enjoy!

Regards,

Slava Imeshev

(No) Comments: Post a response

Testing Private Methods in Java

Posted: August 19th, 2011 | Author: | Filed under: General

Java 5 provides vargargs API that allows to test private methods in a very concise way. Here is how.

Yesterday, during a meetup at Loopt, a question of an API for testing private methods was brought up. I used to use JUnit add-on PrivateAccessor [1] when I was testing private methods. The problem was that applying PrivateAccessor produced prohibitively verbose code, so you had to think really hard to understand what the test was doing.

invoke() with varargs

Luckily, Java 5 provides vargargs API that allows to write really concise code for invoking private methods and return results. Here is the invoke() method that makes use of vargargs. Feel free to use it in your code:

   /**
    * Invokes a private method on an object.
    *
    * @param obj        the object on what to invoke the method.
    * @param methodName the name of the method to invoke.
    * @param parameters a list of parameter
    * @return the result of dispatching the method represented by the object.
    */
   public static <T> T invoke(Object obj, String methodName, Object... parameters) {

      try {

         // Create a list of parameter classes
         final Class[] parameterTypes = new Class[parameters.length];
         for (int i = 0; i < parameters.length; i++) {
            parameterTypes[i] = parameters[i].getClass();
         }

         // Get method
         Class clazz = obj.getClass();
         Method method = clazz.getDeclaredMethod(methodName, parameterTypes);
         method.setAccessible(true);

         // Call method
         return (T)method.invoke(obj, parameters);
      } catch (RuntimeException e) {

         throw e;
      } catch (Exception e) {

         throw new IllegalArgumentException(e);
      }
   }

Complete Example

Below is a complete JUnit tests that tests StringBuilder’s private method append(). Notice how neatly the invocation of append() fits into a short single line:

import java.lang.reflect.Method;
import junit.framework.TestCase;

public final class StringBuilderTest extends TestCase {


   /**
    * Object under test
    */
   private StringBuilder object;


   /**
    * Tests String's private method 'append(StringBuilder sb)'.
    */
   public void testAppend() {
      
      StringBuilder toAppend = new StringBuilder("Test");
      StringBuilder result = invoke(object, "append", toAppend);
      assertEquals("Test", result.toString());
   }


   /**
    * Invokes a private method on an object.
    *
    * @param obj        the object on what to invoke the method.
    * @param methodName the name of the method to invoke.
    * @param parameters a list of parameter
    * @return the result of dispatching the method represented by the object.
    */
   public static <T> T invoke(Object obj, String methodName, Object... parameters) {

      try {

         // Create a list of parameter classes
         final Class[] parameterTypes = new Class[parameters.length];
         for (int i = 0; i < parameters.length; i++) {
            parameterTypes[i] = parameters[i].getClass();
         }

         // Get method
         Class clazz = obj.getClass();
         Method method = clazz.getDeclaredMethod(methodName, parameterTypes);
         method.setAccessible(true);

         // Call method
         return (T)method.invoke(obj, parameters);
      } catch (RuntimeException e) {

         throw e;
      } catch (Exception e) {

         throw new IllegalArgumentException(e);
      }
   }


   public void setUp() throws Exception {

      super.setUp();

      object = new StringBuilder(10);
   }
}

Making invoke() reusable

To make invoke() reusable, I suggest to move it to a separate utility class or at least to create a super class for all your tests that extends junit.TestCase and move invoke() there.

Regards,

Slava Imeshev

References

1. JUnit Add-On PrivateAccessor

(7) Comments: Post a response