Monday, November 4, 2013

Monitoring your Tomcat application server, or any other Java process, using JMX and OpenNMS

This post will show you how to monitor a Tomcat application server using JMX MBeans and an SNMP management application. The first part will guide you through securely enabling JMX management on the Java process. Once you have done this, you can use tools like Visual VM to monitor your process and its MBeans. If you want to take a step further and have this data gathered all the time, you might want to continue with the second part and integrate it with OpenNMS.

Requirements

It is assumed that you have Tomcat on a server you want to manage, and you have also got Sun Java 7 SDK (not JRE) to run on your machine (they both can be the same machine, but we keep them separated here). You can apply this to any other Java process with little modification (to the MBean names maybe).

Enable JMX on Server

Add a few Java arguments to your CATALINA_OPTS to enable remote connection to JMX, and then restart the application server:

export CATALINA_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=1100 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.password.file=<JAVA_HOME>/jre/lib/management/jmxremote.password"

Replacing <JAVA_HOME> with the appropriate path for your Java, copy <JAVA_HOME>/jre/lib/management/jmxremote.password.template into <JAVA_HOME>/jre/lib/management/jmxremote.password and set the passwords for monitorRole and controlRole at the end of the file as you wish. We will assume you have set them to MONITOR_PASS and CONTROL_PASS here .

Make sure that your firewall doesn't block port 1100. You may change the port number as you wish, but remember to change it everywhere mentioned in this guide.

Now run a JMX utility to connect to your JMX server, like JConsole or Visual VM. If using JConsole, connect to a remote host providing hostname, port (1100), username (controlRole), and password (CONTROL_PASS). If using Visual VM with a VisualVM-MBeans plugin installed, you might be able to browse among the MBeans and call some operations (click to enlarge):




This might be enough if you want to see what's going on in the JVM at the moment. You can even double click the numbers (only the ones highlighted) to see a graph over time.



But if you want to keep historical records of different things over time, keep reading. We will be demonstrating how to use a network management software to connect to JMX and gather information. This will be based on an open source application called OpenNMS, but you can probably use your own software so long as it supports JSR160.

Install OpenNMS

This document provides instructions for Fedora 16. Please follow the OpenNMS installation guide for other operating systems.

Add OpenNMS repository.
rpm -Uvh http://yum.opennms.org/repofiles/opennms-repo-stable-fc16.noarch.rpm

Install, init, and start PostgreSQL server, if you don't have one.

yum -y install postgresql postgresql-server
/sbin/service postgresql initdb
/sbin/service postgresql start
/sbin/chkconfig postgresql on

To allow OpenNMS which is run as root to connect to PostgreSQL as opennms, you need to relax some access requirements. Edit /var/lib/pgsql/data/pg_hba.conf and make sure you have entries like this (change the last column):

local   all         all                               trust
host    all         all         127.0.0.1/32          trust
host    all         all         ::1/128               trust

Now restart PostgreSQL for these changes to take effect:

/sbin/service postgresql restart

Now you are ready to install OpenNMS:

yum -y install opennms
/opt/opennms/bin/runjava -S /usr/java/latest/bin/java
/opt/opennms/bin/install -dis
/sbin/service opennms start

Discover and Configure

If you have followed the instructions in the previous section, you are now able to access the web interface of OpenNMS via http://<opennms-hostname>:8980/opennms/. Enter admin for both username and password when prompted. Just remember to open port 8980 in your firewall for hosts you will be accessing OpenNMS web interface from.
The easiest way to set up OpenNMS to monitor a JMX service is to hijack the configuration to set up its own JMX interface. The only things we need to change is port number, username, password, and a few names that are different in Tomcat. Please see this wiki page for an explanation.
Please modify your configuration files stated below to make sure that it matches what provided here.

/opt/opennms/etc/capsd-configuration.xml

<protocol-plugin protocol="OpenNMS-JVM" class-name="org.opennms.netmgt.capsd.plugins.Jsr160Plugin" scan="on" user-defined="false">
    <property key="port" value="1100" />
    <property key="factory" value="PASSWORD-CLEAR"/>
    <property key="username" value="controlRole" />
    <property key="password" value="CONTROL_PASS" />
    <property key="protocol" value="rmi"/>
    <property key="urlPath" value="/jmxrmi"/>
    <property key="timeout" value="3000" />
    <property key="retry" value="2" />
    <property key="type" value="default" />
</protocol-plugin>

/opt/opennms/etc/collectd-configuration.xml

<service name="OpenNMS-JVM" interval="300000" user-defined="false" status="on">
        <parameter key="port" value="1100"/>
        <parameter key="factory" value="PASSWORD-CLEAR"/>
        <parameter key="username" value="controlRole" />
        <parameter key="password" value="CONTROL_PASS" />
        <parameter key="retry" value="2"/>
        <parameter key="timeout" value="3000"/>
        <parameter key="protocol" value="rmi"/>
        <parameter key="urlPath" value="/jmxrmi"/>
        <parameter key="rrd-base-name" value="java" />
        <parameter key="ds-name" value="opennms-jvm"/>
        <parameter key="friendly-name" value="opennms-jvm"/>
        <parameter key="collection" value="jsr160"/>
        <parameter key="thresholding-enabled" value="true"/>
</service>

/opt/opennms/etc/poller-configuration.xml

<service name="OpenNMS-JVM" interval="300000" user-defined="false" status="on">
  <parameter key="port" value="1100"/>
  <parameter key="factory" value="PASSWORD-CLEAR"/>
  <parameter key="username" value="controlRole"/>
  <parameter key="password" value="CONTROL_PASS"/>
  <parameter key="retry" value="2"/>
  <parameter key="timeout" value="3000"/>
  <parameter key="rrd-repository" value="/opt/opennms/share/rrd/response" />
  <parameter key="ds-name" value="opennms-jvm"/>
  <parameter key="friendly-name" value="opennms-jvm"/>
</service>

/opt/opennms/etc/jmx-datacollection-config.xml

...
<mbean name="JVM MemoryPool:Eden Space" objectname="java.lang:type=MemoryPool,name=PS Eden Space">
...
<mbean name="JVM MemoryPool:Survivor Space" objectname="java.lang:type=MemoryPool,name=PS Survivor Space">
...
<mbean name="JVM MemoryPool:Perm Gen" objectname="java.lang:type=MemoryPool,name=PS Perm Gen">
...
<mbean name="JVM MemoryPool:Old Gen" objectname="java.lang:type=MemoryPool,name=PS Old Gen">
...

Now restart the server after making these changes.

/sbin/service opennms restart

Once restarted successfully, go to the web interface and perform the following steps:
  • In the Admin tab, click "Add Interface for Scanning", then enter <tomcat-server-hostname> and add.
  • In the Events tab, click "All Events" and look for services being discovered.
  • In the Reports tab, click "Resource Graphs", select Tomcat server in the standard reports, select the opennms-jvm, then click "Graph Selection".



Here are your graphs.



You need to follow the same pattern to monitor other MBeans. For example to graph total compilation time we need to follow these steps.
First, add an entry in jmx-datacollection-config.xml file to query the MBean (use JConsole or JVisualVM to find the name of the MBean you are interested in):

<mbean name="JVM Compilation" objectname="java.lang:type=Compilation">
  <attrib name="TotalCompilationTime" alias="TotCompilationTime" type="gauge"/>
</mbean>

Then, add a report template in the snmp-graph.properties section:

report.jvm.compilation.name=JVM Compilation
report.jvm.compilation.columns=TotCompilationTime
report.jvm.compilation.type=interfaceSnmp
report.jvm.compilation.command=--title="JVM Compilation" \
 DEF:compilationTime={rrd1}:TotCompilationTime:AVERAGE \
 LINE2:compilationTime#0000ff:"Compilation Time" \
 GPRINT:compilationTime:AVERAGE:" Avg \\: %8.2lf %s\\n"

Finally, don't forget to add this new graph to the list of graphs (mind the semicolon and backslash at the end of the line):

reports=mib2.HCbits, mib2.bits, mib2.percentdiscards, mib2.percenterrors, \
...
jvm.gc.copy, jvm.gc.msc, jvm.gc.parnew, jvm.gc.cms, jvm.gc.psms, jvm.gc.pss, jvm.compilation, \
...

Sunday, November 3, 2013

Android app to "Google" images


Some time ago I wrote a simple Android app to search for images using Google Search API, I thought it is worth sharing in case someone else needs to do the same. I had to create a custom search engine (https://www.google.com/cse/) and create an API project (https://code.google.com/apis/console) to get set up. The rest is ordinary stuff you all know.


There is one thing to mention though: Using Java search API you have to specify the web sites you want to search in. You can't search the whole Internet using this API. You could do this using image search API, but since it is deprecated it is not worth investing in.


The complete working code along with a pre-built .apk is also provided. I have implemented other interesting features too like search suggestions and infinite loading, which will become handy down for you the road.

Simply clone https://github.com/normanatashbar/imagesearch.git or download by clicking here. Please remember to change the search engine ID and API key with your own if you are using this code as a base.

Wednesday, October 2, 2013

How to start investigating Java's OutOfMemoryError

This blog post addresses services/support people and doesn't provide too much detailed information about memory management in Java.

You get out of memory, now what? First of all you need to understand what kind of OutOfMemory is it. You may run out of OS virtual memory, native memory allocated to Java process, or Java heap. The error message normally gives a good indication of specifics, see a few examples:
 
OutOfMemoryError: Java heap space
OutOfMemoryError: PermGen space
OutOfMemoryError: unable to create new native thread
OutOfMemoryError: requested XXX bytes for ChunkPool::allocate

A typical Java process has a heap where Objects go into, which is also divided into different sections. Depending on the implementation of JVM they might be called: Eden Space (New), From Space (Survivor 1), To Space (Survivor 2), Old Generation (Tenured), and Perm Generation (it is considered outside Heap in some implementations). For a detailed explanation see this page or this page. On top it add memory required for class-loaders, garbage collection process, threads stack, JNI, native memory buffers, etc. For a detailed explanation see this page.

Here, I'm going to list what you need to know before analyzing an OutOfMemoryError:

1- What is the architecture of your machine and the JVM running your Java process: 32-bit, 64-bit, or else? How to proceed from here depends on answers to this question, because a 32-bit Java process is limited to about 3GB of usable virtual memory in user space with default settings, and 4GB in best case.
 
$ uname -a
Linux houman-laptop 3.9.10-100.fc17.x86_64 #1 SMP Sun Jul 14 01:31:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
$ java -version
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

The commands above tell me that I'm running a 64-bit JVM on a 64-bin machine.

2- How much memory is allocated to the Java process?
 
$ ps awwwxo pid,user,%mem,%cpu,vsz,rss,cmd | head -1; ps awwwxo pid,user,%mem,%cpu,vsz,rss,cmd | grep tomcat
  PID USER     %MEM %CPU    VSZ   RSS CMD
 2885 houman   10.2  1.0 3405352 826028 /usr/java/jdk1.7.0_25/bin/java ... org.apache.catalina.startup.Bootstrap start
 5774 houman    0.0  0.0 109408   872 grep --color=auto tomcat

The command above suggests that my Java process is allocated 3.4GB of memory by OS (VSZ) and is currently utilizing 0.8GB (RSS) of it.

3- How much memory is allocated to Java heap?
 
$ /usr/java/jdk1.7.0_25/bin/jmap -heap 2885
Attaching to process ID 2885, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 23.25-b01
using thread-local object allocation.
Parallel GC with 4 thread(s)
Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 536870912 (512.0MB)
   NewSize          = 1310720 (1.25MB)
   MaxNewSize       = 17592186044415 MB
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 134217728 (128.0MB)
   MaxPermSize      = 268435456 (256.0MB)
   G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
PS Young Generation
Eden Space:
   capacity = 55836672 (53.25MB)
   used     = 44561864 (42.49750518798828MB)
   free     = 11274808 (10.752494812011719MB)
   79.8075214797902% used
From Space:
   capacity = 59637760 (56.875MB)
   used     = 51424800 (49.042510986328125MB)
   free     = 8212960 (7.832489013671875MB)
   86.2285907451923% used
To Space:
   capacity = 59637760 (56.875MB)
   used     = 0 (0.0MB)
   free     = 59637760 (56.875MB)
   0.0% used
PS Old Generation
   capacity = 300941312 (287.0MB)
   used     = 124813120 (119.03106689453125MB)
   free     = 176128192 (167.96893310546875MB)
   41.47423933607361% used
PS Perm Generation
   capacity = 209780736 (200.0625MB)
   used     = 124733744 (118.95536804199219MB)
   free     = 85046992 (81.10713195800781MB)
   59.459103051292566% used
37997 interned Strings occupying 4151064 bytes.

It says my Java process has 287MB in old generation (119MB used) and 200MB in perm generation (118MB used) for example. On top it also indicates what are the ultimate limits for the Java process. When utilization of these 2 sections get high and the capacity is near the maximum available (Eden + From + To + PS Old grow towards MaxHeapSize, and PS Perm grows towards MaxPermSize), chances are you are running out of heap space, one way or another. Referring to this picture might help you understand the output better:


This command will tell you about garbage collections performed and the time spent doing so:
 
$ /usr/java/jdk1.7.0_25/bin/jstat -gcutil 2885
  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT   
  0.00   86.34  20.72  41.71  59.46 83      1.991   6      1.832    3.823

This command will tell you about biggest objects that live in heap:
 
$ /usr/java/jdk1.7.0_25/bin/jmap -histo 2885 | head -20
 num     #instances         #bytes  class name
----------------------------------------------
   1:          1244       81546688  [Lorg.apache.activemq.command.DataStructure;
   2:        229010       31452120  <constMethodKlass>
   3:        229010       31156304  <methodKlass>
   4:        239863       28604688  [C
   5:         18433       22196816  <constantPoolKlass>
   6:         39290       15346496  [B
   7:         18433       14414408  <instanceKlassKlass>
   8:         14020       13783136  <constantPoolCacheKlass>
   9:         71423        6119320  [Ljava.util.HashMap$Entry;
  10:        237082        5689968  java.lang.String
  11:         50255        5038688  [I
  12:        125650        4020800  java.util.HashMap$Entry
  13:         97931        3917240  java.util.LinkedHashMap$Entry
  14:         42557        3404560  java.lang.reflect.Method
  15:        107680        3022072  [Ljava.lang.String;
  16:         38100        2438400  java.util.LinkedHashMap
  17:         19563        2362808  java.lang.Class

To investigate further, we might need to refer to the application logs, or use a more sophisticate tools (like YourKit, or Eclipse Memory Analyzer).

4- How much more memory are you using besides heap? Now deduct the RSS figure by your heap capacity, and that's what you are using for everything else.
 
$ bc
bc 1.06.95
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
826.028 - (53.25+56.875+56.875+287.0+200.0625)
171.9655

For me, it was 171MB. If this number is too high, it is worth checking threads. Running too many threads can affect memory usage of the application.
 
$ /usr/java/jdk1.7.0_25/bin/jstack 2885
Output is too long...

There is no easy way to figure out what's wrong if native memory usage is too high, there are some methods mentioned here though.

Thursday, September 19, 2013

Simple JMS Topic Example

I was recently dealing with some JMS issues, in order to do which I developed a simple utility to send/receive messages using ActiveMQ. Although it is a trivial thing to do, but I though I can share it here in case someone finds it useful.

You can find the complete code and import it to eclipse from here.


Here is a simple topic producer:
package com.test;

import javax.jms.Connection;
import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnectionFactory;

public class SimpleTopicProducer implements Runnable
{
    private final String brokerUrl;
    public final String topicName;

    public SimpleTopicProducer(String brokerUrl, String topicName)
    {
        this.brokerUrl = brokerUrl;
        this.topicName = topicName;
    }

    public void run()
    {
        try
        {
            // Create a ConnectionFactory
            ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerUrl);

            // Create a Connection
            Connection connection = connectionFactory.createConnection();
            connection.start();

            // Create a Session
            Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

            // Create the destination (Topic or Queue)
            Destination destination = session.createTopic(topicName);

            // Create a MessageProducer from the Session to the Topic or
            // Queue
            MessageProducer producer = session.createProducer(destination);
            producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);

            // Create a messages
            String text = "SimpleTopicProducer - From: " + Thread.currentThread().getName() + " : " + this.hashCode();
            TextMessage message = session.createTextMessage(text);

            // Tell the producer to send the message
            System.out.println("Sent message: " + message.hashCode() + " : " + Thread.currentThread().getName());
            producer.send(message);

            // Clean up
            session.close();
            connection.close();
        }
        catch (Exception e)
        {
            System.out.println("Caught: " + e);
            e.printStackTrace();
        }
    }
}

Here is a simple topic consumer:
package com.test;

import javax.jms.Connection;
import javax.jms.Destination;
import javax.jms.ExceptionListener;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnectionFactory;

public class SimpleTopicConsumer implements Runnable, ExceptionListener
{
    private final String brokerUrl;
    private final String topicName;
    private final int lifetime;

    public SimpleTopicConsumer(String brokerUrl, String topicName, int lifetime)
    {
        this.brokerUrl = brokerUrl;
        this.topicName = topicName;
        this.lifetime = lifetime;
    }

    public void run()
    {
        try
        {
            System.out.println("SimpleTopicConsumer, started on " + topicName);

            // Create a ConnectionFactory
            ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerUrl);

            // Create a Connection
            Connection connection = connectionFactory.createConnection();
            connection.start();

            connection.setExceptionListener(this);

            // Create a Session
            Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

            // Create the destination (Topic or Queue)
            Destination destination = session.createTopic(topicName);

            // Create a MessageConsumer from the Session to the Topic or
            // Queue
            MessageConsumer consumer = session.createConsumer(destination);

            long startTime = System.currentTimeMillis();

            while (true)
            {
                long now = System.currentTimeMillis();
                if (now - startTime > lifetime)
                {
                    System.out.println("Time's up, exiting...");
                    break;
                }

                // Wait for a message
                Message message = consumer.receive(1000);

                if (message == null)
                    continue;

                if (message instanceof TextMessage)
                {
                    TextMessage textMessage = (TextMessage) message;
                    String text = textMessage.getText();
                    System.out.println("SimpleTopicConsumer - Received (text): " + text);
                }
                else
                {
                    System.out.println("SimpleTopicConsumer - Received: " + message);
                }
            }

            consumer.close();
            session.close();
            connection.close();
        }
        catch (Exception e)
        {
            System.out.println("Caught: " + e);
            e.printStackTrace();
        }
    }

    public synchronized void onException(JMSException ex)
    {
        System.out.println("JMS Exception occured.  Shutting down client.");
    }
}

This is the main program to put the things together:
package com.test;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;

public class SimpleJmsApp
{
    private static final String BROKER_URL = "tcp://localhost:61616?jms.prefetchPolicy.all=1000";
    private static final int CONSUME_LIFE_TIME_IN_MS = 3600 * 1000;
    private static final boolean START_PRODUCERS = false;
    private static final Set TOPICS = new HashSet<>(Arrays.asList("test_topic"));

    public static void main(String[] args) throws Exception
    {
        if (args.length > 0)
        {
            TOPICS.clear();
            TOPICS.addAll(Arrays.asList(args));
        }

        System.out.println("Now starting consumers...");
        for (String topic : TOPICS)
        {
            SimpleTopicConsumer consumer = new SimpleTopicConsumer(BROKER_URL, topic, CONSUME_LIFE_TIME_IN_MS);
            thread(consumer, false);
        }

        if (START_PRODUCERS)
        {
            Thread.sleep(1000);
            System.out.println("starting producers...");
            for (String topic : TOPICS)
            {
                SimpleTopicProducer producer = new SimpleTopicProducer(BROKER_URL, topic);
                thread(producer, false);
            }
        }
    }

    public static void thread(Runnable runnable, boolean daemon)
    {
        Thread brokerThread = new Thread(runnable);
        brokerThread.setDaemon(daemon);
        brokerThread.start();
    }
}

Thursday, June 20, 2013

Google OAuth 2.0 using Scribe

If you need to use Google's OAuth 2.0 API, you can use Scribe to do that. It doesn't support OAuth 2.0 for Google out of the box, therefore I finalized and tested my changes (I had written about before) and applied them to my copy of the library code. I will be sending a pull request soon so that we can have something similar in the next releases.

It also takes care of refreshing tokens. You might need to use something like this in order to use it:

package org.scribe.examples;

import java.util.Scanner;

import org.scribe.builder.ServiceBuilder;
import org.scribe.builder.api.Google2Api;
import org.scribe.model.OAuthRequest;
import org.scribe.model.Response;
import org.scribe.model.Token;
import org.scribe.model.Verb;
import org.scribe.model.Verifier;
import org.scribe.oauth.OAuthService;

public class Google2Example
{
    private static final String NETWORK_NAME = "Google";
    private static final String PROTECTED_RESOURCE_URL = "https://www.googleapis.com/oauth2/v1/userinfo?alt=json";
    private static final String SCOPE = "https://mail.google.com/ https://www.googleapis.com/auth/userinfo.email";
    private static final Token EMPTY_TOKEN = null;

    public static void main(String[] args)
    {
        boolean refresh = true;
        boolean startOver = true;

        // TODO: Put your own API key, secret, and callback URL here.
        String apiKey = "407408718192.apps.googleusercontent.com";
        String apiSecret = "";
        String callbackUrl = "https://developers.google.com/oauthplayground";

        OAuthService service = new ServiceBuilder().provider(Google2Api.class).apiKey(apiKey).apiSecret(apiSecret)
                .callback(callbackUrl).scope(SCOPE).offline(true).build();
        Scanner in = new Scanner(System.in);

        System.out.println("=== " + NETWORK_NAME + "'s OAuth Workflow ===");
        System.out.println();

        Verifier verifier = null;
        // TODO: Put your own token information here, if you don't want to start over the whole process.
        Token accessToken = new Token("ACCESS_TOKEN", "REFRESH_TOKEN");

        if (startOver)
        {
            // Obtain the Authorization URL
            System.out.println("Fetching the Authorization URL...");
            String authorizationUrl = service.getAuthorizationUrl(EMPTY_TOKEN);
            System.out.println("Got the Authorization URL!");
            System.out.println("Now go and authorize Scribe here:");
            System.out.println(authorizationUrl);
            System.out.println("And paste the authorization code here");
            System.out.print(">>");
            verifier = new Verifier(in.nextLine());
            System.out.println();

            // Trade the Request Token and Verfier for the Access Token
            System.out.println("Trading the Request Token for an Access Token...");
            accessToken = service.getAccessToken(EMPTY_TOKEN, verifier);
            System.out.println("Got the Access Token!");
            System.out.println("(if your curious it looks like this: " + accessToken + " )");
            System.out.println();
        }

        if (refresh)
        {
            try
            {
                // Trade the Refresh Token for a new Access Token
                System.out.println("Trading the Refresh Token for a new Access Token...");
                Token newAccessToken = service.getAccessToken(accessToken, verifier);
                System.out.println("Got the Access Token!");
                System.out.println("(if your curious it looks like this: " + newAccessToken + " )");
                System.out.println();
                accessToken = newAccessToken;
            }
            catch (Exception e)
            {
                e.printStackTrace();
            }
        }

        // Now let's go and ask for a protected resource!
        System.out.println("Now we're going to access a protected resource...");
        OAuthRequest request = new OAuthRequest(Verb.GET, PROTECTED_RESOURCE_URL);
        service.signRequest(accessToken, request);
        Response response = request.send();
        System.out.println("Got it! Lets see what we found...");
        System.out.println();
        System.out.println(response.getCode());
        System.out.println(response.getBody());

        System.out.println();
        System.out.println("Thats it man! Go and build something awesome with Scribe! :)");

    }
}

Tuesday, May 28, 2013

How to read dates from Oracle database?

Last week I came across this piece of code, which made the alarm bells ring for me. Can you see the issue with this code?




/**
 * Given a date, strip it of its timezone, and return the date as if it was GMT0
 * @param oracleTimestamp
 * @param localTimeZone
 * @return
 */
private Date toLocalTime(Date oracleTimestamp, TimeZone localTimeZone)
{
    if (oracleTimestamp == null)
        return null;
    Calendar local = localTimeZone == null ? Calendar.getInstance() : Calendar.getInstance(localTimeZone);
    local.clear();
    long localToUtcDelta = local.getTimeZone().getOffset(oracleTimestamp.getTime());
    return new Date(oracleTimestamp.getTime() + localToUtcDelta);
}

The fundamental issue is that there is no concept of locality or time zone in the Date object. This method gets a single and unique moment in time and returns another single and unique moment in time with a possible Gap of a few hours. Since the event has happened in one moment only, therefore one of these values are wrong. It means we start off a wrong value and then try to amend it later. Let us review some background in order to understand the issue a little more.
java.util.Date is time-zone independent and is represented as a long number for the number of milliseconds passed since Epoch, in UTC. As an example, if you put 1,369,656,000,000 in the epoch converter you will get these values back:




So if you had to do magical conversion like this, it is a sign that you (or someone else you have been relying on) have failed to create the right Date object out of this alternative representation. Two common cases are parsing from String and reading from database. When parsing from String values, it is easy to parse using a DateFormat which is set for GMT:

DateFormat gmtDateFormat = new SimpleDateFormat("yyyyMMddHHmmss");
DateFormat localDateFormat = new SimpleDateFormat("yyyyMMddHHmmss");
gmtDateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
Date date = gmtDateFormat.parse("20130528160000");
System.out.println("GMT: " + gmtDateFormat.format(date) + " Local: " + localDateFormat.format(date));

Let's focus on the more tricky case for the rest of this post, which is reading dates from database (Oracle in this case).

The way DATE and TIMESTAMP Oracle types work is a bit different though. They store year, month, day, ... values separately, as if you had stored the string representation directly. These two fields don't store time zone information. There are other types like "TIMESTAMP WITH TIME ZONE" that do store time zone and are a better fit for times when you care about time zone, but let's assume that we can't use them right now.

JDBC driver converts these two separate formats together for us, but it uses the local time zone of the environment it runs in. If your code runs in GMT+10 time zone, when you store Date(1,369,656,000,000), you end up with "Mon 27 May 2013 10:00:00" in database (with time zone information lost here). Then when you read it back it will be converted back to the right original Date value. This will all break when the time zone in read time is different than that of write time. In order to prevent that, we all agree to store DATE and TIMESTAMP values in GMT, meaning if you look into database you will see "Mon, 27 May 2013 12:00:00".

In plain JDBC it is really easy to read and write correct Date values, without having to jump through hoops, like this:

// WRITING
Timestamp nowTimestamp = new Timestamp(nowDate.getTime());
PreparedStatement insertStmt = conn.prepareStatement(
    "INSERT INTO DATE_TEST_TABLE (ID, DATE_COLUMN, TIMESTAMP_COLUMN) VALUES (?, ?, ?)");
try
{
    insertStmt.setInt(1, getSerial());
    insertStmt.setTimestamp(2, nowTimestamp, cal);
    insertStmt.setTimestamp(3, nowTimestamp, cal);
    insertStmt.executeUpdate();
}
// READING
PreparedStatement selectStmt = conn.prepareStatement(
    "SELECT ID, DATE_COLUMN, TIMESTAMP_COLUMN FROM DATE_TEST_TABLE ORDER BY ID");
ResultSet result = null;
try
{
    result = selectStmt.executeQuery();
    while (result.next())
    {
        System.out.println(
            String.format("%2s, %s, %s",
                result.getInt(1),
                result.getTimestamp(2, cal).toString(),
                result.getTimestamp(3, cal).toString()
            ));
    }
}

In JPA, it is not as straightforward, but not hard though and requires only a row mapper:

class DateTestObjectRowMapper implements RowMapper
{
    public Object mapRow(ResultSet rs, int rowNum) throws SQLException
    {
        Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("GMT"));
        int id = rs.getInt(1);
        Date dateField = rs.getTimestamp(2, calendar);
        Date timestampField = rs.getTimestamp(3, calendar);
        return new DateTestObject(id, dateField, timestampField);
    }
}
// READING
String sql = "SELECT ID, DATE_COLUMN, TIMESTAMP_COLUMN FROM DATE_TEST_TABLE ORDER BY ID";
List<?> objects = getJdbcTemplate().queryForList(sql);
System.out.println("Objects (wrong values): " + objects);
List<?> objectsMappedGMT = getJdbcTemplate().query(sql, new DateTestObjectRowMapper());
System.out.println("Objects (right values, using row mapper and GMT calendar): " + objectsMappedGMT);

If you have used queryForList() method, you are using your local time zone to convert/parse dates which are stored in GMT. You get the WRONG values back, it is not only that they are not local dates, there is no such thing as local date.


Download the complete code to experiment more for yourself. Run OracleDatePlainJdbcTest and OracleDateJpaTest classes to get started.