Monday, August 13, 2012

Android app to send emails from GMail using OAuth for authorisation

UPDATE: This no longer works, since OAuth 1 is no longer supported by Google.
 
It is appalling to see how few documents and sample codes exist for something this simple (check this): I want an app to authorise me to GMail through OAuth and let me send emails.

I am going to release parts of my project that deals with this as a complete working application, sounds good ha? The bad news is that I'm using OAuth 1 which is officially deprecated by Google, yet they don't advise on how to do the same with OAuth 2 as many things are different in the new protocol.

Feel free to download my code here or check it out from SVN:
svn checkout http://tinywebgears-samples.googlecode.com/svn/trunk/email-oauth/ tinywebgears-samples-read-only
You can import it in Eclipse once you have checked it out, enjoy.


While you are here, it is worthwhile to take you through some important parts of my code. Everything mentioned here is already put in the SVN module, so you don't need to copy/paste from here.

As you can see I'm using a patched version of commons-net library, to be able to pass "AUTH XOAUTH BASE64_TOKEN" to the SMTP server (update: this feature is available in commons-net, from this revision onwards):
diff -ru commons-net-3.1-src/src/main/java/org/apache/commons/net/smtp/AuthenticatingSMTPClient.java commons-net-3.1-src-patched/src/main/java/org/apache/commons/net/smtp/AuthenticatingSMTPClient.java
--- commons-net-3.1-src/src/main/java/org/apache/commons/net/smtp/AuthenticatingSMTPClient.java 2012-02-15 03:11:49.000000000 +1100
+++ commons-net-3.1-src-patched/src/main/java/org/apache/commons/net/smtp/AuthenticatingSMTPClient.java 2012-07-25 00:36:30.809267711 +1000
@@ -208,6 +208,10 @@
             }
             return SMTPReply.isPositiveCompletion(sendCommand(
                 new String(Base64.encodeBase64(password.getBytes()))));
+        }
+        else if (method.equals(AUTH_METHOD.XOAUTH))
+        {
+            return SMTPReply.isPositiveIntermediate(sendCommand(username));
         } else {
             return false; // safety check
         }
@@ -243,7 +247,9 @@
         /** The standarised (RFC2195) CRAM-MD5 method, which doesn't send the password (secure). */
         CRAM_MD5,
         /** The unstandarised Microsoft LOGIN method, which sends the password unencrypted (insecure). */
-        LOGIN;
+        LOGIN,
+        /** XOAuth method which accepts a signed and base64ed OAuth URL. */
+        XOAUTH;

         /**
          * Gets the name of the given authentication method suitable for the server.
@@ -258,6 +264,8 @@
                 return "CRAM-MD5";
             } else if (method.equals(AUTH_METHOD.LOGIN)) {
                 return "LOGIN";
+            } else if (method.equals(AUTH_METHOD.XOAUTH)) {
+                return "XOAUTH";
             } else {
                 return null;
             }
The main OAuth functions are performed by a helper class, which comprise, by their appearing order:
  • creating a request token,
  • making an authorization url,
  • getting an access token using the verifier code passed back by Google,
  • making a secure request to the user details API to get user's email address,
  • and finally making an XOAUTH string every time an email is due to be sent.
One thing that took me a while to figure out was that the access token is long-lived and almost never expires (unless the user revokes it), but the XOAUTH string has a very short life-time and hence must be generated quite frequently.

Let's have a look at important parts of the code. Using Scribe library my code is much less than when I used Signpost:

package com.tinywebgears.gmailoauth.util;

import org.scribe.builder.ServiceBuilder;
import org.scribe.builder.api.GoogleApi;
import org.scribe.model.OAuthRequest;
import org.scribe.model.Response;
import org.scribe.model.Token;
import org.scribe.model.Verb;
import org.scribe.model.Verifier;
import org.scribe.oauth.OAuthService;

public class OAuthHelperV1 implements OAuthHelper
{
    public static final String OAUTH_APP_KEY = "anonymous";
    public static final String OAUTH_APP_SECRET = "anonymous";
    public static final String URL_OAUTH_AUTHORIZE = "https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=";
    public static final String OAUTH_PARAM_VERIFIER = "oauth_verifier";

    private OAuthService service;
    private Token requestToken;

    private static OAuthHelperV1 oauthHelper;

    static
    {
        if (oauthHelper == null)
        {
            String scope = OAUTH_SCOPE_EMAIL + " " + OAUTH_SCOPE_GMAIL;
            oauthHelper = new OAuthHelperV1(OAUTH_APP_KEY, OAUTH_APP_SECRET, scope, OAUTH_CALLBACK_URL, OAUTH_APP_NAME);
        }
    }

    public static OAuthHelperV1 get()
    {
        return oauthHelper;
    }

    public OAuthHelperV1(String consumerKey, String consumerSecret, String scope, String callbackUrl, String appname)
    {
        service = new ServiceBuilder().provider(GoogleApi.class).apiKey(consumerKey).apiSecret(consumerSecret)
                .scope(scope).callback(callbackUrl).build();
    }

    @Override
    public String getAuthorizationUrl()
    {
        requestToken = service.getRequestToken();
        return URL_OAUTH_AUTHORIZE + requestToken.getToken();
    }

    @Override
    public String extractVerifier(String uri)
    {
        if (uri.indexOf(OAUTH_PARAM_VERIFIER + "=") != -1)
            return UriUtil.getParam(uri, OAUTH_PARAM_VERIFIER);
        return null;
    }

    @Override
    public Token getAccessToken(String verifier)
    {
        Token accessToken = service.getAccessToken(requestToken, new Verifier(verifier));
        return accessToken;
    }

    @Override
    public String makeSecuredRequest(Token accessToken, String url)
    {
        OAuthRequest request = new OAuthRequest(Verb.GET, url);
        service.signRequest(accessToken, request);
        request.addHeader("GData-Version", "3.0");
        Response response = request.send();
        System.out.println("Got it! Lets see what we found...");
        System.out.println();
        System.out.println(response.getCode());
        System.out.println(response.getBody());
        return response.getBody();
    }

    @Override
    public void signRequest(Token accessToken, OAuthRequest request)
    {
        service.signRequest(accessToken, request);
    }
}

Here is also the code that sends emails:

package com.tinywebgears.gmailoauth.mail;

import java.io.PrintWriter;
import java.io.Writer;
import java.security.AccessController;
import java.security.Provider;
import java.security.Security;

import org.apache.commons.net.PrintCommandListener;
import org.apache.commons.net.smtp.AuthenticatingSMTPClient;
import org.apache.commons.net.smtp.SMTPReply;
import org.apache.commons.net.smtp.SimpleSMTPHeader;

import android.util.AndroidRuntimeException;
import android.util.Log;

public class OAuthGMailSender
{
    private static final String TAG = "OAuthGMailSender";

    private String mailhost = "smtp.gmail.com";
    private Integer mailport = 587;
    private String xoauthToken;

    static
    {
        Security.addProvider(new JSSEProvider());
    }

    public OAuthGMailSender(String xoauthToken)
    {
        this.xoauthToken = xoauthToken;

    }

    public synchronized void sendMail(String subject, String body, String sender, String recipient) throws Exception
    {
        Log.d(TAG, "Sending email...");
        SimpleSMTPHeader header = new SimpleSMTPHeader(sender, recipient, subject);

        AuthenticatingSMTPClient client = new AuthenticatingSMTPClient();
        client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
        client.connect(mailhost, mailport);
        client.login();
        client.execTLS();
        client.auth(AuthenticatingSMTPClient.AUTH_METHOD.XOAUTH, xoauthToken, null);

        if (!SMTPReply.isPositiveCompletion(client.getReplyCode()))
        {
            Log.e(TAG, "SMTP Authentication failed: " + client.getReplyCode() + ":" + client.getReplyString());
            client.disconnect();
            throw new AndroidRuntimeException("SMTP Authentication failed.");
        }

        client.setSender(sender);
        client.addRecipient(recipient);

        Writer writer = client.sendMessageData();
        if (writer != null)
        {
            writer.write(header.toString());
            writer.write(body);
            writer.write("\n.\n");
            writer.close();
            client.completePendingCommand();
        }

        client.logout();

        client.disconnect();
    }

    @SuppressWarnings("serial")
    public static final class JSSEProvider extends Provider
    {
        public JSSEProvider()
        {
            super("HarmonyJSSE", 1.0, "Harmony JSSE Provider");
            AccessController.doPrivileged(new java.security.PrivilegedAction<Void>()
            {
                public Void run()
                {
                    put("SSLContext.TLS", "org.apache.harmony.xnet.provider.jsse.SSLContextImpl");
                    put("Alg.Alias.SSLContext.TLSv1", "TLS");
                    put("KeyManagerFactory.X509", "org.apache.harmony.xnet.provider.jsse.KeyManagerFactoryImpl");
                    put("TrustManagerFactory.X509", "org.apache.harmony.xnet.provider.jsse.TrustManagerFactoryImpl");
                    return null;
                }
            });
        }
    }
}


Read less interesting stuff yourselves!

Sunday, August 5, 2012

Google OAuth 2 with Scribe on Android

Update: See this post for the latest on this, and links to the source code.

I started off with Nilvec's valuable blog post to do oauth on my android application using signpost library, which was great except it only supported the first version of oauth (already deprecated). The problem with it (rather than deprecation) is not having anything like a refresh token. Once the access token expires, after an hour, your application stops working and you have to wake up the user to re-grant you, such a pain! Update: I was wrong. The access token is actually long-lived but you need to regenerate a  fresh XOAUTH string to give to SMTP server. You don't need the concept of refresh token in OAuth1, that's why it isn't there!

Then I went to try Scribe (version 1.3.1, available here). It is a very nice library, but unfortunately it doesn't inherently support Google's OAuth 2 at the time of writing. Although it is not very hard to implement one, but I happened to be lucky enough to find one already done. I had to slightly modify it to get it to work with my installed application client ID (where there is no client secret):

package org.scribe.builder.api;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.scribe.exceptions.OAuthException;
import org.scribe.extractors.AccessTokenExtractor;
import org.scribe.model.OAuthConfig;
import org.scribe.model.OAuthConstants;
import org.scribe.model.OAuthRequest;
import org.scribe.model.Response;
import org.scribe.model.Token;
import org.scribe.model.Verb;
import org.scribe.model.Verifier;
import org.scribe.oauth.OAuth20ServiceImpl;
import org.scribe.oauth.OAuthService;
import org.scribe.utils.OAuthEncoder;
import org.scribe.utils.Preconditions;

/**
 * Google OAuth2.0
 * Released under the same license as scribe (MIT License)
 * @author yincrash
 *
 */
public class Google2Api extends DefaultApi20 {

    private static final String AUTHORIZE_URL = "https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=%s&redirect_uri=%s";
    private static final String SCOPED_AUTHORIZE_URL = AUTHORIZE_URL + "&scope=%s";

    @Override
    public String getAccessTokenEndpoint() {
        return "https://accounts.google.com/o/oauth2/token";
    }

    @Override
    public AccessTokenExtractor getAccessTokenExtractor() {
        return new AccessTokenExtractor() {

            public Token extract(String response) {
                Preconditions.checkEmptyString(response, "Response body is incorrect. Can't extract a token from an empty string");

                Matcher matcher = Pattern.compile("\"access_token\" : \"([^&\"]+)\"").matcher(response);
                if (matcher.find())
                {
                    String token = OAuthEncoder.decode(matcher.group(1));
                    return new Token(token, "", response);
                }
                else
                {
                    throw new OAuthException("Response body is incorrect. Can't extract a token from this: '" + response + "'", null);
                }
            }
        };
    }

    @Override
    public String getAuthorizationUrl(OAuthConfig config) {
        // Append scope if present
        if (config.hasScope()) {
            return String.format(SCOPED_AUTHORIZE_URL, config.getApiKey(),
                    OAuthEncoder.encode(config.getCallback()),
                    OAuthEncoder.encode(config.getScope()));
        } else {
            return String.format(AUTHORIZE_URL, config.getApiKey(),
                    OAuthEncoder.encode(config.getCallback()));
        }
    }

    @Override
    public Verb getAccessTokenVerb() {
        return Verb.POST;
    }

    @Override
    public OAuthService createService(OAuthConfig config) {
        return new GoogleOAuth2Service(this, config);
    }

    private class GoogleOAuth2Service extends OAuth20ServiceImpl {

        private static final String GRANT_TYPE_AUTHORIZATION_CODE = "authorization_code";
        private static final String GRANT_TYPE = "grant_type";
        private DefaultApi20 api;
        private OAuthConfig config;

        public GoogleOAuth2Service(DefaultApi20 api, OAuthConfig config) {
            super(api, config);
            this.api = api;
            this.config = config;
        }

        @Override
        public Token getAccessToken(Token requestToken, Verifier verifier) {
            OAuthRequest request = new OAuthRequest(api.getAccessTokenVerb(), api.getAccessTokenEndpoint());
            switch (api.getAccessTokenVerb()) {
                case POST:
                    request.addBodyParameter(OAuthConstants.CLIENT_ID, config.getApiKey());
                    // TODO HA: API Secret is optional
                    if (config.getApiSecret() != null && config.getApiSecret().length() > 0)
                        request.addBodyParameter(OAuthConstants.CLIENT_SECRET, config.getApiSecret());
                    request.addBodyParameter(OAuthConstants.CODE, verifier.getValue());
                    request.addBodyParameter(OAuthConstants.REDIRECT_URI, config.getCallback());
                    request.addBodyParameter(GRANT_TYPE, GRANT_TYPE_AUTHORIZATION_CODE);
                    break;
                case GET:
                default:
                    request.addQuerystringParameter(OAuthConstants.CLIENT_ID, config.getApiKey());
                    // TODO HA: API Secret is optional
                    if (config.getApiSecret() != null && config.getApiSecret().length() > 0)
                        request.addQuerystringParameter(OAuthConstants.CLIENT_SECRET, config.getApiSecret());
                    request.addQuerystringParameter(OAuthConstants.CODE, verifier.getValue());
                    request.addQuerystringParameter(OAuthConstants.REDIRECT_URI, config.getCallback());
                    if(config.hasScope()) request.addQuerystringParameter(OAuthConstants.SCOPE, config.getScope());
            }
            Response response = request.send();
            return api.getAccessTokenExtractor().extract(response.getBody());
        }
    }
}

My modifications are marked by the "TODO HA" string. I also had to modify a few other places to allow null/empty client secret.

package org.scribe.builder;

import java.io.*;
import org.scribe.builder.api.*;
import org.scribe.exceptions.*;
import org.scribe.model.*;
import org.scribe.oauth.*;
import org.scribe.utils.*;

/**
 * Implementation of the Builder pattern, with a fluent interface that creates a
 * {@link OAuthService}
 * 
 * @author Pablo Fernandez
 *
 */
public class ServiceBuilder
{
  private String apiKey;
  private String apiSecret;
  private String callback;
  private Api api;
  private String scope;
  private SignatureType signatureType;
  private OutputStream debugStream;
  
  /**
   * Default constructor
   */
  public ServiceBuilder()
  {
    this.callback = OAuthConstants.OUT_OF_BAND;
    this.signatureType = SignatureType.Header;
    this.debugStream = null;
  }
  
  /**
   * Configures the {@link Api}
   * 
   * @param apiClass the class of one of the existent {@link Api}s on org.scribe.api package
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder provider(Class<? extends Api> apiClass)
  {
    this.api = createApi(apiClass);
    return this;
  }

  private Api createApi(Class<? extends Api> apiClass)
  {
    Preconditions.checkNotNull(apiClass, "Api class cannot be null");
    Api api;
    try
    {
      api = apiClass.newInstance();  
    }
    catch(Exception e)
    {
      throw new OAuthException("Error while creating the Api object", e);
    }
    return api;
  }

  /**
   * Configures the {@link Api}
   *
   * Overloaded version. Let's you use an instance instead of a class.
   *
   * @param api instance of {@link Api}s
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder provider(Api api)
  {
   Preconditions.checkNotNull(api, "Api cannot be null");
   this.api = api;
   return this;
  }

  /**
   * Adds an OAuth callback url
   * 
   * @param callback callback url. Must be a valid url or 'oob' for out of band OAuth
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder callback(String callback)
  {
    Preconditions.checkNotNull(callback, "Callback can't be null");
    this.callback = callback;
    return this;
  }
  
  /**
   * Configures the api key
   * 
   * @param apiKey The api key for your application
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder apiKey(String apiKey)
  {
    Preconditions.checkEmptyString(apiKey, "Invalid Api key");
    this.apiKey = apiKey;
    return this;
  }
  
  /**
   * Configures the api secret
   * 
   * @param apiSecret The api secret for your application
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder apiSecret(String apiSecret)
  {
    // TODO HA: Commented out
    //Preconditions.checkEmptyString(apiSecret, "Invalid Api secret");
    this.apiSecret = apiSecret;
    return this;
  }
  
  /**
   * Configures the OAuth scope. This is only necessary in some APIs (like Google's).
   * 
   * @param scope The OAuth scope
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder scope(String scope)
  {
    Preconditions.checkEmptyString(scope, "Invalid OAuth scope");
    this.scope = scope;
    return this;
  }

  /**
   * Configures the signature type, choose between header, querystring, etc. Defaults to Header
   *
   * @param scope The OAuth scope
   * @return the {@link ServiceBuilder} instance for method chaining
   */
  public ServiceBuilder signatureType(SignatureType type)
  {
    Preconditions.checkNotNull(type, "Signature type can't be null");
    this.signatureType = type;
    return this;
  }

  public ServiceBuilder debugStream(OutputStream stream)
  {
    Preconditions.checkNotNull(stream, "debug stream can't be null");
    this.debugStream = stream;
    return this;
  }

  public ServiceBuilder debug()
  {
    this.debugStream(System.out);
    return this;
  }
  
  /**
   * Returns the fully configured {@link OAuthService}
   * 
   * @return fully configured {@link OAuthService}
   */
  public OAuthService build()
  {
    Preconditions.checkNotNull(api, "You must specify a valid api through the provider() method");
    Preconditions.checkEmptyString(apiKey, "You must provide an api key");
    // TODO HA: Commented out
    // Preconditions.checkEmptyString(apiSecret, "You must provide an api secret");
    return api.createService(new OAuthConfig(apiKey, apiSecret, callback, signatureType, scope, debugStream));
  }
}

package org.scribe.oauth;

import org.scribe.builder.api.*;
import org.scribe.model.*;

public class OAuth20ServiceImpl implements OAuthService
{
  private static final String VERSION = "2.0";
  
  private final DefaultApi20 api;
  private final OAuthConfig config;
  
  /**
   * Default constructor
   * 
   * @param api OAuth2.0 api information
   * @param config OAuth 2.0 configuration param object
   */
  public OAuth20ServiceImpl(DefaultApi20 api, OAuthConfig config)
  {
    this.api = api;
    this.config = config;
  }

  /**
   * {@inheritDoc}
   */
  public Token getAccessToken(Token requestToken, Verifier verifier)
  {
    OAuthRequest request = new OAuthRequest(api.getAccessTokenVerb(), api.getAccessTokenEndpoint());
    request.addQuerystringParameter(OAuthConstants.CLIENT_ID, config.getApiKey());
    // TODO HA: API Secret is optional
    if (config.getApiSecret() != null && config.getApiSecret().length() > 0)
          request.addQuerystringParameter(OAuthConstants.CLIENT_SECRET, config.getApiSecret());
    request.addQuerystringParameter(OAuthConstants.CODE, verifier.getValue());
    request.addQuerystringParameter(OAuthConstants.REDIRECT_URI, config.getCallback());
    if(config.hasScope()) request.addQuerystringParameter(OAuthConstants.SCOPE, config.getScope());
    Response response = request.send();
    return api.getAccessTokenExtractor().extract(response.getBody());
  }

  /**
   * {@inheritDoc}
   */
  public Token getRequestToken()
  {
    throw new UnsupportedOperationException("Unsupported operation, please use 'getAuthorizationUrl' and redirect your users there");
  }

  /**
   * {@inheritDoc}
   */
  public String getVersion()
  {
    return VERSION;
  }

  /**
   * {@inheritDoc}
   */
  public void signRequest(Token accessToken, OAuthRequest request)
  {
    request.addQuerystringParameter(OAuthConstants.ACCESS_TOKEN, accessToken.getToken());
  }

  /**
   * {@inheritDoc}
   */
  public String getAuthorizationUrl(Token requestToken)
  {
    return api.getAuthorizationUrl(config);
  }
}

Here is a sample java code that uses this to get a user's email address:

package org.scribe.examples;

import org.scribe.builder.ServiceBuilder;
import org.scribe.builder.api.FacebookApi;
import org.scribe.builder.api.Google2Api;
import org.scribe.model.*;
import org.scribe.oauth.OAuthService;

import java.util.Scanner;

public class Google2Example
{
    private static final String NETWORK_NAME = "Google";
    private static final String PROTECTED_RESOURCE_URL = "https://www.googleapis.com/oauth2/v1/userinfo?alt=json";
    private static final String SCOPE = "https://mail.google.com/ https://www.googleapis.com/auth/userinfo.email";
    private static final Token EMPTY_TOKEN = null;

    public static void main(String[] args)
  {
      // Client ID for web applications
      // String apiKey = "random-string.apps.googleusercontent.com";
      // String apiSecret = "client-secret-generated-by-google";
      // String callbackUrl = "https://code.google.com/oauthplayground";
      // Client ID for installed applications (there is no secret)
      String apiKey = "random-string.apps.googleusercontent.com";
      String apiSecret = "";
      String callbackUrl = "http://localhost";

      OAuthService service = new ServiceBuilder()
                                  .provider(Google2Api.class)
                                  .apiKey(apiKey)
                                  .apiSecret(apiSecret)
                                  .callback(callbackUrl)
                                  .scope(SCOPE)
                                  .build();
    Scanner in = new Scanner(System.in);

    System.out.println("=== " + NETWORK_NAME + "'s OAuth Workflow ===");
    System.out.println();

    // Obtain the Authorization URL
    System.out.println("Fetching the Authorization URL...");
    String authorizationUrl = service.getAuthorizationUrl(EMPTY_TOKEN);
    System.out.println("Got the Authorization URL!");
    System.out.println("Now go and authorize Scribe here:");
    System.out.println(authorizationUrl);
    System.out.println("And paste the authorization code here");
    System.out.print(">>");
    Verifier verifier = new Verifier(in.nextLine());
    System.out.println();
    
    // Trade the Request Token and Verfier for the Access Token
    System.out.println("Trading the Request Token for an Access Token...");
    Token accessToken = service.getAccessToken(EMPTY_TOKEN, verifier);
    System.out.println("Got the Access Token!");
    System.out.println("(if your curious it looks like this: " + accessToken + " )");
    System.out.println();

    // Now let's go and ask for a protected resource!
    System.out.println("Now we're going to access a protected resource...");
    OAuthRequest request = new OAuthRequest(Verb.GET, PROTECTED_RESOURCE_URL);
    service.signRequest(accessToken, request);
    Response response = request.send();
    System.out.println("Got it! Lets see what we found...");
    System.out.println();
    System.out.println(response.getCode());
    System.out.println(response.getBody());

    System.out.println();
    System.out.println("Thats it man! Go and build something awesome with Scribe! :)");

  }
}

Sorry if there is no complete code as I'm still experimenting with all this stuff.