Understanding the ‘unresolved constraint’, ‘missing resource’ message from Apache Felix

It’s pretty common while developing an OSGi bundle that your imports and exports won’t quite match what you need or what exists in the server you’re deploying to. This can show up as NoClassDefFoundError, ClassNotFoundException or as log output in a stacktrace from bundle resolution. Hall, Pauls, McCullough and Savage did a great job of covering NCDFE and CNFE in “OSGi In Action” (chapter 8), let’s take a look at figuring out what the bundle resolution stacktrace is telling us. (I make nothing from the sales of “OSGi In Action” and suggest it to anyone interested in OSGi.)

Just like learning to read the stacktrace from an exception in Java is key to debugging, so is true about the dependency resolution messages from an OSGi container. Below is the output from Apache Felix when it encountered a missing dependency required by a bundle:

ERROR: Bundle org.sakaiproject.nakamura.webconsole.solr [124]: Error starting slinginstall:org.sakaiproject.nakamura.webconsole.solr-1.2-SNAPSHOT.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle org.sakaiproject.nakamura.webconsole.solr [124]: Unable to resolve 124.0: missing requirement [124.0] package; (package=org.apache.solr.client.solrj) [caused by: Unable to resolve 84.0: missing requirement [84.0] package; (package=org.sakaiproject.nakamura.api.lite) [caused by: Unable to resolve 86.0: missing requirement [86.0] package; (&(package=com.google.common.collect)(version>=9.0.0)(!(version>=10.0.0)))]])
org.osgi.framework.BundleException: Unresolved constraint in bundle org.sakaiproject.nakamura.webconsole.solr [124]: Unable to resolve 124.0: missing requirement [124.0] package; (package=org.apache.solr.client.solrj) [caused by: Unable to resolve 84.0: missing requirement [84.0] package; (package=org.sakaiproject.nakamura.api.lite) [caused by: Unable to resolve 86.0: missing requirement [86.0] package; (&(package=com.google.common.collect)(version>=9.0.0)(!(version>=10.0.0)))]]
    at org.apache.felix.framework.Felix.resolveBundle(Felix.java:3443)
    at org.apache.felix.framework.Felix.startBundle(Felix.java:1727)
    at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1156)
    at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:264)
    at java.lang.Thread.run(Thread.java:619)

What you have here is a stacktrace with a lengthy message. The important part of the stacktrace for us is the message.

ERROR: Bundle org.sakaiproject.nakamura.webconsole.solr [124]: Error starting slinginstall:org.sakaiproject.nakamura.webconsole.solr-1.2-SNAPSHOT.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle org.sakaiproject.nakamura.webconsole.solr [124]: Unable to resolve 124.0: missing requirement [124.0] package; (package=org.apache.solr.client.solrj) [caused by: Unable to resolve 84.0: missing requirement [84.0] package; (package=org.sakaiproject.nakamura.api.lite) [caused by: Unable to resolve 86.0: missing requirement [86.0] package; (&(package=com.google.common.collect)(version>=9.0.0)(!(version>=10.0.0)))]])

This message is pretty simple but the structure is common for nastier messages (i.e. deeper resolution paths before failure). Let’s pull it apart to see what’s happening in there.

ERROR: Bundle org.sakaiproject.nakamura.webconsole.solr [124]: Error starting slinginstall:org.sakaiproject.nakamura.webconsole.solr-1.2-SNAPSHOT.jar

This very first part tells us that an error occurred while trying to load the org.sakaiproject.nakamura.webconsole.solrbundle. Nice start, but not quite the crux of the matter. Let’s keep reading.

org.osgi.framework.BundleException: Unresolved constraint in bundle org.sakaiproject.nakamura.webconsole.solr [124]: Unable to resolve 124.0: missing requirement [124.0] package; (package=org.apache.solr.client.solrj) [caused by: Unable to resolve 84.0: missing requirement [84.0] package; (package=org.sakaiproject.nakamura.api.lite) [caused by: Unable to resolve 86.0: missing requirement [86.0] package; (&(package=com.google.common.collect)(version>=9.0.0)(!(version>=10.0.0)))]])

Phew, that’s a lot of text! This is the heart of what we need though, so let’s break it down to make more sense of it.

(
    org.osgi.framework.BundleException: Unresolved constraint in bundle org.sakaiproject.nakamura.webconsole.solr [124]: Unable to resolve 124.0: missing requirement [124.0] package; (package=org.apache.solr.client.solrj)
        [
            caused by: Unable to resolve 84.0: missing requirement [84.0] package; (package=org.sakaiproject.nakamura.api.lite)
            [
                 caused by: Unable to resolve 86.0: missing requirement [86.0] package; (&(package=com.google.common.collect)(version>=9.0.0)(!(version>=10.0.0)))
            ]
        ]
)

What are those [number]s in the message?

The numbers in the message tell us the bundle ID on the server.

Unresolved Package Name	Bundle ID Where Resolution Failed
org.apache.solr.client.solrj	124
org.sakaiproject.nakamura.api.lite	84
com.google.common.collect	86

Once you pull apart the message it becomes more obvious that it has structure and meaning! The structure of the message tells us that bundle 124 depends on a package from bundle 84 which depends on a package from bundle 86 which is unable to resolve com.google.common.collect;version=[9.0.0, 10.0.0). The innermost/very last message tells us the root of the problem; the dependency resolver was unable to find com.google.common.collect at version=[9.0.0, 10.0.0). Now we have somewhere to start digging.

How To Fix This

I suggest one of the following steps:

1. Add a bundle that exports the missing package with a version that matches the required version
2. Change the version to match an exported package already on the server

In this particular environment, com.google.common.collect;version=10.0.0 is what our server has deployed. The descriptor above specifically blocks any version not in the 9.x.x range. We generate the OSGi manifest by using the Maven Bundle Plugin which uses the BND tool to generate the manifest. In BND version > 2.1.0, the macro for versions was changed. Our solution has ranged from rolling back to bnd version=2.1.0 OR define the macro differently. The results are the same; the version segment in the manifest header becomes com.google.common.collect;version>=9.0.0 which finds our bundle of com.google.common.collect;version=10.0.0.

Notes about environment

The above message and stacktrace originated from a Sakai OAE environment which is built on Apache Sling and thusly Apache Felix. We use an artifact ID that is the root package of the bundle (org.sakaiproject.nakamura.webconsole.solr). This has the side effect that our bundle names look like package names in the message but gives a very clear naming convention.

Using pygraphviz to plot OSGi bundle dependencies

I’ve been working on an OSGi project for the last few years. As with any project, evolutionary changes will eventually require some cleanup. As new bundles have been added over time, the graph of dependencies is starting to get unwieldy in places. Even with good management of these dependencies, a nice visual layout of things can really help you see how your bundles are interconnected and give you the power to start separating some connections if you graph starts to hit cyclical dependencies.

I drove around a few different visual tools in Eclipse (PDE visualization tool, m2eclipse dependency graph) but needed something that would narrow the view to just my project. I not only wanted to see what things my bundles depend on in the project but I wanted to see what depends on a given bundle.

This was my first project with pygraphviz but after I figured out which way the grain runs, I was able to give it a basic graph of my project and generate the diagrams I wanted. I finally let pygraphviz handle the graph traversing and things got better (less code, faster results).

Since I wanted to analyze the runtime dependencies of my server, I installed the Felix Remote Shell bundle along with the Felix Shell bundle to allow remote connections to management functionality. With these in place, I was able to connect via telnet and query for package level information to build my graph.

Using the code below, I was able to generate a graph of the entire project (successors) and a graph for each bundle in the graph (predecessors and successors). Some sample images are below the source code. Eventually I’ll add saving and opening of dot files to allow for analysis without a running server.

#! /usr/bin/env python
import re
from sets import Set
import os
import pygraphviz as pgv
import sys
import telnetlib

# [   1] [Active     ] [   15] org.sakaiproject.nakamura.messaging (0.11.0.SNAPSHOT)
bundle_from_ps = re.compile('^\[\s*(?P<bundle_id>\d+)\]\s\[.+\]\s(?P<bundle_name>.+)\s')

# org.sakaiproject.nakamura.api.solr; version=0.0.0 -> org.sakaiproject.nakamura.solr [11]
bundle_from_req = re.compile('^.*-\> (?P<bundle_name>.*) \[(?P<bundle_id>.*)\]$')

def get_sakai_bundles():
    """Get a list of bundles that are create as part of Sakai OAE.
    Returns a dictionary of dict[bundle_name] = bundle_id.
    """
    tn = telnetlib.Telnet('localhost', '6666')
    tn.write('ps -s\nexit\n')
    lines = [line for line in tn.read_all().split('\r\n') if 'org.sakaiproject' in line]

    bundles = {}
    for line in lines:
        m = bundle_from_ps.match(line)
        bundles[m.group('bundle_name')] = m.group('bundle_id')
    return bundles

def get_package_reqs(bundle_id):
    """Gets the requirements (imports) for a given bundle.
    Returns a dictionary of dict[bundle_name] = bundle_id.

    Keyword arguments:
    bundle_id -- Bundle ID returned by the server in the output of
                 get_sakai_bundles()
    """
    tn = telnetlib.Telnet('localhost', '6666')
    tn.write('inspect package requirement %s\nexit\n' % (bundle_id))
    lines = [line for line in tn.read_all().split('\r\n') if line.startswith('org.sakaiproject') and line.endswith(']')]

    reqs = {}
    for line in lines:
        m = bundle_from_req.match(line)
        reqs[m.group('bundle_name')] = m.group('bundle_id')
    return reqs

def build_bundle_graph():
    """Build a graph_attr (nodes, edges) representing the connectivity
    within Sakai bundles
    """
    sakai_bundles = get_sakai_bundles()
    bundles = {}
    for b_name, b_id in sakai_bundles.items():
        reqs = get_package_reqs(b_id)
        bundles[b_name] = reqs.keys()
    return bundles

def draw_subgraph(name, graph, filename, successors = True):
    """Draw (write to disk) a subgraph that starts with or ends with
    the specified node.

    Keyword arguments:
    name -- name of the node to focus on
    graph -- graph of paths between bundles
    filename -- filename to write subgraph to
    successors -- whether to lookup successors or predecessors
    """
    if successors:
        nbunch = graph.successors(name)
    else:
        nbunch = graph.predecessors(name)

    if nbunch:
        nbunch.append(name)
        subgraph = graph.subgraph(nbunch)
        subgraph.layout(prog = 'dot')
        subgraph.draw('graphviz/%s' % filename)

def main():
    if not os.path.isdir('graphviz'):
        os.mkdir('graphviz')

    bundles_graph = build_bundle_graph()

    # print the whole graph
    graph = pgv.AGraph(data = bundles_graph, directed = True)
    graph.layout(prog = 'dot')
    graph.draw('graphviz/org.sakaiproject.nakamura.png')

    for b_name, reqs in bundles_graph.items():
        draw_subgraph(b_name, graph, '%s.png' % b_name)
        draw_subgraph(b_name, graph, '%s-pred.png' % b_name, False)

if __name__ == '__main__':
    main()

Sakai OAE Bundle Dependency Graph

Sakai OAE Solr Bundle Predecessors

Sakai OAE Presence Bundle Successors

Committer Status on Apache Sling

I woke up this morning to the fantastic news that I’ve been offered committer status on Apache Sling! I graciously and excitedly accepted the opportunity to become an Apache committer.
The various points of processing are in motion, so expect to see more Sling posts from me as I get ramped up into being a more productive member of Apache Sling.

Basics of OSGi and Declarative Services

This is a slide deck for a talk I gave at the Sakai 2010 conference in lovely Denver, Colorado. The title to the talk was “Using OSGi in Nakamura” which was my original intention, but after reviewing the slides I seemed to have given a talk about the basics of OSGi. Good information under a wrong marquee.

No matter the intention, I found today that I needed to refer to the slides for setting up an activator, so I’m reposting it here to give myself and hopefully others faster access. Be on the lookout for a repeat performance of this presentation at the Sakai 2011 conference in Los Angeles. I might even try to show it around a bit once the European conference is announced.

When To Immediately Activate An OSGi Component

OSGi has a fantastic feature for immediate components ^[1] and delayed components ^[2]. This allows components to delay their possibly expensive activation until the component is first accessed. At the very least it allows the OSGi platform to consume resources as needed. No sense is sucking up those server resources for something that can wait.

As developers start their adventure into OSGi, I’ve noticed a pattern creeping up that I’m not sure folks are aware of: just because you like your component doesn’t mean it needs to start immediately. Unless your component has something it needs to do before other components can do their work, there’s little need to start a component immediately. A lot of times, the work can be moved into the bundle activator or is fine being delayed.

People often see that a component does something in its activation method and assume it should start immediately. The questions to ask yourself are:

• Will the things that need to use this component have a reference to it?
  Yes? Delayed activation.
• Before activation, will functionality be missing that should be available as soon as possible?
  Yes? Immediate activation.
• Can something else do the job of activating the component at a later time?
  Yes? Delayed activation.

Registering with an external service, such as JMS, is a perfect scenario for an immediate component. If the JMS service were in OSGi, declarative services could do the work of making sure service trackers know your component exists, but OSGi has no jurisdiction over external resources. Binding to a JMS topic requires manual interaction and therefore should happen by way of immediate component activation.

Getting and setting of properties is not a case for immediate component activation. Neither is because the service is referenced by something else.

When using declarative services, a reference to your not-yet-started component will be given to anything that has said it wants to collect an interface you implement. This requires no effort on the part of the service being referenced and thusly does not require the referenced service to be activated immediately. Fret not! On first access of this service by any consumer the service will get activated.

OSGi has not been the fastest thing for me to learn but understanding the nuances of little things like this really helps me understand why it is a good approach at decoupled services and modular software construction.

<sub>1. OSGi compendium spec version 4.2, section 112.2.2
2. OSGi compendium spec version 4.2, section 112.2.3</sub>

LDAP Authentication & Authorization Dissected and Digested

LDAP is one of those things that I’ve integrated with a few times but never put enough energy into to really get the details or understand it much.  There’s always been someone I can bounce questions off of and thankfully those people were available again as I started working out the details of performing LDAP authentication.

The steps below are general enough to be used by anyone and will hopefully shed some light into the steps performed in LDAP authentication.  The process below also includes some steps for authorization.

Authentication

1.  Get a connection to the LDAP server.
With the host and port for the LDAP server, create a connection to it.  This can be  a simple direct connection or a pooled connection.  If more than a basic test, it is best to use a pooled connection.  Log and fail if a connection cannot be created.

2.  Bind as the application user.
Bind the connection to the application user.  This user should have enough permissions to search the area of LDAP where users are located.  This user may also have permissions to search for things outside of the users area (groups, authorization).  Log and fail if the application user cannot bind.

3.  Search for the DN (distinguished name) of the user to be authenticated.
This is where we verify the username is valid.  This does not authenticate the user but simply makes sure the requested username exists in the system.  Log and fail if the user’s DN is not found.

4.  Bind as user to be authenticated using DN from step 3.
Now for the moment of truth.  Bind to the connection using the DN found in step 3 and the password supplied by the user.  Log and fail if unable to bind using the user’s DN and password.

Authorization

5.  Re-bind as application user.
To check the authorization of a user, we need to read attributes from the user’s account. To do this, we need to re-bind as the application user.

6.  Search for user and require attributes.
A filter is used to search for the user like was done in step 3 but we’ll add an extra check to the query to look for the attributes that show we’re authorized.

Example Code

The code shown here is using the JLDAP library from Novell.  Interesting includes are noted at the top. The utility class used for escaping the search filter can be found in the Sakai Project’s Nakamura codebase.

import com.novell.ldap.LDAPAttribute;
import com.novell.ldap.LDAPConnection;
import com.novell.ldap.LDAPEntry;
import com.novell.ldap.LDAPException;
import com.novell.ldap.LDAPSearchResults;
// ...
String baseDn = "ou=People,o=MyOrg";
String userFilter = "uid = {}";
String authzFilter = "authzAttr=special:Entitlement:value";
// ...
public boolean authenticate(Credentials credentials) throws RepositoryException {
    boolean auth = false;
    if (credentials instanceof SimpleCredentials) {
      // get application user credentials
      String appUser = connMgr.getConfig().getLdapUser();
      String appPass = connMgr.getConfig().getLdapPassword();

      // get user credentials
      SimpleCredentials sc = (SimpleCredentials) credentials;

      long timeStart = System.currentTimeMillis();

      String userDn = LdapUtil.escapeLDAPSearchFilter(userFilter.replace("{}",
          sc.getUserID()));
      String userPass = new String(sc.getPassword());

      LDAPConnection conn = null;
      try {
        // 1) Get a connection to the server
        try {
          conn = connMgr.getConnection();
          log.debug("Connected to LDAP server");
        } catch (LDAPException e) {
          throw new IllegalStateException("Unable to connect to LDAP server ["
              + connMgr.getConfig().getLdapHost() + "]");
        }

        // 2) Bind as app user
        try {
          conn.bind(LDAPConnection.LDAP_V3, appUser, appPass.getBytes(UTF8));
          log.debug("Bound as application user");
        } catch (LDAPException e) {
          throw new IllegalArgumentException("Can't bind application user [" + appUser
              + "]", e);
        }

        // 3) Search for username (not authz).
        // If search fails, log/report invalid username or password.
        LDAPSearchResults results = conn.search(baseDn, LDAPConnection.SCOPE_SUB, userDn,
            null, true);
        if (results.hasMore()) {
          log.debug("Found user via search");
        } else {
          throw new IllegalArgumentException("Can't find user [" + userDn + "]");
        }

        // 4) Bind as user.
        // If bind fails, log/report invalid username or password.

        // value is set below. define here for use in authz check.
        String userEntryDn = null;
        try {
          LDAPEntry userEntry = results.next();
          LDAPAttribute objectClass = userEntry.getAttribute("objectClass");

          if ("aliasObject".equals(objectClass.getStringValue())) {
            LDAPAttribute aliasDN = userEntry.getAttribute("aliasedObjectName");
            userEntryDn = aliasDN.getStringValue();
          } else {
            userEntryDn = userEntry.getDN();
          }

          conn.bind(LDAPConnection.LDAP_V3, userEntryDn, userPass.getBytes(UTF8));
          log.debug("Bound as user");
        } catch (LDAPException e) {
          log.warn("Can't bind user [{}]", userDn);
          throw e;
        }

        if (authzFilter.length() > 0) {
          // 5) Return to app user
          try {
            conn.bind(LDAPConnection.LDAP_V3, appUser, appPass.getBytes(UTF8));
            log.debug("Rebound as application user");
          } catch (LDAPException e) {
            throw new IllegalArgumentException("Can't bind application user [" + appUser
                + "]");
          }

          // 6) Search user DN with authz filter
          // If search fails, log/report that user is not authorized
          String userAuthzFilter = "(&(" + userEntryDn + ")(" + authzFilter + "))";
          results = conn.search(baseDn, LDAPConnection.SCOPE_SUB, userAuthzFilter, null,
              true);
          if (results.hasMore()) {
            log.debug("Found user + authz filter via search");
          } else {
            throw new IllegalArgumentException("User not authorized [" + userDn + "]");
          }
        }

        // FINALLY!
        auth = true;
        log.info("User [{}] authenticated with LDAP in {}ms", userDn,
            System.currentTimeMillis() - timeStart);
      } catch (Exception e) {
        log.warn(e.getMessage(), e);
      } finally {
        connMgr.returnConnection(conn);
      }
    }
    return auth;
  }

Setting Up RVM, Ruby, Gem and Rails on Ubuntu

This is centered on Ubuntu only because aptitude is used once. If you’re comfortable with your package manager feel free to replace the aptitude command with your local command.

I’m beginning to dig deeper into ruby, rails and the whole suite of tools available for them and have hit my knee on several things along the way. I’m no stranger to downloading things to download things to do things; I’ve been doing that in Java since 1996. I’ve also been kicking the tires on Ubuntu sense back before they got their naming scheme together and Debian even before that. I totally dig the Debian/Ubuntu way of install by deb packages. But with ruby, like Java, I feel it is best setup a machine by minimizing the use of apt (apt-get, aptitude, etc) and downloading the tools you need directly. I don’t let aptitude install Eclipse, Xalan, servlet-api or any of the other tools/libraries I use on top of the JVM and this is the route I have ended up going with when setting up ruby as well.

TL;DR: short list of what to do

This short list is for Ruby 1.9.2. If you want to install an earlier version, be sure to follow the extra steps about installing gem from rubygems.org.

1. Install RVM [details]:
   

   bash < > ~/.bashrc  # This loads RVM into a shell session.
   source ~/.bashrc

2. Install a C compiler for RVM [details]:
   

   sudo aptitude install gcc

3. Install Ruby 1.9.2 using RVM [details]:

   See below for instructions on installing RubyGems when using Ruby < v1.9.2
   

   rvm pkg install zlib
   rvm install 1.9.2 --default -C --with-zlib-dir=$HOME/.rvm/usr

4. Install Rails using RubyGems [details]:

   gem install rails

Details:

Install RVM

Note: The full install instructions have much better details.

RVM (Ruby Version Manager) gives us a way to install Ruby without getting it from aptitude. Being that Ruby is our starting point for everything after RVM and RVM is our means of getting Ruby, everything is downhill after getting RVM. You could install ruby using aptitude then install rvm as a gem but what’s the point of downloading Ruby just to download stuff to eventually download Ruby (again) and remove the system local stuff that was installed the first time. Let’s just cut out all the cruft in the middle.

Install a C compiler for RVM

Since RVM setups up the Ruby interpreter, it has to act 1 level deeper than the interpreter and C is that next level down. GCC is a very common tool to have on a linux machine, so this step is mostly for setting up a new machine. It’s always a good idea to have GCC installed. And vi. But this is a tech blog and not for theology, so I won’t go into that.

 

Install Ruby 1.9.2 using RVM

The big moment! All this work to get Ruby installed so we can begin our journey into RoR-land. This will take some time to perform as it needs to download, configure and compile Ruby.

 

Install RubyGems from rubygems.org for Ruby < v1.9.2

Note: The full install instructions have much better details.

wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz
tar zxf rubygems-1.3.7.tgz
cd rubygems-1.3.7
ruby setup.rb

Ruby 1.9.2 includes RubyGems 1.3.7, so we don’t have to install it when we install Ruby, but for previous versions we’ll need to install this bit ourselves. You can install it from aptitude but it locks you from doing system updates and some other functionality I wasn’t keen on losting. Installing it from rubygems.org gives back the freedom to break my system in whatever way I choose.

Install Rails using RubyGems

This is where things get down to Ruby’s style of easy. Package management that looks like an already very easy package manager (aptitude) is a bonus in my book. Installing gems will become something you most likely get very comfortable with as this is the easiest way to get the libraries you need for your Ruby installation. Read more into RVM Gemsets to see where this stuff gets even cooler.
Note: If you get this error message:

ERROR:  Loading command: install (LoadError)
    no such file to load -- zlib

take a look at this page for installing Ruby with zlib. I tripped over this nicely.