Analysis of move to OpenSAML2 library in UNICORE

Introduction

OpenSAML 2 is a big library covering handling of SAML assertions, implementing SAML protocols along with support for several SAML profiles. Internally OpenSAML 2 is a set of three closely coupled libraries (build by the same development team):

  • opensaml Core SAML functionality: allows to produce SAML objects using API, to marshal them to DOM objects and to unmarshall SAML objects from XML DOM objects.
  • java-openws Simple web service (SOAP) stack including WS-Security support.
  • xml-tooling XML parsing along with support for XML signatures and XML encryption.

OpenSAML libraries altogether have 172.5 KLOC.

In UNICORE a very small samly2 library is used to provide SAML support. samly2 provides support for producing SAML object using API, to convert them to and from XMLBeans objects and DOM objects. Also support for XML digital signing is provided. samly2 does not provide any web service support and uses XMLBeans library to manipulate XML easily.

samly2 has 2.8 KLOC.

Even looking at bare number of code lines number (OpenSAML is over 60 times bigger then samly2) we can be sure that differences are big. First of all web service stack in samly2 is not needed as it is provided by other specialized libraries in UNICORE. The 2nd difference is the XMLBeans adoption. It allowed to mostly eliminate all the need for a separate solution which is implemented by xml-tooling. Finally samly2 implements only those parts of SAML specification that are anyhow interesting from the UNICORE PoV, not everything that was defined.

What was analysed

The following actions were undertaken to verify if the exchange of samly2 with OpenSAML makes sense. The order here is not the same in which the actions were undertaken, it was modified to provide a more clear evaluation below.

  1. OpenSAML library and API was initially evaluated.
  2. One typical sequence of operations which is implemented in samly2 was implemented using OpenSAML2 and the results were compared.
  3. Tricks and hacks which are programmed in samly2 to work around problems which were observed in various UNICORE components (e.g. due to buggy 3rd party libraries or bug in a legacy code) were tried to be applied in OpenSAML 2.
  4. Dependencies of OpenSAML2 were compared with dependencies which are used in UNICORE.
  5. Performance of both solutions was compared, though in a very limited way.
  6. Effort to maintain samly2 was assessed by checking all commits during the last 12 months and this was compared with the expected effort to integrate and maintain implementation based on OpenSAML2.

Results of evaluation

Generally p. (1) showed (as expected) that coverage of OpenSAML is complete and all required SAML functionalities are there. API turned out to be verbose and rather low level. Regular user documentation is mediocre: there are few pages with incomplete examples. JavaDoc documentation is quite good. Generally finding out a possible way to program something with OpenSAML2 is quite easy. Finding the best way is more tricky as one must dig the source code. For the experienced developer it is acceptable.

Implementation done in point (2) involved creation of an assertion with an API, setting issuer and subject, adding one attribtue, finally signing the assertion. Here code without the last signature creation is compared (it will be explained below why).

OpenSAML implementation required ca 25 lines of code to realize the task. The code which implements it in samly2 is 34 lines. NOTE: this compares usage of OpenSAML library with implementation inside samly2. To realize this task using samly2 one needs 5 lines of code.

The next task was to check whether certain workarounds which can be found in samly2 can be applied in OpenSAML2 based implementation. There is not much of them and only one of those looks problematic. It is related to namespace declaration in XML signature inserted into SAML assertion. It looks that to work around it the OpenSAML digital signature support would have to be ignored, and samly2 code which uses xmlsec API directly (as OpenSAML under the covers) just left as is. This is why in point one comparison of digital signature was not performed.

Another option is to ignore the samly2 workaround (what would caused older UNICORE Gateways incompatible with new stack). Then usage of OpenSAML2 code to generate signature is nearly 30. The similar code inside samly2 is around 50 lines. Usage of samly2 to sign the assertion: 1 line.

Here it is possible to summarize line comparison of the test implementation (rounded in any case of any doubt in favour of OpenSAML2):

- Implementation inside samly2 Implementation using samly2 Implementation using OpenSAML 2
Lines of code 84 6 52
Percentage 161% 11% 100%

Dependency analysis showed three potentially severe problems: openSAML2 uses Spring Framework core library, version 2.0. In UNICORE a version 3.0 is used. On the another hand UNICORE is using wss4j library (providing WS-Security support). The latest version of this library still uses OpenSAML 1 library. It is very hard to verify if at runtime some errors can occur, but unfortunately it is possible: OpenSAML 2 uses in many cases the same namespaces (Java package names) as its predecasor. According to OpenSAML web page the versions are fully incompatible. Finally the probably the most important issue was observed by accident: OpenSAML 2 uses DOM level 3 API. This is unsupported by XMLBeans library which forms a fundamental base of the whole UNICORE software. It is possible that some very difficult to overcome problems may come up during integration phase.

Performance of the two implementations was compared only briefly. Two implementations doing the same sequence of operations in OpenSAML and samly2 were compared: create assertion, add issuer, subject, one attribute without a value, convert to DOM element. Also another test was run were the code was additionally signing the assertions. 100 of warm up iterations for both libraries were run. The tests evaluated 10000 repetitions of the task (of course on the same decent system, JDK etc.). Several runs proofed that results presented below are stable:

Test samly2 OpenSAML
without dsig 2.4s 3.8s
with dsig 21.4s 24.1s

We can see that samly2 outperforms OpenSAML, however the difference is small. In fact creation of a digital signature is dominating and as both solutions use the same xmlsecurity library underneth the results makes sense. Additionally one more observation was made. OpenSAML library required 1.7s to initialize on the test machine which is a quite powerful workstation (3.17GHz, IC2). It can be expected that on less powerful notebook the startup can consume more then 2s. This is not relevant for the server side however for command line utilities the penalty is an issue.

Effort to maintain the samly2 library in the year 2010 was measured using SVN commits. There were altogether 17 of them. Out of them 8 were actual bugfixes or improvements; the rest contained only semi-automatic changes (like update of version for a release, change of an updated dependency) or fixes of typos in JavaDocs. The real 8 changes were small (few lines of code). We can roughly estimate that yearly maintenance of the samly2 library takes between 0.5 to 3 work days.

Conclusion

TODO

-- KrzysztofBenedyczak - 21-Dec-2010

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2010-12-22 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback