Tuesday, July 30, 2013

CXF security: integrate PKI to Security Token Service

Introduction

In previous blog CXF security: getting certificates from central PKI we have seen how to use Public Key Infrastructure and XKMS service to locate certificates in message encryption scenario. This blog is continuation of previous one and explains the integration of central PKI into SecurityTokenService (STS) for the authentication.

WS-Trust and SecurityTokenService

In authentication scenarios client generates or obtains security token and sends it to the service for verification inside the message. Security token can be either user name and password, SAML assertion or Kerberos ticket. Generation of security token sometimes is non-trivial task, requires access to external systems and using third party libraries. Therefore it makes a lot of sense to free service participants from implementing any security processing logic on their own. This logic can be delegated to SecurityTokenService (STS) offering functionality to issue, validate, renew or remove Security Tokens.The STS is defined within the OASIS WS-Trust specification.

The communication between client, service and STS is depicted on following figure:

The typical authentication scenario using WS-Trust requires the following steps:

Client obtains credentials and requests security token from STS.
STS verifies credentials using external IDM or other mechanism.
STS generates security token (for instance SAML), signs it with own private key and sends it back to client.
Client injects security token into protocol security header (SOAP or REST HTTP) and sends request message to the service.
Service extracts security token and validates it locally or using remote call to STS. As far as service and STS are in trusted relationship, it is enough just to validate STS signature of the security token.

You can find more details about STS service in CXF WS-Trust documentation and Oli Wulff blog.
Typically STS call is transparent for the client and triggered by appropriate WS-Policy assertion (IssuedToken).

When STS validates user's X509 certificate?

In some scenarios STS receives user certificate and uses it for authentication and for generation of SAML assertion:

Client sends user certificate in STS request as credentials for own authentication. In this case client must additionally sign part of request with private key to proof of possession:   <wsse:Security
    xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" soap:mustUnderstand="1">
    <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary"
        ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3"
        wsu:Id="X509-3CBA5F87FE6EAABD0E13661135280161">...</wsse:BinarySecurityToken>
    <wsu:Timestamp wsu:Id="TS-1">
        <wsu:Created>2013-04-16T11:58:48.003Z</wsu:Created>
        <wsu:Expires>2013-04-16T12:03:48.003Z</wsu:Expires>
    </wsu:Timestamp>
    <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
        Id="SIG-2">
        <ds:SignedInfo>
            ...
            <ds:Reference URI="#TS-1">
                ...
            </ds:Reference>
        </ds:SignedInfo>
        <ds:SignatureValue>...</ds:SignatureValue>
        <ds:KeyInfo Id="KI-3CBA5F87FE6EAABD0E13661135280202">
            <wsse:SecurityTokenReference
                wsu:Id="STR-3CBA5F87FE6EAABD0E13661135280223">
                <wsse:Reference URI="#X509-3CBA5F87FE6EAABD0E13661135280161"
                    ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" />
            </wsse:SecurityTokenReference>
        </ds:KeyInfo>
    </ds:Signature>
</wsse:Security>
ds:KeyInfo>
</ds:Signature>
...

You can see that BinarySecurityToken has X509 type and contains user certificate. Timestamp is signed with KeyInfo referencing to this certificate URI="#X509-3CBA5F87FE6EAABD0E13661135280161".
Client sends user certificate to be included into SAML token as SubjectConfirmationData/KeyInfo element. This certificate can be referenced and used for validation of XML signatures:

<saml2:Assertion xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion"
    xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    ID="_C5CE27CE46751556E113662938382792" IssueInstant="2013-04-18T14:03:58.279Z"
    Version="2.0" xsi:type="saml2:AssertionType">
    ...
    <saml2:Subject>
        <saml2:NameID
            Format="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified"
            NameQualifier="http://cxf.apache.org/sts">alice@EXAMPLE.COM</saml2:NameID>
        <saml2:SubjectConfirmation
            Method="urn:oasis:names:tc:SAML:2.0:cm:holder-of-key">
            <saml2:SubjectConfirmationData
                xsi:type="saml2:KeyInfoConfirmationDataType">
                <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                    <ds:X509Data>
                        <ds:X509Certificate>...</ds:X509Certificate>
                    </ds:X509Data>
                </ds:KeyInfo>
            </saml2:SubjectConfirmationData>
        </saml2:SubjectConfirmation>
    </saml2:Subject>
</saml2:Assertion>

<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
    Id="SIG-11">
    <ds:SignedInfo>
    ...
    <ds:KeyInfo Id="KI-CC7D54F0FC1CF34D4913662938383496">
        <ns3:SecurityTokenReference
            xmlns:ns3="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
            xmlns:wsse11="http://docs.oasis-open.org/wss/oasis-wss-wssecurity-secext-1.1.xsd"
            wsse11:TokenType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0">
            <ns3:KeyIdentifier
                ValueType="http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLID">_C5CE27CE46751556E113662938382792
            </ns3:KeyIdentifier>
        </ns3:SecurityTokenReference>
    </ds:KeyInfo>
</ds:Signature>
...

You can see that KeyIdentifier into Signature references SAML token ID="_C5CE27CE46751556E113662938382792". That means, the certificate from SAML SubjectConfirmation must be used to verify this signature.

In both scenarios STS should validate the user certificate.

User Certificate Validation

Validation through local java keystore

STS validates client certificate using verifyTrust() method of WSS4J Crypto interface. Default implementation of Crypto interface is Merlin. Merlin provider is keystore based, that means, user certificate will be validated using local STS java keystore. Merlin checks first is user certificate itself in STS keystore. If yes, validation is successfully finished. If no, Merlin proves if keystore contains all trusted chain certificates.

The problem in this approach is that STS local java keystore should contain either every client certificate or all certificates from certificate trusted chain. Furthermore, administrator should manage revocation lists for all client certificates in STS keystore.

That is not really acceptable for enterprise environments hosting a lot of clients. One possibility to resolve this issue is using Crypto XKMS implementation accessing central Public Key Infrastructure (PKI) to validate user certificate.

Validation through PKI

Using of XKMS based Crypto implementation is depicted bellow:

You can see that STS uses XKMS based Crypto implementation instead Merlin. XKMS Crypto still gets STS private key from local java keystore, but it uses central PKI to validate user certificates. W3C specifies standard SOAP interface to access and administrate keys remotely: XML Key Management Specification (XKMS 2.0). XKMS Crypto provider invokes XKMS service to validate user certificate. Dependent on XKMS service implementation, it either validates user certificate locally or delegates validation call to remote PKI. Administrator can manage certificates and revocation lists either directly in PKI or through XKMS registration interface (XKRSS). Note, that beginning from CXF 3.0.0, XKMS service and client implementation as well as XKMS Crypto Provider are available in CXF distribution.

Configure STS to use XKMS Crypto Provider

How to say STS that it must use XKMS Crypto Provider instead default keystore based one (Merlin)?

The XKMS Crypto Provider should be instantiated and set as property either in StaticSTSProperties object or in STS jaxws:endpoint. That can be done either programmatic or via Spring/Blueprint configuration.
The finished tutorial source code for STS configured with XKMS crypto provider is prepared on xkms_symmetric_tutorial. It is slightly modified code from Glen Mazza blog.

Sample spring configuration looks like:

jaxws:client is configuration of XKMS client, it is used as constructor argument of xkmsCryptoProviderFactory. Argument "/stsKeystore.properties" of xkmsCryptoProvider is necessary to create default keystore based crypto provider. It is responsible to get STS private key from the local keystore. Lookup and validation of the public keys (X509) will be delegated to XKMS crypto provider and XKMS service.

Alternatively XKMS Crypto Provider can be set directly into STS jaxws:endpoint:

...

<jaxws:endpoint id="transportSTS"
       implementor="#transportSTSProvider"
        address="/SecurityTokenService/Transport"
        wsdlLocation="wsdl/ws-trust-1.4-service.wsdl"
        xmlns:wst="http://docs.oasis-open.org/ws-sx/ws-trust/200512/"
        serviceName="wst:SecurityTokenService"
        endpointName="wst:Transport_Port">
        <jaxws:properties>

...

</jaxws:endpoint>

Using XKMS for STS scenario with Symmetric Key

This feature can be especially useful for STS scenario with SymmetricKey. With this scenario, the STS and the WS consumer negotiate a symmetric key.


STS scenario with SymmetricKey (used from SAP resource)

The WS-Client authenticates himself to STS and contributes material to the creation of symmetric key.
The STS verifies WS-Client authentication and generates symmetric key using material received from WS-Client
The STS encrypts symmetric key using WS-Service public key and inserts the encrypted key together with security token into SAML assertion. The STS signs SAML assertion and sends it together with key material for generation symmetric key to the WS-Client.
The WS-Client generates short-lived symmetric key from own material and the key material from the STS.
The WS-Client inserts the SAML token, into the message header. It encrypts the message texts or/and signs the message with the generated symmetric key. It then sends the user's message to the WS-Service.
The WS-Service checks the signature in the SAML token and uses its private key to decrypt the symmetric key contained in the SAML token.
The WS-Service verifies the signature of the WS-Client (Holder-of-Key) with the decrypted symmetric key. In this way, the STS confirms that the Holder-of-Key is the subject (the user) in the assertion. The WS-Service uses the symmetric key to decrypt the message text.

On the step (3) STS needs the public key (certificate) of target WS-Service. Normally STS servers not only one, but multiple services (restricted by url patterns in TokenServiceProvider). This can be a serious drawback to manage public certificates of all services into STS local keystore.
XKMS Crypto provider provides elegant solution of this using following configuration:

encryptionUsername (in StaticSTSProperties or jaxws:endpoint properties) should be set into special value: useEndpointAsCertAlias (STSConstants.USE_ENDPOINT_AS_CERT_ALIAS)
encryptionCrypto should be set to XKMS Crypto implementation
Service certificates should be saved into XKMS under service endpoint (use Application "urn:apache:cxf:service:endpoint" and service endpoint as identifier)

Spring configuration looks like:

<bean id="xkmsExtensions"
        class="org.apache.cxf.xkms.model.extensions.AdditionalClassesFactory" />

    <jaxws:client xmlns:serviceNamespace="http://www.w3.org/2002/03/xkms#wsdl"
        id="xkmsClient" serviceClass="org.w3._2002._03.xkms_wsdl.XKMSPortType"
        serviceName="serviceNamespace:XKMSService" endpointName="serviceNamespace:XKMSPort"
        address="http://localhost:8080/xkms/XKMS/">
        <jaxws:properties>
            <entry key="jaxb.additionalContextClasses">
                <bean class="java.lang.Object" factory-bean="xkmsExtensions"
                    factory-method="create" />
            </entry>
        </jaxws:properties>
    </jaxws:client>

    <bean id="xkmsCryptoProviderFactory" class="org.apache.cxf.xkms.crypto.impl.XkmsCryptoProviderFactory">
        <constructor-arg ref="xkmsClient"/>
    </bean>

    <bean id="xkmsCryptoProvider" class="org.apache.cxf.xkms.crypto.impl.XkmsCryptoProvider"
          factory-bean="xkmsCryptoProviderFactory"
          factory-method="create">
          <constructor-arg type="java.lang.String" value="/stsKeystore.properties" />
    </bean>
    

    <bean id="mySTSProviderBean"
        class="org.apache.cxf.sts.provider.DefaultSecurityTokenServiceProvider">
        <property name="stsProperties" ref="mySTSProperties" />
        <property name="services" ref="myServiceList" />
    </bean>

    <bean id="myServiceList" class="org.apache.cxf.sts.service.StaticService">
        <property name="endpoints" ref="wspAllowedEndpoints" />
    </bean>

    <util:list id="wspAllowedEndpoints">
        <value>http://localhost:8080/doubleit/services/doubleit.*</value>
    </util:list>

    <bean id="mySTSProperties" class="org.apache.cxf.sts.StaticSTSProperties">
        <property name="signatureUsername" value="mystskey" />
        <property name="callbackHandlerClass" value="sts.PasswordCallbackHandler" />
        <property name="issuer" value="DoubleItSTSIssuer" />
        <property name="encryptionUsername" value="useEndpointAsCertAlias"/>
        <property name="encryptionCrypto" ref="xkmsCryptoProvider" />
        <property name="signatureCrypto" ref="xkmsCryptoProvider" />
    </bean>

    <jaxws:endpoint id="CXFSTS" implementor="#mySTSProviderBean"
        address="/STS" wsdlLocation="/WEB-INF/wsdl/DoubleItSTSService.wsdl"
        xmlns:ns1="http://docs.oasis-open.org/ws-sx/ws-trust/200512/"
        serviceName="ns1:SecurityTokenService" endpointName="ns1:STS_Port">
        <jaxws:properties>
            <entry key="ws-security.callback-handler" value="sts.PasswordCallbackHandler" />
            <entry key="ws-security.signature.properties" value="stsKeystore.properties" />
            <entry key="ws-security.signature.username" value="mystskey" />
            
            <entry key="ws-security.encryption.username" value="useReqSigCert" />
        </jaxws:properties>
    </jaxws:endpoint>

In this case STS recognizes encryptionName constant and will ask XKMS Crypto for the service certificate using AppliesTo endpoint address. XKMS will locate service certificate using this endpoint address.
STS can server multiple WS-Services and doesn't care about services certificates locally - they are stored and managed in central XKMS repository.

Conclusion

In this blog we have discussed use case and possible solution to use central certificates management infrastructure inside Security Token Service (STS) to validate and locate X509 certificates.

Monday, April 1, 2013

CXF security: getting certificates from central PKI

CXF security uses asymmetric algorithms for different purposes: encryption of symmetric keys and payloads, signing security token and messages, SSL transport bindings.

If you look in samples and tutorials, the public keys (in form of X509 certificates) are normally stored in java keystores.

For example, if sender encrypts the message payload sending to the receiver, he should have access to receiver certificate saved in local keystore. The sender uses this certificate for message encryption and receiver decrypts request with corresponded own private key:

Seems to be OK? Imagine now that you have production environment with 100 different clients of this service and service certificate is expired. You should reissue and replace certificate in ALL client keystores! Even more, if keystores are packaged into war files or OSGi bundles – they should be unpackaged and updated. Not really acceptable for enterprise environments.

Therefore large service landscapes have concept of central certificates management. It means that X509 certificates are not stored locally in keystores, but are provided and administrated centrally.

Normally it is a responsibility of PublicKey Infrastructure (PKI) established in organization. PKI is responsible for create, manage, store, distribute, synchronize and revoke public certificates and certification authorities (CAs).

W3C specifies standard SOAP interface to access and administrate keys remotely: XML Key Management Specification (XKMS 2.0). XKMS contains two parts:
a) Key Information Service providing keys locating and validating functionality
b) Key Registration Service responsible for registration, revocation, recovery and reissuing of keys.
Note, that beginning from CXF 3.0.0, XKMS service and client implementation as well as XKMS Crypto Provider (see next chapter) will be available in CXF distribution.

WSS4J Crypto Providers
Fine, assume your organization has established simple PKI infrastructure and provides some kind of remote access to look up and validate the certificates (Rest or SOAP XKMS based).

The question now is the following: how to configure my CXF client and service to use public certificates not from local keystore, but from central PKI? Is there any easy way to do it?

Fortunately yes! CXF uses WSS4J for most of security aspects and WSS4J has own concept of crypto providers (don’t mix them with crypto providers of Java Cryptography Architecture).

WSS4J crypto providers are responsible to obtain X509 certificates and private keys, verify X509 trust chain and construct X509 certificates.

WSS4J crypto providers must implement Crypto interface. WSS4J also provides base abstract class with common functionality CryptoBase and keystore based implementation of crypto provider Merlin.

If you do not configure anything in CXF, WSS4J uses keystore based Merlin crypto provider by default. But, remember, we would like to get our certificates from PKI instead of keystore. Therefore it will be necessary to create own WSS4J crypto provider for this purpose. Interesting for us are two following methods:

java.security.cert.X509Certificate[] getX509Certificates(CryptoType cryptoType)

boolean verifyTrust(java.security.cert.X509Certificate[] certs, boolean enableRevocation)

Frist method looks up and returns X509 certificates based on CryptoType identifier. CryptoType specifies certificate identifier as subject DN, issuer DN and serial number, alias, thumbprint or SKI bytes.

We still obtain private keys from keystore, therefore corresponded methods will be delegated to standard Merlin provider. Skeleton implementation of PKI crypto provider can look like:

public class PKICryptoProvider extends CryptoBase {
    private Crypto defaultCrypto;
    private Properties keystoreProps;

    public PKICryptoProvider() {
        try {
            // load keystore properties and alias for private keys
            // …
            defaultCrypto = CryptoFactory.getInstance(keystoreProps);
        } catch (WSSecurityException e) {
            throw new IllegalStateException(
                    "Cannot instantiate default crypto provider: "
                            + e.getMessage(), e);
        }
    }

    @Override
    public X509Certificate[] getX509Certificates(CryptoType cryptoType)
            throws WSSecurityException {
        CryptoType.TYPE type = cryptoType.getType();
        X509Certificate[] certs = new X509Certificate[0];
        switch (type) {
        case SUBJECT_DN: {
            // get X509 certificate remotely from PKI on the base of subjectDN
            break;
        }
        case ALIAS: {
            // get X509 certificate remotely from PKI on the base of alias
            break;
        }
        case ISSUER_SERIAL: {
            // get X509 certificate remotely from PKI on the base of issuer DN
            // and serial number
            break;
        }
        default: {
            throw new IllegalArgumentException("Not supported cryptoType: "
                    + cryptoType);
        }
        }
        return certs;
    }

    @Override
    public String getX509Identifier(X509Certificate cert) {
        return cert.getSubjectDN().getName();
    }

    @Override
    public PrivateKey getPrivateKey(X509Certificate certificate,
            CallbackHandler callbackHandler) throws WSSecurityException {
        // …
        return defaultCrypto.getPrivateKey(identifier, password);
    }

    @Override
    public PrivateKey getPrivateKey(String identifier, String password)
            throws WSSecurityException {
        // …
        return defaultCrypto.getPrivateKey(identifier, password);
    }

    @Override
    public boolean verifyTrust(X509Certificate[] certs)
            throws WSSecurityException {
        return verifyTrust(certs, false);
    }

    @Override
    public boolean verifyTrust(X509Certificate[] certs, boolean enableRevocation)
            throws WSSecurityException {
        // call PKI remote method to validate certificate trust chain
        return false;
    }

    @Override
    public boolean verifyTrust(PublicKey publicKey) throws WSSecurityException {
        // call PKI remote method to validate certificate trust chain
        return false;
    }
}

As far as crypto provider makes a lot of remote calls, it makes sense to care about certificate caching.

Using PKI Crypto provider our message encryption picture will change in following way:

CXF distributions (starting from version 3.0.0) will contain XKMS based WSS4J Crypto implementation, so you can just reuse it.

Configure custom WSS4J Crypto Providers
How CXF and WSS4J know that they should use our custom crypto provider instead standard Merlin? There are some ways to configure it.

Keystore properties
You can specify own crypto provider in keystore properties file.
clientKeystore.properties:

org.apache.ws.security.crypto.provider=org.company.security.PKICryptoProvider
org.apache.ws.security.crypto.merlin.keystore.type=jks
org.apache.ws.security.crypto.merlin.keystore.password=secret
org.apache.ws.security.crypto.merlin.keystore.private.password=secret
org.apache.ws.security.crypto.merlin.keystore.alias=myclient
org.apache.ws.security.crypto.merlin.keystore.file=./etc/keystores/clientstore.jks

After it you configure keystore properties for CXF client or service as usual. For example in Spring it will be looks like:

<jaxws:client id="MyClient" xmlns:serviceNamespace="http://services.company.org/MyService"
    serviceClass="org.company.services.MyService" serviceName="serviceNamespace:MyServiceProvider"
    endpointName="serviceNamespace:MyServicePort" address="http://localhost:8080/services/MyService">
    <jaxws:properties>
        <entry key="ws-security.encyption.properties" value="clientKeystore.properties" />
        <entry key="ws-security.signature.properties" value="clientKeystore.properties" />
    </jaxws:properties>
</jaxws:client>

ws-security.encryption.properties says CXF that clientKeystore.properties will be used for the encryption. WSS4J will instantiate and call our PKICryptoProvider automatically. The same keystore properties can be specified for signature as well (ws-security.signature.properties).

Crypto provider object in Spring configuration
Sometimes more convenience way is to instantiate custom crypto provider explicitly. For example, if you want to inject proxy to communicate with PKI into PKICryptoProvider or use non-default constructor. For such cases CXF provides ws-security.encryption.crypto and ws-security.signature.crypto properties. You can pass pre-constructed PKICryptoProvider object using these properties.

Spring configuration in this case looks like:

<bean id="pkiCryptoProvider" class="org.company.security.PKICryptoProvider">
<!—-inject proxies, fields here -->
</bean>

<jaxws:client id="MyClient" xmlns:serviceNamespace="http://services.company.org/MyService"
    serviceClass="org.company.services.MyService" serviceName="serviceNamespace:MyServiceProvider"
    endpointName="serviceNamespace:MyServicePort" address="http://localhost:8080/services/MyService">
    <jaxws:properties>
        …
        <entry key="ws-security.encyption.crypto" value-ref=" pkiCryptoProvider " />
        <entry key="ws-security.signature.crypto" value-ref=" pkiCryptoProvider " />
    </jaxws:properties>
</jaxws:client>

Of course you can use the same approach to configure PKICryptoProvider for CXF service (jaxws:endpoint).

Specify Crypto provider object programmatically
Crypto provider object can be also passed programmatically. Client code will look like:

PKICryptoProvider pkiCryptoProvider = new PKICryptoProvider();

SOAPService ss = new SOAPService(wsdlURL, SERVICE_NAME);
Greeter port = ss.getSoapPort();
Map<String , Object> requestContext =
    ((javax.xml.ws.BindingProvider)port).getRequestContext();
    requestContext.put(SecurityConstants.ENCRYPTION_CRYPTO,
    pkiCryptoProvider);
    requestContext.put(SecurityConstants.SIGNATURE_CRYPTO,
    pkiCryptoProvider);

Remember that this code is not thread safe. To use thread safe request context it is necessary to add following property:

((BindingProvider)proxy).getRequestContext().put("thread.local.request.context", "true");

On the service side you can use this code:

…
@Resource
private WebServiceContext wsContext;
public ResponseData businessMethod(requestData) {
    PKICryptoProvider pkiCryptoProvider = new PKICryptoProvider();

    wsContext.getMessageContext().put(SecurityConstants.ENCRYPTION_CRYPTO, pkiCryptoProvider);
    wsContext.getMessageContext().put(SecurityConstants.SIGNATURE_CRYPTO, pkiCryptoProvider);
}

And in interceptor:

public class CustomSecurityInterceptor extends AbstractPhaseInterceptor<Message> {
    public CustomSecurityInterceptor () {
        super(Phase.PRE_LOGICAL);
    }

    @Override
    public void handleMessage(Message message) throws Fault {
       PKICryptoProvider pkiCryptoProvider = new PKICryptoProvider();

       message.put(SecurityConstants.ENCRYPTION_CRYPTO, pkiCryptoProvider);
       message.put(SecurityConstants.SIGNATURE_CRYPTO, pkiCryptoProvider);
}

Conclusion
In this blog we have discussed requirements and use cases to use central certificates management infrastructure, showed how to create custom PKI based crypto provider and configure it in CXF clients/services. PKI crypto provider can be used for communication with Security Token Service (STS) and inside STS to validate X509 certificates as well. But it is topic for the next blog.

Sunday, December 16, 2012

Using WS-Policy in CXF projects

WS-Policy provides flexible mechanism to activate desired functionality on the client or service sides. Article describes how to define policies in custom CXF projects, implement policy-aware interceptors and explains some aspects of internal CXF design regarding WS-Policy.

How to define policies

There are basically 3 main possibilities to define WS-Policy in CXF projects:

WSDL Policy attachment
Spring configuration
Dynamically via message context property

Let look into them in details.

WSDL Policy attachment

WS-Policies can be attached and referenced in WSDL elements. Web Services Policy 1.5 - Attachment standard describes all possible alternatives. WS-Policies can be placed inside WSDL itself or referenced as external documents. CXF will automatically recognize, read and use policies defined or referenced in WSDL. Sample of attached policy is shown below:

<wsdl:definitions name="HelloWorld" targetNamespace="http://apache.org/hello_world_soap_http" 
…
<wsdl:service name="SOAPService">
    <wsdl:port binding="tns:Greeter_SOAPBinding" name="SoapPort">
        <soap:address location="http://localhost:9000/SoapContext/SoapPort"/>
        <wsp:Policy xmlns:wsp="http://www.w3.org/ns/ws-policy">
             <wsam:Addressing xmlns:wsam="http://www.w3.org/2007/02/addressing/metadata">
                 <wsp:Policy/>
              </wsam:Addressing>
         </wsp:Policy>
    </wsdl:port>
</wsdl:service>
</wsdl:definitions>

Spring configuration

It is possible to define policies directly in Spring configuration of client and service as jaxws feature. CFX will recognize and use configured WS-Policies:
Client:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:jaxws="http://cxf.apache.org/jaxws"
       xmlns:cxf="http://cxf.apache.org/core"
       xmlns:p="http://cxf.apache.org/policy"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
http://cxf.apache.org/core http://cxf.apache.org/schemas/core.xsd
http://cxf.apache.org/policy http://cxf.apache.org/schemas/policy.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd"
http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd">
    <jaxws:client id="CRMServiceClient" name="{http://services.talend.org/CRMService}CRMServiceProvider"
            xmlns:serviceNamespace="http://services.talend.org/CRMService"
            serviceClass="org.talend.services.crmservice.CRMService"
            serviceName="serviceNamespace:CRMServiceProvider"
            endpointName="serviceNamespace:CRMServicePort"
            address="${endpoint.prefix}/CRMServiceProvider">
            <jaxws:features>
                <p:policies>
                    <wsp:PolicyReference xmlns:wsp="http://www.w3.org/ns/ws-policy" URI="classpath:/saml.policy"/>
                </p:policies>
            </jaxws:features>
    </jaxws:client>
</beans>

Service:

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:jaxws="http://cxf.apache.org/jaxws"
       xmlns:cxf="http://cxf.apache.org/core"
       xmlns:p="http://cxf.apache.org/policy"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
http://cxf.apache.org/core http://cxf.apache.org/schemas/core.xsd
http://cxf.apache.org/policy http://cxf.apache.org/schemas/policy.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd"
http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd">
    <jaxws:endpoint id="CRMService"
            xmlns:serviceNamespace="http://services.talend.org/CRMService"
            serviceName="serviceNamespace:CRMServiceProvider"
            endpointName="serviceNamespace:CRMServicePort"
            implementor="#CRMServiceBean"
            address="/CRMServiceProvider">
            <jaxws:features>
                <p:policies>
                    <wsp:PolicyReference xmlns:wsp="http://www.w3.org/ns/ws-policy" URI="classpath:/saml.policy"/>
                </p:policies>
            </jaxws:features>
    </jaxws:endpoint>
</beans>

Dynamically through message property

Sometimes policies cannot be configured statically, because they are obtained or calculated dynamically for concrete message (for example using Policy Server or Service Registry). For such cases CXF provide a possibility to load policy dynamically and set it into the message context property. It can be done for example in custom interceptor that fulfils the following:

Get policy from external location and build it for current message.
Parse WS-Policy XML using Neethi library.
Store result Policy object into PolicyConstants.POLICY_OVERRIDE message content property.
Important is that this custom policy interceptor is called before CXF PolicyInInterceptor or PolicyOutInterceptor. Than CXF will automatically recognize Policy stored into this property and use it with highest priority.

I have published a small sample illustrating how to apply policy dynamically.

Create custom policy assertions and associate interceptors

It is quite easy to define own policy assertions and associate interceptors with it. Topic is already well described in CXF document http://cxf.apache.org/docs/developing-assertions.html, I just provide a list of steps necessary to do:

Provide Assertion Builder class for custom assertion implementing AssertionBuilder<T> interface.
Interface type can be Element, XMLStreamReader or OMElement.
Interface contains two methods: build() and getKnownElements().
Implementation of build() method should construct Assertion from the incoming type. It can be PrimitiveAssertion (without attributes or child elements), NestedPrimitiveAssertion (without attributes but with nested policy element) and JaxbAssertion (assertion described by any XML schema).
getKnownElements() method must return QNames of assertion elements from which assertion can be built.
Implement policy interceptor provider class extending AbstractPolicyInterceptorProvider class. The main task of policy interceptor provider is to say which interceptors must be activated for specified policy assertion. Policy interceptor provider constructor gives assertions QNames as argument of super constructor and adds corresponded interceptors using getters:

public class AuthorizationInterceptorProvider extends AbstractPolicyInterceptorProvider {
    private static final long serialVersionUID = -5248428637449096540L;
    private static final AuthorizationInInterceptor IN_AUTHZ_INTERCEPTOR = new AuthorizationInInterceptor();
    private static final AuthorizationInInterceptor OUT_AUTHZ_INTERCEPTOR = new AuthorizationOutInterceptor();
    private static final Collection<QName> ASSERTION_TYPES;
    static {
        ASSERTION_TYPES = new ArrayList<QName>();
        ASSERTION_TYPES.add(AuthorizationConstants.AUTHORIZATION_ASSERTION);
    }
    public AuthorizationInterceptorProvider() {
        super(ASSERTION_TYPES);
        getInInterceptors().add(IN_AUTHZ_INTERCEPTOR);        
        getOutInterceptors().add(OUT_AUTHZ_INTERCEPTOR);        
    }
}

Assertion builder and policy interceptor provider can be registered using CXF bus extension mechanism: just create a file META-INF/cxf/bus-extensions.txt containing the following:

org.company.AuthorizationInterceptorProvider::true
org.company.AuthorizationAssertionBuilder::true

Boolean value at the end specifies lazy loading strategy.
CXF automatically recognizes the assertion builder and policy interceptor provider and store them into registries: AssertionBuilderRegistry and PolicyInterceptorProviderRegistry. Since CXF 2.6.0 it is possible to register multiple interceptor providers for single assertion.

How and where CXF processes policies

As I already mentioned, CXF provides two interceptors: org.apache.cxf.ws.policy.PolicyInInterceptor and org.apache.cxf.ws.policy.PolicyOutInterceptor. These interceptors are responsible to load policy from destination, parse, merge them and add all associated interceptors into message interceptor chain. Functionality of policy interceptors are represented on the following figure:

Briefly, policy interceptors make following steps:

Check message property PolicyConstants.POLICY_OVERRIDE.
If PolicyConstants.POLICY_OVERRIDE contains policy, it will be taken for further processing.
If property is empty, policy will be asked from ServiceModel. Here CXF loads policies attached to WSDL or provided via Spring configuration.
If any policy on step 2 or step 3 is found, EffectivePolicy will be created. Appropriate WS-policies will be merged for the current message and built into Neethi Policy object.
All interceptors registered for result policy assertions will be added to message interceptor chain.

Additionally, CXF verifies satisfied policy assertions in PolicyVerfificationInInterceptor, PolicyVerificationInFaultInterceptor, PolicyVerificationOutInterceptor. If assertion is not processed and not satisfied in corresponded interceptor, than In- interceptors throw Fault and Out- interceptors provide appropriate log messages.
The practical using of WS-Policy is illustrated in ws_policy and ws_security CXF samples.

Friday, November 30, 2012

CXF XSLT Feature

XSLT Feature

CXF XSLT Feature is introduced as alternative of lightweight CXF Transformation Feature.
The most transformation use cases can be implemented using Transformation Feature: dropping the namespace of the outbound messages; qualifying the incoming message; changing namespaces; appending or dropping elements; converting attributes to elements.

But it can be not enough for some advanced transformations. If you have to implement non-trivial transformation, not supported by Transformation Feature - it is use case for XSLT Feature. Here the full power of XSL is available.
Therefore decision which feature should be used is very easy: CXF Transformation Feature for simple transformations and CXF XSLT Feature for advanced transformations, not supported by lightweight variant.

Configuration

Spring configuration of XSLT Feature is very easy, it is necessary just to configure XSL script for inbound and/or outbound transformation:

<bean id="xsltFeature" class="org.apache.cxf.feature.transform.XSLTFeature">          
      <property name="inXSLTPath" value="requestTransformation.xsl" />          
      <property name="outXSLTPath" value="responseTransformation.xsl" />  
</bean>

After it, feature can be added to client or endpoint:

<jaxws:client id="customerService" serviceName="customer:CustomerServiceService"
  endpointName="customer:CustomerServiceEndpoint" address="http://localhost:9091/CustomerServicePort"
  serviceClass="com.example.customerservice.CustomerService">
  <jaxws:features>
     <ref bean="xsltFeature" />
  </jaxws:features>
</jaxws:client>

<jaxws:endpoint xmlns:customer="http://customerservice.example.com/"
  id="CustomerServiceHTTP" address="http://localhost:9090/CustomerServicePort"
  serviceName="customer:CustomerServiceService" endpointName="customer:CustomerServiceEndpoint"
  implementor="com.example.customerservice.server.CustomerServiceImpl">
  <jaxws:features>
     <ref bean="xsltFeature" />
  </jaxws:features>
</jaxws:endpoint>

It is also possible to add XSLT interceptors directly in Java code:

 CustomerServiceService service = new CustomerServiceService();
 CustomerService customerService = service.getCustomerServicePort();
 Client client = ClientProxy.getClient(customerService);
 XSLTOutInterceptor outInterceptor = new XSLTOutInterceptor(Phase.PRE_STREAM, StaxOutInterceptor.class, null,
                                                            XSLT_REQUEST_PATH);
 client.getOutInterceptors().add(outInterceptor);

By default XSLT interceptors run on POST_STREAM and PRE_STREAM phases.
In some cases it may be needed to change the phase, for example, the in transformation has to be applied after the encrypted payload has been decrypted and its signature checked.
For such transformations to succeed XLSTInInterceptor/XSLTOutInterceptor will need to be created with a constructor accepting a 'phase' String parameter.

The XSLT interceptors support following message contents:

InputStream/OutputStream;
Reader/Writer;
XMLStreamReader/XMLStreamWriter.

Therefore the interceptors can be used on different phases and can also work with JMS Transport using JMS Text messages (produces Reader/Writer message contents).

Wednesday, September 19, 2012

CXF Performance on Sun Solaris Platform: some numbers

Introduction

I would like to show some performance benchmarks for Apache CXF on SPARC/Solaris 11 Platform. CXF was deployed in OSGi Karaf container and uses Jetty as Web Server.
JMS communication was done using Apache ActiveMQ broker, database access was based on EclipseLink JPA with Apache Aries managed transactions and Oracle DB server.

Acknowledgments

A lot of thanks for colleges Marian Kueffner for composing and integration of benchmark results and Christian Schneider for supporting me with the tests.

Benchmark Goals

Measure maximum CXF service invocation throughput on a particular hardware configuration: SPARC/Solaris 11
Identify bottlenecks, areas of tuning and enhancements

Source Code
Source code is available on GitHub: cxf.benchmarks

Hardware

SPARC / Solaris 11 SUN-T4-2-FAMILY-0 30351017 SPARC T4-2 512GB – 8 GbE Ports – 4 FC Ports – 4 10GbE *604532* Ports

2x SPARC T4 8-Core 2,85 Ghz. processor

512GB RAM (32x 16GB DDR3 DIMM MODULES)

6x 600GB SAS2 10k HDD

2x Qlogic FC 8Gbps HBA PCIe

2x 10 GbE SFP+ HBA PCIe

1x 1 GbE Quad Gigabit Ethernet HBA PCIe

Software

Deployment: Karaf OSGi (Equinox based) container

Versions:

Java: 6 (build 1.6.0_26-b3)
CXF: 2.6.0
Karaf: 2.2.6
Jetty: 7.5.4
Aries: 0.3.1
ActiveMQ: 5.6.0
EclipseLink: 2.3.0

Scenarios

Benchmarks were measured for three main scenario:

(A) JAX-WS based Web Services (SOAP over http)
(B) Messaging over JMS/ActiveMQ (SOAP over JMS)
(C) Database access via Service (EclipseLink, container managed transaction, Oracle)

Every scenario was executed with messages of three sizes: small (1 KB), medium (10 KB) and large (1 MB). Service implementation simply redirects request to response and in scenario (C) additionally fulfills DB access operations. The focus of benchmarks was DOM-based message model, but for some scenarios service was tested also using stream oriented message processing. Throughput on the service side and CPU load are measured.

Benchmark results

Scenario A: small messages with DOM model; HTTP

Scenario A: small messages with stream model; HTTP

Scenario A: medium messages with DOM model; HTTP

Scenario A: large messages with DOM model; HTTP

Special Scenario: maximal service load

In this scenario service was loaded using 4 clients: three SoapUI and one CXF. Clients were activated sequentially with 1 minute interval. You can see how service load changing when the next client becomes active.

Scenario B: small messages with DOM model; JMS

Scenario B: medium messages with DOM model; JMS

Scenario B: large messages with DOM model; JMS

Scenario C: Inserts in Oracle DB using EclipseLink JPA and Aries transactions

Scenario C: Selects from Oracle DB using EclipseLink JPA and Aries transactions

Conclusion

Generally CXF shows high performance in described scenarios on Sun Solaris Platform.

I have some remarks regarding the benchmark results:

Single thread shows pretty low performance, but system is highly scalable. Perhaps Solaris should be additionally tuned for single thread.

Service (provider) is very scalable. As you can see in the Test Case 5, adding more consumers just increases the service throughput. I reach more than 10K messages per second, but it still was not the top.

JMS scenario with ActiveMQ shows relative low performance with default configuration. The numbers are much higher if persistence is deactivated or at least immediate synchronization is switched off.

Stream oriented message processing increases throughput on 25-30% even for small messages.