Skip to main content

DynamicReports and Cocoon integration

This is one more tutorial on how to exploit DynamicReports reporting library in an existing web application. This time the application is based on Cocoon 2.2 framework. This post is a continuation to the previous posts: Choosing Java reporting tool - part 2 and DynamicReports and Spring MVC integration. The complete code won't be provided here but only the essential code snippets together with usage remarks.

First of all I'm writing the plan of this how to:
  1. Description of existing application data flow.
  2. Adding project dependencies.
  3. Implementing export servlet.
  4. Implementing XSLT extension for the export data source.
  5. Implementing XSLT for postprocessing.
  6. Modifying JX template.
  7. Modifying Cocoon sitemap.
Description of existing application data flow
Cocoon framework organizes data flow using the concept of pipelines. For details please refer to Cocoon documentation. My application uses the following component sequence in a pipeline:
  • Standard JX template generator produces xquery code.
  • Custom XQuery generator executes searching xquery over Sedna XML DB and returns resulting xml data as an html table.
  • Standard XSLT transformer performs required postprocessing of data (formatting numbers and fetching data from sources other than XML DB) and returns ready to use html table with data.
  • Finally the generated html table is being asynchronously inserted into the web page (this happens outside of Cocoon pipeline and is performed by AJAX means of YUI - it won't be covered here being outside of this post's scope).
I'm going to provide some sources below for better understanding.

sitemap.xmap:
<map:pipelines>
    <map:pipeline id="services">
        <map:match pattern="xquery-macro/**">
            <map:generate src="xquery/{1}.jx" type="jx"/>
            <map:serialize type="text"/>
        </map:match>
        
        <map:match pattern="xquery/**">
            <map:generate src="cocoon:/xquery-macro/{1}" type="queryStringXquery">
                <map:parameter name="contextPath" value="{request:contextPath}"/>
            </map:generate>
            <map:transform src="xslt/postprocessXqueryResults.xslt" type="saxon"/>
            <map:serialize type="xml"/>
        </map:match>
    </map:pipeline>
</map:pipelines>

<map:components>
    <map:serializers>
        <map:serializer logger="sitemap.serializer.xml" mime-type="text/xml" name="xml" src="org.apache.cocoon.serialization.XMLSerializer">
            <encoding>UTF-8</encoding>
        </map:serializer>
    </map:serializers>
    <map:transformers>
        <map:transformer name="saxon" src="org.apache.cocoon.transformation.TraxTransformer">
            <xslt-processor-role>saxon</xslt-processor-role>
        </map:transformer>
    </map:transformers>
    <map:generators>
        <map:generator name="queryStringXquery" src="org.lagivan.prototype.generator.QueryStringXQueryGenerator">
            <map:parameter name="db-url" value="${xmldb.url}"/>
            <map:parameter name="db-user" value="${xmldb.user}"/>
            <map:parameter name="db-password" value="${xmldb.password}"/>
            <map:parameter name="cache-validity" value="${xmldb.cache-validity}"/>
        </map:generator>
    </map:generators>
</map:components>
In the sitemap you can see the first three steps described above and several components being configured inside map:components tag. I omitted mentioning text and xml serializers just for simplicity. Also I won't provide javascript that launchs this pipeline as it's out of this post's scope.

Adding project dependencies
We do use Maven to manage project dependencies. Actually in this case dependencies (that are related to DynamicReports) are exactly the same as in the previous post.

Implementing export servlet
This is the main part of export functionality that is implemented according to the official example. Servlet implementation requires writing a Java class extending HttpServlet below and also a piece of configuration that is Cocoon specific.

ExportServlet.java:
package org.lagivan.prototype.export;

import net.sf.dynamicreports.jasper.builder.JasperReportBuilder;
import net.sf.dynamicreports.report.exception.DRException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.OutputStream;
import java.util.HashMap;
import java.util.Map;

/**
 * @author Ivan Lagunov
 */
public class ExportServlet extends HttpServlet {

    private static final Logger log = LoggerFactory.getLogger(ExportServlet.class);

    private static final String PARAMETER_TYPE = "type";
    private static final String VALUE_TYPE_PDF = "pdf";
    private static final String VALUE_TYPE_XLS = "xls";

    private static final Map<String, String> FILE_TYPE_2_CONTENT_TYPE = new HashMap<String, String>();
    static {
        FILE_TYPE_2_CONTENT_TYPE.put(VALUE_TYPE_PDF, "application/pdf");
        FILE_TYPE_2_CONTENT_TYPE.put(VALUE_TYPE_XLS, "application/vnd.ms-excel");
    }

    private IExportDataSource dataSource;

    public void setDataSource(IExportDataSource dataSource) {
        this.dataSource = dataSource;
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
        String fileType = request.getParameter(PARAMETER_TYPE);
        log.info("Exporting {} report", fileType);

        response.setContentType(FILE_TYPE_2_CONTENT_TYPE.get(fileType));
        OutputStream out = response.getOutputStream();
        try {
            JasperReportBuilder jrb = createJasperReport();

            if (VALUE_TYPE_PDF.equals(fileType)) {
                jrb.toPdf(out);
            } else if (VALUE_TYPE_XLS.equals(fileType)) {
                jrb.toExcelApiXls(out);
            }
        } catch (DRException e) {
            throw new ServletException(e);
        }
        out.close();
    }

    private JasperReportBuilder createJasperReport() {
        // Here I used DynamicReports API to build a report 
        // and to fill it with the datasource.
    }
}
The servlet must be defined properly in the Spring servlet configuration file.

myblock-servlet-service.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:servlet="http://cocoon.apache.org/schema/servlet"
       xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://cocoon.apache.org/schema/servlet http://cocoon.apache.org/schema/servlet/cocoon-servlet-1.0.xsd">

    <bean class="org.apache.cocoon.sitemap.SitemapServlet" name="org.lagivan.prototype.myblock.service">
        <servlet:context context-path="blockcontext:/myblock/" mount-path="/myblock">
            <servlet:connections>
                <entry key="ajax" value-ref="org.apache.cocoon.ajax.impl.servlet"/>
                <entry key="forms" value-ref="org.apache.cocoon.forms.impl.servlet"/>
                <entry key="export" value-ref="org.lagivan.prototype.export.service"/>
            </servlet:connections>
        </servlet:context>
    </bean>

    <bean class="org.lagivan.prototype.export.ExportServlet" name="org.lagivan.prototype.export.service">
        <servlet:context context-path="blockcontext:/myblock/" mount-path="/export"/>
        <property name="dataSource" ref="exportDataSource"/>
    </bean>
</beans>
You might have got a question where exportDataSource bean is defined. Well, it's defined in the Spring application context configuration file.

myblock-application-context.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">

    <bean class="org.lagivan.prototype.export.ExportDataSource" id="exportDataSource" scope="session">
        <aop:scoped-proxy proxy-target-class="false"/>
    </bean>

    <bean class="org.lagivan.prototype.xslt.ExportDataSourceExtension" id="exportDataSourceExtension">
        <property name="dataSource" ref="exportDataSource"/>
    </bean>
</beans>
Here you can see two beans: exportDataSource and exportDataSourceExtension. The first is a storage for the data source that implements an interface IExportDataSource to access stored data; the second is a Java XSLT extension that simply delegates calls to the data source object (sources for these files will be given in the next section below). Also it's worth mentioning that exportDataSource needs to be proxied with aop:scoped-proxy due to its session scope (refer the Spring reference for the details).

Implementing XSLT extension for the export data source
Thus, Java XSLT extension is meant for saving xquery search results data into the session-scoped data source property. The same data source object is used then by the export servlet to generate the final report document. In this section I'll provide sources for Java XSLT extension and the Java data source.

IExportDataSource.java:
package org.lagivan.prototype.export;

import net.sf.jasperreports.engine.JRRewindableDataSource;

/**
 * @author Ivan Lagunov
 */
public interface IExportDataSource extends JRRewindableDataSource {

    // To be used inside ExportServlet.createJasperReport() to set columns. 
    String[] getColumns();

    // To be called from Java XSLT extension to set columns and their values.
    void setColumns(String... columns);
    void setColumnValue(int columnIndex, String value);

    // To be called from Java XSLT extension after all column values for a single table row are already set.
    void onColumnValueCompleted();
}
ExportDataSource.java:
/**
 * DynamicReports - Free Java reporting library for creating reports dynamically
 *
 * Copyright (C) 2010 - 2011 Ricardo Mariaca
 * http://dynamicreports.sourceforge.net
 *
 * This file is part of DynamicReports.
 *
 * DynamicReports is free software: you can redistribute it and/or modify
 * it under the terms of the GNU Lesser General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * DynamicReports is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License
 * along with DynamicReports. If not, see <http://www.gnu.org/licenses/>.
 */
package org.lagivan.prototype.export;

import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JRField;

import java.util.*;

/**
 * @author Ricardo Mariaca (dynamicreports@gmail.com)
 * @author Ivan Lagunov
 */
public class ExportDataSource implements IExportDataSource {
    private String[] columns;
    private String[] columnValues;
    private List<Map<String, Object>> values;
    private Iterator<Map<String, Object>> iterator;
    private Map<String, Object> currentRecord;

    public ExportDataSource() {
    }

    @Override
    public String[] getColumns() {
        return columns;
    }

    @Override
    public void setColumns(String... columns) {
        this.columns = columns;
        this.values = new ArrayList<Map<String, Object>>();
        this.columnValues = new String[columns.length];
        Arrays.fill(columnValues, "");
    }

    @Override
    public void setColumnValue(int columnIndex, String value) {
        columnValues[columnIndex] = value;
    }

    @Override
    public void onColumnValueCompleted() {
        add(columnValues);
        Arrays.fill(columnValues, "");
    }

    @Override
    public Object getFieldValue(JRField field) throws JRException {
        return currentRecord.get(field.getName());
    }

    @Override
    public boolean next() throws JRException {
        if (iterator == null) {
            this.iterator = values.iterator();
        }
        boolean hasNext = iterator.hasNext();
        if (hasNext) {
            currentRecord = iterator.next();
        }
        return hasNext;
    }

    @Override
    public void moveFirst() throws JRException {
        this.iterator = null;
    }

    private void add(Object... values) {
        Map<String, Object> row = new HashMap<String, Object>();
        for (int i = 0; i < values.length; i++) {
            row.put(columns[i], values[i]);
        }
        this.values.add(row);
    }
}
The data source code is based on net.sf.dynamicreports.examples.DataSource taken from DynamicReports zip archive. It can be easily set for a report using JasperReportBuilder.setDataSource(dataSource) later on (that's how I did in ExportServlet). As you can see, the ExportDataSource simply saves String values for each column of a row and stores them at once when onColumnValueCompleted happens.

ExportDataSourceExtension.java:
package org.lagivan.prototype.xslt;

import org.lagivan.prototype.export.IExportDataSource;

/**
 * @author Ivan Lagunov
 */
public class ExportDataSourceExtension {

    private static IExportDataSource dataSource;

    public void setDataSource(IExportDataSource dataSource) {
        ExportDataSourceExtension.dataSource = dataSource;
    }

    public static void setColumns(String... columns) {
        dataSource.setColumns(columns);
    }

    public static void setColumnValue(int columnIndex, String value) {
        dataSource.setColumnValue(columnIndex - 1, value);
    }

    public static void onColumnValueCompleted() {
        dataSource.onColumnValueCompleted();
    }
}
You can refer to this tutorial about extending XSLT with Java for more details. The main thing to understand that it's recommended to make a method static to be able to call it simply from XSLT. In spite of being a static field, the dataSource will still be session-scoped as soon as you mark the Spring bean with aop:scoped-proxy.

Implementing XSLT for postprocessing
Now let's turn to the XSLT code that saves data in the ExportDataSource using the ExportDataSourceExtension.

postprocessExport.xslt:
<?xml version="1.0" encoding="UTF-8"?>
<!--
  Author: Ivan Lagunov
  This stylesheet saves XQuery search results in ExportDataSource
-->
<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:exp="java:org.lagivan.prototype.xslt.ExportDataSourceExtension">

  <xsl:output method="html" omit-xml-declaration="yes"/>

  <!-- set columns -->
  <xsl:template match="thead">
    <xsl:copy>
      <xsl:value-of select="exp:setColumns(data(tr/th[not(@exportIgnore)]/text()))"/>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!-- set column values -->
  <xsl:template match="tbody">
    <xsl:copy>
      <xsl:for-each select="tr">
        <xsl:for-each select="td[not(@exportIgnore)]">
          <xsl:choose>
            <xsl:when test="a/text()">
              <xsl:value-of select="exp:setColumnValue(position(), data(a/text()))"/>
            </xsl:when>
            <xsl:when test="normalize-space(.)">
              <xsl:value-of select="exp:setColumnValue(position(), normalize-space(.))"/>
            </xsl:when>
          </xsl:choose>
        </xsl:for-each>
        <xsl:value-of select="exp:onColumnValueCompleted()"/>
      </xsl:for-each>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!-- remove exportIgnore attributes -->
  <xsl:template match="@exportIgnore"/>

  <!-- copy all nodes and attributes which are not processed by one of available templates -->
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:apply-templates/>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>

Modifying JX template
Here is a sample of JX template that the postprocessExport.xslt can be applied to.

xquery_search.jx:
<?xml version="1.0" encoding="UTF-8"?>


  <jx:macro name="xquery-macro">
    <set-results/>
    return
    <return-results/>
  </jx:macro>

  
    <![CDATA[
    let $results := collection("products")/ProductItem
    ]]>
  

  
    <![CDATA[
    <table style="width: 100%" id="search-results-table">
      <caption class="header">
        <span style="float:left">{count($results)} result(s) found</span>
        <span style="float:right">
          <a title="Export to Excel" href="{$contextPath}/export?type=xls">
            <img style="border: none;" src="resource/external/icons/mimetypes/xls_16.png"/>
          </a>
          <a title="Export to PDF" href="{$contextPath}/export?type=pdf">
            <img style="border: none;" src="resource/external/icons/mimetypes/pdf_16.png"/>
          </a>
        </span>
      </caption>
      <thead>
        <tr>
          <th class="header">ID</th>
          <th class="header">Product name</th>
          <th class="header" exportIgnore="">Download</th>
        </tr>
      </thead>
      <tbody>
      {
        for $result in $results
        let $id := data($result/@id)
        order by $id
        return 
        <tr id="{$id}">
          <td><a href="{$contextPath}/myblock/product/preview.html?id={$id}" target="_blank">{$id}</a></td>
          <td>{$result/Name/text()}</td>
          <td exportIgnore="">
            <a href="{$contextPath}/myblock/product/download/{$id}.xml" target="_blank">
              <img src="resource/external/icons/mimetypes/xml_16.png"/>
            </a>
          </td>
        </tr>
      }    
      </tbody>
    </table>
    ]]>
  

As you can see, this template produces a valid xquery code to output products data in the html table format. It worked flawlessly even before adding the export feature. As soon as export feature was implemented, I added exportIgnore attribute to filter out unnecessary column in export results and also links in the caption to trigger the ExportServlet.

Modifying Cocoon sitemap
Finally it's only the Cocoon sitemap left to be updated. We need to process XQuery results with postprocessExport.xslt and to enable ExportServlet to process "{$contextPath}/export" URL. I provided the initial sitemap.xmap contents above and here are the changes.

sitemap.xmap:
<map:pipelines>
    <map:pipeline id="services">
        <map:match pattern="xquery/**">
            <map:generate src="cocoon:/xquery-macro/{1}" type="queryStringXquery">
                <map:parameter name="contextPath" value="{request:contextPath}"/>
            </map:generate>
            <map:transform src="xslt/postprocessXqueryResults.xslt" type="saxon"/>
            <map:transform src="xslt/postprocessExport.xslt" type="saxon"/>
            <map:serialize type="xml"/>
        </map:match>

        <map:match pattern="export">
            <map:generate src="servlet:export:/"/>
        </map:match>
    </map:pipeline>
</map:pipelines>

Comments

  1. One small note about using jx:templates to generate the XQuery. I chose that approach because I was not sure wheter it made sense to store application specific (but nonetheless reusable) xquery-parts as modules in Sedna XMLDB itself. But after having done both, I feel that this approach was the wrong one and storing xquery modules makes much more sense. Even if they are application specific, one could separate app-specific functions in a separate module. The benefit from using modules is that you can more easily import functions at any time.

    ReplyDelete
  2. Thanks for the remark!

    I've just imagined how much it will influence the development process.

    The advantage is an opportunity to deploy XMLDB modules without any significant delay, thus no interruption during the application work. But it's not that much valuable in our case.

    However, in this post XSLT processing logic is closely related to the syntax generated by XQuery. I feel like it's not a nice design solution to divide application logic and move application specific code to XML DB. Moreover, if we're going to set up any Continuous Integration tool finally (which is worth doing btw - to be discussed), we may face some related issues here.

    ReplyDelete
  3. You make a valid statement. I don't even care if they are stored in the XMLDB itself. The problem is that you can only have a single source where to import xquery modules from. At least this is the case for Sedna. If we would be able to store actual modules inside our Cocoon app as well, and somehow configure we would import them from our own app, that would be the best solution.

    ReplyDelete
  4. I got your idea. I agree it would be really nice to have such an opportunity. However, it's indeed not available in Sedna XML DB for now. I've created a feature request for them: http://sourceforge.net/tracker/?func=detail&atid=713733&aid=3446873&group_id=129076

    ReplyDelete
  5. Nice. I actually did investigate this possibility in the past and stumbled upon this article explaining that vendors could implement such a feature: http://www.stylusstudio.com/xquery/xquery_functions.html

    E.g.:
    import module namespace hr = "http://hr.example.com/" at "hr-module.xquery";

    You might as well have found this mail ;-)
    http://article.gmane.org/gmane.text.xml.sedna/2318

    ReplyDelete

Post a Comment

Popular posts from this blog

Connection to Amazon Neptune endpoint from EKS during development

This small article will describe how to connect to Amazon Neptune database endpoint from your PC during development. Amazon Neptune is a fully managed graph database service from Amazon. Due to security reasons direct connections to Neptune are not allowed, so it's impossible to attach a public IP address or load balancer to that service. Instead access is restricted to the same VPC where Neptune is set up, so applications should be deployed in the same VPC to be able to access the database. That's a great idea for Production however it makes it very difficult to develop, debug and test applications locally. The instructions below will help you to create a tunnel towards Neptune endpoint considering you use Amazon EKS - a managed Kubernetes service from Amazon. As a side note, if you don't use EKS, the same idea of creating a tunnel can be implemented using a Bastion server . In Kubernetes we'll create a dedicated proxying pod. Prerequisites. Setting up a tunnel.

Notes on upgrade to JSF 2.1, Servlet 3.0, Spring 4.0, RichFaces 4.3

This article is devoted to an upgrade of a common JSF Spring application. Time flies and there is already Java EE 7 platform out and widely used. It's sometimes said that Spring framework has become legacy with appearance of Java EE 6. But it's out of scope of this post. Here I'm going to provide notes about the minimal changes that I found required for the upgrade of the application from JSF 1.2 to 2.1, from JSTL 1.1.2 to 1.2, from Servlet 2.4 to 3.0, from Spring 3.1.3 to 4.0.5, from RichFaces 3.3.3 to 4.3.7. It must be mentioned that the latest final RichFaces release 4.3.7 depends on JSF 2.1, JSTL 1.2 and Servlet 3.0.1 that dictated those versions. This post should not be considered as comprehensive but rather showing how I did the upgrade. See the links for more details. Jetty & Tomcat. JSTL. JSF & Facelets. Servlet. Spring framework. RichFaces. Jetty & Tomcat First, I upgraded the application to run with the latest servlet container versio

Extracting XML comments with XQuery

I've just discovered that it's possible to process comment nodes using XQuery. Ideally it should not be the case if you take part in designing your data formats, then you should simply store valuable data in plain xml. But I have to deal with OntoML data source that uses a bit peculiar format while export to XML, i.e. some data fields are stored inside XML comments. So here is an example how to solve this problem. XML example This is an example stub of one real xml with irrelevant data omitted. There are several thousands of xmls like this stored in Sedna XML DB collection. Finally, I need to extract the list of pairs for the complete collection: identifier (i.e. SOT1209 ) and saved timestamp (i.e. 2012-12-12 23:58:13.118 GMT ). <?xml version="1.0" standalone="yes"?> <!--EXPORT_PROGRAM:=eptos-iso29002-10-Export-V10--> <!--File saved on: 2012-12-12 23:58:13.118 GMT--> <!--XML Schema used: V099--> <cat:catalogue xmlns:cat=