Wrapping up all the work

As reaching the hard deadline of the pencil down day, I have clear up the code and, write tests and finished writing the documents. As requested by the Lluis previously i have created separate pull requested for my Reporting.REST module works.

I have created separate branches for each pull requested.

• Report def: https://github.com/openmrs/openmrs-module-reportingrest/pull/6
• Cohort def: https://github.com/openmrs/openmrs-module-reportingrest/pull/7
• Datasetdef: https://github.com/openmrs/openmrs-module-reportingrest/pull/8
• Report resource: https://github.com/openmrs/openmrs-module-reportingrest/pull/9
• Report processor: https://github.com/openmrs/openmrs-module-reportingrest/pull/10
• Evaluated report: https://github.com/openmrs/openmrs-module-reportingrest/pull/11
• Evaluated cohort: https://github.com/openmrs/openmrs-module-reportingrest/pull/12
• Evaluated dataset: https://github.com/openmrs/openmrs-module-reportingrest/pull/13
• Report request: https://github.com/openmrs/openmrs-module-reportingrest/pull/14
• Dataset req: https://github.com/openmrs/openmrs-module-reportingrest/pull/15
• Cohort req: https://github.com/openmrs/openmrs-module-reportingrest/pull/16
• Test configure: https://github.com/openmrs/openmrs-module-reportingrest/pull/17

Repository for the Pentaho data integration for the OpenMRS https://github.com/sashrika/openmrs-contrib-reportingpdiplugin.

pull request for the PDI plugin https://github.com/djazayeri/openmrs-contrib-reportingpdiplugin/pull/1

Additional documents 

Proposal page for the Reporting.REST module https://wiki.openmrs.org/display/docs/Proposal+after+Design+Forum+June+12,+2013

project page for the Reporting.REST module https://wiki.openmrs.org/display/projects/Reporting+REST+web+services+Enhancements+-+Project+Plan

Documentation on Reporting.REST  : https://wiki.openmrs.org/display/~sashrika/Reporting.REST+enhancements

Design page for the PDI plugin https://wiki.openmrs.org/display/~sashrika/Pentaho+Data+Integration+for+OpenMRS+project+page

User manual for the PDI plugin :
https://wiki.openmrs.org/display/~sashrika/PDI+plugin+for+OpenMRS+-+User+Manual

You Tube Video for PDI plugin

http://www.youtube.com/watch?v=QbVQNy8i6_s

http://youtu.be/_z-OcvNEViM

http://youtu.be/EgMSjCM3rWY

You Tube Video for Reporting.REST web services

http://www.youtube.com/watch?v=vUNcBO14c0s

Finishing PDI plugin

As mentioned in the Proposal page, I finished works in the PDI plugin.

Now it can pull data from OpenMRS server via it Reporting.REST web services.
The PDI plugin now can get EvaluatedReportDefinition data, EvaluatedCohortDefinition data and EvaluatedDataSetDefinition data to the Pentaho Spoon. It is working like a Input step.

You can also give parameters for the execution.

Guidelines for use Pentaho Data Integration plugin

• Buid the Project using maven
• Go to the project folder in your machine
• In "openmrs-contrib-reportingpdiplugin\target\classes" there will be a plugin.xml file and openmrs_logo_tiny.png file. Copy both files to a seperate folder.
• Go to "openmrs-contrib-reportingpdiplugin\target" there will be a openmrs-contrib-reportingpdiplugin-1.0-SNAPSHOT.jar. Copy that file to the previous folder.
• Go to Pentaho installation folder. go to "design-tools\data-integration\plugins\steps folder" in the installation folder.
• past your folder to that location.
• Restart spoon.
• click File->new transformation
• In the transformation window click "input"
• "OpenMRS lookup" step will be shown in the list.
• Drag and drop that icon to the workspace.

• Double click on the icon
• A diolog box will be opened.
• set server url,password and username.

• if you want to get the evaluated data of a report definition, put the uuid of the report definition in the ReportDefinition text area.

• if you want to get the evaluated data of a cohort definition, put the uuid of the cohort definition in the CohortDefinition text area.

• if you want to get the evaluated data of a DataSet definition, put the uuid of the DataSet definition in the DataSetDefinition text area.

• if you want to get the evaluated data of a DataSet definition along with a Cohort Definition, put the uuid of the DataSet definition in the DataSetDefinition text area, and put the uuid of the Cohort Definition in the CohortDefinition text area.

• If any definition require parameters, give the parameters in the Parameter text area.

          Eg:-
          minAge=10;maxAge=31

• parameters should separate by a ";". you should use the "name" of the parameter. not the "label" of that parameter.

• Then you can perform any transformation to the OpenMRS data which is given by the OpenMRS step plugin.

Finishing minor works in the Reporting.REST

I realized that I need to finish the EvaluatedReportResource in Reporting.REST module. We need to pull Evaluated Report data in tabular format for do transformations in the Pentaho Data Integration. 

So I completed the EvaluatedReportResource  and also the Report/Cohort/DataSet Requests classes. 

Data Sets are shown independently. 

I added search queried like

GET q={definitionName}

GET user={requestBy}

GET recent={hours}

in Request Resources. This Week I hope to finish the major parts in the pentaho project. 

PDI story Contd...

This week i tried to build, deploy and run the PDI plugin of OpenMRS. It seems it has bit old file structure. As I saw no one has touched that after February of 2012.

Still I unable to build it properly because its  tests fail. It gives a
" JsonMappingException: Can not deserialize instance of java.util.ArrayList out of START_OBJECT token " exception. http://pastebin.com/HgDZbrr9 . As I understood, it throws the error because of the structure of our JSON output we producing. I have to find a way to build it properly.

The existing plugin only accept a DataSet Definition and a CohortDefinition. I hope we can extend this to accept a ReportDefinition and parameters.

And I don't think we can happy about the way of  testings on the PDI module. it directly access the OpenMRS server data for testing purposes. I think it is better to use a xml test data file instead of the OpenMRS server data.

 After discussing with Darius I hope to start the implementation in the upcoming week.

Migrating to Pentaho

After working several weeks with RESTful web services, I started to work on my second phase of my project. Writing a plugin for Kettle.

First I got some undestand on Pentaho Data Integration(Kettle project).
I downloaded the Pentaho Data Integration and did some transformations with spoon.

I read the pdi_embed_extend_guide.pdf. It gives a good idea to how to start to write a plugin.
I deploy and run the example plugin.

Then I moved on to our existing OpenMRS plugin. I compared it with the given pentaho example plugin.
It has thoroughly followed the Pentaho conventions. It uses a RestClient to pull data from the REST API of OpenMRS. It is a Input step.

So my next target will be identify the improvidence which need in this release.
Hope to speedup my coding in next few days.

Reporting REST Module – Current Status

                           

                             This document describes the operations that I have implemented and the operations that are to implemented.

•    In ReportDefinitionResource class, “baseCohort” was added to the representation. In                        ReportDefinitionResource class, “dataSetDefintion” was added to the representation.

•  In DataSetDefinitionResource class, “columnNames” was added to the representation. In the Design page it only has the “name” attribute under the “columnNames”. But in the DataSetMetaDataConvertor class has 3 attributes as name,lable,dataType under the “columnNames”. So I added lable,dataType in addition to the name attribute under the “columnNames”.

•  I added a new ReportResource class as in the design page. All necessary methods have overriden in that class.  Special attributes like ContentType, Extension,Contentes were added to the representation.

          GET/ ReportResource

          GET/ ReportResource/uuid were implemented.

          Purge/delete/save operations are not yet implemented.

• I added a new ReportProcessorResource class as in the design page. I added few special attributes to the representation like ProcessorType, Configuration, ProcessorMode, RunOnSuccess, RunOnError

          GET/reportProcessor

          GET/ reportProcessor /uuid were implemented.

          DELETE operation is not yet implemented.

•   In ReportRequestResource class

     GET/reportRequest

     GET/ reportRequest?recent=

     Were implemented.  reportDefinition/ isProcessAutomatically/schedule attributes are added to the                   representation.

           POST/ reportRequest should be implemented.

•  I added a new DataSetRequestResource class as in the design page.

          It now supports

          GET/ DataSetRequest

          DELETE/ DataSetRequest /uuid

          POST/ DataSetRequest

          GET/ DataSetRequest /uuid

          Currently GET/ DataSetRequest /uuid only give the status of the DataSetRequest. It should be                       modified to give the result of the DataSetRequest, if it is COMPLETE.                       

• I added a new CohortRequestResource class as in the design page.

         It now supports

         GET/ CohortRequest

         DELETE/ CohortRequest /uuid

         POST /CohortRequest

         GET/ CohortRequest /uuid

        Currently GET/ CohortRequest /uuid only give the status of the CohortRequest. It should be modified           to give the result of the DataSetRequest, if it is COMPLETE.           

Getting ready for the Mid

This week I started to working on DataSetRequestResource class CohortRequestResource class. There is no any notion about a DataSetRequest or a CohortRequest in the Reporting Module. So I had to implement these resources with the help of the existing resources. As decided in the conversation with Michael Seaton and Darius Jazayeri, I wrapped Cohort/DataSet definition in a report request and process it as a Report request. 

This week I implemented DELETE DataSetRequest or a CohortRequest,

GET DataSetRequest or a CohortRequest and POST DataSetRequest or a CohortRequest.

But still there is bit more works to do in these classes. 

After finishing these works I am planning to write test cases for my classes, in next week. Some re factoring needs to the code. I hope I will able to fill the javadoc for the classes.

I hope that would be great, if OpenMRS has  some guidelines for the interns to get a idea for them, about how the coding should go. Specially telling whether javadocs are needed for the new code. Or mentioning the need of writing test cases.