Grails, AJAX and Client Templates

Grails just gets better by the day for me. Experimenting with AJAX especially with dynamically updating parts of screen, I have come across following set of problems:

• How to transfer the data?
  
• How to render the data?
  
• How to add more dynamic functions to dynamically loaded portions of the page?
  

My initial attempts included rendering HTML on the server and transferring the code to the client, and then replacing innerHTML of the loaded page. This worked  OK, until I also wanted to invoke Javascript functions after loading the HTML. Before I realized what was happeninng I ended up with loads of HTML Snippets on the Server and Javascript Spaghetti Nightmare all over the place.

After some research i came across TrimPath. TrimPath is a template engine written entirely in Javascript and therefore can run on the client. It is quite similar to technologies like Velocity, JSP, or ASP and uses a Javascript object graph as a context for the dynamic parts. 

Folllowing Setup has proven to work for me:

Include Trimpath Library and Templates in the root HTML

Trimpath is loaded from a single Javascript file, I used the GSP tag for this:

Each template (i.e. page parts which will be dynamically updated) I want to use, is embedded in a hidden div tag, and has an ID. To simplify this step, I created a custom Tag for this:

[...]

To further streamline the process I added a number of additional tags with abstract looping, conditions, "inline edit boxes" and the likes. Currently all of this code is built around the Yahoo UI (YUI) library. I plan to abstract this like the other Grails AJAX tags.

Create Base Mechanism to Render Templates and Reload Page Portions

Using the Yahoo UI Scoping and Layout mechanism and Trimpath Library the Javascript is a no brainer:

YAHOO.ST.editor.renderTemplate = function (templateId, contextObject){

   var template = TrimPath.processDOMTemplate(templateId, contextObject)  

   YAHOO.ST.global.layout.getUnitByPosition('center').body.firstChild.innerHTML = template

};

Thats it. 

Invoke the Reload

All which is left on the client side now is to get the data to render (called contextObject in the previously mentioned code) and decide which template to show (called templateId above). In my example it is done in an Yahoo AJAX callback passing the JSON response as the contextObject:

success: function(oResponse) {

   var oResults = eval("("+oResponse.responseText+")")

   YAHOO.ST.editor.renderTemplate('template_edit", oResults)

}        

Provide JSON data 

Grails provides the easiest possible way to provide JSON formatted results. Import the JSON format:

import grails.converters.JSON  

and then in the controller render the result like follows:

render returnMap as JSON    

Done!

I am still tinkering on how to distribute logic: Should the client only pull data and decide what to do with it, or should the server send code and invoke methods on the client? If the latter is the case, I would think that a RJS like extension for GRAILS would be desirable.

Grails, Javascript Templating and YahooUI make it so easy to create Rich Internet Applications by "pure programing", that it never ceases to amaze me. Keep up the good work!

GORM Inheritance & Relationships

Trying out GRAILS today, I came across an odd bug – or is it a feature? Before I get into details a class diagram of the problem at hand

I have the super class Shape, which should reference a list of ShapeAttributes. I therefore expect all sub classes  of Shape (i.e. Square and Triangle) to also have a reference to a list of ShapeAttributes. On top of that I also want to have a bi-directional relationship, so I can list all shapes (regardless of concrete implementation) from a single ShapeAttribute instance.

Here are my domain classes:

class Shape {       

static hasMany = [attributes:ShapeAttribute]

}

class Square extends Shape {

}

class Triangle extends Shape{

}

class ShapeAttribute{

String name

String value

static hasMany = [shapes: Shape]

static belongsTo = Shape

}

To test all of this I wrote the following integration test:

void testSimpleInheritance() {

def triangle = new Triangle()

def square = new Square()

def shape = new Shape()

def lengthAtt = new ShapeAttribute(name: "length", value: "value")

def shapeAtt = new ShapeAttribute(name: "type", value: "not specific")

def triangleAtt = new ShapeAttribute(name: "type", value: "triangle")

def squareAtt = new ShapeAttribute(name: "type", value: "square")

                                                                            

shape.addToAttributes shapeAtt

shapeAtt.addToShapes(shape)

triangle.addToAttributes lengthAtt

triangle.addToAttributes triangleAtt

square.addToAttributes lengthAtt

square.addToAttributes triangleAtt 

    

shape.save()

triangle.save()

square.save()          

           

assert shape.attributes.size() == 1

assert triangle.attributes.size() == 2

assert square.attributes.size() == 2

assert shapeAtt.shapes.size() == 1

assert lengthAtt.shapes.size() == 2

assert triangleAtt.shapes.size() == 1

assert squareAtt.shapes.size() == 1

}   

But the compilation of all this fails due to:

     [copy] Copying 1 file to /Users/stefansiprell/.grails/1.0.3/projects/scratchpad

org.codehaus.groovy.grails.exceptions.GrailsDomainException: No owner defined between domain classes [class Triangle] and [class ShapeAttribute] in a many-to-many relationship. Example: def belongsTo = ShapeAttribute

It seems to me, that GRAILS knows that Triangle should have a list of Attributes, but it does not know that Triangle should be the managing (i.e. belongsTo) part. What am I doing wrong? Or where is GRAILS failing?

Memory Leaks and Annotations

I admit that I was a bit forgetful maintaining my blog. But I just discovered that I actually had my first comment which motivated me beyond any expectation!

This is just a quick one: One thing I really love about Java is its simplicity, especially in terms of memory management. It works great in 99.9% of the time, the remaining 0.1% is a real time killer though.

There is an amount of tools and mechanisms available to view the heap, count objects, and the likes. JMX does not really much in counting objects and Eclipse TPTP is just too "sandboxy".  Either of them really dont help much when the problem is visible, which typically is too late (thousands of out of memory exceptions a minute). It also proved to be very hard to recreate most of the production scenarios in a test environment.

As a Java Developer I try to anticipate the lifecycle of my objects. When I know that I will be instantiating millions of objects, I will try to either use a pattern (Flyweight, Object Pool or the likes) or I will try to make the object as small as possbile (Struct Like Structure,  Masking Bits for multi boolean fields, etc. ). I think any developer does this, so there should be only little effort in telling the VM what I think should happen. I am thinking of an annotation which tells the VM: "OK, expect this object to be instantiated a couple million times."

But why?

1. Smarter Generation Handling

If the garbage collector knows that the object will be instantiated quite often, it might as well create it outside of "Eden" right of way and save some collection CPU cycles later on.

2. Space Detection

If a class is instantiated too often (configurable threshold) and the instances linger around too long, the garbage collector will check if the class is marked as "Bulk Instancable" (either by pre declared package like java.lang or using an annotation). If this is not the case, a warning is issued informing the developerl / administrator that something bad is about to happen

3. Static Style Checks

Using tools like PMD or the likes, eclipse can warn the user that he or she is breaking certain best practices for writing classes with minimal memory requirements.

2. and 3. can be easily done in a seperate Open Source project. Instrumentation and PMD come to mind. The first idea (optimized memory utilization) is beyond my know how. I am sure that memory layout and the likes can help a lot as well, but then again,  I am too spoiled by Java to know much about physical memory management. 

Ia there anything similar out there?

CSV in Java – Why is it so "not standard"

XML is getting there. Libraries to consume and create XML are available on just about every IT platform out there, more and more Real World problems are being solved with XML, and the developers can finely tune their applications to speed and memory constraints. 

But XML is not the only  document exchange format out there — far from it. CSV remains very dominant. The simplicity of its format, the human readable format, the very compressed data structure still make CSV the best format for exchanging large bulks of data. 

As much as I like the attitude of the Java Community to standardize common solutions using upcoming technologies, I regret that the community seems to have forgotten all about CSV. There are discontinued solutions out there, such as CSV JDBC which allows read-only access to CSV files, or simpler CSV File parsers.

But they all lack basic plausibility/integrity checks, column count verification, type security, and much more I like from working with XML. For each project I had to recode parts of  the mentioned libraries. Rewriting libraries instead of configuring them, is not my typical approach to coding Java applications. 

I would like to see a solution, where I can attach/link an XSD to a CSV file. A special parser would then create a DOM object or SAX stream based on the XSD file. I could access columns as XML elements, have type safety, defined nullable attributes, use Marshalling frameworks for POJO Bean binding, use XSLT to create hierarchical structures and many more.

Is something available like this? Can we quit maligning CSV processing and lift it to the XMLstandard?