JPA Myth: EclipseLink and Hibernate are compatible. Not true: Hibernate does NOT follow the JPA Spec

Please take a look at the Hibernate JIRA issue EJB-441, seen it?

Now, please tell me if this is correct:

According to Gavin King (author of Hibernate) the JPA spec the @Column(nullable=false) is just a schema generation hint, and therefore JPA implementations should NOT use it for object level validation.

I think that means that if an entity object with a null property marked with the @Column(nullable=false) is persisted using entityManager.persist() it should crash with a java.sql.SQLException as the root exception, thrown when the "INSERT" statement failed after it was sent to the database by the JDBC driver (as it happens in EclipseLink), and not because some implementation specific mechanism "pre-validated" it and prevented it from reaching the database (as it happens in Hibernate thanks to org.hibernate.engine.Nullability.checkNullability).

Is that correct? Because I don’t know anymore!

Lets say that is correct, does that means that the following is correct too?:

Validation should be entirely the responsibility of the JSR-303, and the properties of an entity should only be checked for nullability before sending to the database if a JSR-303 implementation is there to indicate that.

Is that true? And if it is, then:

What is exactly the difference between @Basic(optional=true) and @Column(nullable=true)? What happens when this 2 are applied to the same property, but they contradict each other? (As in : @Basic(optional=false) and @Column(nullable=true) or in @Basic(optional=true) and @Column(nullable=false)

I could not find a statisfactory answer in the spec documentation, would anyone be so kind as to resolve this issue? Because it Hibernate Forums they have decided that the way to deal with this inconsistencies and contradictions in Hibernate and in the JPA standard is to plain ignore me since May of 2009.

As you can see I tried to raise the issue with 2 JIRAS, forum post and easy to reproduce unit-tests, but nothing seems to work… what should I do then?

Conceptual Queries extension for JPAQL: JPAQL queries are too sensitive to model changes

When a query is written in JPAQL, it still depends to much on the domain model (I think JPAQL could be extended to support Conceptual Queries).

For example, lets say you want to:

Find each white cat that lives in a house with doors made of wood

A cat is of one color, it lives in one house, a house can have 1 door and the door can be made of 1 material

So, the query in JPAQL looks like this:

select c from Cat c where c.color=”white” and c.house.door.material.name = "wood"

But then, lets say our customer changes the requirements:

Find each white cat that lives in a house with doors made of wood

A cat is of one color, it lives in many houses, a each house has many doors and the door is made of one or more materials (for example half glass, half wood)

Since our relationships are now @OneToMany so we write their names in plural, and we are newbies in JPAQL we try doing this (and it, of course does not work):

select c from Cat c Cat c where c.color=”white” and c.houses.doors.materials.name = "wood"

Now, we can of course solve this problem using subqueries and the exists keyword, but that makes the resulting query way more complex, and even if the above worked, it still is a different query, but, in our english example, the query didn’t change:

Find each white cat that lives in a house with doors made of wood

So, why we can not write in JPAQL something like:

select c from Cat c where c with color= “white” and lives in House has Doors made of Material with name = "wood"

That way the query wouldn’t have to change even if the multiplicity of the relationships changed. Of course now the question is, from where do I get the “with”,“lives in”, “has”, “made of” and “with” well, simple:

• The with operator is available for each non @Entity member of the class (for strings, integers, etc).
• For relationships with other entities we just add the conceptual name of the relationship name as an attribute in the @OneToMany or @ManyToOne annotations:

Example:

Before:

public class Cat{

@Column

private String color;

@ManyToOne(conceptualName=”lives in ”)

private House houses;

}

After:

public class Cat{

@Column

private String color;

@OneToMany(mappedBy=”cat”, conceptualName=”lives in ”)

private Set<House> houses;

}

This way changes in the object model do not necessarily affect our queries. What do you think? Is this a good solution?

Ruby is younger therefore better than Java?

I wrote this as a response to From Java to Ruby: Programmer's view, but I couldn't post it because of a bug in that site, so I decided to post it in my own blog:

Isn't this a simplified view of the advantages of Ruby over Java? For example the lack of choice in Ruby means that if the "one Ruby way" to do stuff is not good for you project... you will have to go to other technologies (Java for example). And that can happen pretty often:

• Hibernate has many more options for integration with legacy databases than Ruby's Active Record... almost all databases now have JDBC drivers (you can not say the same about Ruby database support).
• Spring offers integrated transaction handling that makes it possible to switch from JDBC transactions to JTA transactions without changing a single line of Java code (you just need to modify around 5 lines in an XML file) what is the equivalent for that in Ruby?
• Calling stored procedures is not that easy with Ruby... what is the equivalent of HQL (JPAQL) for Ruby... can you honestly say that it can handle all the special cases HQL can... and with the same efficiency?
• With java I can build a web application GWT style, JSF style or plain JSP style... and each style has advantages and disadvantages... do I have all that power with Rails? (Of course, those frameworks can be re-built in Ruby, but the question is, do I have them now?)

I think you are right when you say that it's healthy to start with a clean slate and rebuild based on a cleaner, simpler foundation, but before saying that the new foundation is actually better than the older one, you have to be sure that your new foundation is actually capable of handling all the special cases the old foundation was capable of handling... or remember that maybe if you remove all the abilities the old foundation has to handle special cases you might end up realizing that your new foundation is just a replica of the state that the old foundation had when it was younger. (And even then the older foundation has the advantage that you can use it as it was used in the past, but you can not use a new foundation as it will be used in the future)

Data Transfer Object Injection

Data Transfer Object injection is a programming error which results in security holes., it is to a Remote Object Service based applications which use object graphs what SQL Injection is to web-based applications which use databases.

DTO injection could happen where there is a remote object service that allows a client system to send and and object graph that is automatically converted by an object relational mapper in to SQL statements.

Instead of sending a valid object graph, the attacker can send a different object graph, representing alterations to the database that go well beyond his security level. For example, a remote object service receives an object graph that represent changes in the objects that represent new users, or new permissions granted for existing users of the system.

To prevent this problem it should be possible to specify at the object relational mapping level, which entities can be saved by the current user... many object relational mappers, or xml relational mappers automatically write the changes represented by the object graph to the database, without caring if the current application user has the privileges required to persist those objects... we can not rely on RDBMS security, because most remote object services use the same user for all the calls... and I think it that connecting with a different user for each remote object service would be bad for connection pooling (decreasing performance)

I wonder if anyone else thinks this is a common security problem... Mmmm... I will add this to C2... I wonder how (or if it) will evolve.

RIAs: Faulting &Uniquing (or Merging?) (Granite, Ajax)

Today I realized that lazy loading support for Granite Data Services is in its infacy... is more like "Partial Loading" (it will load everything not initialized, and not initialized stuff will remain "unloaded" forever).

I am thinking this leads to a pattern like this:

1. I need to work with persons, so I fetch a list of them from a remote service.
2. I choose to work with the person with id "3";
3. present the contacts of person "3". (here is the tricky stuff, all the contacts that I load have a reference to person "3", what do I do about that? do I re fetch it, creating a different object and breaking uniquing, or do I look for a way to prevent that "same but different object" in my application? )

I guess that we will need something like Faulting & Uniquing , and a Client Side EditingContext (or Client Side EntityManger)... to control data in the client side... (our own idea of LDS DataStore ?)

But... until granite has that... what could we do as a first step? it would be nice if we could "merge" a recently obtained object with one a fetched before... something like ADO.NET DataSet... (I can not believe I am writing that I miss the DataSet)

I have been thinking... a fully "AJAX" traditional JavaScript based application would have the same problems if it had a complex enough UI... but I haven't heard of anything like it, it seems that most AJAX application developers build applications so simple that they don't even care about having to write and re-write client side data manipulation code... (or... maybe those applications don't even enough client side behavior to need it?).

I guess that until Granite has his own "Data Management" the way to handle data will be... to imitate the practices of traditional AJAX applications?

(Mmmm, now with Google Gears... will we see how JavaScript based frameworks for automatic handling of DTOs and ORM start to appear everywhere? Parhaps this will revive the interest in something like SDO?)

Okey, things I miss about NHibernate

Lately I have been working without a formal object relational mapping (you know, using custom "hand made" business objects, and fetching data from the database using stored procedures)

Here are the things I miss from NHibernate:

• Automatic caching of objects by primary key
• Easy support for primary keys with multiple fields
• Polymorphic Queries (and all the benefits of persistent inheritance)
• Queries with "." navigation "object.Relationship.Relationship.Field = :parameter"
• Prefecthing relationships: "load Products with their to one relationship with Vendor preloaded"
• Easy pagination (instead of trying to figure out how to do it in combination with stored procedures for a particular version of a particular database (that can be a real nightmare))
• Interactive single and multicolumn sorting (try to do it with stored procedures, and of course without cheating by using the execute command, remember if you use execute all the so called "security benefits" of stored procedures vs dynamic sql dissapear)
• Concurrency support (automatical increment of a "version" field when saving changes, even if those changes are not just field changes, but relationship changes, custom exception type for concurrency errors, optimistic locking)
  
• Query by example (no need to concatenate strings by hand based on whether a field is null, no need to worry about upper case or lower case comparision when comparing each field, no need to update a query definition if a column is added or dropped from a table)

And, if somehow you have to work with a database that doesn't have support for stored procedures (and you can't use NHibernate)... things get even more "interesting":

• Alias for ugly table names and ugly field names (instead of "select * from PDT where PCKT = 1" you write "select * from Products where PackageType = 1"
• Named Parameters: Instead of "select * from PDT where PCKT = ? and RQTY = ?" you can write: "select * from Products where PackageType= :packageType and RemainingQuantity = :remainingQuantity" this is specially a nightmare when one has to use OleDb, and it gets worse with Inserts or Updates: "INSERT INTO PDT (NN,PCKT,RQTY,DRP,PRNBR) VALUES (?,?,?,?,?)" instead of the simple: "session.save(product)"

I will update this list as it increases...

Object Relational Mapper vs Business Entity Manager

In my opinion the best two are Hibernate and EOF, I think a mix between them could create "the ultimate" Object Relational Mapper... or the ultimate Business Entity Manager...?

EOF (the best BEM?)

• General Overview
• Manual
  

Hibernate (the best ORM?)

• General Overview
• Manual
  

An interesting topic for analysis could be the limits between an Object Relational Mapper and an Business Entity Manager. Some people think that Hibernate shouldn't evolve to become more like EOF, because it should aim to do one and only one thing (be the best ORM) and other people think that it would be nice to have the services that an Business Entity Manager gives you:

• automatic connection handling
• automatic transactions handling
• multiple undo levels
• real nested transactions
• client and server objects
• distributed non-transactional lazy loading
• etc,etc
  

There are also some people that think that a Business Entity Manager is more closer to the MDA "objective", so I think it is important to take a look at (Using Borland's ECO to develop model-powered applications for .NET) an many related articles, the specially interesting thing for me about ECO is its OCL support (that can be used, for example to configure constraints that help ensuring that only consistent data is saved into the database... it is kind of a Design by Contract for the domain model)

IMO the Active Record pattern is getting too much popularity (perhaps thanks to Ruby , but I think it leads to an anemic domain model ) and Domain Model based in Business Entity Managers is not as popular (thanks to the problems in EJB1 and EJB2) but it is important to remember that EJB is not the only way to build an Business Entity Manager, there are ways to build Business Entity Managers that are a lot more lightweight and user friendly than EJB, (just investigate more about EOF and ECO)

UI Design: Edit then List vs List then Edit

Today I was thinking about the way I build UIs (List then Edit) and the way most people I know build UIs (Edit then List) and the relation that the way you build you UI has with the way you store the data you manipulate in you UI.

List then Edit

In this "pattern" for UI structure, after the user selects an option in the menu he sees a window that allows searching with some controls (generally in the top of the window),and with some control in the bottom that represents the search results, then the user selects one of the search results (an object, or row if you prefer) and he proceeds to work (edit, modify) it, for that he opens a new window "the editor" (perhaps by double clicking the selected object, perhaps by clicking an "edit" button in the list window) in the editor he may modify the object being edited as much as he likes, and then click "save" if he wants to commit his changes or "cancel" if he wants to rollback them.
If the user wants to create a new object, the list has a "new" button that also calls the "editor" so that the user can set the initial values for the fields before the object is committed to the database for the first time

Edit then List

In this other "pattern" for UI structure, after we select an option in the menu we are right there in the editor, but all the controls that could allow us to modify the current object (or row) are disabled, because we haven't searched for any (or created a new one) from here the user can choose to create a new object that action has the effect of enabling all the control in the editor, or search for an already persisted object (or row) by clicking the "search button", that shows the search "list" window, in the top of that window there are controls to configure the search criteria, and in the top a control to represent the search results, from here the user can choose one of the search results (an object or row), and click on the "accept" button, which takes him back to the editor that now is displaying the previously selected object, here the use may choose to make some changes and click "save", or just cancel click "cancel" but actions have the effect of disabling the controls in the editor UI until the buttons "search" or "new" are used again.


Edit in List
When an application generally follows List the Edit, sometimes, if the info in the selected object is very simple, the user may modify it right there, but for for objects with medium to complex (several fields, to one or to many relations with other objects) calling a window to modify the object is more comfortable for the user

Search (List) in Editor
Some times, the developer likes to use the same UI to Edit and to Search, this works like some kind of "query by example", that is when user enters the editor, instead of being disabled an waiting for a click on the "new" or "search" button, the controls are enabled, and if the user writes some data in to them and clicks "search" the already persisted object (or row) that is more similar to the partial information already written in the editor is loaded. Sometimes there are many objects that match the query by example, here is when a navigator control can be a nice thing to have.

Master List, Detail Editor
This is also a very common configuration, both the list and the editor are in the same window, the list generally in the top, and the editor in the bottom, I think this somewhat corresponds with Two Panel Selector.

Persistence
Okey, until now I have exposed what I think are the some of the common "patterns" used to create UIs that manipulate data, now... what I find more interesting about this is the the way this "patterns" affect the way the data is stored or retrieved from persistence:

• If we use Edit then List:
  • The Edit is displayed on the screen
  • The user clicks "search" and the search List window appears
  • The user configures the search criteria in the List window
  • The objects (or row) matching the criteria are shown in the List window
  • The user selects one of the matching objects and click in the "accept button" in the List window
  • We return to the Edit window, that now is displaying the selected object.
  • We can make changes to the current object by clicking "save" or click "cancel" to dismiss the changes, if we do either action the Edit controls are disabled and we again have to click "search" or "new" to work with an object
  • The problem here, is that the user might want to see the same search results he used the last time, but each time we call the search List window, we are creating a new object, and all of the configuration from the last time we called it is already lost (this disadvantage has a "nice side", because we don't have to worry about showing "stale data" to our user)
  • Another disadvantage is that maybe not all the controls we use to manipulate the data in the edit windows are "databindable" so we have to manually reset the state of the Edit window, and if we don't do it correctly we could have bugs that "transfer information between edits". (First I edit John and set his age to 24, then I edit Mary, and her age is also shown as 24, but her age is 56, and we had no intention of changing it, it happened because we forgot to clear the text-box value)
    
• If we use List and then Edit:
  • The list is displayed on the screen
  • The user configures the search criteria in the List window
  • The objects (or row) matching the criteria are shown in the List window
  • The user selects one of the matching objects and click in the "edit button" in the List window.
  • The "Edit" window appears on top of the List window, and is displaying the selected object.
  • We can make changes to the current object by clicking "save" or click "cancel" to dismiss the changes, if we do either action the Edit window is closed and we return to the List window
  • One of the advantages of this approach is that we don't have to worry about the "transfer between edits" bug, because each time we call the Edit window, it is a new window, without any data, ready to configure itself to match the data contained in the object we are going to edit.
  • Then problem of keeping the search results to reuse them is also automatically solved, because the search List window was never closed, it was there all time, behind the edit window.
  • But the problem now is that some of data shown there may not be updated, or perhaps, it has data that has never been on the database, and that we do not want on the database, how can that be? well, we edited the object in the edit window, and the databinding ensured that the changes were written to the object before we clicked "save" or "cancel", if our intention is to keep those changes and we clicked "save" in the Edit window then we don't have a problem... but if we clicked cancel, now we need to rollback in memory changes and some of this changes could have modified relationships with other objects, or properties of the other objects, we could have created, deleted or updated a complex graph of objects but "only in memory" and now, we need to rollback all this changes, if we are using an Object Relational Mapper (ORM) that supports this (like Apple's EOF) or a relational cache that allows for in memory transactions (like the .NET DataSet) we can solve this problem easily (of course the DataSet has other disadvantages), but if we are using an ORM like NHibernate that AFAIK does not rollback in memory changes then you have a problem, you have to re-fetch the information from the database.

This seems like a big omission from the NHibernate guys... but it isn't exactly so, everything I have exposed here, has been on the assumption that we are working in a "Smart-Client" that holds local information, and Hibernate was born in "the web world". In the web, you "need" to re-fetch the information on each request (so when you go from editor to list or from list to editor you always reload the data) and you don't really pass the object you are going to edit from the list to the edit, it easier, and more efficient to just pass the primary key, of course the problem there is that you can only do that with objects previously persisted in the database, if you object is new it has to be serializable and some times that just isn't advisable (but that is an issue for another discussion)...

The thing with web based application is that the controls in the UI don't actually store the information directly in your object, their store it in the view-state, or in the query-string, in cookies, in the session or in an object that "represents the form", and only when you finally want to save, you extract the information from the web-controls and write it in to your object (at least that is more/less was the way I did it in the Java servlet world) but this is a "feature" that might be going away... the problem is that with the new JSF databinding, or with the new databinding facilities of ASP.NET 2.0, you can actually bind you controls directly with you datasources... and then how will we rollback the in memory changes? should we build a framework on top of NHibernate, a kind of "in memory object context" that handles the commits and rollbacks in memory?

More Questions

• And what about the case when a list window calls an edit window that has an embedded list control that calls another edit window? (complex multiple level or nested interfaces) should the object relational mapper make it easier for me to build this kind of UIs?
• Or Should a new kind of framework deal with the problem of communicating the persistent objects with the UI?
• is using DTOs really the solution? Does Apple's EOF go beyond the responsibility of an ORM by providing in memory transactions?
• If NHibernate can almost transparently persist objects to the UI, shouldn't this other framework be able do the same and transparently present the object in to the UI without having to manually create objects to do this job?
  
  I am thinking about publishing this post as an article in the Wikipedia to see how it evolves... I would love to read some comments about this article to improve it.
  
  
  
  
  
  

NHiberntate Attributes + MSBuild = SQL DDL Generation

All right... I am building a system that uses NHibernate
as the persistence mecanism to save in to the database
and i decided that the best way to go... was to draw the
object model using the new VisualStudio 2005 Class Designer
the add the attributes... and then generate the SQL... but the
problem is that I have to run an extenal tool to do that... and I want the
DDL SQL to be re-generated each time I compile...
so I guess I will have to create a new MSBuild Task that integrates the
SchemaExport with MsBuild

So... I have written my task... but it does not work... now I am reading
How to: Write a Task to find out what am I missing... perhaps it is because
my task depends on an external assembly (NHibernate)... mmm no... that was not the problem... the problem was that I was not specifying the correct path for the source assembly.

It works.... well it worked... but only once... it seems that the source assembly gets blocked... lets think... how could that be fixed...

Why in ADO.NET the DataSet has to work with DataRow and not an Interface

Right now... there are two ways of handling an XML file in an easily and with object orientation:

1. the intrusive (but more dynamic) TypedDataSet, generated bye the xsd.exe
2. And object serialization generated by XSDObjectGenerator.
   

Each one has its advantages and disadvantages... but I wonder... why we can not combine the advantage of the by changing the internal design of ADO.NET... How?

Well the DataSet... is a collection of DataTables... the DataTable is a Collection of DataRows....
why it is not built to be more flexible? i.e. :

• The DataSet could be an implementation of IDataSet
• The default implementation for IDataSet " the DataSet" could be a collection of IDataTables
• The default implementation for IDataTable "the DataTable" could be a collection of IDataRows
• The default implementation of IDataRow... could be "the DataRow"

This way... I wouldn't have to worry about having to inherit from the DataSet, DataTable or DataRow classes... if that is not what I want... and this way I could create my own independant object hierarchy and... if I want to handle it as a DataSet well, all I have to do is implement the appropriate interfaces...

Of course... making Microsoft change the ADO.NET API to add this flexibility... could be too much to ask (after all there are a lot of applications built on top of the "legacy" ADO.NET) but... what about Mono? they could create an extensible non intrusive interface based dataset... don't you think?

ADO.NET vs ObjectRelationalMapping

So... why I don't like DataSets? Well, first of all, it is not that I think that DataSets are useless, it is just that I feel that Object Relational Mapping (ORM) is a far better solution, because, when properly built:

1. You don't have to worry about primary keys of foreign keys (or the proprietary identity mechanism of a particular database)
2. You don't have to worry about creating the master row first, and the detail rows later (master detail relationships are automatically handled by the ORM)
3. You don't have to worry about saving all the changes in your data in the proper order(CRUD operations are automatically ordered following master detail relationships)
4. You can easily add optimistic locking to you code without having to change all your SQL "UPDATE" or "DELETE" code.
5. Your domain model is more abstracted from the idea that is being read from a database (a really good ORM should be able to read from a different relational source... or perhaps even a hierarchical source (Object Hierarchical Mapper?)... for example... from an XML file.
6. You automatically get "cache" benefits, transparently, and sometimes you can even plug special caching strategies.