Architecture Rituals and Patterns: On the Importance of the Middle Way Once and Only Once Balances Yagni - Part 2

...he heard a master teaching a girl how to play a string instrument. The master told her that if the string was too loose, it would not sound, but if the string was too tight, it would break: the string had to be at just the right tension to produce music and harmony. At that moment, he understood the middle way...

In the previous part of this series, I told you about YAGNI and OAOO and said that OAOO and YAGNI are like Yin and Yang. They balance each other. Without OAOO, YAGNI would slowly strangle us. With OAOO, you always have a system that you can improve (a phrase by Ron Jeffries).

I also "tied the cat," built systems as I saw others build them, was at the Shu level of ShuHaRi when programming enterprise systems, put my DAOs, put my Service, and put my Controller.

All the "well-architected" systems I knew did this, so I did it too. If a well-architected system looks like this in the end, it must be because it looked like this from the beginning... right?

Well, no. Things don't look the same when you're building them as when they're finished. If, for example, you try to build an arch without extra supports (which you will later remove), you could wrongly conclude that if you can't build it "without using supports," you're doing it wrong... That's not the case.

I discovered I was wrong almost by chance, I had already read that OAOO and YAGNI are like Yin and Yang. They balance, but I hadn't understood it beyond the sound of the words in the phrase... When I understood it, I wrote that article about epicycles.

I learned it when I came into contact with PHP programmers...

Speak your truth quietly and clearly; listen to others, even the dull and the ignorant; they too have their story. Desiderata

Those PHP programmers didn't know what YAGNI was... they knew nothing about the principle of building the simplest thing that could work (KISS)... they had never visited c2, and some of them had difficulty reading English...

And yet...

Yet precisely because of that, they followed YAGNI and KISS to a fascinating extreme, they did absolutely nothing that wasn't essential to finish on time the work assigned to them. Yes, their code was almost unbearable to the sight of someone like me at that time...

But what I needed months to build, they did in weeks; what I needed weeks for, they did in days; and what I needed days for, they did in hours...

On the other hand, their intense (and intuitive?) adherence to YAGNI and KISS also had its price, the technical debt they accumulated was brutal, and while they could put functionality in the hands of the user in record time, they also had to redo everything almost from scratch if they were asked for a change, there simply was no reuse...

There was no balance... But for me, it was a shock to look at the other extreme, and the other extreme looked back at me... I could spend hours refactoring code to eliminate redundancy to the maximum, whether the system was ready for the delivery date was a secondary factor... I was all OAOO... they were all YAGNI... I like to think that we learned a lot from each other..

Yagni means that we should only implement what we need now, not what we foresee that we are going to need. This strategy (when it works) eliminates some costs because it does not allow them to come into existence (they were things that we actually never came to need), and it delays the costs of those things that we will eventually need to a point in a future date. However, a commonly used counter-argument is that it is better to implement generic things now than in the future...

If we implement everything incrementally by invoking YAGNI, the system can become increasingly difficult to modify, especially if every time we implement something new, we do it always in the simplest possible way

It is good to do the simplest thing that could work, however, we must remember to reapply that rule to keep the whole system as simple as possible. This means that we will refactor the code so that nothing violates OAOO.

This refactoring, by centralizing any functionality into a single class or method, leaves the system well positioned for the next change. Anything you need to do afterward, you can be almost sure that you can do it better by improving or replacing a specific and isolated point of your code.

Now let's see an example:

• I have to make a registration screen, which includes
  

  a <select> element that displays the countries of the world, so that when a person registers on my site, I can present them with content relevant to their country (or countries) of origin. As I'm going to apply YAGNI, I won't do more than this:

• Controller:

public List<Country> getCountriesList() {
    logger.info("Getting countries!");
    List<Country> countries = getSimpleJdbcTemplate().query(
        "select id, name from country", 
        new CountryMapper());
    return countries;
}

Following YAGNI, I don't have to do more. If this ends up being the only screen in my entire system that displays countries, I won't have wasted my time on complexity without benefits.

But then, it turns out that in another part of the system, on the screen to register orders, when I have to put in the address where I want the product I'm purchasing to be delivered, and among the information to indicate in the address, I have to include the country

It is then, and only then, -when I already have 2 conceptually separate places, and for which it is not convenient to use the same controller- that I add a service:

• Countries Service:

public List<Country> getListOfCountries() {
    logger.info("Getting countries!");
    List<Country> countries = getSimpleJdbcTemplate().query(
        "select id, name from country", 
        new CountryMapper());
    return countries;
}

• Controller: User Registration

public List<Country> getProductsToList() {
    return getCountryServices().getListOfCountries();
}

• Controller: Order Address Registration

public List<Country> getProductsToList() {
    return getCountryServices().getListOfCountries();
}

This is why when a system is finished, it gives the impression that "everything is built with layers from the beginning." But that's a mistake, a product of seeing the system as something static, as something that "was born" with the form it had at a moment in time, and not as something organic, whose structure changes as it adapts to the needs of the user who acquired it.

The architecture of the project is not something that is predefined before understanding the system. The architecture is an emergent structure, derived from the user stories that are gradually built as the system is shaped. If the system does not include among its stories the need to operate on multiple different databases (Oracle, SQL Server, etc.), then the DAO layer should never have to exist. If it doesn't add value, it's superfluous.

In the case of the previous example, we can go even further. Nowadays, most applications are RIAs built on HTML5 and JavaScript, where the logic in Java (or in C#, or in Groovy, or in Scala...) merely acts as a layer of data exchange and security between the persistence layer (typically a database) and the business logic and presentation layer (which are usually located entirely in JavaScript)

So why do we insist on using POJOs that don't add value? Why doesn't our code look like this:

public List<Map<String, ?>> getCountriesList() {
    logger.info("Getting countries!");
    List<Map<String, ?>> countries = getSimpleJdbcTemplate().queryForList("select id, name from country");
    return countries;
}

Why complicate our existence with one (or several) POJO (or DTO) if our system is so simple (at the Java level) that the methods exposed by our controllers only do CRUD and it is the JavaScript code that really brings the data to life? What's the point of working with POJOs, if our domain model is anemic?

Approaches like OData, sweep away in one fell swoop all this paraphernalia of making controllers to slowly and step by step build a query API that could be automatically derived from the data model. But OData is just a query mechanism, which comes to eliminate the need to keep "tying the cat"... that is, to make a custom layer of controllers each time.

Why don't Oracle, SQLServer, MySQL, PostgreSQL, etc. directly offer an OData API (or something similar) instead of forcing us to reinvent it each time?

It seems that our human traditions make us live in cycles within cycles:

1. We invented Cobol, to expose our data through procedures.
2. We invented relational databases (whose implementation we never completed) to allow access to data through query expressions (SQL).
3. We then invented stored procedures, to procedurally encapsulate our query expressions.
4. We then invented object-relational mappers, to achieve compatibility between multiple databases and be able to query them with a unified query language.
5. We then invented SOAP, to procedurally encapsulate our objects.
6. We discovered that in parallel, someone had invented REST, which allows us to query resources using query expressions represented by a URL.

And so... again and again, procedures, expressions, procedures, expressions, procedural approach, declarative approach... and every time we reinvent the wheel, we feel like we are breaking new ground... when in reality, we are just giving the wheel of fashion another turn.

Will we ever find the balance? (Or perhaps we have already found it... and just need to realize it...)

 Originally published in spanish in Javamexico

 

 

Architecture Rituals and Patterns: On the Importance of tying up the cat - Part 1

The Zen master and his disciples began their evening meditation. The cat that lived in the monastery made so much noise that it distracted the monks from their practice, so the master ordered that the cat be tied up during the entire evening practice. When the master died years later, the cat continued to be tied up during the meditation session. And when, eventually, the cat died, another cat was brought to the monastery and was tied up during the practice sessions. Centuries later, scholars descendants of the Zen master wrote treatises on the profound spiritual meaning of tying up a cat for meditation practice.

A few years ago, I attended an event organized by Sun to promote Java here in Mexico (I think it was called Java Dev Days). At that time, I entered a presentation, I think it was by Sang Shin (the creator of JavaPassion), in which he explained how simple it was to use JRuby with Ruby on Rails to create a "hello world" page. It was something like this video: http://www.youtube.com/watch?v=sas3KCKwyHI. It was the first time I saw RoR, so on that side, it seemed educational... however, most of the audience... was falling asleep!

That caught my attention... why were they all so bored, from what I had been able to observe the previous days, many of them were enterprise programmers, with the kind of clients that I have had, who give you very little information to work with, demand a lot, and want everything for yesterday..., so I started to listen to the comments that they exchanged quietly (or not so quietly) between yawns:

    Is that simple? It took like 15 minutes to make a hello world screen!
    How boring, I could do that with JSP in 1 minute
    Where does this guy get that this is simple? Did you see the mega directory structure that those commands created? That's not simple at all!
    You can tell this presenter spends his time giving courses, but he's never worked in real life, I'd like to see him with his RoR having to get the job done at 3 in the morning

At least the 2 rows sitting in front of me were not remotely impressed with RoR. And why would they be if in JSP you can make a hello world simply by putting something like this:

<HTML>
 <HEAD>
  <TITLE>Hello World</TITLE>
 </HEAD>
 <BODY>
  <H1><%= "Hello World!"  %> </H1></BODY>
</HTML>

Certainly, arguing that the 15 minutes of the multi-step video are simpler than this is absurd.

Well, some will say, (as part of me thought) the point of the example with RoR is not to show how simple it is to make a silly "Hello World" but to show the potential of RoR's MVC architecture to do much more complex things...

Let me see if I understood... are you telling me that I'm going to complicate my life by making something easy much more difficult and that's going to result in a simplification of the difficult? More complexity today == Less complexity tomorrow? That violates YAGNI. (Yagni means: You aren't going to need it)

And it's not that you have to avoid violating YAGNI just to follow tradition, as in the case of the cat in the monastery. Yagni has a reason for being: in systems development, change is the only constant, and if you get ahead and build something before you have a clear idea of whether you really need it, when you get to the point where you are really going to use it, it is very likely that what you really need will be very different from what you had predicted. This doesn't mean that your code shouldn't be flexible, but you have to be very careful, because you can end up with an over-engineered solution, ready for situations that will never occur. If you use too much energy today, fighting with the ghosts of things that will never come, when the real enemies finally arrive, you won't have the strength to face them.

Now, one of the big problems we have in learning programming is that when we are learning, we have to build examples like a Hello World in RoR to understand how it works. But as instructors, we have to be very careful in saying "look, with this example I'm going to show you how simple this is...". No. You don't make a hello world in RoR because it's simple. Let's not confuse our students. We don't go to the gym to sweat because it's less tiring than staying at home watching TV. We go because by doing so we will improve our skills, we go to prepare ourselves for future situations. So, in programming, not only Yagni rules, we also have another principle: OAOO. (OAOO means, once and only once).

OAOO is the root cause of the apparent complexity of RoR, and the archenemy of copy&paste. OAOO has other names, it is also known as the DRY principle (Do not repeat yourself). It's not good to have copies of the same code scattered throughout the project, because if there is a bug in one of them, and you didn't follow OAOO, you'll have to find and fix them all, instead of solving the problem in one place.

OAOO and YAGNI are like Yin and Yang. They balance each other. Without OAOO, YAGNI would slowly strangle us. With OAOO, you always have a system that you can improve (a phrase by Ron Jeffries).

OAOO alone would also kill us, leading us to an eternal refactoring of code searching for the holy grail of the generic code with the fewest possible lines without ever stopping...

We need both. We need balance.

How many times have you faced code like this in your work:

DAO:
    public List<Product> getProductList() {
        logger.info("Getting products!");
        List<Product> products = getSimpleJdbcTemplate().query(
                "select id, description, price from products",
                new ProductMapper());
        return products;
    }

Service:
    public List<Product> getListOfProducts() {
        logger.info("Getting products!");
        List<Product> products = getProductsDao().getProductList()
        return products;
    }


Controller:
    public List<Product> getProductsToList() {
        logger.info("Getting products!");
        List<Product> products = getProductsServices().getListOfProducts()
        return products;
    }

What value does this add? Why can't we simply do it like this (no Service layer, no DAOs layer):

Controller:
    public List<Product> getProductsToList() {
        logger.info("Getting products!");
        List<Product> products = getSimpleJdbcTemplate().query(
                "select id, description, price from products",
                new ProductMapper());
        return products;
    }

Why is it that if you don't tie up the cat before meditating... I mean, why is it that the principles of multi-layer architecture dictate that it should be done this way... really? Is that really the answer? NO!

If it doesn't satisfy an immediate need and doesn't help OAOO (obviously the redundancy in this doesn't help OAOO), we shouldn't do it. We are burning precious time and energy on "busy work" that nobody will thank us for.

Heresy! Many have told me when I tell them to stop having so many layers just because. Layers are NECESSARY. You can't remove them.

Of course, you can, and I'm not the only one who thinks so. JBoss Seam was designed precisely with the idea of eliminating so much unnecessary layering. And that was YEARS ago. Why hasn't it dawned on most people that making layers for the sake of making layers is not correct? Why do we still tie up the cat and buy a new cat every time we start a new project? Tradition.

Now, traditions are not all bad, (those cultures with traditions that don't help their survival simply become extinct). Where does the idea come from that putting so many layers helps in any way?

I will talk about that in my next post in this series...

(originally published in spanish on Javamexico)

Find me a circle

A circle is a simple shape of Euclidean geometry consisting of those points in a plane which are equidistant from a given point called the center. The common distance of the points of a circle from its center is called its radius.

But Circles can not, and do not exist in reality, anything that seems to be a Circle is actually a polygon with a very high number of sides... the closest to a Circle that we can get is to have each side of the polygon the size of quantum distance.

Just try and draw a Circle in The Gimp (or in Paint.NET).

Now zoom 600% in to it. You will see it is actually a polygon, composed of thousands of little pixels, the same is true for "circles" in reality, they are composed of thousands of little quantum pixels.

So, Circles do not exist, they are just a math abstraction. And that is why we end up needing unattainable numbers like Pi to deal with the area inside of a circle... because there is no such thing as the area inside of a circle (because there are no circles).

The area inside of anything in reality, that looks like a circle, can be calculated by dividing it in "quantum pixel areas", and the adding those areas to get the total area inside of the supposed circle. Now, that would require a lot of effort, that is the reason we believe in circles, because they are a very useful abstraction (even if it is an abstraction that fails and produces Pi as a result of its failure)

Among Archimedes mathematical accomplishments is the computation of pi, he used the method of exhaustion as a way to compute the area inside a circle by filling the circle with a polygon of a greater and greater number of sides. The quotient formed by the area of this polygon divided by the square of the circle radius can be made arbitrarily close to Pi as the number of polygon sides becomes large, proving that the area inside the circle of radius r is Pi * (r*r),Pi being defined as the ratio of the circumference to the diameter.

When a circle's diameter is 1 unit, its circumference is Pi units. Why Pi is such simple concept but such a strange "number"? because Circles do not exist, Pi is an example of a Mathematics (Geometry?) abstraction leak.

As we increase the number of sides, we approximate more, and more to the circle, but we can never really have a circle.

This is why we can never know the complete value of Pi, because there is now way to have a circle in a discrete quantum based reality, "circles" are a good model for this things with too many sides to count easily, but circles are just a model, not a reality

That is something, that as software developers we should always keep in mind: The Map (the Model) is not The Territory (the Reality)

OO Principles

Hi! Today I am reading the book Head First Object-Oriented Analysis and Design, I specially liked the OO Principles, I have already read about them in c2, but I really like the way they are summarized in this book:

• OCP: Classes should be open for extension, but closed for modification. (to avoid new requirements breaking old tested code)
  
• DRY: Don't repeat yourself, avoid duplicate code by abstracting out things that are common and placing them in a single location. (to avoid having to fix the same thing in different places, or solving the same problema again, and again and again... I believe this is releated to YAGNI balances OAOO.... i guess I could say that YAGNI balances DRY, and therefore DRY equals OAOO)
  
• SRP: Single responsability principle, every object in your system should hava a single responsibility, and all the object's services shoul be focused on carry in out that single responsability. (to avoid Big Ball of Mud, a huge object that does everything, that is hard to extend, hart to understand, and lots of stupid little object that do nothing, mmmm, this reminds me of a pricinciple I read about a long time ago about balancing the intelligence between your objects)
  
• LSP: Subclasses should be suitable for their base clases (to avoid confusing code, on which you believe you can use a member of class hierarchy, and after trying you realize thet the result is not the expected one)
  

OMG: Object Orientation and Stored Procedures

OMG! how many times have I read the argument "using stored procedures is object oriented programming" just think of the database as an object, and think that each stored procedure is an method of the database....
Now... I am not going to say that using stored procedures is wrong... and I am not going to say that the only correct way of solving software problems is using object orientation... ( I do prefer to use object orientation, I a do prefer to use an object relational mapper instead of a stored procedures, specially for OLTP applications, I think application built that way are more maintainable, more portable between database, can be built faster, scale better, etc,etc)

But for batch processing, stored procedures have a clear advantage, and in my experience many business problems are solved by using an OLTP application, and a short but very important list of batch process... (of course the problem here is that if you built you application using an ORM, probably you have a lot of you business logic already implemented in a presentation-independent form, and, depending on the size of you database, you could program you batch process using your business objects... that has its disadvantages (slower performance that stored procedures) and its advantages (reutilization)... I believe reutilization and maintenance are more important that performance... specially because it is possible to worry about performance in the wrong way: for example Premature Optimization )

But the main point of this post, is that, first, I didn't think that using stored procedures was object oriented programming... it was "procedural programming"... or perhaps even "relational programming" but certainly not object oriented programming (where is the inheritance? where is the polymorphism?) but... after giving it some more thought... I realized that while it could not be "object oriented" it could be "object based"... stored procedures can encapsulate your tables (one of the major advantages of stored procedures) and you can see them as methods to a your "database" object...

So... now... your (OLTP?) system is a thin presentation layer on top of the "database object"... now... is there something in favor... or against that in object oriented design theory?.... mmm.... it achieves Presentation/Business Logic separation... and that is good... but I still feel there is something that doesn't feel right...

OMG! Well, it turns out there is! : Now the database is a God Object.
From Wikipedia: A God object is an object that knows too much or does too much. The God object is an example of an anti-pattern, it goes against divide & conquer ... using a God object is analogue of failing to use subroutines in procedural programming languages, because so much code references it, it makes maintenance difficult...

So... the next time someone tells you that using stored procedures is good object oriented programming... you could answer him or her: Oh My God (Object)! (You could think of it as object based, but, form an object oriented point if view.. it is a recommended solution? couldn't it be seen as an anti-pattern? I am not saying it is wrong, I am just saying that from an object oriented point of view, it might not be the best way to solve the problem... of course, using object orientation might not be the right way to solve this particular problem, after all, object orientation is not a Silver Bullet)

Object Relational Mapper vs Business Entity Manager

In my opinion the best two are Hibernate and EOF, I think a mix between them could create "the ultimate" Object Relational Mapper... or the ultimate Business Entity Manager...?

EOF (the best BEM?)

• General Overview
• Manual
  

Hibernate (the best ORM?)

• General Overview
• Manual
  

An interesting topic for analysis could be the limits between an Object Relational Mapper and an Business Entity Manager. Some people think that Hibernate shouldn't evolve to become more like EOF, because it should aim to do one and only one thing (be the best ORM) and other people think that it would be nice to have the services that an Business Entity Manager gives you:

• automatic connection handling
• automatic transactions handling
• multiple undo levels
• real nested transactions
• client and server objects
• distributed non-transactional lazy loading
• etc,etc
  

There are also some people that think that a Business Entity Manager is more closer to the MDA "objective", so I think it is important to take a look at (Using Borland's ECO to develop model-powered applications for .NET) an many related articles, the specially interesting thing for me about ECO is its OCL support (that can be used, for example to configure constraints that help ensuring that only consistent data is saved into the database... it is kind of a Design by Contract for the domain model)

IMO the Active Record pattern is getting too much popularity (perhaps thanks to Ruby , but I think it leads to an anemic domain model ) and Domain Model based in Business Entity Managers is not as popular (thanks to the problems in EJB1 and EJB2) but it is important to remember that EJB is not the only way to build an Business Entity Manager, there are ways to build Business Entity Managers that are a lot more lightweight and user friendly than EJB, (just investigate more about EOF and ECO)

UI Design: When do you save?

Hi!

How do developers save?? I know I know, after reading some of the comments on UI Design: Edit then List vs List then Edit I learned that most developers believe that the UI Design is a user concern, not a developer concern (build the UI as the User wants it, not as the developer wants it)... I have been wondering...

• Do the users really know what they want?
• Do the users really appreciate some of the internal behaviors the UI provides?
• Do all software projects have enough budget and type to make real usability test?
• Do all customers pay an amount of money for a project that justifies the extra effort of building a well designed UI?
• Do you as a developer have a framework that make all this issues moot and for you is so extremely easy to build a good UI that you always built it in the absolute best way?

Lets take, for example the "Save" or "Cancel" button in a typical CRUD application, and lets say the UI is build following List then Edit:

• The user search all customers named "John"
• The user selects the first one from the list
• The user clicks the edit button on the tool-bar (in the List window)
• The Edit window appears, the user modifies everything (from some fields, to complex to many, to one, many to many and many to many relationships) and then...
• • The user clicks "save" because he is happy with his changes
  • The user clicks "cancel" because he is not happy with his changes.

Now... a well behaved system, shouldn't write anything into the database until the "save" button is clicked (regardless of the complexity of the "edit customers" UI) and a well designed persistence API should make it as easy as "editingContext.Rollback()" to achieve that effect... and, if we weren't editing an customer, and from a performance perspective, if as a result of the editing of customer we created new "detail" objects(or rows) and we decided to cancel the operation, the server shouldn't even be aware that we created those objects, and then "roll-backed" its existence.

But the questions now is:

• Do we all always check that nothing is written on the database until we click "save"? and "Does the user care?"

I know I always try to check that if and only if the save button is clicked I save... but I have met lots of other developers that do not care... if for example, they have to add new addresses to the customer they write the new addresses into the database the moment they create them (from inside the edit customer ui!) and after the "cancel" button is clicked, this to many relationship is not rollbacked... they just don't care...

Now... I have to admit that I have done this sometimes (when in a extreme hurry to deliver a new form) , and "i feel" like it is enough if I add a message box warning (of those the user never reads) saying "if you add a new address all your changes will be committed to the database" ... the user, as always, just dismisses the warning and proceeds to add addresses... and perhaps click cancel later... nevertheless as soon as I have time, I fix the form so that it only saves when the save button is clicked... (and sometimes, with complex nested UIs that is a pretty hard job)

But... the thing is that I have seen lots of systems built with the idea "if you write it on the screen, and you do as much as click any button your changes will be saved" I have seen it specially in web based systems, but also in smartclients... and I have had the opportunity to hear all kinds of complains of this systems, and I have even replaced some of them with "only if you click save you save" systems... but I have never heard a single user saying "it is nice that the cancel button really works now" or "i hated the previous system because I clicked cancel and it saved my changes" but... strangely I have heard some of them complaining "I added 50 addresses to a customer, and then by mistake i clicked cancel, and then by mistake again I answered "yes" to the question "Are you sure you want to cancel your changes" and my changes were lost!!!!

But that has been my experience... maybe you do know a user that appreciates a consistent behavior in the "save" and "cancel" buttons...

How do you decide the transactional UI boundaries of your system? Do you really ask the user "do you want the cancel button" to really work? Do you always have the time to build 2 o 3 different UI prototypes and really test usability? Do your users really appreciate the extra 2 o 3 days that may take you to build a transactionally consistent UI? or the they prefer an UI that is built faster and if they made any mistakes, they "edit again" and rollback the mistakes manually (and they are happy with that behavior)? have you ever received a complain from a user saying "i want a cancel button that really works"? Do you implement cancel by calling an API similar to "editingContext.Rollback()" or do you cancel in "a custom way" (by explicitly deleting changes already made to the database) ? do you offer real "save" and "cancel" buttons even in your web based applications?

UI Design: Edit then List vs List then Edit

Today I was thinking about the way I build UIs (List then Edit) and the way most people I know build UIs (Edit then List) and the relation that the way you build you UI has with the way you store the data you manipulate in you UI.

List then Edit

In this "pattern" for UI structure, after the user selects an option in the menu he sees a window that allows searching with some controls (generally in the top of the window),and with some control in the bottom that represents the search results, then the user selects one of the search results (an object, or row if you prefer) and he proceeds to work (edit, modify) it, for that he opens a new window "the editor" (perhaps by double clicking the selected object, perhaps by clicking an "edit" button in the list window) in the editor he may modify the object being edited as much as he likes, and then click "save" if he wants to commit his changes or "cancel" if he wants to rollback them.
If the user wants to create a new object, the list has a "new" button that also calls the "editor" so that the user can set the initial values for the fields before the object is committed to the database for the first time

Edit then List

In this other "pattern" for UI structure, after we select an option in the menu we are right there in the editor, but all the controls that could allow us to modify the current object (or row) are disabled, because we haven't searched for any (or created a new one) from here the user can choose to create a new object that action has the effect of enabling all the control in the editor, or search for an already persisted object (or row) by clicking the "search button", that shows the search "list" window, in the top of that window there are controls to configure the search criteria, and in the top a control to represent the search results, from here the user can choose one of the search results (an object or row), and click on the "accept" button, which takes him back to the editor that now is displaying the previously selected object, here the use may choose to make some changes and click "save", or just cancel click "cancel" but actions have the effect of disabling the controls in the editor UI until the buttons "search" or "new" are used again.


Edit in List
When an application generally follows List the Edit, sometimes, if the info in the selected object is very simple, the user may modify it right there, but for for objects with medium to complex (several fields, to one or to many relations with other objects) calling a window to modify the object is more comfortable for the user

Search (List) in Editor
Some times, the developer likes to use the same UI to Edit and to Search, this works like some kind of "query by example", that is when user enters the editor, instead of being disabled an waiting for a click on the "new" or "search" button, the controls are enabled, and if the user writes some data in to them and clicks "search" the already persisted object (or row) that is more similar to the partial information already written in the editor is loaded. Sometimes there are many objects that match the query by example, here is when a navigator control can be a nice thing to have.

Master List, Detail Editor
This is also a very common configuration, both the list and the editor are in the same window, the list generally in the top, and the editor in the bottom, I think this somewhat corresponds with Two Panel Selector.

Persistence
Okey, until now I have exposed what I think are the some of the common "patterns" used to create UIs that manipulate data, now... what I find more interesting about this is the the way this "patterns" affect the way the data is stored or retrieved from persistence:

• If we use Edit then List:
  • The Edit is displayed on the screen
  • The user clicks "search" and the search List window appears
  • The user configures the search criteria in the List window
  • The objects (or row) matching the criteria are shown in the List window
  • The user selects one of the matching objects and click in the "accept button" in the List window
  • We return to the Edit window, that now is displaying the selected object.
  • We can make changes to the current object by clicking "save" or click "cancel" to dismiss the changes, if we do either action the Edit controls are disabled and we again have to click "search" or "new" to work with an object
  • The problem here, is that the user might want to see the same search results he used the last time, but each time we call the search List window, we are creating a new object, and all of the configuration from the last time we called it is already lost (this disadvantage has a "nice side", because we don't have to worry about showing "stale data" to our user)
  • Another disadvantage is that maybe not all the controls we use to manipulate the data in the edit windows are "databindable" so we have to manually reset the state of the Edit window, and if we don't do it correctly we could have bugs that "transfer information between edits". (First I edit John and set his age to 24, then I edit Mary, and her age is also shown as 24, but her age is 56, and we had no intention of changing it, it happened because we forgot to clear the text-box value)
    
• If we use List and then Edit:
  • The list is displayed on the screen
  • The user configures the search criteria in the List window
  • The objects (or row) matching the criteria are shown in the List window
  • The user selects one of the matching objects and click in the "edit button" in the List window.
  • The "Edit" window appears on top of the List window, and is displaying the selected object.
  • We can make changes to the current object by clicking "save" or click "cancel" to dismiss the changes, if we do either action the Edit window is closed and we return to the List window
  • One of the advantages of this approach is that we don't have to worry about the "transfer between edits" bug, because each time we call the Edit window, it is a new window, without any data, ready to configure itself to match the data contained in the object we are going to edit.
  • Then problem of keeping the search results to reuse them is also automatically solved, because the search List window was never closed, it was there all time, behind the edit window.
  • But the problem now is that some of data shown there may not be updated, or perhaps, it has data that has never been on the database, and that we do not want on the database, how can that be? well, we edited the object in the edit window, and the databinding ensured that the changes were written to the object before we clicked "save" or "cancel", if our intention is to keep those changes and we clicked "save" in the Edit window then we don't have a problem... but if we clicked cancel, now we need to rollback in memory changes and some of this changes could have modified relationships with other objects, or properties of the other objects, we could have created, deleted or updated a complex graph of objects but "only in memory" and now, we need to rollback all this changes, if we are using an Object Relational Mapper (ORM) that supports this (like Apple's EOF) or a relational cache that allows for in memory transactions (like the .NET DataSet) we can solve this problem easily (of course the DataSet has other disadvantages), but if we are using an ORM like NHibernate that AFAIK does not rollback in memory changes then you have a problem, you have to re-fetch the information from the database.

This seems like a big omission from the NHibernate guys... but it isn't exactly so, everything I have exposed here, has been on the assumption that we are working in a "Smart-Client" that holds local information, and Hibernate was born in "the web world". In the web, you "need" to re-fetch the information on each request (so when you go from editor to list or from list to editor you always reload the data) and you don't really pass the object you are going to edit from the list to the edit, it easier, and more efficient to just pass the primary key, of course the problem there is that you can only do that with objects previously persisted in the database, if you object is new it has to be serializable and some times that just isn't advisable (but that is an issue for another discussion)...

The thing with web based application is that the controls in the UI don't actually store the information directly in your object, their store it in the view-state, or in the query-string, in cookies, in the session or in an object that "represents the form", and only when you finally want to save, you extract the information from the web-controls and write it in to your object (at least that is more/less was the way I did it in the Java servlet world) but this is a "feature" that might be going away... the problem is that with the new JSF databinding, or with the new databinding facilities of ASP.NET 2.0, you can actually bind you controls directly with you datasources... and then how will we rollback the in memory changes? should we build a framework on top of NHibernate, a kind of "in memory object context" that handles the commits and rollbacks in memory?

More Questions

• And what about the case when a list window calls an edit window that has an embedded list control that calls another edit window? (complex multiple level or nested interfaces) should the object relational mapper make it easier for me to build this kind of UIs?
• Or Should a new kind of framework deal with the problem of communicating the persistent objects with the UI?
• is using DTOs really the solution? Does Apple's EOF go beyond the responsibility of an ORM by providing in memory transactions?
• If NHibernate can almost transparently persist objects to the UI, shouldn't this other framework be able do the same and transparently present the object in to the UI without having to manually create objects to do this job?
  
  I am thinking about publishing this post as an article in the Wikipedia to see how it evolves... I would love to read some comments about this article to improve it.