Architecture Rituals and Patterns: On the Importance of the Middle Way Once and Only Once Balances Yagni - Part 2

...he heard a master teaching a girl how to play a string instrument. The master told her that if the string was too loose, it would not sound, but if the string was too tight, it would break: the string had to be at just the right tension to produce music and harmony. At that moment, he understood the middle way...

In the previous part of this series, I told you about YAGNI and OAOO and said that OAOO and YAGNI are like Yin and Yang. They balance each other. Without OAOO, YAGNI would slowly strangle us. With OAOO, you always have a system that you can improve (a phrase by Ron Jeffries).

I also "tied the cat," built systems as I saw others build them, was at the Shu level of ShuHaRi when programming enterprise systems, put my DAOs, put my Service, and put my Controller.

All the "well-architected" systems I knew did this, so I did it too. If a well-architected system looks like this in the end, it must be because it looked like this from the beginning... right?

Well, no. Things don't look the same when you're building them as when they're finished. If, for example, you try to build an arch without extra supports (which you will later remove), you could wrongly conclude that if you can't build it "without using supports," you're doing it wrong... That's not the case.

I discovered I was wrong almost by chance, I had already read that OAOO and YAGNI are like Yin and Yang. They balance, but I hadn't understood it beyond the sound of the words in the phrase... When I understood it, I wrote that article about epicycles.

I learned it when I came into contact with PHP programmers...

Speak your truth quietly and clearly; listen to others, even the dull and the ignorant; they too have their story. Desiderata

Those PHP programmers didn't know what YAGNI was... they knew nothing about the principle of building the simplest thing that could work (KISS)... they had never visited c2, and some of them had difficulty reading English...

And yet...

Yet precisely because of that, they followed YAGNI and KISS to a fascinating extreme, they did absolutely nothing that wasn't essential to finish on time the work assigned to them. Yes, their code was almost unbearable to the sight of someone like me at that time...

But what I needed months to build, they did in weeks; what I needed weeks for, they did in days; and what I needed days for, they did in hours...

On the other hand, their intense (and intuitive?) adherence to YAGNI and KISS also had its price, the technical debt they accumulated was brutal, and while they could put functionality in the hands of the user in record time, they also had to redo everything almost from scratch if they were asked for a change, there simply was no reuse...

There was no balance... But for me, it was a shock to look at the other extreme, and the other extreme looked back at me... I could spend hours refactoring code to eliminate redundancy to the maximum, whether the system was ready for the delivery date was a secondary factor... I was all OAOO... they were all YAGNI... I like to think that we learned a lot from each other..

Yagni means that we should only implement what we need now, not what we foresee that we are going to need. This strategy (when it works) eliminates some costs because it does not allow them to come into existence (they were things that we actually never came to need), and it delays the costs of those things that we will eventually need to a point in a future date. However, a commonly used counter-argument is that it is better to implement generic things now than in the future...

If we implement everything incrementally by invoking YAGNI, the system can become increasingly difficult to modify, especially if every time we implement something new, we do it always in the simplest possible way

It is good to do the simplest thing that could work, however, we must remember to reapply that rule to keep the whole system as simple as possible. This means that we will refactor the code so that nothing violates OAOO.

This refactoring, by centralizing any functionality into a single class or method, leaves the system well positioned for the next change. Anything you need to do afterward, you can be almost sure that you can do it better by improving or replacing a specific and isolated point of your code.

Now let's see an example:

• I have to make a registration screen, which includes
  

  a <select> element that displays the countries of the world, so that when a person registers on my site, I can present them with content relevant to their country (or countries) of origin. As I'm going to apply YAGNI, I won't do more than this:

• Controller:

public List<Country> getCountriesList() {
    logger.info("Getting countries!");
    List<Country> countries = getSimpleJdbcTemplate().query(
        "select id, name from country", 
        new CountryMapper());
    return countries;
}

Following YAGNI, I don't have to do more. If this ends up being the only screen in my entire system that displays countries, I won't have wasted my time on complexity without benefits.

But then, it turns out that in another part of the system, on the screen to register orders, when I have to put in the address where I want the product I'm purchasing to be delivered, and among the information to indicate in the address, I have to include the country

It is then, and only then, -when I already have 2 conceptually separate places, and for which it is not convenient to use the same controller- that I add a service:

• Countries Service:

public List<Country> getListOfCountries() {
    logger.info("Getting countries!");
    List<Country> countries = getSimpleJdbcTemplate().query(
        "select id, name from country", 
        new CountryMapper());
    return countries;
}

• Controller: User Registration

public List<Country> getProductsToList() {
    return getCountryServices().getListOfCountries();
}

• Controller: Order Address Registration

public List<Country> getProductsToList() {
    return getCountryServices().getListOfCountries();
}

This is why when a system is finished, it gives the impression that "everything is built with layers from the beginning." But that's a mistake, a product of seeing the system as something static, as something that "was born" with the form it had at a moment in time, and not as something organic, whose structure changes as it adapts to the needs of the user who acquired it.

The architecture of the project is not something that is predefined before understanding the system. The architecture is an emergent structure, derived from the user stories that are gradually built as the system is shaped. If the system does not include among its stories the need to operate on multiple different databases (Oracle, SQL Server, etc.), then the DAO layer should never have to exist. If it doesn't add value, it's superfluous.

In the case of the previous example, we can go even further. Nowadays, most applications are RIAs built on HTML5 and JavaScript, where the logic in Java (or in C#, or in Groovy, or in Scala...) merely acts as a layer of data exchange and security between the persistence layer (typically a database) and the business logic and presentation layer (which are usually located entirely in JavaScript)

So why do we insist on using POJOs that don't add value? Why doesn't our code look like this:

public List<Map<String, ?>> getCountriesList() {
    logger.info("Getting countries!");
    List<Map<String, ?>> countries = getSimpleJdbcTemplate().queryForList("select id, name from country");
    return countries;
}

Why complicate our existence with one (or several) POJO (or DTO) if our system is so simple (at the Java level) that the methods exposed by our controllers only do CRUD and it is the JavaScript code that really brings the data to life? What's the point of working with POJOs, if our domain model is anemic?

Approaches like OData, sweep away in one fell swoop all this paraphernalia of making controllers to slowly and step by step build a query API that could be automatically derived from the data model. But OData is just a query mechanism, which comes to eliminate the need to keep "tying the cat"... that is, to make a custom layer of controllers each time.

Why don't Oracle, SQLServer, MySQL, PostgreSQL, etc. directly offer an OData API (or something similar) instead of forcing us to reinvent it each time?

It seems that our human traditions make us live in cycles within cycles:

1. We invented Cobol, to expose our data through procedures.
2. We invented relational databases (whose implementation we never completed) to allow access to data through query expressions (SQL).
3. We then invented stored procedures, to procedurally encapsulate our query expressions.
4. We then invented object-relational mappers, to achieve compatibility between multiple databases and be able to query them with a unified query language.
5. We then invented SOAP, to procedurally encapsulate our objects.
6. We discovered that in parallel, someone had invented REST, which allows us to query resources using query expressions represented by a URL.

And so... again and again, procedures, expressions, procedures, expressions, procedural approach, declarative approach... and every time we reinvent the wheel, we feel like we are breaking new ground... when in reality, we are just giving the wheel of fashion another turn.

Will we ever find the balance? (Or perhaps we have already found it... and just need to realize it...)

 Originally published in spanish in Javamexico

 

 

Architecture Rituals and Patterns: On the Importance of tying up the cat - Part 1

The Zen master and his disciples began their evening meditation. The cat that lived in the monastery made so much noise that it distracted the monks from their practice, so the master ordered that the cat be tied up during the entire evening practice. When the master died years later, the cat continued to be tied up during the meditation session. And when, eventually, the cat died, another cat was brought to the monastery and was tied up during the practice sessions. Centuries later, scholars descendants of the Zen master wrote treatises on the profound spiritual meaning of tying up a cat for meditation practice.

A few years ago, I attended an event organized by Sun to promote Java here in Mexico (I think it was called Java Dev Days). At that time, I entered a presentation, I think it was by Sang Shin (the creator of JavaPassion), in which he explained how simple it was to use JRuby with Ruby on Rails to create a "hello world" page. It was something like this video: http://www.youtube.com/watch?v=sas3KCKwyHI. It was the first time I saw RoR, so on that side, it seemed educational... however, most of the audience... was falling asleep!

That caught my attention... why were they all so bored, from what I had been able to observe the previous days, many of them were enterprise programmers, with the kind of clients that I have had, who give you very little information to work with, demand a lot, and want everything for yesterday..., so I started to listen to the comments that they exchanged quietly (or not so quietly) between yawns:

    Is that simple? It took like 15 minutes to make a hello world screen!
    How boring, I could do that with JSP in 1 minute
    Where does this guy get that this is simple? Did you see the mega directory structure that those commands created? That's not simple at all!
    You can tell this presenter spends his time giving courses, but he's never worked in real life, I'd like to see him with his RoR having to get the job done at 3 in the morning

At least the 2 rows sitting in front of me were not remotely impressed with RoR. And why would they be if in JSP you can make a hello world simply by putting something like this:

<HTML>
 <HEAD>
  <TITLE>Hello World</TITLE>
 </HEAD>
 <BODY>
  <H1><%= "Hello World!"  %> </H1></BODY>
</HTML>

Certainly, arguing that the 15 minutes of the multi-step video are simpler than this is absurd.

Well, some will say, (as part of me thought) the point of the example with RoR is not to show how simple it is to make a silly "Hello World" but to show the potential of RoR's MVC architecture to do much more complex things...

Let me see if I understood... are you telling me that I'm going to complicate my life by making something easy much more difficult and that's going to result in a simplification of the difficult? More complexity today == Less complexity tomorrow? That violates YAGNI. (Yagni means: You aren't going to need it)

And it's not that you have to avoid violating YAGNI just to follow tradition, as in the case of the cat in the monastery. Yagni has a reason for being: in systems development, change is the only constant, and if you get ahead and build something before you have a clear idea of whether you really need it, when you get to the point where you are really going to use it, it is very likely that what you really need will be very different from what you had predicted. This doesn't mean that your code shouldn't be flexible, but you have to be very careful, because you can end up with an over-engineered solution, ready for situations that will never occur. If you use too much energy today, fighting with the ghosts of things that will never come, when the real enemies finally arrive, you won't have the strength to face them.

Now, one of the big problems we have in learning programming is that when we are learning, we have to build examples like a Hello World in RoR to understand how it works. But as instructors, we have to be very careful in saying "look, with this example I'm going to show you how simple this is...". No. You don't make a hello world in RoR because it's simple. Let's not confuse our students. We don't go to the gym to sweat because it's less tiring than staying at home watching TV. We go because by doing so we will improve our skills, we go to prepare ourselves for future situations. So, in programming, not only Yagni rules, we also have another principle: OAOO. (OAOO means, once and only once).

OAOO is the root cause of the apparent complexity of RoR, and the archenemy of copy&paste. OAOO has other names, it is also known as the DRY principle (Do not repeat yourself). It's not good to have copies of the same code scattered throughout the project, because if there is a bug in one of them, and you didn't follow OAOO, you'll have to find and fix them all, instead of solving the problem in one place.

OAOO and YAGNI are like Yin and Yang. They balance each other. Without OAOO, YAGNI would slowly strangle us. With OAOO, you always have a system that you can improve (a phrase by Ron Jeffries).

OAOO alone would also kill us, leading us to an eternal refactoring of code searching for the holy grail of the generic code with the fewest possible lines without ever stopping...

We need both. We need balance.

How many times have you faced code like this in your work:

DAO:
    public List<Product> getProductList() {
        logger.info("Getting products!");
        List<Product> products = getSimpleJdbcTemplate().query(
                "select id, description, price from products",
                new ProductMapper());
        return products;
    }

Service:
    public List<Product> getListOfProducts() {
        logger.info("Getting products!");
        List<Product> products = getProductsDao().getProductList()
        return products;
    }


Controller:
    public List<Product> getProductsToList() {
        logger.info("Getting products!");
        List<Product> products = getProductsServices().getListOfProducts()
        return products;
    }

What value does this add? Why can't we simply do it like this (no Service layer, no DAOs layer):

Controller:
    public List<Product> getProductsToList() {
        logger.info("Getting products!");
        List<Product> products = getSimpleJdbcTemplate().query(
                "select id, description, price from products",
                new ProductMapper());
        return products;
    }

Why is it that if you don't tie up the cat before meditating... I mean, why is it that the principles of multi-layer architecture dictate that it should be done this way... really? Is that really the answer? NO!

If it doesn't satisfy an immediate need and doesn't help OAOO (obviously the redundancy in this doesn't help OAOO), we shouldn't do it. We are burning precious time and energy on "busy work" that nobody will thank us for.

Heresy! Many have told me when I tell them to stop having so many layers just because. Layers are NECESSARY. You can't remove them.

Of course, you can, and I'm not the only one who thinks so. JBoss Seam was designed precisely with the idea of eliminating so much unnecessary layering. And that was YEARS ago. Why hasn't it dawned on most people that making layers for the sake of making layers is not correct? Why do we still tie up the cat and buy a new cat every time we start a new project? Tradition.

Now, traditions are not all bad, (those cultures with traditions that don't help their survival simply become extinct). Where does the idea come from that putting so many layers helps in any way?

I will talk about that in my next post in this series...

(originally published in spanish on Javamexico)