Simplicity and Hubris: Web vs Enterprise Development

I would find it really funny that most people do not understand the difference between intelligent and control interfaces if would not mean that lots of time and money are wasted trying to make intelligent interfaces work as control interfaces, or Web 2.0 Internet solutions work for Enterprise intranet problems.

It is the typical "since Google/Apple/Facebook/Twitter/SomeFreeAndPublicWebSite uses it for X/Y/Z/Q it should be great for the system I am building" syndrome.

Well, it might be great for them when they are trying to solve the X/Y/Z/Q problem, but as it turns out, many developers do not work at places that build the kind of systems this companies build, and are not trying to solve the kind of problems this companies are trying to solve.

Some people think that since our problems do not seem as technologically  impressive as those targeted by free public websites they must be easier, or that since a solution worked for the super huge public web site X then it should also work for the tiny intranet system we are building, well, the truth is that most of the times this solutions do not help at all. Please don't get me wrong, some of their ideas are usable, but we need to remember that in the dreaded Intranet Enterprise development world rules are very different:

• We do not have huge hardware resources at our disposal
• We do not have a huge budget and the hiring power to get the best and only the best experts to work with us
• Our systems are not of the"just register in this form this once and after this use the site with the mouse" kind, our users hate the mouse, because they are going to be capturing data all day, and for that, the keyboard is king (and the mouse is irrelevant)
• We need to interact directly with "special hardware" (such as scanners, printers, finger print readers etc) and browsers do not know how to talk to those, a browser can not even print with precision without help from Acrobat Reader.
• We need the behavior of our systems to be consistent and always the same (Web Search engine users do not care if the search results present different information or the same information in different order as long as the "relevant" stuff is in the first pages, Enterprise users expect search results to be exactly the same as last time, unless they have done something to alter that, and when they do, the change they expect is predictable)
• Our users want access to their data now, and do not care for excuses like "sorry but there is no access to the super massively great cloud because our infinitum connection/cable modem/whatever is failing"
• Our users want data to be confidential (although they do not even understand the meaning of security and its costs). Having all your data "in the cloud" sounds great, until the country where the clouds exists decides that it wants to apply "economic sanctions" to yours and begins by forbidding you access to the data in the cloud because the company that owns it is controlled by their laws. The day I see Google store its internal mission critical strategic information in Amazon's servers or vice-versa is the day I will believe that the Cloud is a safe place to store that kind of information.
  

All those "Web 2.0" companies have done little to help this stuff (In fact, they have created lots of problems, by forcing us intranet Enterprise developers to use primitive runtime platforms (web browsers) to deliver our applications . It is not that they are evil, it is just that they are not targeting the kind of problems Enterprise developers need to solve. And I hate when I see people recommending approaches that worked fine for this companies for problems that just can not be solved with them.

How can I use PowerShell to delete all the files with a particular extension

Just run the command:

get-childitem rootdir -include *.extension -recurse | foreach ($_) {remove-item $_.fullname}

And that is it. Thanks Scripting Guy.

This is specially useful for me because very often, after I make changes in project reference in VS.NET some .dll files are left “out of sync” and and make parts your project act as if you did not re-compiled them. Sadly there is no easy way to tell VS.NET to get rid of all the .dll files that may be causing problems.

Make that PDF secure: Toy Security

Make that PDF secure! That is one of the funniest requests I have received from a boss at one of my jobs… I answered by asking: “Secure? How secure?” and my boss told me: well, I want the user to be unable to modify it, copy it or print it.

Why do I say that this request is funny? Well, because this was a web application, the typical HTML+JavaScript+JSON/XML+Choose-the-Server-Side-Tech-You-Like application. Now, of course you can not really “display” a PDF with that technology, you need to have a PDF viewer like Acrobat Reader so that your user can “see” the PDF, and, of course, all the indented users for this intranet web applications had it installed.

My boss had read that Acrobat had an option to make a PDF read-only with a password, and that it was even possible to “protect” the document so that it could not be printed… sounds like the solution… no? well, first of all , if you can see it, it means that the structure of the document is available to the reader, which means that the document is not really encrypted, and that means the any tool capable of reading and writing .pdf will be able to remove the password without even worrying about trying to guess it using a brute force (or even an heuristic) attack.

What can you do then? Well you can actually encrypt it… that will prevent the user from modifying it… but it will also prevent it from being seen… unless the user has the password… which also means he/she can un-encrypt the document and remove the protection.

There is no thing that can be done then? well, if security of this document is actually a priority, you could create a custom reader unable to print… that should work should it? Well, not exactly, the user can always just press the “printscreen” key that is available in all keyboards and get a copy of the document… I have read that it is possible to install some DRM level protections in some OSes that could detect that, but when you take it to that level, a new questions is brought in to the table: is it worth the cost? does the document actually needs that kind of security? if not, it means this is a case of toy security, your boss just wants this feature because it sounds like a feature that “would impress the user” but not something that is really required by the application, he/she has not really give this features (and its implications) some thought.

Lets say that at my boss says: “Yes, that level of security is really needed” (very unlikely, but could happen), well, that brings in the next question: what if the user just goes, takes one of then great modern digital cameras, and takes the picture from the screen monitor? (and then spends a few days retouching it in the Gimp or Photoshop).That means it will be able to steal the information, and even be able to print it. How can you prevent that from happening? How can you make a document that is actually secure even after that?

Well, the answer is that if what you want to prevent is the theft of information, the only way is to not allow the user to see the document, if the user can see it, the user can copy it, one way of another (there is even the possibility that the user has photographic memory) and there is no way you can make the user “forget” that he saw the document.

But lets say that what you want to do is detect if a document is a forgery that is, right at the moment that you see it printed on a paper, you want to be able to know if it came from your system (and it was altered by someone after it was copied or not)… the only way to do that, is embedding the same information that the document has in a 2D barcode (including something that uniquely identifies the document), and also embed in another barcode a digital signature that is able to validate the data. That is the only way I know you can demonstrate that the document is a forgery… obviously, by the time you have explained this to you boss (if he/she has managed to make sense from it) the most likely thing is that he/she is unwilling to ask you to do all that it is needed to achieve this goal, mostly because it is nothing like what he/she wanted in the first place: it something much more abstract, and requires a barcode reader to verify the document.

How to find files containing a string with Powershell

A while ago a wrote a small post to remind me of How to find files containing a string in Unix.

Now, I have learned how to do pretty much the same thing but with Powershell:

get-childitem {directory} –include {file pattern} -recurse | select-string -pattern "{string to Find}" -casesensitive

You can write a path in {directory} or nothing to use the current directory.

Understanding REST Verbs and its relationship with CRUD

I used to believe that REST verbs mapped to CRUD operations more or less like this:

GET -> Read
DELETE -> Delete
POST->Insert/Update
PUT ->Insert/Update

Yes, I mapped both POST and PUT map to the same CRUD operation, but now I am not that sure this mapping is completely right…

I have learned that the difference between POST and PUT is the URI, in PUT the URI identifies the entity enclosed with the request (the entity you wan to Insert or Update), while in POST the URI identifies the resource that will handle the enclosed entity.

So, POST is like saying "Resource X, do something with this Entity", while PUT is more like "Insert/Update this Entity as Resource X"

So POST is more like “Tell ‘X’ to do something with ‘E’” and that has really no direct mapping with a CRUD operation… of course, a POST, internally, can do one, or many of the CRUD operations using the data of the entity that is being passed to it as a parameter… Does that mean that POST maps to all CRUD operations?

I guess I need to learn more about this…

Greenspuns Tenth Rule Of Programming and ADO.NET Data Services

This is a humorous observation once made by Philip Greenspun:

Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of CommonLisp.

Or the database variant (Greencodds Tenth Rule Of Programming):

Every sufficiently complex application/language/tool will either have to use a database or reinvent one the hard way

I sometimes I feel that:

Any sufficiently complicated procedural api contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of query language.

Take for example, Web Services, they started a procedural thing where people created their own custom verbs for each service, then people realized that the semantics of HTTP constitutes a coordination language which is is sufficiently for any computational communication pattern, they realized that GET / PUT / POST / were enough for any program, but they again forgot about the query part, so after a while they reinvented a query language (like SQL) by creating create a query language based on URL query strings, as in ADO.NET Data Services)

Sometimes I feel all IT industry is like a dog chasing its own tail… we invent Cobol to handle data procedurally only to invent SQL to handle data declaratively only to invent stored procedures and web services to handle it procedurally to then invent REST to handle it declaratively I wonder what will be the name of the procedural silver bullet that will replace REST…

JEE Development Deployment: In the order wrong

So, you start a new we project in Eclipse 3.4.1 (or in Netbeans 6.5.1) and you run it, what happens is that basically the project gets copied in to the directory the application server uses to “keep” the applications it runs, then the application server starts, looks in to its directory for applications and starts your application, and you are able to see your project,if you are using Tomcat, or JBoss as lot people do, you will be able to see you web project on port 8080. Right? Wrong!

what really happens is that then the application server starts and then the project gets copied in to the directory the application server uses to “keep” the applications it runs, then the IDE opens the web browser, that connects to the port 8080 using an url that includes the context of the application, its “name” so to speak, and then, the application server then realizes there a new application with that name in on its “directory applications” and starts it.

Noticed the difference:

What one typically believes it happens:

1. the project gets copied in to the directory the application server uses to “keep” the applications it runs
2. the IDE starts the application server
3. looks in to its directory for applications and starts your application…
4. now you will be able to see you web project on port 8080

What really happens:

1. the IDE starts the application server
2. the project gets copied in to the directory the application server uses to “keep” the applications it runs
3. then the IDE opens the web browser, that connects to the port 8080 using an url that includes the context of the application
4. the application server then realizes there a new application with that name in on its “directory applications” and starts it

Now, why is it so important to understand in what order this happens?

Well, because during development this processes is repeated again and again, until the application reach a point where it can be delivered to the customer the problem is that some times, one of the iterations of this process ends with an application so dysfunctional that it is able to crash the application server (and that will not stop until we get a real virtual machine for java).

But until we do get a better virtual machine, there is a more pressing problem, if we deploy an application that crashes the application server, the order in which the process is currently done makes it impossible to deploy the fixed version, unless we delete the previous version completely. Why do I say that: well lets analyze it, lets say we start with a fresh application server, where we have never installed our application:

1. then the application server starts
2. the project gets copied in to the directory the application server uses to “keep” the applications it runs
3. then the IDE opens the web browser, that connects to the port 8080 using an url that includes the context of the application
4. the application server then realizes there a new application with that name in on its “directory applications” and starts it

This first version of our application is far from finished, but it does not have any bug capable of crashing the application server, so we add som more functionality, and we ask the IDE to run our application again, and what does the IDE do?:

1. then the application server starts
2. since the application server already knows it has our (failed) application installed, it does not need a visit from the browser to start it, in starts it immediately
3. the project gets copied in to the directory the application server uses to “keep” the applications it runs, since the application is already there, only the changed files are copied
4. if the changes are of a particular kind (alterations to web.xml for example) the applications server restarts the application so that it starts working with the new configuration
5. then the IDE opens the web browser, that connects to the port 8080 using an url that includes the context of the application

This second version of our application actually has a nasty bug, so after step 4, the application server hangs, and we are forced to kill it using the services provided by the operating system to deal with misbehaving processes, then we go back to our code, fix the problem, and we ask the IDE to run our application again, and what does the IDE do?:

1. the IDE starts the application server
2. since the application server already knows it has our (failed) application installed, it does not need a visit from the browser to start it, in starts it immediately
3. The application server hangs.

And that is it, the project files are never copied in to the directory the application server uses to “keep” the applications it runs, the critical problem here is that the IDE seems to wait for the application server to start correctly before copying the new files (it should copy them any way, that of course would not prevent it from hanging, but, after we had killed the process, the next time we ran the application server, if we had fixed the bug, it would not hang any more… but even then the behavior would be anti-intuitive, why the process is not like this is a mystery to me:

1. the project gets copied in to the directory the application server uses to “keep” the applications it runs, since the application is already there, only the changed files are copied
2. the IDE starts the application server
3. since the application server already knows it has our (failed) application installed, it does not need a visit from the browser to start it, in starts it immediately
4. Since we have fixed the bug, everything runs fine

Do you know why it does not work like this? If you do, please… would you explain it to me?