10 thoughts on successful server programming

What defines succesful server software and how can it be programmed?

How to program server software successfully

Everyone uses server software nowadays, at least in the internet, but most time people are not aware of it. Therefore "successful" server software is boring. Not the technology or the creation process itself, but server software is succesful when it behaves boring. Or, in other words, it simply works. Most times you do not know it's there and if it behaves as it should one tends not to see it.

So at first, a definition of "server software" would be helpful. Server software(s) are programs that do not run on your computer directly. Results of input are computed on another peace of hardware but then the results are shown in your own program (or internet browser, this very page you are reading is nothing else than a computed result from the servers of blogspot.com).

But how are programs prepared to work in the daily server world? Just a few (apparently obvious) thoughts to achieve this goal.

10 thoughts on successful server programming

1. Be aware that a server program or script is not comparable with software running on your computer at home. Server software has to run self-sufficient to a high degree.
2. Think of an appropriate logging to relieve later-on forensic fault diagnostics. Better to log too much than nothing, users who access your server will do this in ways you never thought of while programming it. And there is a good chance that you need to narrow down what exactly happened.
3. Use all features your programming language provides to write solid software. If exception handling is available, use it. Finally-blocks to clean up your code? Definitely. And there is much more you can utilize, from assertions to the usage of complete testing systems etc.
   
4. In any case avoid memory leaks. If you program C / C++ be very careful, but even if you use Java / .NET or scripting languages this may get an issue for long running server software (just assume that a collection of data in Java / .NET will only get longer and longer and never shorter).
   
5. When you use external resources (i.e. databases, networks or file systems) be aware that they may fail. The least should be an explanative error message for your users. The best would be an email notification to the administrator and a retry / recover of the failing system automatically (to a second database, for example). The parts in programming I need to rely on external resources are the parts I spend the most time developing them, create fallbacks for or even use logging at most.
   
6. Try to get the configuration as precise as possible. The configuration(s) of your systems are the base your software is running on, keep it as simple as possible and as complex as necessary. To ease your way changing your configuration later on simply follow an old saying from Antoine de Saint Exupery: "Perfection is reached not when there is nothing left to add, but when there is nothing left to take away."
   
7. If possible try to set up a test system. If you do not have a spare server for that, do it on your own (development) computer. The moment you notice erratical behaviour is the moment where you do not want to mess around setting up a system to replay and find the error.
   
8. Make Backups. Nobody wants backup but everyone cries for recovery. Do not forget that you write server software and, in case something bad happened, you have to be up and running as fast as possible. Backups may help to recover fast, then you got time to search for the problem.
   
9. If you follow all these steps do not forget that "releasing" has other implications for server software. The good news is that you may not need to release client-software on hundreds or thounsands of computers. The bad news is that you might have to take your server down to release new program(part)s and believe it or not: Users don't like that ;-) . So releasing should be as fast and stable as possible. Best would be that it is rollback-able to an older (working) release.
   
10. Alas, there is one very unpopular thought left: Document it. Start documenting the structure (i.e. involved server machines, a top-down documentation of the different modules/parts of your software etc.) and the programming code inline itself. There are many guides and tools out there that help you get your documentation right, search and follow the ones that help you most here.
    

Though this all sounds pretty obvious it is astounding that you nearly never see systems that follow all this tips. Even if you want to you may not be able to follow all of them because of restrictions from the outer world (timelines, new "prio zero" features needed yesterday etc.). But often it helps just to keep them in mind in your daily work and introduce them eventually. And now, happy server-software-coding!

How to successful log in programming

In a previous article, "Thoughts on successful logging", I wrote about why and how logging is done and, rudimentarily, about best practices in logging. This article now shows how successful logging can be done: Where and what should be logged and how the impact of logging on the overall performance (meaning: usage of resources and time to interpret log-messages) can be optimized.


Where and what to log or: What happens if logging goes wrong.

The simple answers to these two questions "Where?" and "What?" would be: Log everywhere, log all, then you have all the informations about your program in your log at any time. But most times, alas, there are no simple answers in life (or especially: in programming) and following these two answers will result in a couple of problems:

1. The log will explode with log messages, finding crucial messages will become difficult.
2. The log will get verbose because one incident raising a log condition might be logged more often than once (see below for an explanation).
3. The verbosity might be misleading. This can be the case when one incident is logged in different places in the code and therefore seems to deliver different meanings (at first sight).
   
4. Logging methods will be called more often than actually necessary and (thereby) increase the size of memory usage / log files etc. .
5. Gathering the informations more often than needed for the log message (i.e. retrieving additional informations, concating the message from them ...) and calling the log method will decrease the overall performance of the program or of parts of it.
   

Taking these problems serious and pondering a little about them will reveal that successful logging also demands a certain programming style, namely the kind that allows programmers to put the logging lines onto the right place. So let's first talk about the "where" to log, finding this answer will additionally help to clear how to log, seen from a programmers point of view.


A first example: Where to log or not to log. That is not the question.

Logging is like error (exception) handling: If you do it in the wrong place, you get too much or, alternatively, too less information. So the question you have to ask yourself is: Am I in a state of "misbehaviour" (or success) at this point that justifies a log message and - even more important - do I have all informations in this place to produce a log message I can read and am sure what exactly happened? Remember: The moment you do not work on the program every day you should be able to interpret the log message in a couple of seconds, too.

Let us assume we have methods openFile() and createFile() which return a file handle if the file could be opened / created or null if this is not possible. Then, owning a file handle we have the following methods on it: fileHandle.readContent(), fileHandle.writeContent() and fileHandle.deleteFile(). We then want to write the methods emptyFile() and moveFile(). The pseudo.code for this might look like:

function fileHandle prepareFile(String fileName) {
fileHandle = openFile(fileName);
if(fileHandle == null) {
logError("Cannot open a file.");
fileHandle = createFile(fileName);
if(fileHandle == null) {
 logError("Cannot open a file.");
}
return null;
}
// Eventually do some additional action here which justifies this method
return fileHandle;
}

function boolean emptyFile(String emptyFileName) {
fileHandle = prepareFile(emptyFileName);
if(fileHandle!=null) {
fileHandle.writeContent("");
fileHandle.close();
return true;
}
return false;
}

function boolean moveFile(String source, String destination) {
sourceFileHandle = prepareFile(source);
destinationFileHandle = prepareFile(destination);
if(sourceFileHandle==null || destinationFileHandle==null) {
return false;
}
else {
destinationFileHandle.writeContent(sourceFileHandle.readContent());
sourceFileHandle.deleteFile();
sourceFileHandle.close();
destinationFileHandle.close();
return true;
}
}

Obviously, this code will also produce misbehaviour (just look at the moveFile() - method and imagine what will happen if the source file cannot be opened). But moreover the log messages will not help you finding an error. At first, you only get the message "Cannot open file." - which file? But only changing the log line to logError("Cannot open file " + fileName + "."); will not do the trick either, though it is a lot better.

That obviously is because you can call one of the methods (emptyFile() or moveFile() ) and always only get the same log message. You do not exactly know where the error happened. The method prepareFile() is not the right place to log the message because it has not all informations needed ("am I emptying or moving a file?") to create a significant conclusion of the error that happened. The scope of the method prepareFile() is not suffice to log enough information to be precise.

Let us rewrite the methods for the sake of better logging:

function fileHandle prepareFile(String fileName) {
fileHandle = openFile(fileName);
if(fileHandle == null) {
fileHandle = createFile(fileName);
// Do not log here - this would be verbose!
return null;
}
// Eventually do some additional action here which justifies this method
return fileHandle;
}

function boolean emptyFile(String emptyFileName) {
fileHandle = prepareFile(emptyFileName);
if(fileHandle==null) {
logError("emptyFile: File " + emptyFile + "could not be prepared.");
return false;
}
else {
fileHandle.writeContent("");
fileHandle.close();
return true;
}
}

function boolean moveFile(String source, String destination) {
sourceFileHandle = prepareFile(source);
if(sourceFileHandle==null) {
logError("moveFile: Cannot prepare source file " + source + ".");
return false;
}
destinationFileHandle = prepareFile(destination);
if(destinationFileHandle==null) {
logError("moveFile: Cannot prepare destination file " + destination + ".");
sourceFileHandler.close();
return false;
}
destinationFileHandle.writeContent(sourceFileHandle.readContent());
sourceFileHandle.deleteFile();
sourceFileHandle.close();
destinationFileHandle.close();
return true;
}

As you can see, two things happened: Firstly, the logging will create better error messages, that kind you really know what happened. And secondly, more astonishing, the error handling improved. This, clearly, is based on the fact that logging and error-handling often go hand-in-hand (and you surely could do better code using exception handling and finally blocks). Or, asked in an other way: Why does anybody in 90 percent of all cases want to log? Yes: In case errors happened.

The improvement, of course, has it's price too: We needed an additional five lines to accomplish it. But this should only show one thing: Next time you think of the whereabouts of logging, simply think of the whereabouts of error-handling.


What to log.

The answer of the "where to log" already gave some answers to the "what to log" question. Errors, better said exceptional behaviour of the code (or the state of the program) surely is worth to be logged. As listet in the previous article, the purpose of logging changes as the programming work changes too. From simple checks if the program code works as desired to maintainance of the program, i.e. performance logging and logging of the proper (or improper) working of complete modules. This kind of logging should be added from the beginning on too, to a certain degree. It helps you to better understand your code at develop time and maintain the program later on in one big shot. Only make sure that verbosity will not be an issue at any time.

This can be achieved in quite a few ways and always depends on the functional blocks you want to monitor with logging. Let us assume you want to watch the processing time of a page created by your web application. A good idea would be to define a maximum time you allow the page to be rendered. Log only if this time-limit is exceeded, this way your log will only get flooded in case the program works too slow. In other cases, when performance is not an issue, try to get the basic conditions on how logging is useful (like "used time" while logging the performance) and follow them, this will keep your log slim and dandy.


Logging: A conclusion.

Let us conclude with a list of "best practices" for logging. These points merely are explained in the article and summarized in the following six points:

1. Try to formulate your logging messages that way that you can quickly recall the cause for the logging only by reading the message.
2. Treat logging like error (or success) handling: Find the right place for your logging message that has all informations you want to know. Improving the error/exceptional code handling will naturally create the places you can log easy and good. Don't log at every opportunity.
3. Often, less is more. Try to concentrate on vital log messages and avoid verbosity.
4. Sometimes, more is more. Think of the life-cycle of your program not only the time you code it, your program will go productive and certain logging messages (i.e. performance logging for memory or speed or database usage etc.) can too be very useful later on.
   
5. Use a logging (logging-API or Framework, self-written logging) that has different on-and-offswitchable log levels ("debug", "info", "error", "fatal") and is able to log into a kind of persistence layer, be it files, databases or something similar. The moment your program goes productive you will love this option for latter error analysis.
6. Use a logging (API, Framework, self-written) that interferes as little as possible with the rest of your code. It must not use too much resources or be overcomplex to use. Anyway, it should be able to be configured at run-time, i.e. switching debug-levels on or off, switching file logging on or off etc. .

So, happy logging! And might the information be with you... .

Thoughts on successful logging

Dealing with logging the last days, again, I thought it would be time to b-log about it too. So let us dive into the questions why programmers log, how programmers log and what the best and worst practices are on how to log.


Why logging?

Nearly every program and nearly every programmer does logging. And because of that, the purpose of logging changes through the lifecycle of a program.

At first, in "programming mode", logging normaly serves as status output and is used to validate the functioning of the newly written code. The simplest example for this would be a log-line in a block of code that is placed there to prove that the block is called anyway.
Introspection is another important thing at this point: If a programmer is not sure about the variables and values of a program at a certain point, he may simply output the values, the "state" of his program, using a log.
Logging also is useful to show off errors that do not directly occur in the programming code, such as a subsystem (like a database connection) went down etc. To know about these things is vital while programming: You never can be sure that your program works fine and dandy if the environment it works in isn't.

But as the program and the programming code evolves and grows more and more to a productive system, the intention of logging changes. The goal is not to monitor the code precisely any more but rather the modules of a program and their proper functionality. The main task of logging changes to: Survey the duration of operations, their failing or their success. This kind of log-mutation often comes smoothly and not abrupt or as a sheer matter of choice because the newly needed informations are usually already needed while programming, too.


How logging is done.

Logging exists as long as programming itself does. It starts by an ordinary output to the console via "println" and does not even end with high-level logging engines that are retrievable via network connections or external logging-client programs.

It always starts the same way. At first, just a couple of informations are merely printed to the console, after that, a little package is written that fulfills this task. Then, with rising complexity, the informations need to be categorized, otherwise crucial facts are lost in too many log-lines. By the way, logging means to trace the activity of a program over time, so the output has to be preserved in files or databases or repositories of any kind. And what about distributed systems on more than one computer, shouldn't they log to one place so that surveillance of only this log-source is possible? What started fairly simple can get pretty complex over time.

To not reinvent the wheel at this stage, there are a lot of logging APIs out there in the world. But they all boil down to some base functionality:

• The methods to log are - more or less - simple to call
  
• Different log-levels are introduced such as "debug", "trace", "info", "warn", "error", "fatal" so that the importance of a log-line can be classified
• Most loggings have different channels that can be switched on or of for observations
• Often, the log output can be canalized into a file or similiar data sinks

Anyway, fulfilling these preconditions do not define a good logging mechanism or API.


Best (and worst) practices doing the logdance.

Inserting a line of code that logs something always arises the same question: Is this correct, at this place, this time? Will this spam the log later on or is it accurately set here?

In first sight it is a good thing to log. If you create a system that will work for some time or reaches a certain degree of complexity it always is good to (at least vaguely) know what the system is doing right now. Or which conditions might have led to a failure of the code. Computer programs are systems that react on user or data and interact with their environment on many different variables and states. To write a program without logging is like having a black box you put input in and expect the correct output to be delivered. But then if something doesn't work proper you only have that black box to look at, without any further information why your program did not work.

But, as mentioned before, logging all and everything will not work out, either. Here are some suggestions on logging successful:

• Try to log a state or error just once. Log (only) at the end of an action or an error-handling block. This will reduce verbosity and makes the log much more readable.
  
• Think about to introduce prefixes to the log messages which will give a hint where the log message is placed in the code. Example: You have a module that parses URLs, so every message might start with "ParseURL:...". This way you directly know where the logging message comes from.
  
• Do not only think of the place in the program code where you want to log a message, but consider the environment (variables, states, etc.) to be logged, too. This way it is more easy to recollect not only the place but the circumstances from a log message itself.
  
• Keep in mind that the intention of logging will change as the program matures (described above). Sometimes spending 10 seconds more for a log message (i.e. to additionally show variables and states in the message) will greatly pay off later on.

So much for the "where to log" and "when to log", let's determine "how to log". As mentioned before, you can use own code or prebuild logging packages. Anyway logging should be made easy for the programmer (otherwise no one will use it) and treat computer ressources with respect (otherwise it will draw the performance of the program down).

Using prebuild packages is not a bad idea. Many functions that will be needed are available and can be used later on, even if at the start you are not aware that they will be useful. But be careful selecting the logger you want to use: It's use will spread through the whole program and changing it afterwards is nearly impossible. In general the development history of logging packages displays the idea what logging really should achive a little bit: From very simple modules to "all-in-one device suitable for every purpose" loggers, they nowadays return to a k.i.s.s-approach. Doing the log dance? Sure! But keep it simple, fast and easy-to-use, otherwise it will be counterproductive and no one will use it.

This article is continued at "How to successful log in programming"