Architecting System Logs and Logging - Finding Proper Balance

System logs and logging is one of the least sexy things in software development.  Nevertheless, they are highly valuable.  The ultimate challenge in a logging architecture is balancing information with performance.  That is why you want to tackle this during the architecture process.  It is time-consuming when left until implementation or beyond.

The architecture part of logging often boils down to location.  There are facets of storage like archiving, rollover, and format.  However, these are often driven by storage decisions.  These include options like a file or database solution, a single file or one per system, and accessibility considerations.  Any of these options can be valid.  On the other hand, they may not be the best choice for your specific system.  Keep in mind that a single large file may be easy to design, but it will be challenging to use for debugging.

Log files are information-rich by design.  That means they can be useful to hackers that are trying to find access credentials, loopholes, or even reverse engineer your system.  You may need to partition out information to different file locations or limit the data logged.  There are also compliance issues like HIPPA or PCI to guide you in what should not be sent out to a log.

Most modern logging libraries include settings to allow you to set the amount of data logged form none to profuse.  These features are excellent.  However, you want to do what you can to lead the implementation team to follow a consistent approach.  While this can be covered in coding standards, you can also create an architecture that leads them in the right direction.

System logs are a feature of an application that often are ignored until something goes wrong.  Thus, it is not uncommon for maintainers to complain about poor or limited logging in a system.  When you keep these features in mind while architecting the solution, it is more likely you will avoid this weakness.