A Distributed Network Logging Topology

Network logging is used to monitor computer systems for potential problems and threats by network administrators. Research has found that the more logging enabled, the more potential threats can be detected in the logs. However, generally it is considered too costly to dedicate the manpower required to analyze the amount of logging data that it is possible to generate. Current research is working on different correlation and parsing techniques to help filter the data, but these methods function by having all of the data dumped in to a central repository. Central repositories are limited in the amount of data they are able to receive without losing some of the data. In large networks, the data limit is a problem, and industry standard syslog protocols could potentially lose data without being aware of the loss, potentially handicapping network administrators in their ability to analyze network problems and discover security risks. This research provides a scalable, accessible and fault-tolerant logging infrastructure that resolves the centralized server bottleneck and data loss problem while still maintaining a searchable and efficient storage system.