Image may be NSFW.
Clik here to view.
How many logs/second can usually be handled by syslog servers? By syslog servers, I am referring to rsyslog, syslog-ng, splunk etc. The intent of the question is to find out at what logs/second rate the OS (Linux kernel >=3.0) becomes bottle-neck.
So far, I am able to forward about 10,000 logs/sec using UDP. If I increase the rate, logs start getting dropped. I don’t know what is causing the drops, is it the application (it is a custom build log server) or is it the OS?
Can you please share your experience with regards to how many logs/second you are able to transfer without seeing any drops?
Image may be NSFW.
Clik here to view.
You can increase the kernel UDP buffers, but this will not solve the problem. If you want to forward this amount of messages reliably, you should forget UDP syslog and use TCP instead.
We have measured over 500k events per second with nxlog when transferring syslog messages over TCP.
Check more discussion of this question.
