

Etsy is known for helping people “make a living making things”. The engineers at Etsy also specialize in making things, particularly when it comes to managing the website’s servers. While some companies have embraced fancier logging tools like Flume, Scribe, or Logstash, Etsy has decided to stick with a tried and true technology: Syslog-ng. The engineers have perfected it for the company’s large-scale needs with some clever performance tuning.
Etsy runs syslog on its own server and configures all of its other servers to send log data to the syslog server. The central syslog server is an 8-core system with 12 GB of RAM. It currently absorbs approximately 60,000 events per second and peaks at about 25 percent of the server’s CPU load. These events may include Apache web server access and error logs, Squid proxy logs, and other important server information.
Some examples of the performance tuning Etsy is doing include:
Overall, Etsy was able to reduce CPU consumption from 70 percent to 15 percent with a little research and testing to find the best ways to aggregate server logs. All of the technical details, including graphs, are available on the Etsy developer blog.
Support our open free content by sharing and engaging with our content and community.
Where Technology Leaders Connect, Share Intelligence & Create Opportunities
SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.