kishorvpatil

[kishorvpatil]

Apache Storm New UI Features

By Kishorkumar Patil, Apache Storm PMC and Committer

Apache Storm has come a long way since its release in 2011. Over the past year, much work has been done to improve Storm’s scalability and performance. Many new user facing features have also been added, enhancing ease of use and debuggability in secure multi-tenant environments.  Important user facing features we’ve implemented at Yahoo include:

1. Dynamic Profiling of JVMs: With hundreds of JVMs running across the cluster, it is important to be able to take jstacks, heap-dumps and run JVM profiler without logging to each machine. Now, we support gathering these inputs directly from the UI.

2. Deep Search in Topology Logs: This enhances users’ ability to search errors across all logs for living/dead topologies directly from the UI.

3. Routing Logs to Mining systems: With gigabytes of logs being produced, routing logs to mining systems Logstash and ElasticSearch enables users to choose the best tool for log mining, analysis and visualization.

4. Dynamic Log Level for debugging: The ability to dynamically define new loggers and set log levels using the UI eliminates time consuming and expensive step of re-launching of topologies for debugging.