Academia.eduAcademia.edu

Figure 2 – uploaded by Ben Blamey

See full PDFdownloadDownload figure
Fig. 2: Architecture / Network Topology Comparison between the frameworks and streaming source intregrations — focusing on major network traffic flows. Note that with Spark and a TCP streaming source, one of the worker nodes is selected as ‘receiver’ and manages the TCP connection to the streaming source. Note that neither the monitoring and throttling tool (which communicates with all components) are shown, nor is additional metadata traffic between nodes (for scheduling, etc.). Note the adaptive queue- and P2P-based message processing in HarmoniclO.

Figure 2 Architecture / Network Topology Comparison between the frameworks and streaming source intregrations — focusing on major network traffic flows. Note that with Spark and a TCP streaming source, one of the worker nodes is selected as ‘receiver’ and manages the TCP connection to the streaming source. Note that neither the monitoring and throttling tool (which communicates with all components) are shown, nor is additional metadata traffic between nodes (for scheduling, etc.). Note the adaptive queue- and P2P-based message processing in HarmoniclO.

Related Figures (4)

Fig. 3: Performance of Apache Spark (under TCP, File stream- ing, and Kafka integrations), and HarmoniclO, by maximum message throughput frequency over the domain under study. Under each setting, the highest frequency is shown in bold, with color-coding according to which framework/integration was able to achieve the best frequency. Compare with Fig. 1.
Fig. 3: Performance of Apache Spark (under TCP, File stream- ing, and Kafka integrations), and HarmoniclO, by maximum message throughput frequency over the domain under study. Under each setting, the highest frequency is shown in bold, with color-coding according to which framework/integration was able to achieve the best frequency. Compare with Fig. 1.
Fig. 5: Maximum frequency by message size, for Spark (with TCP, File Streaming, Kafka) and HarmonicIO under varying message size; normalized as a fraction of the best performing framework for the particular parameter values. Fig. 4: Maximum stream processing frequencies for Spark (with TCP, File Streaming, Kafka) and HarmonicIO by message size; for selection of CPU cost/message.
Fig. 5: Maximum frequency by message size, for Spark (with TCP, File Streaming, Kafka) and HarmonicIO under varying message size; normalized as a fraction of the best performing framework for the particular parameter values. Fig. 4: Maximum stream processing frequencies for Spark (with TCP, File Streaming, Kafka) and HarmonicIO by message size; for selection of CPU cost/message.
Related topics:
Computer ScienceArchitecture
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved