@swirlAI Thank you for writing the article. I have the following questions.
1) How does the flink receives/processes stream of data? Do task managers work on top of kafka queues?
2) Why do we need network managers for stream processing? I believe that the actor system already provides a way to move data and coordinate among tasks.
3) Why does flink uses concept of timeslots on the jobs. I don't think kafka stream processor uses timeslots concept?
4) The artictecture described is good for batch processing, but you mentioned that flink is used for stream processing?
These are SAI Notes that are meant for high level overview of Data related concepts. Explanation of Flink architecture is enough to fully grasp the picture of all the moving parts that compose the cluster (hence the architecture), detailed explanations are for long form deep dives.
@swirlAI Thank you for writing the article. I have the following questions.
1) How does the flink receives/processes stream of data? Do task managers work on top of kafka queues?
2) Why do we need network managers for stream processing? I believe that the actor system already provides a way to move data and coordinate among tasks.
3) Why does flink uses concept of timeslots on the jobs. I don't think kafka stream processor uses timeslots concept?
4) The artictecture described is good for batch processing, but you mentioned that flink is used for stream processing?
this article is full capitalized terms and doesn't explain the core concepts in. any meaningful manner.
Thank you for the feedback.
These are SAI Notes that are meant for high level overview of Data related concepts. Explanation of Flink architecture is enough to fully grasp the picture of all the moving parts that compose the cluster (hence the architecture), detailed explanations are for long form deep dives.