Perspective on how people out there are marrying Python and Kafka together ideas (AC)
Python client libraries but you have to do it yourself All in on Faust which does everything for you Fuast emphasizes properly sized/structured messages like protobufs but Prefect is into pickling a n y t h i n g Ruby environment experience (PC)
handled mostly with the client libraries if you want to control them in more detail probably need to go to the original Java implementation Interface to look into the events
Confluent has one that we could base it off of (PC) () Where does the cursor/polling/event transferance etc actually happen? (PC)
in the FlowRunner (idea 1) in something akin to the scheduler that will submit a new flow run (PIN 14 idea) is it in the streaming services itself (like do we have an integration Kafka consumer published that knows how to call create_flow_run and then just leave everything meaningful on the streaming infra side?) The UI situation
is there a way we can make multiple flows that are associated with one stream look different in the UI? As above, is there a research project with Confluent or other systems to see how this is designed elsewhere and then we can back into how hard copying it/deriving it would be in our UI? 