Python Forum
subscribing to kafka topic/key
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
subscribing to kafka topic/key
#1
hi all

has anyone been able to subscribe to a topic, but only want the data for a specific key coming back... without having to specify the partition.

now i can subscribe to topic and simply filter the data for my key... rather want this filter done by the brokers.

G
Reply
#2
???
Reply
#3
Larz, whats meant by ???

G

(Jan-02-2020, 10:13 AM)Larz60+ Wrote: ???
Reply
#4
Hello,
there is way too few information provided for (I would say most of people here) to be able to provide help.
We don't know what technology/protocol you are using. It could be MQTT from what you say. If that's the case, you should mention it (even better yet, put it in topic title), and hope there is someone in the forums who is familiar with it. If the question is about your code, you should post it. If it is about using the technology in general, it belongs to Bar, since it's not a Python coding question.
Reply
#5
look at the subject, I did mention Kafka as the technology. It has concept of a topic, a partition and a key, people having worked with Kafka will know exactly what I'm meaning.

G
Reply
#6
Good, thanks for explanation. Is it a Python coding question though? Or Python related in any way?
Reply
#7
Python accessing Kafka functionality, so python usage related.

G
Reply
#8
Please don't be cryptic.
It's difficult to determine what you're actual question is.
Please read https://python-forum.io/misc.php?action=help&hid=19
Reply
#9
Sorry but there is nothing cryptic bout what I’m asking, if you for one did not read my subject that specifies Kafka As per your first response where you actually said post it in the subject (where it was already)... don’t call it cryptic.

Kafka is a message bus, it uses topics and message keys to manage message flows to consumers and consumer groups.

I’m was very clear in my original post. I want to subscribe to a topic and just get message that match my specified key.

And as I’m asking this on a python forum I’d assume everyone would understand python is the language being used.


(Jan-02-2020, 08:37 PM)Larz60+ Wrote: Please don't be cryptic.
It's difficult to determine what you're actual question is.
Please read https://python-forum.io/misc.php?action=help&hid=19
Reply
#10
georgelza, I realize this thread hasn't been the most helpful for you, and I hope that changes, but I suggest you be mindful of your tone. Even if it's justified, it's likely to scare off question answerers. (I usually skip threads that get this bad, am only replying because I don't agree with how the mods handled things up to this point.)

All that said, your question is cryptic because (1) you asked how to do something I expect to be clearly documented and (2) you didn't provide any code, so I have no idea what you're doing. It's really easy for conversations about code to degenerate if the code isn't provided and for some reason no one asks for it.

When you provide your code, it should be as minimal as possible. Remove any unused imports, input() calls, file I/O (if it's related to the question) and hard-code inputs for the simplest possible thing for us to play with. I assure you we do want to help, and we'll make an effort if we get what we need to be helpful.

On top of this, I recommend you at the very least link to resources for getting Kafka going, the easier the better. For this kind of question we're likely to want to test our suggestions, and many of us won't go the extra mile to install extra software, especially if the question asker doesn't provide help with it.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Confusing in [for loop] topic Sherine 11 3,382 Jul-31-2021, 02:53 PM
Last Post: deanhystad
Photo Integration of apache spark and Kafka on eclipse pyspark aupres 1 3,701 Feb-27-2021, 08:38 AM
Last Post: Serafim
  [split] New thread/topic rajp1497 1 1,845 Sep-24-2020, 01:55 AM
Last Post: micseydel
  How do I create a actor if I subscribe to a specific topic? sdf1444 0 1,673 Aug-01-2019, 09:29 PM
Last Post: sdf1444
  Help with finding correct topic in Python learning yahya01 1 2,168 Jun-06-2019, 05:01 PM
Last Post: buran
  Topic Modelling - Document Labels Nicson 0 1,685 Nov-20-2018, 04:56 PM
Last Post: Nicson
  Projected Surface in 2D [Difficult topic] Hans_K 6 3,792 Aug-02-2017, 09:16 AM
Last Post: Hans_K

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020