Producing messages with key:value pairs from console producer

[vicradon]

Please how can I specify the key-value pairs for a console producer?
For example, I want to produce messages with string:integer, key:value pairs types. Below is what I inferred from the docs.

rpk topic produce streams-plaintext-input -f "string:int"

[vuldin]

I haven't used the CLI to produce events yet, but here is a link to a node script that reads from a file to produce events:

/~https://github.com/redpanda-data/redpanda-examples/blob/69d795c9696a34298722a5f10ed12b7d9d5ebbd9/clients/js/producer.js

You can run the above app with the following commands:

git clone /~https://github.com/redpanda-data/redpanda-examples.git
cd clients/js
npm install
node producer.js --brokers localhost:19092

Replace localhost:19092 in the last command with your own hostname:port.

[vuldin]

The CSV file the app reads is here: /~https://github.com/redpanda-data/redpanda-examples/blob/69d795c9696a34298722a5f10ed12b7d9d5ebbd9/spark/scala/src/main/resources/spx_historical_data.csv

You can just replace that data with whatever data you have and run the app.

[twmb]

The js is doing something a bit more advanced than can be done by rpk: read a row, convert it to json, and publish. rpk can't do any conversion from one shape to another. The parser right now can pick out the key / value / topic needles from a bunch of arbitrary text, but there is no action taken once the needles are picked out (perhaps there can be, though the DSL might be a bit odd).

[coral-waters]

doc ticket to make sure this makes it into the doc - #4144

[emaxerrno]

i think @twmb responded with rpk produce ... '%k:%v' you need a format string that coral will add to the docs.

[twmb]

If the input string is guaranteed not to contain a colon, the parsing format can be %k:%v\n, the first bit to parse the key until a colon is seen, and then parse a value until a newline is seen.