In HDP, KAFKA_HOME=/usr/hdp/current/kafka-broker/
1. Start server
./bin/kafka-server-start.sh config/server.properties
2. Topic operations:
Create a new topic:
./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic jsontest
List all topics:
./bin/kafka-topics.sh --list --zookeeper localhost:2181
Delete a topic:
kafka-topics --zookeeper localhost:2181 --delete --topic post
You will see the post topic is marked for deletion, not not happen yet.
Add
Then restart kafka and zookeeper. You will see the topic is deleted.
Delete a topic:
kafka-topics --zookeeper localhost:2181 --delete --topic post
You will see the post topic is marked for deletion, not not happen yet.
Add
delete.topic.enable = true
in Kafka config/server.properties
Then restart kafka and zookeeper. You will see the topic is deleted.
./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic jsontest \
--property parse.key=true \
--property key.separator=,
4. Sample consumer
./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic jsontest \
--from-beginning \
--property print.key=true \
--property print.value=true
--from-beginning: If the consumer does not already have an established offset to consume from, start with the earliest message present in the log rather than the latest message.
5. Shut down broker
./bin/kafka-run-class.sh kafka.admin.ShutdownBroker --zookeeper <zookeeper_host:port/namespace> --broker <brokerID>
If the broker has the lead partition shut down, this tool transfers the leadership proactively to other in-sync replicas on another broker. If there is no in-sync replica available, the tool will fail to shut down the broker in order to ensure no data is lost.
6. Start Schema Registry
./bin/schema-registry-start etc/schema-registry/schema-registry.properties
7. Override Topic Config
$ kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics --entity-name my-topic \
--add-config retention.ms=10
$ kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics --entity-name my-topic \
--delete-config retention.ms
8. Describe Consumer Group
$ /usr/bin/kafka-consumer-groups --zookeeper zk01.example.com:2181 --describe --group flume
No comments:
Post a Comment