apache-kafka - Apache Mesos with Kafka - answerstu - answerstu.com answerstu

Apache Mesos with Kafka

I am working on integrating Mesos with Kafka, as per the git hub documentation scheduler can be run on a independent machine which has mesos and kafka installed and with the help of configuration registration to mesos-master can be done.

How do we run brokers on slave, from scheduler and what configuration is required to run broker on slave.

how to configure slave as broker.

2 Answers

  1. Martin- Reply

    2019-11-13

    If you're already using Docker, you can make use of the Docker image as described in

    You can launch the Kafka Scheduler image with Marathon, and then either use the REST API or CLI to start brokers/topics.

    The dispatching of the actual brokers will then be done by the Mesos software itself. Using the brokers is done via the defined Zookeeper node name.

  2. Matthew- Reply

    2019-11-13

    Thanks, I was able to run the scheduler through shell and as well as docker. I had two questions though

    sudo docker run -t -p 5900:5900 --net=host root/kafka-mesos ./kafka-mesos.sh \ 
      scheduler --master=192.168.1.115:5050 --zk=192.168.1.115:2181 \
      --api=http://192.168.1.118:5900 --storage=zk:/kafka-mesos \
      --log=/var/log/mesos.log --debug=true
    

    I have used above command in marathon to run the container, if I remove --net=host the schedule registers with docker containers ip with the master but after adding --net=host it uses the slave's ip and port forwards it to docker container. Is there any better way?

    The second question is how many schedulers can we bring up in the one mesos cluster say 3 master 5 slaves i.e. can we scale or support scheduler cluster?

Leave a Reply

Your email address will not be published. Required fields are marked *

You can use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>