Troubleshooting onyx-kafka does not write to the topic. Howto run kafka in docker swarm. Error setting volume size at runtime (/ dev / shm)?

I am trying i) to eliminate a simple onyx-kafka work not recording the topic. See more details here . And you can try in this sample project . I think the reason is that there is only one kafka node.

So, I tried ii) starting kafka confluentinc/cp-kafka:3.3.1(with zookeeper confluentinc/cp-zookeeper:3.3.1), working with docker (09.17.0-ce, build afdb6d4) in swarm mode . But then I get this error.

Warning: space is running low in /dev/shm (shm) threshold=167,772,160 usable=58,716,160

A) With the docker layout, I could simply configure the installed disk /dev/shmfor a larger capacity with a configuration parameter shm_size: 1G(see here ).

Is there an equivalent for rock dockers? I just need to control the size of the disk volume. But I do not see such an option in the docker configuration deploy(see here ). The option docker service --mount does not seem to solve the problem. Any options here?

B) I do not quite understand how to do this <<27> here . Therefore, I do not know if this is a workable option.

C) Thanks to Andrew Mulholland (@itwasntandy) from the devops-engineersSlack channel , I can fake it by passing --mount type=tmpfs,dst=/dev/shm,tmpfs-size=768000000as call parameters docker service create ...(see here ).

, tmpfs docker stack deploy. docker, . .

. , docker-compose.yml ?

docker stack deploy --compose-file docker-compose.yml my-app. . . , :

  • i) , , onyx-kafka, (. ). , , ...
  • ii) (17.09.0-ce, build afdb6d4) , kafka confluentinc/cp-kafka:3.3.1 ( zookeeper confluentinc/cp-zookeeper:3.3.1). .
+1

Source: https://habr.com/ru/post/1016504/


All Articles