Ts=T15:06:58.031631393Z caller=memberlist_logger.go:74 level=warn msg="Was able to connect to ingester-1-d816f38c but other probes failed, network may be misconfigured" Ts=T15:06:53.030784722Z caller=memberlist_logger.go:74 level=warn msg="Was able to connect to metrics_generator-ba2cb484 but other probes failed, network may be misconfigured" Ts=T15:06:48.030561892Z caller=memberlist_logger.go:74 level=warn msg="Was able to connect to ingester-2-a06d7368 but other probes failed, network may be misconfigured" Ts=T15:06:43.03000974Z caller=memberlist_logger.go:74 level=warn msg="Was able to connect to ingester-3-43da4322 but other probes failed, network may be misconfigured" The logs, for example from the digester: level=info ts=T15:06:35.368651832Z caller=memberlist_client.go:542 msg="joined memberlist cluster" reached_nodes=4 Ts=T09:49:56.071293583Z caller=memberlist_logger.go:74 level=error msg="Push/Pull with distributor-96cf3228 failed: dial tcp 10.0.0.102:7946: connect: connection refused"Įven with the interface or address set, I am still seeing warnings flood Level=error ts=T09:12:57.061880819Z caller=rate_limited_logger.go:27 msg="pusher failed to consume trace data" err="DoBatch: InstancesCount <= 0"
#Teamspeak failed to resolve hostname how to
Note the mix of ingress and docker_gwbridge IPs.Ĭan anyone offer a suggestion on how to resolve these warnings and have it only use the valid NIC and IPs, or point me to where in the documentation I can find an answer? My problem looks similar to issue 927, but I didn’t find a solution there.Īn internal network with no exposed ports isn’t of much use and the only solution I can think of is adding something like HA Proxy to the mix as at the moment the stack can’t consume spans, with the distributor logging errors like this: The issue seems to be down to the container having multiple NICs in these scenarios, but no amount of fiddling with http_listen_address, grpc_listen_address, instance_interface_names, interface_names, advertise_addr, or bind_addr and trying to “lock” the executing service to eth0 has fixed them, although they do have an effect as when I set everything to be eth0 (including the advertise/bind IP), I get errors like this:Įvel=warn ts=T08:32:28.387377885Z caller=tcp_transport.go:428 component="memberlist TCPTransport" msg="WriteTo failed" addr=10.0.0.9:7946 err="dial tcp 10.0.0.9:7946: i/o timeout" level=warn ts=T08:32:28.394078007Z caller=tcp_transport.go:428 component="memberlist TCPTransport" msg="WriteTo failed" addr=172.18.0.9:7946 err="dial tcp 172.18.0.9:7946: i/o timeout" The IPs getting logged seem to relate to the docker_gwbridge network. If I expose any port, I see errors like this: If the network is not declared as internal, I see errors like this in the logs: # If this is not internal, the multiple NICs mess up Tempo opt/tempo/configs/tempo.yml:/etc/tempo.yml:roĬommand: "-target=query-frontend -config.file=/etc/tempo.yml"Ĭommand: "-target=querier -config.file=/etc/tempo.yml"Ĭommand: "-target=compactor -config.file=/etc/tempo.yml"Ĭommand: "-target=metrics-generator -config.file=/etc/tempo.yml" # No ports exposed, the multiple NICs mess up Tempo This is the Swarm YAML file I am using: version: '3.8'Ĭommand: "-target=distributor -config.file=/etc/tempo.yml" However I am seeing continual network problems. I have taken the example distributed compose and converted it over to Docker Swarm.
0 kommentar(er)
