hi
I'm working on geting my Windows nodes (EKS) mo...
# supportn
Nadav Mavor
01/12/2025, 8:16 PMhi
I'm working on geting my Windows nodes (EKS) monitoring the Linux nodes in the cluster are working with no issues looking at the Windows I see an error
Copy code
Name: my-release-k8s-infra-otel-agent-psv7k
Namespace: k8s-infra
Priority: 0
Service Account: my-release-k8s-infra-otel-agent
Node: ip-10-200-92-137.il-central-1.compute.internal/10.200.92.137
Start Time: Sun, 12 Jan 2025 14:32:00 -0500
Labels: <http://app.kubernetes.io/component=otel-agent|app.kubernetes.io/component=otel-agent>
<http://app.kubernetes.io/instance=my-release|app.kubernetes.io/instance=my-release>
<http://app.kubernetes.io/name=k8s-infra|app.kubernetes.io/name=k8s-infra>
controller-revision-hash=54f56899c4
pod-template-generation=1
Annotations: checksum/config: e36604974160324c46f777a1165d6961b31fec4a868929b286f003df9e974bb6
Status: Pending
IP:
IPs: <none>
Controlled By: DaemonSet/my-release-k8s-infra-otel-agent
Containers:
my-release-k8s-infra-otel-agent:
Container ID:
Image: <http://docker.io/otel/opentelemetry-collector-contrib:0.109.0|docker.io/otel/opentelemetry-collector-contrib:0.109.0>
Image ID:
Ports: 13133/TCP, 8888/TCP, 4317/TCP, 4318/TCP
Host Ports: 13133/TCP, 8888/TCP, 4317/TCP, 4318/TCP
Command:
/otelcol-contrib
Args:
--config=/conf/otel-agent-config.yaml
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 100m
memory: 100Mi
Liveness: http-get http://:13133/ delay=10s timeout=5s period=10s #success=1 #failure=6
Readiness: http-get http://:13133/ delay=10s timeout=5s period=10s #success=1 #failure=6
Environment:
OTEL_EXPORTER_OTLP_ENDPOINT: my-release-signoz-otel-collector.platform:4317
OTEL_EXPORTER_OTLP_INSECURE: true
OTEL_EXPORTER_OTLP_INSECURE_SKIP_VERIFY: false
OTEL_SECRETS_PATH: /secrets
K8S_CLUSTER_NAME: kubernetes-ilcentral1-prod-01
DEPLOYMENT_ENVIRONMENT: production
K8S_NODE_NAME: (v1:spec.nodeName)
K8S_POD_IP: (v1:status.podIP)
K8S_HOST_IP: (v1:status.hostIP)
K8S_POD_NAME: my-release-k8s-infra-otel-agent-psv7k (v1:metadata.name)
K8S_POD_UID: (v1:metadata.uid)
K8S_NAMESPACE: k8s-infra (v1:metadata.namespace)
SIGNOZ_COMPONENT: otel-agent
OTEL_RESOURCE_ATTRIBUTES: signoz.component=$(SIGNOZ_COMPONENT),k8s.cluster.name=$(K8S_CLUSTER_NAME),k8s.node.name=$(K8S_NODE_NAME),host.name=$(K8S_NODE_NAME)
Mounts:
/conf from otel-agent-config-vol (rw)
/hostfs from hostfs (ro)
/var/lib/docker/containers from varlibdockercontainers (ro)
/var/log from varlog (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kzbqp (ro)
Conditions:
Type Status
PodReadyToStartContainers False
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
otel-agent-config-vol:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: my-release-k8s-infra-otel-agent
Optional: false
varlog:
Type: HostPath (bare host directory volume)
Path: /var/log
HostPathType:
varlibdockercontainers:
Type: HostPath (bare host directory volume)
Path: /var/lib/docker/containers
HostPathType:
hostfs:
Type: HostPath (bare host directory volume)
Path: /
HostPathType:
kube-api-access-kzbqp:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: op=Exists
<http://node.kubernetes.io/disk-pressure:NoSchedule|node.kubernetes.io/disk-pressure:NoSchedule> op=Exists
<http://node.kubernetes.io/memory-pressure:NoSchedule|node.kubernetes.io/memory-pressure:NoSchedule> op=Exists
<http://node.kubernetes.io/not-ready:NoExecute|node.kubernetes.io/not-ready:NoExecute> op=Exists
<http://node.kubernetes.io/pid-pressure:NoSchedule|node.kubernetes.io/pid-pressure:NoSchedule> op=Exists
<http://node.kubernetes.io/unreachable:NoExecute|node.kubernetes.io/unreachable:NoExecute> op=Exists
<http://node.kubernetes.io/unschedulable:NoSchedule|node.kubernetes.io/unschedulable:NoSchedule> op=Exists
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 40m default-scheduler Successfully assigned k8s-infra/my-release-k8s-infra-otel-agent-psv7k to ip-10-200-92-137.il-central-1.compute.internal
Warning FailedCreatePodSandBox 40m kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cb27d0d9180127bc9db8a9d20d5d4ebf21c0defffca9c505041cb5b3aa02bbdc": plugin type="vpc-bridge" name="vpc" failed (add): failed to parse Kubernetes args: failed to get pod IP address my-release-k8s-infra-otel-agent-psv7k: error executing k8s connector: error executing connector binary: exit status 1 with execution error: pod my-release-k8s-infra-otel-agent-psv7k does not have label <http://vpc.amazonaws.com/PrivateIPv4Address|vpc.amazonaws.com/PrivateIPv4Address>
Warning FailedCreatePodSandBox 40m kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "dfd1bf178c0c11672ded8734cb509100c54d11b37e10e5a65b081dad0a9045ea": plugin type="vpc-bridge" name="vpc" failed (add): failed to parse Kubernetes args: failed to get pod IP address my-release-k8s-infra-otel-agent-psv7k: error executing k8s connector: error executing connector binary: exit status 1 with execution error: pod my-release-k8s-infra-otel-agent-psv7k does not have label <http://vpc.amazonaws.com/PrivateIPv4Address|vpc.amazonaws.com/PrivateIPv4Address>Nadav Mavor
01/12/2025, 8:17 PMand looking at the nodes, its looking good
Copy code
Name: ip-10-200-92-137.il-central-1.compute.internal
Roles: <none>
Labels: <http://beta.kubernetes.io/arch=amd64|beta.kubernetes.io/arch=amd64>
<http://beta.kubernetes.io/instance-type=c5.xlarge|beta.kubernetes.io/instance-type=c5.xlarge>
<http://beta.kubernetes.io/os=windows|beta.kubernetes.io/os=windows>
<http://failure-domain.beta.kubernetes.io/region=il-central-1|failure-domain.beta.kubernetes.io/region=il-central-1>
<http://failure-domain.beta.kubernetes.io/zone=il-central-1c|failure-domain.beta.kubernetes.io/zone=il-central-1c>
<http://k8s.io/cloud-provider-aws=f75f73fdee9d99be663751437b625d58|k8s.io/cloud-provider-aws=f75f73fdee9d99be663751437b625d58>
<http://kubernetes.io/arch=amd64|kubernetes.io/arch=amd64>
<http://kubernetes.io/hostname=ip-10-200-92-137.il-central-1.compute.internal|kubernetes.io/hostname=ip-10-200-92-137.il-central-1.compute.internal>
<http://kubernetes.io/os=windows|kubernetes.io/os=windows>
<http://node.kubernetes.io/instance-type=c5.xlarge|node.kubernetes.io/instance-type=c5.xlarge>
<http://node.kubernetes.io/windows-build=10.0.17763|node.kubernetes.io/windows-build=10.0.17763>
<http://topology.ebs.csi.aws.com/zone=il-central-1c|topology.ebs.csi.aws.com/zone=il-central-1c>
topology.k8s.aws/zone-id=ilc1-az3
<http://topology.kubernetes.io/region=il-central-1|topology.kubernetes.io/region=il-central-1>
<http://topology.kubernetes.io/zone=il-central-1c|topology.kubernetes.io/zone=il-central-1c>
Annotations: <http://alpha.kubernetes.io/provided-node-ip|alpha.kubernetes.io/provided-node-ip>: 10.200.92.137
<http://csi.volume.kubernetes.io/nodeid|csi.volume.kubernetes.io/nodeid>: {"<http://ebs.csi.aws.com|ebs.csi.aws.com>":"i-0f74f3f574869cb84"}
<http://node.alpha.kubernetes.io/ttl|node.alpha.kubernetes.io/ttl>: 0
<http://volumes.kubernetes.io/controller-managed-attach-detach|volumes.kubernetes.io/controller-managed-attach-detach>: true
CreationTimestamp: Sun, 12 Jan 2025 14:31:59 -0500
Taints: <http://node.kubernetes.io/os=windows:NoSchedule|node.kubernetes.io/os=windows:NoSchedule>
Unschedulable: false
Lease:
HolderIdentity: ip-10-200-92-137.il-central-1.compute.internal
AcquireTime: <unset>
RenewTime: Sun, 12 Jan 2025 15:13:46 -0500
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Sun, 12 Jan 2025 15:10:37 -0500 Sun, 12 Jan 2025 14:31:55 -0500 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Sun, 12 Jan 2025 15:10:37 -0500 Sun, 12 Jan 2025 14:31:55 -0500 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Sun, 12 Jan 2025 15:10:37 -0500 Sun, 12 Jan 2025 14:31:55 -0500 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Sun, 12 Jan 2025 15:10:37 -0500 Sun, 12 Jan 2025 14:31:59 -0500 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 10.200.92.137
InternalDNS: ip-10-200-92-137.il-central-1.compute.internal
Hostname: ip-10-200-92-137.il-central-1.compute.internal
Capacity:
cpu: 4
ephemeral-storage: 52426748Ki
memory: 8080924Ki
pods: 110
<http://vpc.amazonaws.com/PrivateIPv4Address|vpc.amazonaws.com/PrivateIPv4Address>: 224
Allocatable:
cpu: 4
ephemeral-storage: 48316490877
memory: 7978524Ki
pods: 110
<http://vpc.amazonaws.com/PrivateIPv4Address|vpc.amazonaws.com/PrivateIPv4Address>: 224
System Info:
Machine ID: EC2AMAZ-RB0KTMB
System UUID: EC277401-15BA-506B-EC00-DED85ACE4D64
Boot ID: 816
Kernel Version: 10.0.17763.6659
OS Image: Windows Server 2019 Datacenter
Operating System: windows
Architecture: amd64
Container Runtime Version: <containerd://1.7.14>
Kubelet Version: v1.30.7-eks-59bf375
Kube-Proxy Version: v1.30.7-eks-59bf375
ProviderID: aws:///il-central-1c/i-0f74f3f574869cb84
Non-terminated Pods: (3 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
ipol-staging webapp-service-69664d45bd-csm64 1250m (31%) 4 (100%) 2400Mi (30%) 6600Mi (84%) 20m
k8s-infra my-release-k8s-infra-otel-agent-psv7k 100m (2%) 0 (0%) 100Mi (1%) 0 (0%) 41m
kube-system ebs-csi-node-windows-n5ms5 30m (0%) 0 (0%) 120Mi (1%) 768Mi (9%) 41m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 1380m (34%) 4 (100%)
memory 2620Mi (33%) 7368Mi (94%)
ephemeral-storage 0 (0%) 0 (0%)
<http://vpc.amazonaws.com/PrivateIPv4Address|vpc.amazonaws.com/PrivateIPv4Address> 2 2
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 41m kube-proxy
Normal Starting 41m kubelet Starting kubelet.
Normal NodeHasSufficientMemory 41m (x2 over 41m) kubelet Node ip-10-200-92-137.il-central-1.compute.internal status is now: NodeHasSufficientPID
Normal NodeReady 41m kubelet Node ip-10-200-92-137.il-central-1.compute.internal status is now: NodeReady
Normal Synced 41m cloud-node-controller Node synced successfully
Normal ControllerVersionNotice 41m vpc-resource-controller The node is managed by VPC resource controller version v1.6.2
Normal RegisteredNode 41m node-controller Node ip-10-200-92-137.il-central-1.compute.internal event: Registered Node ip-10-200-92-137.il-central-1.compute.internal in ControllerNadav Mavor
01/12/2025, 8:17 PMim not even close to max out the node
Copy code
Allocatable:
cpu: 4
ephemeral-storage: 48316490877
memory: 7978524Ki
pods: 110
<http://vpc.amazonaws.com/PrivateIPv4Address|vpc.amazonaws.com/PrivateIPv4Address>: 22428 Views