Track and tune EPP resource requests

[rambohe-ch]

Summary

The EPP (Endpoint Picker) Deployment in charts/modeldeployment currently sets:

epp:
  resources:
    requests:
      cpu: 100m
      memory: 256Mi

These values are inherited as-is from the upstream GAIE inferencepool chart and have not been validated against this fork's actual runtime footprint.

Concern

There is one EPP Deployment per modeldeployment. When several are scheduled together, they are likely to be packed onto the first node Karpenter provisions. With unmeasured requests (and no limits) this risks either overcommitting that node or, conversely, under-requesting and getting evicted under pressure.

Action items

• Measure actual EPP CPU/memory usage under representative load.
• Adjust epp.resources.requests based on the measurements.
• Decide whether to set epp.resources.limits.
• Consider pod anti-affinity / topology spread if EPP packing on a single node becomes a problem.

References

• charts/modeldeployment/templates/epp-deployment.yaml
• charts/modeldeployment/values.yaml (epp.resources)