Only one Gateway instance supported

[neotheprogramist]

Describe the bug
The bug is simple, I wanted to have 2 Gateway resources in separate namespaces, that handle different hostnames.
Gateways that uses different GatewayClasses can be deployed succesfully.
It seems that there is support for only ONE Gateway per GatewayClass.

kubectl get gateway -A
NAMESPACE     NAME                CLASS              ADDRESS   PROGRAMMED   AGE
default       different-gateway   production-nginx             False        37s
dev-gateway   dev-gateway         production-nginx             True         88s

kubectl describe gateway/different-gateway 
Name:         different-gateway
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  gateway.networking.k8s.io/v1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2024-01-27T17:25:37Z
  Generation:          1
  Resource Version:    1694
  UID:                 c1e607c6-f529-4421-89ce-d84de503c3ba
Spec:
  Gateway Class Name:  production-nginx
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  Same
    Hostname:  visoft.dev
    Name:      visoft-dev-https
    Port:      443
    Protocol:  HTTP
Status:
  Conditions:
    Last Transition Time:  2024-01-27T17:26:13Z
    Message:               The resource is ignored due to a conflicting Gateway resource
    Observed Generation:   1
    Reason:                GatewayConflict
    Status:                False
    Type:                  Accepted
    Last Transition Time:  2024-01-27T17:26:13Z
    Message:               The resource is ignored due to a conflicting Gateway resource
    Observed Generation:   1
    Reason:                GatewayConflict
    Status:                False
    Type:                  Programmed
Events:                    <none>

To Reproduce
Steps to reproduce the behavior:

    - name: Create Gateway API CRD
      kubernetes.core.k8s:
        src: https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

    - name: Install NGINX Gateway Fabric Helm Chart for development
      kubernetes.core.helm:
        name: dev-gateway
        namespace: dev-gateway
        chart_ref: oci://ghcr.io/nginxinc/charts/nginx-gateway-fabric
        atomic: true
        create_namespace: true
        wait: true
        values: "{{ lookup('file', '../resources/values/dev-gateway.yaml') | from_yaml }}"

    - name: Install NGINX Gateway Fabric Helm Chart for production
      kubernetes.core.helm:
        name: production-gateway
        namespace: production-gateway
        chart_ref: oci://ghcr.io/nginxinc/charts/nginx-gateway-fabric
        atomic: true
        create_namespace: true
        wait: true
        values: "{{ lookup('file', '../resources/values/production-gateway.yaml') | from_yaml }}"

    - name: Create the Gateway
      kubernetes.core.k8s:
        namespace: default
        src: ../resources/gateway.yaml

resources/gateway.yaml

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: dev-gateway
  namespace: dev-gateway
spec:
  gatewayClassName: production-nginx
  listeners:
    - name: localhost-http
      protocol: HTTP
      port: 80
      hostname: localhost
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: different-gateway
spec:
  gatewayClassName: production-nginx
  listeners:
    - name: visoft-dev-https
      protocol: HTTP
      port: 443
      hostname: visoft.dev

Expected behavior
The different-gateway PROGRAMMED status is True

Your environment

• Version of the NGINX Gateway Fabric - release version or a specific commit. The first line of the nginx-gateway container logs includes the commit info. - repository: ghcr.io/nginxinc/nginx-gateway-fabric tag: 1.1.0
• Version of Kubernetes - kindest/node:v1.29.0
• Kubernetes platform (e.g. Mini-kube or GCP) - kind
• Details on how you expose the NGINX Gateway Fabric Pod (e.g. Service of type LoadBalancer or port-forward)

service:
  ## Creates a service to expose the NGINX Gateway Fabric pods.
  create: true
  ## The type of service to create for the NGINX Gateway Fabric.
  type: NodePort
  ## The externalTrafficPolicy of the service. The value Local preserves the client source IP.
  # externalTrafficPolicy: Local
  ## The annotations of the NGINX Gateway Fabric service.
  annotations: {}

  ## A list of ports to expose through the NGINX Gateway Fabric service. Update it to match the listener ports from
  ## your Gateway resource. Follows the conventional Kubernetes yaml syntax for service ports.
  ports:
    - name: http-tcp
      protocol: TCP
      port: 80
      targetPort: 80
      nodePort: 30080
    - name: http-udp
      protocol: UDP
      port: 80
      targetPort: 80
      nodePort: 30080
    - name: https-tcp
      protocol: TCP
      port: 443
      targetPort: 443
      nodePort: 30443
    - name: https-udp
      protocol: UDP
      port: 443
      targetPort: 443
      nodePort: 30443

[pleshakov]

Hi @neotheprogramist

Thanks for providing the detailed instructions.

We have an issue about supporting multiple Gateways -- #1443 In that case, NGF will provision separate data planes, one per each Gateway. Where data plane is 1 or more NGINX replicas. Could it cover your case?

At the same time, another approach is to merge multiple Gateways and use one data plane for the merged Gateways. However, such merging is not yet fully defined in the Gateway API -- there is an PR with the GEP for it kubernetes-sigs/gateway-api#1863 See also https://gateway-api.sigs.k8s.io/geps/overview/

Additionally

- name: http-udp
- name: https-udp

I wonder if you're planning to use HTTP3 and that's why you have UPD ports exposed

[neotheprogramist]

@pleshakov
Thank you for the answer.

Happy to hear that this is in progress, I'm building basically a cluster for my company for staging environment and for now I'm using just one Gateway with multiple listeners, but Gateway as a namespaced resource I would like to have one in every staging namespace not to build one big Gateway for all.
Here is my cluster architecture provisioning script, and in production I'm using modified version of kubespray.

The UDP was only for testing, but I'm willing to use http3 in the future but is seems it still have not very good support - so that's not a priority, http2 is more than enough for my needs.

If you need any help, testing, or feedback from production I'm happy to provide anything you need and speed up issue fixing ;)

[mpstefan]

We just covered this story in our community meeting. This is technically a duplicate of #1443 which is currently tentatively slotted for 1.3. After our control and data plane split (#1508) we'll be able to pursue this.

Can't guarantee we can deliver it in 1.3 since we'll need to refine both (and we want to get releases out ASAP), but the conversation above will be linked to #1443.