Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

oci_network_load_balancer_backend 404-NotAuthorizedOrNotFound but the backeds are correctly created #1593

Open
garutilorenzo opened this issue May 19, 2022 · 4 comments · May be fixed by #1595
Open

oci_network_load_balancer_backend 404-NotAuthorizedOrNotFound but the backeds are correctly created #1593

garutilorenzo opened this issue May 19, 2022 · 4 comments · May be fixed by #1595
Labels
bug In-Progress Terraform Team is working on the reproduce & fix

Comments

@garutilorenzo
Copy link

garutilorenzo commented May 19, 2022 •
edited

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version and Provider Version

Terraform v1.1.6
on linux_amd64

  • provider registry.terraform.io/hashicorp/oci v4.64.0
  • provider registry.terraform.io/hashicorp/oci v4.76.0
  • provider oracle/oci 4.76.0
  • provider registry.terraform.io/hashicorp/template v2.2.0

i've tried also oracle/oci module v.4.64.0 and the latest oracle oci module

Affected Resource(s)

oci_network_load_balancer_backend

Terraform Configuration Files

lb.tf

resource "oci_network_load_balancer_network_load_balancer" "k3s_load_balancer" {
  compartment_id = var.compartment_ocid
  display_name   = var.k3s_load_balancer_name
  subnet_id      = oci_core_subnet.oci_core_subnet11.id

  is_private                     = true
  is_preserve_source_destination = false

  freeform_tags = {
    "provisioner"           = "terraform"
    "environment"           = "${var.environment}"
    "${var.unique_tag_key}" = "${var.unique_tag_value}"
  }
}

resource "oci_network_load_balancer_listener" "k3s_kube_api_listener" {
  default_backend_set_name = oci_network_load_balancer_backend_set.k3s_kube_api_backend_set.name
  name                     = "k3s kube api listener"
  network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
  port                     = var.kube_api_port
  protocol                 = "TCP"
}

resource "oci_network_load_balancer_backend_set" "k3s_kube_api_backend_set" {
  health_checker {
    protocol = "TCP"
    port     = var.kube_api_port
  }

  name                     = "k3s kube api backend"
  network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
  policy                   = "FIVE_TUPLE"
  is_preserve_source       = true
}

resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
  depends_on = [
    oci_core_instance_pool.k3s_servers,
  ]

  count                    = 2
  backend_set_name         = oci_network_load_balancer_backend_set.k3s_kube_api_backend_set.name
  network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
  port                     = var.kube_api_port

  target_id = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].id
}

data.tf

data "oci_core_instance_pool_instances" "k3s_servers_instances" {
  depends_on = [
    oci_core_instance_pool.k3s_servers,
  ]
  compartment_id   = var.compartment_ocid
  instance_pool_id = oci_core_instance_pool.k3s_servers.id
}

instance_pool.tf

resource "oci_core_instance_pool" "k3s_servers" {
  depends_on = [
    oci_identity_dynamic_group.compute_dynamic_group,
    oci_identity_policy.compute_dynamic_group_policy
  ]

  lifecycle {
    create_before_destroy = true
    ignore_changes        = [load_balancers, freeform_tags]
  }

  display_name              = "k3s-servers"
  compartment_id            = var.compartment_ocid
  instance_configuration_id = oci_core_instance_configuration.k3s_server_template.id

  placement_configurations {
    availability_domain = var.availability_domain
    primary_subnet_id   = oci_core_subnet.default_oci_core_subnet10.id
    fault_domains       = var.fault_domains
  }

  size = 2

  freeform_tags = {
    "provisioner"           = "terraform"
    "environment"           = "${var.environment}"
    "${var.unique_tag_key}" = "${var.unique_tag_value}"
    "k3s-cluster-name"      = "${var.cluster_name}"
    "k3s-instance-type"     = "k3s-server"
  }
}

Panic Output

│ Error: 404-NotAuthorizedOrNotFound, Unknown resource Entity of type Backend with key ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycg2zjcqxbsbe26rwvc6szowce7u7qpi2m5gaqxm53bwza.6443 not found 
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend 
│ Request Target: GET https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends/ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycg2zjcqxbsbe26rwvc6szowce7u7qpi2m5gaqxm53bwza.6443 
│ Provider version: 4.76.0, released on 2022-05-21.  
│ Service: Network Load Balancer Backend 
│ Operation Name: GetBackend 
│ OPC request ID: b55d15500a84243335655841e09c3ab3/2B8A5720ED91321A19ADF40B894FAD48/AC30D14BAA69E1C642354767A3470EA3 
│ 
│ 
│   with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0],
│   on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│   36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│ 
╵
╷
│ Error: 404-NotAuthorizedOrNotFound, Unknown resource Entity of type Backend with key ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycfdk2fs3stowtrhca2dmtz2auu6rkx5qlwfrqxurxlhaq.6443 not found 
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend 
│ Request Target: GET https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends/ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycfdk2fs3stowtrhca2dmtz2auu6rkx5qlwfrqxurxlhaq.6443 
│ Provider version: 4.76.0, released on 2022-05-21.  
│ Service: Network Load Balancer Backend 
│ Operation Name: GetBackend 
│ OPC request ID: 521d57cf4abd2ac90d40ac9d7c1a6ded/8B5292856DD61813672E31F59476D54D/74ABF142B8425A581854E9E355706EC8 
│ 
│ 
│   with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1],
│   on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│   36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│

If i try to apply again:

module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [2m41s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [2m51s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m1s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m11s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m21s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m31s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m41s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [3m51s elapsed]
module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0]: Still creating... [4m1s elapsed]
╷
│ Error: 409-NotAuthorizedOrResourceAlreadyExists, Conflict.  For nlb-id Duplicate backend IP/id + port combinations not allowed: Backend(name=null, ipVersion=Ipv4, ipAddress=null, targetId=ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycfdk2fs3stowtrhca2dmtz2auu6rkx5qlwfrqxurxlhaq, port=6443, weight=1, isDrain=false, isBackup=false, isOffline=false) 
│ Suggestion: The resource is in a conflicted state. Please retry again or contact support for help with service: Network Load Balancer Backend
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend 
│ Request Target: POST https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends 
│ Provider version: 4.76.0, released on 2022-05-21.  
│ Service: Network Load Balancer Backend 
│ Operation Name: CreateBackend 
│ OPC request ID: de113a2f981f3412fc646ea3667b1edf/42712CB6BCC726E45949054091989B81/70843B5EC104486CD24F74190B741DC2 
│ 
│ 
│   with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1],
│   on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│   36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│ 
╵
╷
│ Error: 409-NotAuthorizedOrResourceAlreadyExists, Conflict.  For nlb-id Duplicate backend IP/id + port combinations not allowed: Backend(name=null, ipVersion=Ipv4, ipAddress=null, targetId=ocid1.instance.oc1.eu-zurich-1.an5heljr5kjm7pycg2zjcqxbsbe26rwvc6szowce7u7qpi2m5gaqxm53bwza, port=6443, weight=1, isDrain=false, isBackup=false, isOffline=false) 
│ Suggestion: The resource is in a conflicted state. Please retry again or contact support for help with service: Network Load Balancer Backend
│ Documentation: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/network_load_balancer_backend 
│ Request Target: POST https://network-load-balancer-api.eu-zurich-1.oci.oraclecloud.com/20200501/networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.amaaaaaa5kjm7pyao4axnaunpvssqzrljdopyyuouepvl5dlvsd5d4y7q3ya/backendSets/k3s%20kube%20api%20backend/backends 
│ Provider version: 4.76.0, released on 2022-05-21.  
│ Service: Network Load Balancer Backend 
│ Operation Name: CreateBackend 
│ OPC request ID: 50e0fe689671c85f94d5001fa64469b8/62921F9F81DA9FCC625A918BF0F6872C/8F2367F2069564F8E1629E7A7A0DB7C4 
│ 
│ 
│   with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0],
│   on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│   36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│

Actual Behavior

Backend are correctly created but 404-NotAuthorizedOrNotFound error occured

lb-bug-web2

lb-bug-web

Steps to Reproduce

Apply this module
@garutilorenzo garutilorenzo mentioned this issue May 19, 2022
Closed
@garutilorenzo
Copy link
Author

The same probelm occures if i try to use the private ip ocid as target_id:

│ Error: 404-NotAuthorizedOrNotFound 
│ Provider version: 4.64.0, released on 2022-02-16. This provider is 13 Update(s) behind to current. 
│ Service: Network Load Balancer Backend 
│ Error Message: Unknown resource Entity of type Backend with key ocid1.privateip.oc1.eu-zurich-1.ab5heljrku3dvhwandbamb34s7cthenz4vki52iycqgoj5obfkoexbserhzq.6443 not found 
│ OPC request ID: 73aca760d408945945bcb3ed681d8b8b/EF6781C100C57DB2EF6AC79D40FF19F1/74E7BA9E972DCF1A125522BF258F90E0 
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ 
│ 
│   with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[0],
│   on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│   36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
│ 
╵
╷
│ Error: 404-NotAuthorizedOrNotFound 
│ Provider version: 4.64.0, released on 2022-02-16. This provider is 13 Update(s) behind to current. 
│ Service: Network Load Balancer Backend 
│ Error Message: Unknown resource Entity of type Backend with key ocid1.privateip.oc1.eu-zurich-1.ab5heljryssggfe4rldf26asmvcpjrj2aoktk7fp6yzwdxpngqszpqgugfja.6443 not found 
│ OPC request ID: ba08ecc04d664aa6a44161e9fa453ec3/06469C6E7FA4A2840278859A5C74EB5E/79880A506DB3F22E3DA470E79C826946 
│ Suggestion: Either the resource has been deleted or service Network Load Balancer Backend need policy to access this resource. Policy reference: https://docs.oracle.com/en-us/iaas/Content/Identity/Reference/policyreference.htm
│ 
│ 
│   with module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1],
│   on ../k3slb.tf line 36, in resource "oci_network_load_balancer_backend" "k3s_kube_api_backend":
│   36: resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {

@garutilorenzo
Copy link
Author

garutilorenzo commented May 23, 2022 •
edited

I've found a workaround.
With some reverse enginereeng i've inspected the html of the web console and i've found that the name was setted to the instance name (the name if is not provided is automatically generated).
Inspecting the page i've found that in the id of the html the name instead was setted to $instance_id:$backend_port

oci-reverse-eng

Then i've tried to import the resurce with:

terraform import module.k3s_cluster.oci_network_load_balancer_backend.k3s_kube_api_backend[1] "networkLoadBalancers/ocid1.networkloadbalancer.oc1.eu-zurich-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/backendSets/k3s_kube_api_backend/backends/ocid1.instance.oc1.eu-zurich-1.xxxxxxxxxxxxxxxxxxxxxx:6443"

and the import has correctly imported the backend

Changing the backend name to $instance_id:$backend_port fix the problem, but this is only a workaround.

Also, with this workaround the backand name in the web console is always setted to the instance name.
I think something has changed in the OCI api and at the moment is not documented.

@garutilorenzo
Copy link
Author

UPDATE

The value in the name argument can be any kind of string

name                     = format("%s_%s", "k3s_server", count.index)

or

name                     = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].display_name

so the name argument is a required argument, the value must be "any kind of string".
In the web interface the result is always the same, the instance name is displayed:

oci-backends

This is the working code:

resource "oci_network_load_balancer_backend" "k3s_kube_api_backend" {
  depends_on = [
    oci_core_instance_pool.k3s_servers,
  ]

  count                    = var.k3s_server_pool_size
  backend_set_name         = oci_network_load_balancer_backend_set.k3s_kube_api_backend_set.name
  network_load_balancer_id = oci_network_load_balancer_network_load_balancer.k3s_load_balancer.id
  name                     = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].display_name
  port                     = var.kube_api_port
  target_id                = data.oci_core_instance_pool_instances.k3s_servers_instances.instances[count.index].id
}

garutilorenzo added a commit to garutilorenzo/terraform-provider-oci that referenced this issue May 23, 2022
@garutilorenzo
Updated network_load_balancer_backend.html.markdown fixes oracle#1593
40179c1 
Moved name argument to the Required arguments, fixes oracle#1593
@garutilorenzo garutilorenzo linked a pull request May 23, 2022 that will close this issue
Open
@ravinitp ravinitp added the In-Progress Terraform Team is working on the reproduce & fix label Apr 11, 2023
@ravinitp
Copy link
Member

Thank you for reporting the issue. We have raised an internal ticket to track this. Our service engineers will get back to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug In-Progress Terraform Team is working on the reproduce & fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Updated network_load_balancer_backend.html.markdown fixes #1593 garutilorenzo/terraform-provider-oci
2 participants
@garutilorenzo @ravinitp