consider requeue services reconciled with error

[lwabish]

Is your feature request related to a problem? Please describe.

In my use case, each node has a dedicated nic for openelb. When those nics are down and some loadbalancer service is created, according to the code logic now, error would occur.

openelb/pkg/speaker/layer2/arp.go

Lines 152 to 155 in 0672f10

	hwAddr, err := a.resolveIP(nodeIP)
	if err != nil {
	return fmt.Errorf("failed to resolve ip %s, err=%v", nodeIP, err)
	}

But when we enable the dedicate nic for openelb, which means the lb services created before should work now. But current Reconcile funtion returns ctrl.Result{}, err, which means the service will not trigger the reconcile function anymore even if edited by user, unless delete openelb-manager pod manually, or delete the service.

Describe the solution you'd like

controller runtime ctrl.Result{} has offered us an option to requeue object with reconcile errors, is it reasonable to use something like ctrl.Result{Requeue: true} ?

Describe alternatives you've considered

Additional context