import_attribute(...) in utils.py swallows import errors in custom worker classes when running via the CLI.

[AndreCAndersen]

If you assign a worker_class via the CLI using dot notation for modules, e.g., myproject.myworker.MyWorker, then in the utils.py module there is a function import_attribute(...) that is called. One part of the logic is to automatically try to find the module by chopping off parts of the right hand side of the class path until it finds a valid module. This is done by checking for ImportError exceptions. The problem with this is that if your implementation of the custom worker class has an import error you get a really hard bug to understand just by running the code via the CLI.

You get this top level error:

...
  File "/usr/local/lib/python3.11/site-packages/rq/cli/helpers.py", line 356, in __init__
    self.worker_class = import_attribute(worker_class)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/rq/utils.py", line 120, in import_attribute
    raise ValueError('Invalid attribute name: %s' % attribute_name)
ValueError: Invalid attribute name: MyWorker

Fine, but then you get this thing as a deeper exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/rq/utils.py", line 118, in import_attribute
    attribute_owner = getattr(module, attribute_owner_name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'myproject' has no attribute 'myworker'

This is not at all what the error is. The actual error is that myworker has a faulty import inside of it, but this code swallows the error:

        try:
            module_name = '.'.join(module_name_bits)
            module = importlib.import_module(module_name)
            break
        except ImportError as ex:
            attribute_bits.insert(0, module_name_bits.pop())

rq/rq/utils.py

Lines 96 to 102 in fc4884a

	while len(module_name_bits):
	try:
	module_name = '.'.join(module_name_bits)
	module = importlib.import_module(module_name)
	break
	except ImportError:
	attribute_bits.insert(0, module_name_bits.pop())

It continues, then fails at an unrelated place further down.

rq/rq/utils.py

Lines 117 to 120 in fc4884a

	try:
	attribute_owner = getattr(module, attribute_owner_name)
	except: # noqa
	raise ValueError('Invalid attribute name: %s' % attribute_name)

[selwin]

Mind opening a PR to fix this?