feature: Warn when namespace is not specified for a plugin

[haleemur]

Feature scope

CLI (options, error messages, logging, etc.)

Description

In the current meltano version, the plugin's property namespace is required. However, when a plugin install fails due to namespace not being specified, an error like the following is shown to the user:

<PluginType> '<PluginName>' is not known to Meltano. Try running `meltano lock --update --all` to ensure your plugins are up to date.

Users trying to update existing projects or trying to add new plugins would likely spend hours trying to figure this out if the error message is not improved.

We faced this error when investigating whether our workloads would continue to work as expected with a new meltano version.

[edgarrmondragon]

Hey @haleemur, thanks for raising!

I think the problem is Meltano uses namespace to identify custom plugins:

meltano/src/meltano/core/plugin/project_plugin.py

Lines 109 to 112 in b54c1c2

	if not self.inherit_from and namespace:
	# When not explicitly inheriting, a namespace indicates an
	# embedded custom plugin definition
	self.custom_definition = PluginDefinition(

But otherwise the property is not really required.

That makes me think we may need to:

1. Update the troubleshooting instruction in

   meltano/src/meltano/core/project_plugins_service.py

   Lines 81 to 84 in b54c1c2

   	instruction = (
   	"Try running `meltano lock --update --all` to ensure your plugins are "
   	"up to date"
   	)

   to include something like or add the 'namespace' property to your plugin definition.
2. Warn when namespace is missing, regardless of the type of plugin
3. Have a different, clearer, way of identifying custom plugins

Number 1 and 2 are probably the easiest.

[edgarrmondragon]

Related:

• Make namespace optional in plugin definitions, auto-generate if missing #3296, especially Make namespace optional in plugin definitions, auto-generate if missing #3296 (comment)
• docs: Improve documentation around what a namespace is and does #3116

[haleemur]

Thanks @edgarrmondragon,

I have a slight preference towards option 1 as its the simplest code change, and will suffice. If the documentation is improved according to #3116, maybe we should also update the warning message to point to it.

Option 2 feels a bit incorrect in the sense that the meltano install command succeeds but the user sees a warning. I think it would just create unnecessary doubt for the end-user. So far, we're assuming that the error message is invoked at the right time, but its text does not direct the user towards all the potential root causes.

Option 3 sounds great, I just don't know yet what it would be.

[haleemur]

On further thought, maybe meltano should test whether namespace is present under the scenarios where it needs to be present (is a custom plugin), and emit specific instructions to the user to help resolve installation issues easily.