bug: meltano config set doesn't set correct setting path when an object is passed as value

[XshubhamX]

Meltano Version

3.1.0

Python Version

3.6 (deprecated)

Bug scope

Configuration (settings parsing, validation, etc.)

Operating System

Mac OS

Description

When an object is provided as a value in the set subcommand of config, it doesn't parse the setting path correctly, resulting in the wrong config value.
cc @edgarrmondragon

Code

Command - meltano config meltano set cli '{"log_level": "debug"}'

Expected - Meltano setting 'cli.log_level' was set in `meltano.yml`: "debug"

Actual - Meltano setting 'cli' was set in `meltano.yml`: '{"log_level": "debug"}'

One solution is to traverse the object recursively and exgtract the correct setting value. 

def traverse_nested_settings(
    dict_value,
    nested_settings: list[list[tuple | (set | int)]],
    current_nested_setting=(),
):
    """Create an array of tuples for nested settings."""
    for sub_setting, sub_value in dict_value.items():
        if isinstance(sub_value, dict):
            # If the value is a dictionary, continue traversing
            traverse_nested_settings(
                sub_value,
                nested_settings,
                current_nested_setting + (sub_setting,),
            )
        else:
            # When you reach a leaf node (non-dictionary value),
            # append the accumulated path to final_sub_settings
            nested_settings.append([current_nested_setting + (sub_setting,), sub_value])

[edgarrmondragon]

Thanks for raising @XshubhamX!

I think this could be fixed by adding a cli setting of object kind to the built-in settings in https://github.com/meltano/meltano/blob/49aa50af10afd6293f0e5a8bb7442d01dcd8fcc3/src/meltano/core/bundle/settings.yml:

# CLI
- name: cli
  kind: object

[XshubhamX]

@edgarrmondragon then this would only work for cli related settings for meltano but not for other plugin-related settings. Shouldn't we adapt a generic solution?

[edgarrmondragon]

@edgarrmondragon then this would only work for cli related settings for meltano but not for other plugin-related settings. Shouldn't we adapt a generic solution?

@XshubhamX oh definitely, I'm just thinking about the complexity of a generic solution vs a quick win 😅

[XshubhamX]

@edgarrmondragon then this would only work for cli related settings for meltano but not for other plugin-related settings. Shouldn't we adapt a generic solution?

@XshubhamX oh definitely, I'm just thinking about the complexity of a generic solution vs a quick win 😅

@edgarrmondragon I'd say lets implement a generic one, cause we are almost done with setting config from a file and there also we are accepting a json as a value here.

Wdyt?