Inconsistent result when using $*

[TobyBrull]

I would have expected the following two commands to give the same results:

$ mlr -c --from <( echo $'a,b\n5,6' ) put '$array = $*' then put '${array.a} *= 1000'
a,b,array.a,array.b
5,6,1000,6

$ mlr -c --from <( echo $'a,b\n5,6' ) put '$array = $*' | mlr -c put '${array.a} *= 1000'
a,b,array.a,array.b
5,6,5000,6

[johnkerl]

Hi @TobyBrull and sorry for the long delay!!

The put and filter DSL recognizes the . and sees that array.a is the a slot of array. But all the other verbs do not. :( So there is no field named "array.a".

This is addressed in #1062 (which needs doing).

This is super-weird -- I know. Having Miller accept input and output, in multiple formats, some of which allow nesting (JSON, JSON lines) and some of which don't (CSV, TSV), and trying to do the right thing by default for all combinations, is an exercise in compromise at best.

What's going on here:

• In your first example:
  • There's a field called array which is map-valued with keys a and b.
  • This is all constructed in memory so array is still map-valued when it gets to put
  • put sees a map-valued field named array, nothing named array.a
  • Absent times 1 is 1: https://miller.readthedocs.io/en/latest/reference-main-null-data/#arithmetic-rules
• In your second example:
  • The first mlr, before the pipe, flattens down to CSV as output, so the map-valued field arrays with keys a and b becomes two fields array.a and array.b (use --no-auto-flatten to suppress this)
  • The second mlr, after the pipe, has CSV input and output and no map-valued field names constructed mid-flight so the field names are array.a and array.b

See also
https://miller.readthedocs.io/en/latest/new-in-miller-6/#improved-json-json-lines-support-and-arrays

As a workaround, you can do the following -- this flattens the map-valued field array with keys a and b to two fields array.a and array.b:

$ mlr -c --from <( echo $'a,b\n5,6' ) put '$array = $*' then flatten then put '${array.a} *= 1000'
a,b,array.a,array.b
5,6,5000,6

[TobyBrull]

Makes sense! Thanks for the detailed response!