Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notwithstanding --skip-comments, double-quoted tokens in comment are mishandled with any --c2* option #1562

Closed
sphakka opened this issue May 12, 2024 · 4 comments

Comments

@sphakka
Copy link

sphakka commented May 12, 2024

I'm possibly missing something, but can't get mlr (v6.12.0-dev) to work with comments containing double quotes:

$ echo -e '# Foo "bar" quux'  | mlr --skip-comments  --c2m cat
mlr: parse error on line 1, column 7: bare " in non-quoted-field.

Shouldn't it ignore the comment line altogether?

@sphakka sphakka changed the title Notwithstanding --skip-comments, double-quoted tokens in comment are mishandled with any 'cc' Notwithstanding --skip-comments, double-quoted tokens in comment are mishandled with any --c2* option May 12, 2024
@aborruso
Copy link
Contributor

Hi @sphakka I have your same results.

A way to have an output is to change the input format in csvlite, but it may not be suitable for your CSV input.

echo -e '# Foo "bar" quux\na,b\n1,2' | mlr --skip-comments --icsvlite --omarkdown cat
a b
1 2

@sphakka
Copy link
Author

sphakka commented May 12, 2024

Thanks for the workaround, @aborruso!

@johnkerl
Copy link
Owner

johnkerl commented May 13, 2024

Shouldn't it ignore the comment line altogether?

@sphakka this is related to #1346

In Miller <= 5 (written in C) I had my own CSV parser and I could make it do the right thing.

In Miller 6 (re-written in Go) I use the Go library's CSV parser. Unfortunately, it parses CSV liines as CSV and then lets me decide whether or not they look like comments. This isn't great.

I either need to port my own CSV parser from C to Go and stop using the Go library's CSV parser, or, maybe find some way to intervene in what it's doing ....

@johnkerl johnkerl changed the title Notwithstanding --skip-comments, double-quoted tokens in comment are mishandled with any --c2* option Notwithstanding --skip-comments, double-quoted tokens in comment are mishandled with any --c2* option Jun 8, 2024
@johnkerl
Copy link
Owner

johnkerl commented Jun 8, 2024

Closing this as a duplicate of #1346 (which needs doing) ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants