MOD-6933 MOD-6750 MOD-6749: Lexer consumes some punctuation characters

[nafraf]

Description

Lexer consumes some punctuation characters. This generates wrong queries, and causes errors parsing numeric operators and fixing numeric range syntax:
#4516
#4505

A new parser (v3) was created to support DIALECT 5 with the new behavior

Which issues this PR fixes

1. MOD-6933

Main objects this PR modified

1. To create src/query_parser/v3/ we used src/query_parser/v2/ as base, adding the following changes:
   1.1 In lexer.rl punctuation characters are handle as PUNCTUATION token to be processed by the parser.
   From the ragel-guide, the punct characters are:

   punct – Punctuation. Graphical characters that are not alphanumerics. [!-/:-@[-‘{-~]

   But from the previous list, there are some characters which have an special meaning in the parser and are handled as an specific token.

   Character	Token
   !,#,$,&,',,,.,/,<, =,>,?,@,\,^,_,`	PUNCTUATION (new)
   "	QUOTE
   %	PERCENT
   (	LP
   )	RP
   *	STAR
   -	MINUS
   +	PLUS (new)
   :	COLON
   ;	SEMICOLON
   [	LSQB
   ]	RSQB
   {	LB
   |	OR
   }	RP
   ~	TILDE

   1.2 In lexer.rl the format of NUMBER was modified to support float numbers without leading zero.

   127.0.0.1:6379> FT.explaincli idx '@n:[-.001 +.002]' dialect 2
   1) NUMERIC {-1.000000 <= @n <= 2.000000}
   
   127.0.0.1:6379> FT.explaincli idx '@n:[-.001 +.002]' dialect 5
   1) NUMERIC {-0.001000 <= @n <= 0.002000}

   1.3. In parser.y we created a new rule for unary operators: ~, -, to prevent the use of an operator between the colon after the field name and the parenthesis which specify the query pattern.
   i.e. The following queries are invalid:

   127.0.0.1:6379> FT.SEARCH idx '@t2:-(abc)' dialect 5
   (error) Syntax error at offset 9 near abc
   127.0.0.1:6379> FT.SEARCH idx '@t1:-{abc}' dialect 5
   (error) Syntax error at offset 5 near t1
   127.0.0.1:6379> FT.SEARCH idx '@t2:-abc' dialect 5
   (error) Syntax error at offset 5 near abc

1.4 parser.y: To support numbers with multiple signs +, a new rule was added:
num(A) ::= PLUS num(B)

Mark if applicable

• This PR introduces API changes
• This PR introduces serialization changes

In addition

This PR also merges to the master branch the following PRs:
#4505 (MOD-6750)
#4516 (MOD-6749)

[codecov]

Codecov Report

Attention: Patch coverage is 83.00000% with 102 lines in your changes are missing coverage. Please review.

Project coverage is 85.96%. Comparing base (1466c3e) to head (4ea8fc8).

Files	Patch %	Lines
src/query_parser/v3/lexer.c	82.26%	102 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4604      +/-   ##
==========================================
+ Coverage   85.80%   85.96%   +0.16%     
==========================================
  Files         188      190       +2     
  Lines       33107    34517    +1410     
==========================================
+ Hits        28407    29674    +1267     
- Misses       4700     4843     +143     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[oshadmi]

👍🏼 👏🏼 Nice! Few comments

[github-actions]

Successfully created backport PR for 2.10:

• [2.10] MOD-6933 MOD-6750 MOD-6749: Lexer consumes some punctuation characters #4639