error: unpaired UTF-8 bidirectional control characters detected

[20urc3]

It seems that we have the same issue with encoding UTF8 characters, in f1_c_fuzz.c. ( When trying to compile a js grammar )

f1_c_fuzz.c: In function ‘gen_node_NAUGHTYSTRING’:
f1_c_fuzz.c:13366:53: error: unpaired UTF-8 bidirectional control characters detected [-Werror=bidi-chars=]
13366 |     subnode = node_create_with_val(NODE_TERM__, "<U+202B>test<U+202B>", 6);
      |                                                  ~~~~~~~~    ~~~~~~~~
      |                                                  |           |       |
      |                                                  |           |       end of bidirectional context
      |                                                  |           U+202B (RIGHT-TO-LEFT EMBEDDING)
      |                                                  U+202B (RIGHT-TO-LEFT EMBEDDING)
f1_c_fuzz.c:13376:53: error: unpaired UTF-8 bidirectional control characters detected [-Werror=bidi-chars=]
13376 |     subnode = node_create_with_val(NODE_TERM__, "<U+2066>test<U+2067>", 6);
      |                                                  ~~~~~~~~    ~~~~~~~~
      |                                                  |           |       |
      |                                                  |           |       end of bidirectional context
      |                                                  |           U+2067 (RIGHT-TO-LEFT ISOLATE)
      |                                                  U+2066 (LEFT-TO-RIGHT ISOLATE)
f1_c_fuzz.c:13416:53: error: unpaired UTF-8 bidirectional control characters detected [-Werror=bidi-chars=]
13416 |     subnode = node_create_with_val(NODE_TERM__, "<U+202A><U+202A>test<U+202A>", 7);
      |                                                  ~~~~~~~~~~~~~~~~    ~~~~~~~~
      |                                                  |       |           |       |
      |                                                  |       |           |       end of bidirectional context
      |                                                  |       |           U+202A (LEFT-TO-RIGHT EMBEDDING)
      |                                                  |       U+202A (LEFT-TO-RIGHT EMBEDDING)
      |                                                  U+202A (LEFT-TO-RIGHT EMBEDDING)
f1_c_fuzz.c:13556:57: error: unpaired UTF-8 bidirectional control character detected [-Werror=bidi-chars=]
13556 |     subnode = node_create_with_val(NODE_TERM__, "test<U+2060>test<U+202B>", 10);
      |                                                                  ~~~~~~~~
      |                                                                  |       |
      |                                                                  |       end of bidirectional context
      |                                                                  U+202B (RIGHT-TO-LEFT EMBEDDING)
f1_c_fuzz.c:14116:59: error: unpaired UTF-8 bidirectional control characters detected [-Werror=bidi-chars=]
14116 |   <U+E006E><U+E006F><U+E0070><U+E0071><U+E0072><U+E0073><U+E0074><U+E0075><U+E0076><U+E0077><U+E0078><U+E0079><U+E007A><U+E007B><U+E007C><U+E007D><U+E007E><U+E007F>", 150);
      |
      |                                                                                                                                                                        |
      |                                                                                                                                                                        end of bidirectional context

cc1: all warnings being treated as errors

[h1994st]

Hi @20urc3 , would you mind attaching the js grammar file so that I can have a local test?

[20urc3]

js.json

[20urc3]

This is generated with the python script nautilus to json. Note that it produces the same error with the default javascript.json

[20urc3]

To resolve these issues, you have a few options:

Pair the bidirectional control characters properly:
For example, change:
"<U+202B>test<U+202B>" to "<U+202B>test<U+202C>" (U+202C is the PDF - POP DIRECTIONAL FORMATTING)
"<U+2066>test<U+2067>" to "<U+2066>test<U+2069>" (U+2069 is the PDI - POP DIRECTIONAL ISOLATE)
If these unpaired characters are intentional for testing purposes, you could disable the specific warning:
Add the compiler flag: -Wno-error=bidi-chars or -Wno-bidi-chars

I just added the compiler flag and Im now able to compile the libgrammarmutator-js.so

[20urc3]

Fixed with #52 #52