Using mysql JSON_SEARCH: Why do I have to quote numbers?

I have a "test" table with a PK field "id" and a JSON field "json_data". I make the following insert:

insert into test (json_data) values (JSON_ARRAY(1,2));

However, if I do

select * from test where JSON_SEARCH(json_data, 'all', 2) is not null;

I get "Empty set". Although if I insert

insert into test (json_data) values (JSON_ARRAY("1","2"));

and then repeat the query, I indeed finally get

+----+------------+
| id | json_data  |
+----+------------+
|  2 | ["1", "2"] |
+----+------------+

Why do I have to quote the numbers in order to be able to query on them? I'd rather not do that, is there a way around?