It comes down to the problem of relations (Wikipedia - Relation (Philosophy))
It depends on one’s belief in either Eliminativism, where relations exist in the mind but not the physical world, or Relationism, where relations exist in the physical world.
In Eliminativism, numbers, which are founded on relations, are both real and exist in the mind, but are neither real nor exist in the physical world. Numbers can thereby be explained in a non-reductive physicalist framework.
In Relationism, numbers, which are founded on relations, are both real and exist in the mind, and are both real and exist in the physical world. Numbers cannot thereby be explained in a non-reductive physicalist framework.
In non-reductive physicalism, mental properties, such as relations, cannot be reduced to physical properties.
As there are a number of practical reasons why relations cannot exist in the physical world, Eliminativism must apply, meaning that numbers, which are founded on relations, can be explained in a non-reductive physicalist framework.