Except that sex is NOT binary, it’s bimodal. If sex was binary as you say, then intersex people, people born with a set of sexual chromosomes not matching neither XY nor XX and/or with sexual characteristic associated with a man nor a woman, would not exist. But intersex people do exist, their existance it’s an objective fact, recognised by science, and facts do not care about your feelings. Science is a research method based on evidence, not a buzzword you can throw around to pass you misinterpretation of simplified scientific concepts as facts.
Sex is bimodal (a spectrum between 2 defined points) and not binary (two mutually exclusive options), because nothing in nature can be binary. Nature is based on randomness and as such there is so conscious design behind how it behaves. The concept of “binary” on the other hand is entirely man-made and at odds with nature and can only really exists in computer science, since those are artificially made.
The concept of binary derived from man’s simplification of more complex phenomenons: a man might perceive an object as moving or inhert, but from a physics’ point of view said object is always moving at given velocity, which in some instances might be 0.
So to reiterate, no sex is not binary because human biology and scientific reality do not care about man-made fantasy concepts such as “binary”.