What will you do if someone labels you with something unrelated to you? And if the label is not from a single person but from the entire world itself? The above situation is face by the so called "non-vegetarians"
The prefix "non" may be Latin or French origin meaning "no" or "not". So living thing is anything that has life and non-living thing is something void of life. Again it may seem a very strict example. Let us see another. In schools we have teaching staff and non-teaching staff members. So those who can't or won't teach are labeled "non-teaching".
So given the usage of the prefix "non", it is wrong to label meat-eaters as "non-vegetarians". Almost all of the meat-eaters have no problem in consuming the so called vegetarian foods. That is just like a Doctor teaching in a medical university is both doctor and professor. Being doctor never makes him non-professor and being professor can never make him non-doctor. Likewise the label non-vegetarian is not correct as the meat-eaters never abstain from plant foods. Our diet is not void of veggies. So we are "vegetarians"(who also eat meat).
But the converse is true. The vegetarians abstain from meat. So their diet is void of meat. So it is correct to label them "non-meat-eaters" or "meat-abstainers". Those who choose to renounce or abstain from something should carry "non" prefix and not those who choose to accept any type of food.
In this world of Political Correctness will it be acceptable to call all educated people as "non-illiterate"? Then why there is discrimination against meat-eaters? It is because meat-eaters tend to be more liberal and tolerant towards food habits and people. So it is fair to ask them to carry "non" prefix than to ask the other group!