Well, when you frame it in moral terms things get very tricky, the last thing I want to do is to tell somebody else what is moral.
Having said that, I feel there is a clear difference between being NV and being V. The presence of nervous system makes all the difference to me. A living entity with even a rudimentary nervous system will feel pain. Given there are perfectly good alternatives to sustain life without causing pain, I feel it is a better option for me to avoid causing of pain for the pleasure of my taste buds.
Further, the advent of corporate farming has given rise to unspeakable cruelty towards animals. Chickens live their lives in cages that are no larger than their own body size, with their beaks cut off to prevent them, driven to madness due to such conditions, from harming themselves or other chickens too soon. Pigs and cows are turned into meat making automatons, wallowing in their own filth, drugged up with antibiotics to keep decease away. They are fed products that contain animal proteins from left-over waste from the slaughtering process -- essentially turning them into cannibals -- which is supposed to be a cause of what is known as "mad-cow decease".
From an environmental POV, meat production uses up 3 to 10 times the resources compared to milk products and vegetables.
It is not my place to tell meat-eaters what is moral and what is not. But, to say vegetarians kill plants and so no different from meat-eaters is too simplistic and misses a whole lot of complex issues.
Cheers!