This is interesting. I agree with you that toxic friends aren't necessarily evil, but does not being someone's top priority necessarily make them toxic? It might mean they can't always be there for you the way you might like, but it doesn't necessarily mean that they don't care or that they're toxic.
I thought that way more when I was younger. Now, as an adult in my mid-thirties, I have increased life responsibilities and commitments that make it hard to always prioritize my friends as much as I would like to. I hope I'm not thought of as a toxic friend for that reason.