That does sound unpleasant. Sometimes I try to remember that bossy people can be anxious themselves and it makes them want to throw their weight around trying to control things. For example, maybe she has a germ phobia and fears cleaning bathrooms and it makes her act irrationally--just an example. So I'd start with trying to get my brain to stop seeing the other person as 'enemy #1,' because it can make me a better listener.
Then it may be true that when you start acting in a more assertive manner, she is going to become unpleasant, and you have to work with yourself to accept that. Conflict is inevitable in this situation, if you want change. If you push back, she is likely to get bossy and pushy back. If you can psyche yourself up to just get ready to deal with that, you can handle it. Use only assertive statements, no blaming. "I need for us to split up the work more equitably," NOT "you never clean the bathrooms." If things aren't working out, you could say to her, "we need to see the supervisor about the workload distribution," implying togetherness; that way it won't be behind her back and you won't feel like a tattletale.
It will take some firmness from you though, and I guess that's the part you have to get yourself ready for. But I find that sometimes when we push ourselves in this way really interesting and cool things happen that wouldn't have happened if we just continued on without trying to make change.
Also, if there is an EAP in your facility, you might ask them for some help, just in terms of helping you determine how to go about this.
It sounds very hard, and I wish you luck with it!