What if my kids aren't living with my ex even though she has primary custody?

My ex has primary custody, I get my 1st, 3rd, and 5th weekends. From what my kids tell me, age 6 and 9, their mother isn't living with them any more. Since our divorce 4 years ago, my ex, her sister, her sister's child, my children, and my ex's parents live in one house. But, like I said, the kids say their mother is never around anymore. I've long suspected that she didn't want the kids, but was only interested in child support and the earned income credit, and even then, my ex still uses food stamps for the kids food, and my kids wear hand me downs from their cousins, and my ex spends all the child support money on her self. Well, if she's not living in that house any more, but my children are still living there, is there anything that I can do to try to get custody of my children? I live in Texas by the way, not exactly a dad friendly state. I would think I have a lot more right to my children living with me than my ex's parents.