I do not like my family. Is it okay to just go to Vegas to get married?

I am American and my fiance is Sri Lankan. I do not have a nice family and do not want to torture my fiancee with their rudeness. I would really just like to go to Las Vegas to get married so it is all legal in the US and then maybe have a religious ceremony with his family but without mine in Sri Lanka. I am worried though that his family might think I am strange. I only have 4 close girlfriends who are like my family. I would only invite 4 or 5 people. Would his family think I am pathetic do you think?