Well, my family story goes like this: My great-grandfather was allegedly named Swearingin, and when he asked my great-grandmother to marry him, she said okay, but only if you get the word Gin out of your last name - so he changed it to Swearinger. She was a Women's Christian Temperance Union member etc. and had not much of a sense of humor. Go figure. Whether it is true or not I have no idea - the author of that gossip wasn't very credible generally. However, it almost sounds correct.
The other theory is that we came from Lake Schwering or something like that in Germany.