I don't know anything specific about your ancestors, but I know there are a lot of Potts family members buried in the Rural Cemetery in Dayton Township in Waupaca County. There are a few Brown family members also. Perhaps that could be a starting point to find links to your family members. Good luck!