Richard,
It appears that you may not have access to Census Records. There's a rich history of your Layton Family there in Simpson Mississippi.
Ed and Emma's family in 1940
Maurine (13)
Bonnie Jean (10)
James (7)
Patty Ruth (5)

In 1930, James E and wife Emma are married and living in the household of James' mother.
Martha Layton (60) widowed and head of household
Mary Layton (39) daughter
James E. (28) son of Martha and married at the age of 23
Ernest R. (21) son of Martha
Daniel W (18) son of Martha
Emma (20) daug-in-law, married at the age of 14
Linnie M (3 & 3months) granddaughter of Martha
Bonnie Jean (new born) granddaughter of Martha

In 1920 your Great Grandfather James R. Layton is still alive living on Magee and Pinola St…
James R. (57) head of household
Martha (49) wife
Mary (29) daughter
Dollie (23) daughter
Edd (18) son and your grandfather
Pearl (12) daughter
Earnest (11) son
Dan (9) son

In each census there are other Laytons living next door or very near that I assume are part of this family. Good ancestor hunting!!
Lynda Katonak