Veterans Service Records will not be online. They are accessible to the Veteran while living and next of kin as defined by the The NARA after a Veteran's death. They are the depository of all Government records put in storage.
Here is the link for information how to get his records. But be aware and read about the fire in 1973 that burned 80% of Army recods of the that time.
http://www.archives.gov/veterans/Another thing is all Veterans have been encouraged to file a copy of thier military discharge papers with the County Recorders or Clerks or Family Court of the County they lived in after their discharge and to do the same in each County they move to in the future. So find out where he lived from the time of his release till his death and contact the County to find out if he did file them.
One last place to check for at least a copy of his Honorable Discharge Certificate, if he had Military Honors at his funeral, is the Funeral Home that handled his arrangements and the Cemetery. Also his obit might have some information. The name of the funeral home should be on his death certificate. Some families put the unit on a grave marker, have you seen his?