His father wasn't an affectionate man. The few times he was carried as a toddler, he would remember held by hands that were rough, and smelled of smoke and indistinguishable chemicals. Neither he nor his brother were ever fed by their father, or burped, or had their diapers changed like a normal parent would.
He didn't choose to have an alchemist for a father. Particularly one who held a baby the same way flasks of chemical reagents would.
Frankly, he didn't know what his mother saw in Hoenhime Elric in the first place. For Ed, his world gravitated around his mother and his baby brother. His father was just someone who occasionally stayed at their home and spent more time travelling than caring for his family.
He would see other fathers scoop up their children in their arms and playfully kiss their cheeks until the children would writhe and protest in embarrassment. On the other hand, his own would seem extremely cold and unemotional in comparison. His father always wore gloves whenever he was carried, excused as a safeguard against contamination of delicate alchemical experiments (since whenever Hoenhime was home, he was almost always in the laboratory anyway).
Ed recalls that the first and only time his father displayed some sort of parental affection. The day that his father left the house and never returned.
He can still feel in his cheeks the faint scratch of beard against skin when his father kissed him there. Ed hated the feeling almost immediately. Hoenhime's lips against his cheek were cold, very unlike the warmth of his mother's. It only lasted for a few seconds when his father withdrew, tousled his hair and declared that Ed was the "man of the house" and that he should take care of his mother whenever the father was absent.
Hoenhime hugged his wife and other son goodbye and walked out the door. He never looked back, neither did he try to reassure his wife that he would return, or that he would write letters to them. He didn't care for his family the same way normal fathers did.
Good riddance, Ed thought. He wouldn't have cared less if his father was killed in his journey.