It's true that infant mortality has decreased faster than other mortality in the past century. Even people who survived past infancy had significantly higher mortality rates at the beginning of the 20th century, though.
See "Table 6 - Period Life Tables for the Social Security Area by Calendar Year and Sex" for detailed information from the United States:
In the table series for 1900, male life expectancy at birth was 46.4 years and female life expectancy was 49 years. For the survivors of infant mortality, additional life expectancy at age 6 was 53.2 years and 54.3 years respectively. Most children could not expect to reach their 60th birthday even if they had survived infancy.
At age 18, remaining life expectancy was 41.7 and 42.9 years, for total expected age at death of 59.7 and 60.9 years respectively; weighted by sex distribution at age 18, these young adults had an average life expectancy of 60.3 years.
See "Table 6 - Period Life Tables for the Social Security Area by Calendar Year and Sex" for detailed information from the United States:
https://www.ssa.gov/oact/NOTES/pdf_studies/study120.pdf
In the table series for 1900, male life expectancy at birth was 46.4 years and female life expectancy was 49 years. For the survivors of infant mortality, additional life expectancy at age 6 was 53.2 years and 54.3 years respectively. Most children could not expect to reach their 60th birthday even if they had survived infancy.
At age 18, remaining life expectancy was 41.7 and 42.9 years, for total expected age at death of 59.7 and 60.9 years respectively; weighted by sex distribution at age 18, these young adults had an average life expectancy of 60.3 years.