Washington-Baltimore is actually a very good connection by rail--you'll have a train at least once an hour--except that the station for Baltimore is in the wrong location. Baltimore Penn Station is located far to the north of the city, well outside the Inner Harbor that's going to be the center of tourism and visitors.
What happened is that, historically, Baltimore-Washington was mainly served by the B&O, whose Baltimore station would have been right next to Inner Harbor (Camden Yards is essentially built on old railroad property). When passenger service was consolidated into Amtrak, the Northeast Corridor instead chose to use Pennsylvania Railroad trackage instead of B&O from DC to NYC, and the freight trackage in the corridor consolidated onto the B&O tracks instead. The resulting residual commuter rail line has the standard commuter rail/freight rail politics going on that limits the number of trains that can transit the route.
What happened is that, historically, Baltimore-Washington was mainly served by the B&O, whose Baltimore station would have been right next to Inner Harbor (Camden Yards is essentially built on old railroad property). When passenger service was consolidated into Amtrak, the Northeast Corridor instead chose to use Pennsylvania Railroad trackage instead of B&O from DC to NYC, and the freight trackage in the corridor consolidated onto the B&O tracks instead. The resulting residual commuter rail line has the standard commuter rail/freight rail politics going on that limits the number of trains that can transit the route.