The alternatives were for the most part replaced with plastic because of plastic's remarkable qualities. There's very few relatively inert materials that are cheap to manufacturer, durable, light, can withstand (reasonably) high temperatures, can be formed into whatever shape you want, and can be as hard or flexible as you want.
Glass bottles used to be much more common but they're more expensive, very easy to break, and heavier, which also increases cost of transportation. Paper containers for things can't be allowed to get wet which often makes them impractical. Metals are expensive, potentially reactive, heavy, and inflexible.
Yes plastics are very environmentally problematic, but they do solve a lot of problems that aren't really solved by any other kind of material.
Glass bottles used to be much more common but they're more expensive, very easy to break, and heavier, which also increases cost of transportation. Paper containers for things can't be allowed to get wet which often makes them impractical. Metals are expensive, potentially reactive, heavy, and inflexible.
Yes plastics are very environmentally problematic, but they do solve a lot of problems that aren't really solved by any other kind of material.