The most common "system" as they describe it is probably a database. Since it's not feasible to drop all your data and start over when you want to make a schema change, instead we have "alter table" statements to modify data in place. Often, the only practical changes you can make once you go live are those that can be expressed as database migrations.
Traditional, image-based Smalltalk worked this way. Often debuggers and repl-based languages will support some changes without restarting, but the intention is typically to support debugging and exploration, not to preserve data over the long term. Being able to restart and reload is so taken for granted that every web browser supports it and it's exposed to end users.
A compiler is free to optimize non-persistent data structures to the point where the objects you read about in the code might not actually exist. A persistent system requires that the runtime actually has the data described by the language stored in a way that you can inspect it and write new code that uses it.
Traditional, image-based Smalltalk worked this way. Often debuggers and repl-based languages will support some changes without restarting, but the intention is typically to support debugging and exploration, not to preserve data over the long term. Being able to restart and reload is so taken for granted that every web browser supports it and it's exposed to end users.
A compiler is free to optimize non-persistent data structures to the point where the objects you read about in the code might not actually exist. A persistent system requires that the runtime actually has the data described by the language stored in a way that you can inspect it and write new code that uses it.