It can use new information when performing new tasks. It can store experience based on mistakes it made. It can store knowledge in several ways (context, files) and use it when needed.
There is a chance it will forget something, but the same can be said for humans.
There is a chance it will forget something, but the same can be said for humans.