Comment by lukah

From their implementation it looks like they’re calculating tanh and then applying a weight and bias