Honestly, I'd just make 2 repos if possible. One open source; the other that includes the first one as a library, that has a commercial licence.
Then instead of the "open core" debate (which whenever it can be solved by 2 repos seems specious), you have an open source piece of software and you get points for letting people see your enterprise licenced source code as well.
Then instead of the "open core" debate (which whenever it can be solved by 2 repos seems specious), you have an open source piece of software and you get points for letting people see your enterprise licenced source code as well.