Comment by Manfred

Have you considered using an index directly on language tokens (eg. the abstract language tree representing the file) instead of ngrams on the source text?