I've noticed the same issue on a different (but similarly dark-colored) car with different polish. (In this case was menzerna super intensive polish, or super finish 4000). Unfortunately, I started touching up paint chips with touch up paint before noticing something was in them. (D'oh!) At least I caught the problem before touching up *all* the chips in the hood.
I'm pretty sure I got the residue out of the divots by using either a toothbrush or detailing brush with some sort of appropriate cleaning solution. (I can't remember if that would have been car wash solution, or paint-safe diluted all-purpose cleaner).
I'm pretty sure I also first tried like you did, a quick IPA wipedown with microfiber towel, (and found that didn't work - apparently more agitation down into the divots was necessary.)