Apocalypse Keys is a great fit thematically, but I think adapting it to the trad mystery play of Masks of Nyarlathotep would be very hard. Mystery solving in AK is emergent, where non-specific clues are discovered which the players then piece together to create a possible solution, then dice are rolled to see whether they are right. Mysteries don’t have a predetermined answer, they are created emergently by players. Masks of Nyarlathotep, on the other hand, is built on a sequence of mysteries that have fixed answers and resolutions, and those answers build on each other. So you’d either need to figure out a way to rework the mysteries of MoN such that they are more open, and less interconnected in terms of solutions, while still planting the seeds so that the players are moving through the module. I think it would be very difficult.
As a general rule, PbtA struggles with mystery solving. “Play to find out” is a tough to principle to be true to when progress requires you to solve mysteries, and so there aren’t very many PbtA games based around solving mysteries. Apocalypse Keys, based on Brindlewood Bay, solves this problem by removing the puzzle, mechanically you will eventually solve the mystery because you decide on the solution. Monster of the Week, though its monster hunts are called “mysteries,” kind of gives you a bait and switch. The “investigate a mystery” move gives you clues, but it’s structured such that if you don’t have enough to get you everything you need, eventually, you’ll know where to go. Its monster hunting structure is a bad fit for the kind of sandbox mystery solving that MoN requires. (You bring up Masks, but that’s a TERRIBLE fit for MoN. It’s a game all about teenaged superheros trying to figure out who they are in the world. It has no investigation mechanics and is just about something completely different than MoN). I love, love, love PbtA but I don’t think there’s a published PbtA game that will do this well, and I’d really struggle to hack even the closest for this. The thing is, as sandboxy as MoN is (and it is VERY sandboxy), it’s still a series of mysteries some of which need to be solved to progress, and if NONE of the others are solved it will feel bad for the players. PbtA is a tough fit for this kind of play.
So what would I use? If you’re comfortable with OSR systems, I think Emmy Allen’s Esoteric Enterprises could do this OK. There’s a Spook class that creates monsters and super powered PCs that would work for BPRD and just build from there. You could also use Dark Streets and Darker Secrets (and I think that would be my choice). Character concepts are pretty free form. You have archetypes that give different sorts of powers to the PCs, but they are all very customizable and could be used to create BPRD type characters. Neither has any real mystery mechanics, but it won’t stand in the way, and so long as you’re free with information with the players, they should be able to put the clues together and solve the mysteries. It will be a bit of a meatgrinder, but you can juggle that. If you’re comfortable with going REALLY rules light, I bet Agents of the ODD would do BPRD like gangbusters. I think this is probably what I would do? Now I’m kind of thinking of trying Dark Streets and Darker Secrets.
The more natural fit is Gumshoe, probably Trail of Cthulhu or Fear Itself it you want to lean harder into horror and away from mystery, and you could do that, having them flavor their PCs as BPRD style monsters. I don’t think the BPRD elements would feel as weighty as in the OSR choices since they have no mechanical support, but mystery stuff will work better, less risk of getting stuck.
Anyway, if you do decide to use a PbtA systern, I’d love to hear how it goes. I think it will be an uphill battle, but maybe you’ll figure some stuff out and it will go great.