Fuzzing the Black Box

black box testing closed-source software with LLMs. Can they fuzz the space and get a passing approximation of the real thing?