Human oversight in AI systems is essential as it adds a layer of protection against prompt-injection attacks, ensuring accountability and ethical use.