Well, Dr. D has some great ideas on how to rule the Tri-State area, just horrible planning skills. However, I only believe he's a minor threat because of that, and of the nemesis he's obtained.
Also, if you heard about Time Machine 2, Quantom Boogaloo, he becomes the grand ruler basically. And that's all because Perry failed to defeat him (because future Candace busted Phineas and Ferb in the Rollercoaster episode). This let Dr. D go on with his plans without a Perry to stop him, and the agency fell behind. So, I can say that no, Dr. Heinz Doofenshmirtz is not a minor threat.