The trivial way to derive the private key from a given public key is brute force, that is, you try every possible key to see if it fits the lock (public key), scanning from all zeros to all ones. For example, if the private key was 3 bits long, you would have to try 8 different possible keys: 000, 001, 010, 011, 100, 101, 110 and 111. This process can take a lot of time using current computers if the keys are long enough, something like millions of years. This happens because our computers can only work with bits that are 0 or 1, so they have to keep flipping bits and trying different combinations, just like trying to serially fit all possible keys at the given lock, one by one.

Now imagine a computer that can manipulate special bits called qubits that can be in 0 and 1 states at the same time. Using qubits, quantum computers can store multiple keys at the same time. For example, 3 qubits can store all 8 possible keys I showed before at the same time.

Finally, using qubits quantum computers can process key search in parallel and find the correct key without trying each key in the lock serially. It is like having 8 identical locks and trying all 8 different keys at the same time.

Bottom line, it would take much less time to try all different private key combinations to find the one that fits the given public key. You do not have a exponential problem anymore, that is a problem that scales rapidly if you increase the input size (key size). And if you can find the private key from a public key, all security is gone.

This is why quantum computers pose a risk to classical public key cryptography.