When trying to make SSL and TLS secure, the emphasis has been in ensuring that the implementation of the underlying algorithms is cryptographically correct –that is, to make sure that we are using the right salt, the right hashes, the right steps so that the stream of encrypted bytes cannot be deciphered by brute force, or by clever attacks that exploit cryptographic naivete on the side of the code writers. Heartbleed is a much more conventional vulnerability –a programming error that allows an attacker to retrieve a portion of the server’s private memory. A few decrypted bytes might be in this memory, and it might be of interest to an attacker, but what an attacker is really looking for is for the private keys of the server. SSL traffic is *designed* to be decipherable if the private keys are known (which come in two types, the session symmetric keys, which should be recycled every few minutes, and the server’s private SSL PKI certificate that is renewed every year or two).
The server’s private PKI certificate is of course the most valuable, and it might be (or at least a portion of it) might be loaded into the leaked memory when a connection is first being negotiated. The server’s symmetric session keys, could too be available in this portion of memory.
Thus replacing the buggy SSL version would prevent an attacker from retrieving memory sections of your server as soon as you update OpenSSL, but if the attacker previously got a hand on your server private certificate all new incoming conversations can be decrypted with no problem at all. This is why server certificates should be renewed ahead of time –and renewal of course does mean replacement. But of course, the server private keys need to be secured and their distribution controlled. It would be ironic, but sadly possible, that an organization might end up compromising a new key by trying to get it in a hurry when replacing an old key that might or might not be secure.
A good link with more info: http://www.pcworld.com/article/2142540/website-operators-will-have-a-hard-time-dealing-with-the-heartbleed-vulnerability.html