Blake in the comments notes the issue of phishing -- that is, making a fake website duplicating the look and feel of a real website to capture passwords and identity. As far as I can tell, OpenID has no good solutions (here's a discussion that doesn't end up being that helpful).

Here's a few ideas I've had mulling over this:

• run your own identity server with your own unique look and feel. This may not be too hard with Amazon EC2 type services, particularly if you could do something like run Java Servlets.
  
• Identity servers should mail you new sites that you've accessed, possibly putting a hold on your account if there's suspicious activities
• Identity servers could provide multiple levels of passwords; lesser passwords could be used for establishing identity at untrusted sites; the highest password could be used for direct administration of the identity account

On other security issues:

• preventing replay attacks (summary: it's handled by the best libraries)
  
• trust, spam and signing comments (summary: out of scope)