I've had this site since January, was ranking well then dropped a little. A small slap from Google to position 30 from 2. I thought, OK, I'll switch some things up, remove a couple of links (shitty wikis, remember this was earlier this year when wikis had some power) and it started to climb back. I kinda gave up SEO wise as I was getting more traffic from Facebook/Twitter - it's a clothing site and more geared around summer type of clothing - so I haven't really bothered with SEO on it lately.
I decided to give the site an overhaul today but nothing really needed doing as it was pretty much perfect......so it's definitely a penalty, but a very soft one. I had menskeyword.co.uk and keywordformen.co.uk, so I swapped them over and it's ranking in the top 5 already after switching the domain 2 days ago. It might drop, it might not. Once it's at the top and hopefully I can make it stick until next spring/summer when the high season comes around, then I'll bank a lot. When people hit the site, even when it wasn't ranking so well, I'd get multiple purchases which meant high commissions for me.
So, as the title says: if you have a doubt and everything looks good with on-page SEO, your link-building techniques should work, but you're not getting the rankings and you were previously ranking well. Change the domain. It works. Don't get tied up with branding. Both domains are partial EMDs. EMDs aren't dead. Branding a domain and spending time getting people to notice the brand is a waste of time/money, should you get smacked.