One error that I got was "Unexpected call to method or property access" and it was only present in Internet Explorer 7. Chrome and Firefox worked just fine, which made the client debugging tools for them pretty useless leaving a client side scripting error worst case scenario.
By removing piece by piece of the HTML views building up the failing page, I finally narrowed it down to a small section where I spotted a img tag with a closing tag like so:
<img ... ></img>
That's not normal although I think it's legit according to standards. The code rendering the tag was the ASP.NET MVC System.Web.Mvc.TagBuilder class, which has an rendering option to specify if the tag is self closing or normal. I changed it to self closing which made the img tag to render as <img .. /> and the scripting error was gone.
It turns out that it was the jQuery append() method that was throwing and my guess is that Explorer 7 is pretty sensitive when it comes to the DOM not being "perfect" according to its standards (and it really is ITS standards).
Nasty browser it is.