The only time it can have some SEO impact is when the link to the banner ad is hard coded to the site, that the search engines may pick up the link and count it as a vote towards the site.

But that is only done by the very small sites who have no banner ad server or the few sites who really sell hard coded advertising on their sites. Most sites that accept banner advertising use a banner ad server that allows them to rotate the banner ads on their site — and those are typically javascripts which the search engines cannot read.