It differs per game.
For example:
I play NSMBU on my GamePad, because it's a bit faster than my TV.
I play Nintendo Land on my TV, because both outputs different Video.
That's because AV is less resource-heavy than HDMI.
I have a Component Capture Card and HDMI Capture Card myself, there's no delay on the Component one, while on the HDMI, there's 5 seconds of delay (that's why it requires 2 HDMI Cables, one going to the TV, and the other going to the Console).