Fidelity Freedom Fund 2040 and Vanguard Target Retirement 2040 fund?

Ok, I'll preface this by saying that I know just enough about this stuff to get into trouble. Here's some information, then the question: I have about $20K in the Fidelity Freedom 2040 fund in a Roth IRA. As I get to know more about this "stuff" I'm seeing its expense ratio is 3 to 4 times higher than the Vanguard 2040 fund. Over the long haul (say, 30 years or so) will that difference make a significant impact on my retirement. I'm assuming it does. If so, would it be wise to start buying the Vanguard 2040 (my account is with fidelity now), transfer my IRA to Vanguard, or sell all my Freedom Fund 2040 and put it all into Vanguard 2040...or transfer to Vanguard and then sell all my Fidelity Freedom and then buy the Vanguard 2040...whew...Lemme know what you think. Thanks so much for the advice!