Rankings absolutely matter because they setup your ability to ascend. No matter how good or bad you are, your perception (aka ranking) is what facilitates your speed to ascend up the rankings. Your performance can only do so much because people just won't leapfrog you all that much. Mainly because media types do have agendas....Paul Finebaum is always gonna rank a one loss SEC team ahead of a one loss B1G team, because it's aligned with his agenda.
All that said, ranking suck. The people who make them are really bad at it. This week is a prime example, where the perception that teams like Notre Dame or Texas A&M are "top" teams. Meanwhile they are losing at home to teams that will likely never be worthy of being a ranked team.
It's sad for kids that a bunch of media types who never played a competitive sport dictate these things....and make no mistake a team can only control their own destiny if they win their conference and secure a "decided" berth, which is a maximum of say 6 teams in the country any given year, otherwise you fall victim to the perception lense. A lense decided by people who have minimal "real football" knowledge. It's really quite silly and based on the new playoff proposal the ranking becomes even more important for the "next 6" that get in and the "top 4" to get the bye.
So yes, rankings matter, no matter how much we like them or how non legit they truly are....which is a lot. But they ultimately determine what will happen with non conference champ in the postseason, which is the goal of the regular season.