Podcast on predictive CFB model

[PSU2UNC]

Not sure if this has been discussed before, but there is a podcast called "SEC Fans" who have a predictive model that they use to analyze not only SEC games but also other big games. They have a pod on the OSU-PSU game that you can find below.

http://secfans.com/

What made me laugh was that when the model predicted PSU would win by multiple scores they went out of their way to find issues with the model input data to create a win for OSU. They did this by saying that you have to throw out the App St game data because App St only played 3 games (one was canceled by hurricane) and one of those was FCS (which they don't count in the model).

While statistically, that may be true, App St is by far the best team PSU has played. So by removing the best team PSU has played (which by the way, statistically makes PSU look worse) they were able to fudge the numbers so that OSU wins a shoot out.

I honestly don't understand anyone who thinks PSU doesn't have a very, very good chance at wining this game.

 

[NittPicker]

We'll find out who wins on Saturday. I certainly hope it's Penn State. Playoff predictions are biased as hell and people will use every argument possible to support who they think should be there. Consider that the vast majority of preseason predictions included Alabama, Clemson and Ohio State. The fourth team varies. They people who made those prediction can't admit they aren't very knowledgeable so they'll go to the mat to defend their choices. One loss? Fluke. Two losses? They still deserve to be the first two loss team in the playoff.

As for the mentioned predictive model, if it picks PSU to win but those people choose to ignore it, they're essentially saying the model is worthless. If it's worthless, why talk about it at all?

 

[Beachwineguy]

The model also had Penn State winning by a large margin when including App State, like two TDS. When they took them out, it gave OSU a three point victory.

 

[Ian]

How accurate has it been with previous predictions?

 

[GreggK]

When their numbers spit out something they didn’t agreed with, they just pulled out information until they got the result they wanted.

And then pretended they are all about the data.

 

[fastlax16]

How accurate has it been with previous predictions?

Last season it predicted 33-31 psu.

 

[nitlion6]

Penn State is winning Saturday, but they need to maintain their focus throughout the season. A single slip up and they are probably the fifth team in.

Everybody thank Jimmy Delaney for belittling the value of a Big10 championship.

 

[nitlion6]

View embedded media

Last season it predicted 33-31 psu.

If PSU had played the entire game looking to win they would have won. If they get up on Saturday hopefully they continue to play to dominate and not just beat the clock.

 

[Grant Green]

Not sure if this has been discussed before, but there is a podcast called "SEC Fans" who have a predictive model that they use to analyze not only SEC games but also other big games. They have a pod on the OSU-PSU game that you can find below.

http://secfans.com/

What made me laugh was that when the model predicted PSU would win by multiple scores they went out of their way to find issues with the model input data to create a win for OSU. They did this by saying that you have to throw out the App St game data because App St only played 3 games (one was canceled by hurricane) and one of those was FCS (which they don't count in the model).

While statistically, that may be true, App St is by far the best team PSU has played. So by removing the best team PSU has played (which by the way, statistically makes PSU look worse) they were able to fudge the numbers so that OSU wins a shoot out.

I honestly don't understand anyone who thinks PSU doesn't have a very, very good chance at wining this game.

App St went for 7.5 yd/pass vs PSU which is about average for FBS. They went for 21 yards per pass in their only other data point vs Charlotte - they only threw 14 passes. I know it sounds fishy to remove App, but their game vs Charlotte makes for really strange data. As they said, they like to have 4 data points to run their model and App only has 2.

App St played well, but don't fool yourself into thinking that they are a great team. They are probably somewhere between Louisville and Buffalo.

 

[michnittlion]

How accurate has it been with previous predictions?

That's really all that matters. What is it's verified prediction power?

If I wanted to produce a computer model that predicted a 43-20 PSU victory on Saturday night --- I can do that. I can produce a computer model that can predict ANY output. Doesn't mean it's a good model, of course.

 

[heckmans]

When their numbers spit out something they didn’t agreed with, they just pulled out information until they got the result they wanted.

And then pretended they are all about the data.

On one hand, in science, you have to control for anomalies or your results will be compromised.
On the other hand, remember this the next time you hear about some new study that seems to be biased in nature, and especially the social sciences where it's nearly impossible to repeat the results. Stats can be made to say anything you want, hence the phrase, "Lies, damn lies, and statistics."

 

[PSU2UNC]

That's really all that matters. What is it's verified prediction power?

If I wanted to produce a computer model that predicted a 43-20 PSU victory on Saturday night --- I can do that. I can produce a computer model that can predict ANY output. Doesn't mean it's a good model, of course.

The model appear to be a fairly good predictor (although it's not clear to me how frequently they make qualitative adjustments like discussed above).

 

[PSU2UNC]

App St went for 7.5 yd/pass vs PSU which is about average for FBS. They went for 21 yards per pass in their only other data point vs Charlotte - they only threw 14 passes. I know it sounds fishy to remove App, but their game vs Charlotte makes for really strange data. As they said, they like to have 4 data points to run their model and App only has 2.

App St played well, but don't fool yourself into thinking that they are a great team. They are probably somewhere between Louisville and Buffalo.

Two things:

1) I work with data every day and I almost never discard outliers. The only outliers that get discarded is when I think the data is inaccurate. If the data is "weird" or "strange" it has to stay in the analysis because that is often where the most interesting stories lie.

2) App St is a top 25 team. They will probably win their division and will almost certainly go to (win?) a bowl game.

 

[JJTopp99]

When their numbers spit out something they didn’t agree with, they just pulled out information until they got the result they wanted.

And then pretended they are all about the data.

Sort of what they do with climate change models, lol.

 

[JJTopp99]

Two things:

1) I work with data every day and I almost never discard outliers. The only outliers that get discarded is when I think the data is inaccurate. If the data is "weird" or "strange" it has to stay in the analysis because that is often where the most interesting stories lie.

2) App St is a top 25 team. They will probably win their division and will almost certainly go to (win?) a bowl game.

I want to see Pitt play APP State in a bowl game this year given that half the Lair Board is of the opinion that APP State is "overrated". "Overrated" is probably the most frequently used descriptive word on the Lair by the way because every team over there is overrated (except Pitt of course).

 

[fairgambit]

I honestly don't understand anyone who thinks PSU doesn't have a very, very good chance at wining this game.

Not just a good chance. Not just a very good chance, but a "very, very good chance." Ah, then you don't understand me. That's OK. I would say we have a chance to win, but I expect OSU to win by 10 or more. I obviously hope I'm wrong, but I just believe they have the better team and the Whiteout will not be enough to overcome that. If the game were played in November, I would give us a slight edge, but we have inexperience on defense that I expect will be our downfall.

 

[acg116]

How accurate has it been with previous predictions?

Pretty accurate....I don’t recall the 2017 record but they quote it towards the end of the podcast.

 

[GreggK]

Sort of what they do with climate change models, lol.

No....not really at all.

 

[PSU2UNC]

Not just a good chance. Not just a very good chance, but a "very, very good chance." Ah, then you don't understand me. That's OK. I would say we have a chance to win, but I expect OSU to win by 10 or more. I obviously hope I'm wrong, but I just believe they have the better team and the Whiteout will not be enough to overcome that. If the game were played in November, I would give us a slight edge, but we have inexperience on defense that I expect will be our downfall.

I strenuously object. ;-)

 

[GreggK]

Not just a good chance. Not just a very good chance, but a "very, very good chance." Ah, then you don't understand me. That's OK. I would say we have a chance to win, but I expect OSU to win by 10 or more. I obviously hope I'm wrong, but I just believe they have the better team and the Whiteout will not be enough to overcome that. If the game were played in November, I would give us a slight edge, but we have inexperience on defense that I expect will be our downfall.

They have had a better team than us on paper each of the last three games. And we could have/should have won all three.

 

[fairgambit]

I strenuously object. ;-)

You should know that I have made 114 incorrect predictions in a row. Of course, that means I am overdue for a winner. I hope it doesn't come this week.

 

[heckmans]

No....not really at all.

Not really the thread to be debating it, but it IS actually what happens, with Michael Mann, of PSU at the center.
Google is your friend. Lots of reading here for you.

https://www.google.com/search?q=IPC...e..69i57j33.5727j1j7&sourceid=chrome&ie=UTF-8

 

[fairgambit]

They have had a better team than us on paper each of the last three games. And we could have/should have won all three.

Last year was a 1 point OSU win. Yes, we could have/should have won that one. The previous year was a 3 point Penn State win and they will argue they could have/should have won that one. Three years ago it was a 28 point OSU win. No way we could have/should have won that one.

 

[acg116]

They have had a better team than us on paper each of the last three games. And we could have/should have won all three.

I'll give you the benefit of the doubt that you're forgetting how lopsided 2015 was, but if not, you kind of lose all credibility on this topic if you think we should have won that game.

 

[john4psu]

I want to see Pitt play APP State in a bowl game this year given that half the Lair Board is of the opinion that APP State is "overrated". "Overrated" is probably the most frequently used descriptive word on the Lair by the way because every team over there is overrated (except Pitt of course).

If Pitt were bowl eligible and that is a BIG if.

 

[Grant Green]

Two things:

1) I work with data every day and I almost never discard outliers. The only outliers that get discarded is when I think the data is inaccurate. If the data is "weird" or "strange" it has to stay in the analysis because that is often where the most interesting stories lie.

2) App St is a top 25 team. They will probably win their division and will almost certainly go to (win?) a bowl game.

You probably wouldn't make a professional conclusion based on 2 real data points either. I don't think you can compare early season college football data to whatever data you are crunching. There is a lot of randomness and not a lot of data to iron out those wrinkles. Your data is likely more consistent, reliable, and plentiful.

I think you are seeing App St through the blue and white glasses. I've fallen into that trap before too, where I overrate a team based on their performance vs PSU. App St really only had 2 good quarters against us. For App St to be top 25, they would have to be comparable to a team like Boise St, Missouri, BC (I'm not looking at AP top 25 - that poll sucks). They likely aren't even a true top 50 team.

 

[Westy81585]

Not necessarily defending their picks (though I do agree with them):

But they pulled App State out because of how their system is setup. They average all the FBS level games a team plays, and then match that against the average % of their usual output a teams defense holds their foes too. For example if App State averages 100 yards passing a game, and Penn State holds their opponents to 90% of their average passing output, then their formula calculates that App State will put up 90 yards against PSU.

Unfortunately App State has only played one FBS level team to this point, Penn State. That means Penn State has held them to their exact averages, and it messes up the whole formula.

Honestly their biggest mistake was trying to use the formula at all in this situation. They even mentioned that both of these teams have been plugging in their back ups early, which skews the results dramatically. But what else can you expect; they're SEC fans.

 

[Grant Green]

Unfortunately App State has only played one FBS level team to this point, Penn State. That means Penn State has held them to their exact averages, and it messes up the whole formula.

Honestly their biggest mistake was trying to use the formula at all in this situation. They even mentioned that both of these teams have been plugging in their back ups early, which skews the results dramatically. But what else can you expect; they're SEC fans.

See my post above. App played Charlotte as well and averaged 21 yards per pass, which is crazy. The problem is that PSU actually held App below their average passing because their average is skewed way high. I think they said at one point that if you assumed PSU held App to their average, then the model would have been more reliable.

I think you are right on the 2nd part though. They probably should have just said that the model isn't applicable in this situation because App screws it up. Their non-model handicapping is pretty good.

 

[PSU2UNC]

You probably wouldn't make a professional conclusion based on 2 real data points either. I don't think you can compare early season college football data to whatever data you are crunching. There is a lot of randomness and not a lot of data to iron out those wrinkles. Your data is likely more consistent, reliable, and plentiful.

I think you are seeing App St through the blue and white glasses. I've fallen into that trap before too, where I overrate a team based on their performance vs PSU. App St really only had 2 good quarters against us. For App St to be top 25, they would have to be comparable to a team like Boise St, Missouri, BC (I'm not looking at AP top 25 - that poll sucks). They likely aren't even a true top 50 team.

If I had a model that I believed worked, I wouldn't throw out data points that I "didn't like." Alternatively, the modelers could say "look our model doesn't really work until week 6" (which would be totally acceptable and probably true).

You are entitled to your opinion of App St. In their other two games they have won (against poor competition) by scores of 45-9 and 72-7. I'd compare them favorably with Missouri, BSU and BC.

 

[Grant Green]

If I had a model that I believed worked, I wouldn't throw out data points that I "didn't like." Alternatively, the modelers could say "look our model doesn't really work until week 6" (which would be totally acceptable and probably true).

You are entitled to your opinion of App St. In their other two games they have won (against poor competition) by scores of 45-9 and 72-7. I'd compare them favorably with Missouri, BSU and BC.

They do say in multiple videos that the model isn't effective until at least week 4. In this case, it should probably be more like week 5 or 6 due to Apps lack of quality FBS opponents. I will agree that they should probably have just ditched the model in this case, but I don't think throwing out App st is a terrible alternative. Again, CFB early season data can't be compared to other studies with robust data sets where single outliers aren't going to severely skew results.

It is not just my opinion on App St. I guarantee that App would be a 7 to 14 point underdog to Boise, Missou, and BC on a neutral field. They will probably win their conference and get some AP votes, but that means nothing to me in terms of their actual power ranking.

 

[PSU2UNC]

They do say in multiple videos that the model isn't effective until at least week 4. In this case, it should probably be more like week 5 or 6 due to Apps lack of quality FBS opponents. I will agree that they should probably have just ditched the model in this case, but I don't think throwing out App st is a terrible alternative. Again, CFB early season data can't be compared to other studies with robust data sets where single outliers aren't going to severely skew results.

It is not just my opinion on App St. I guarantee that App would be a 7 to 14 point underdog to Boise, Missou, and BC on a neutral field. They will probably win their conference and get some AP votes, but that means nothing to me in terms of their actual power ranking.

You are entitled to your opinion. Circle back with me at the end of the year to see if you have revised your opinion of App St.

 

[JJTopp99]

If Pitt were bowl eligible and that is a BIG if.

Yes, it is just wishful thinking on my part knowing full well the odds of Pitt going to a bowl game this season are very slim.