Notre Dame's Kyle Brindza kicked the game-winning 27-yard field goal with seven seconds left, capping a two-minute drive by veteran quarterback Tommy Rees to give the Irish (2-0) a 20-17 victory.
Rees, who hadn't played all season, was 3-for-8 on the final drive for 35 yards.
Everett Golson, who had played the entire game until the final series, was 21-for-31 for 289 yards and a touchdown. But his late-fourth quarter fumble set up Purdue's final touchdown.
Purdue quarterback Caleb TerBush had appeared to give the Boilers a shot to win in overtime when he hit Antavian Edison with a 15-yard touchdown pass with 2:12 left that tied the score at 17-17.
TerBush finished 8-for-19 for 79 yards, one touchdown and one interception.
Robert Marve was 11-for-18 for 119 yards and a touchdown. But he hurt his left knee while being sacked late in the fourth quarter, giving TerBush his final chance, and he delivered.
Notre Dame tight end Tyler Eifert, a former Bishop Dwenger standout, had four catches for 98 yards. But he appeared to suffer a head injury and didn't play late.
Purdue (1-1) and Notre Dame played to a 7-7 first-half tie. Marve threw a 2-yard touchdown pass to Edison just before halftime. That countered Golson's 3-yard touchdown run.
The Boilers stuffed the run. Notre Dame had just 18 yards on the ground at halftime. However, Golson was 11-for-15 for 189 yards. Eifert had two catches for 51 yards.
Purdue's quarterback rotation was a first-half dead heat. TerBush was 6-for-11 for 52 yards. He also rushed for 26. Marve was 6-for-10 for 53 yards. He ran for eight yards.
Notre Dame opened the third quarter strong. It stopped Purdue, drove for a touchdown behind a couple of big passes to Eifert, then picked off TerBush and added a field goal. In six minutes, it led 17-7.
The Boilers got inside the Notre Dame 5-yard line in position for a touchdown, but a delay of game penalty and a sack for them to settle for Sam McCartney's 33-yard field goal to make it 17-10.
Facing fourth and 10 with 2:12 left, TerBush hit Edison for a 15-yard tying touchdown pass. It came after the defense forced a Notre Dame fumble.
Then Kelly, never one to tolerate turnovers, turned to Rees. He led the Irish into field goal range and Brindza hit the game-winner.