Joey Dawson, Harry Lewis, Alex Wilkinson and Patrick Virgo were on target in a nine-goal thriller.
Academy manager Tony Daws said: "We went 2-0 up and missed three really good chances where we should have scored, and we controlled the first half, although Rotherham had chances to score.
"The second-half was end-to-end, with both teams creating numerous chances. Rotherham levelled, and then we had two goals disallowed, though I'm not sure why as some strange refereeing decisions went against us.
"This resulted in Rotherham scoring and the game ended 5-4 to them, although it could have been 10-10.
"In possession, a lot of the time we were very good and a number of players really stood out.
"Out of possession, we have a lot of work to do in terms of team shape, compactness, individual roles and understanding our roles and responsibilities."
Team: Balme, Baker, Kemp, Moore, Young, Lewis, Wilson, Gallimore, Dawson, Franklin, Wilkinson.
Coming on at half-time: Crosher, Virgo and Trialist.