Brian Laws saw his side knocked out of the tournament in the first game of his third spell in charge of the club as Matt Fish, Deon Burton, Danny Kedwell and Adam Birchall all struck in the second half.
The Iron entered the match making two changes from the side beaten by MK Dons a week earlier.
Laws brought Mike Grella and Andy Barcham into the starting eleven at the expense of Connor Jennings and David Prutton, who was cup-tied.
Sam Slocombe started in goal, with Christian Ribeiro, David Mirfin, Paul Reid and Tom Newey forming the defence.
The midfield consisted of Mark Duffy, Josh Walker, Michael Collins and Andy Barcham, while Karl Hawley led the attack alongside Grella.
The substitutes were James Severn, Callum Kennedy, Jimmy Ryan, Robbie Gibbons, Niall Canavan, Jennings and Damien Mozika, who had missed the season so far through an ankle injury.
Scunthorpe started well, winning their first corner in the first minute after Barcham's cross was blocked. From the subsequent set-piece, Collins' effort from distance was deflected wide.
Seconds later, the subsequent set-piece wasn't cleared right and it reached Grella, who advanced into the area before drilling a low drive across goal which was tipped away by Stuart Nelson.
In the fifth minute came Scunthorpe's next chance as Walker moved forwards before sending a left footed effort well wide.
The hosts mounted their first attack two minutes later when Chris Whelpdale embarked on a surging run down the right flank but his effort was deflected wide for a corner, from which there was no further danger.
On 11 minutes, Duffy then received the ball on the edge of the area before dribbling down the left hand side and seeing his shot blocked away by Adam Barrett.
The visitors continued to attack and have the better of the game, and Hawley was next to force a save from his former Notts County team-mate Nelson with a header following Collins' cross.
Then, in the 23rd minute, Duffy drilled a powerful effort well wide of goal from the left side of the 18-yard box.
The hosts remained a threat however and had their next opportunity four minutes later when Fish volleyed an attempt narrowly wide after a Myles Weston corner.
The remainder of the first half saw few more chances of note, but Gillingham started the second period with intent as Slocombe was forced into a quick reactionary save as a cross deflected off Walker and goalwards.
The first Scunthorpe opportunity of the second period arrived ten minutes into the second period when Hawley drove forward on a run but he sent his powerful strike well wide of goal.
It was on 62 minutes though when the hosts went ahead as Fish fired home the rebound from close range on the right side of the area after Slocombe tipped away Whelpdale's shot, but it fell into the Gills man's path and he struck the opening goal.
They then made it two just five minutes later as Deon Burton connected with Myles Weston's left-wing cross to send a bullet header beyond Slocombe and into the back of the Iron net.
Grella and Hawley failed to connect with headers as the Iron looked to get back into the game, and then Danny Jackman drove a powerful effort well over. and wide, for the home side.
The Iron made their first substitution with 20 minutes left as Mozika came on for Collins.
But the hosts had a head of steam and were soon to make it three.
Walker felled Jackman in the penalty area and Danny Kedwell stepped up to slam the resulting penalty beyond the dive of Slocombe.
A double Iron alteration followed as Jennings and Ryan replaced Grella and Barcham.
And Jennings had a superb chance two minutes after coming on, getting between the two Gills centre-backs to guide a header over the crossbar from Ribeiro's cross.
There were few more real opportunities, until the hosts made it made it four in the second minute of stoppage time, as Adam Birchall capitalised on a defensive lapse to bundle home from close range.
Scunthorpe are next in action on Tuesday nigh when they visit Walsall.
Gillingham: Nelson, Fish, Payne, Whelpdale, Jackman, Davies, Dack (Lee, 24), Burton (Kedwell, 70), Weston (Birchall, 80), Barrett, Vincelot.
Substitutes not used: Strevens, Frampton, Montrose, Forecast.
Scunthorpe United: Slocombe, Ribeiro, Mirfin, Reid, Newey, Duffy, Walker, Collins (Mozika, 70), Barcham (Ryan, 80), Hawley, Grella (Jennings. 80).
Substitutes not used: Kennedy, Severn, Gibbons, Canavan.
Referee: Andy Woolmer.
Attendance: 4,017 (156 away).