Lewis Grabban scored the game’s only goal in the 83rd minute as Arsenal, who were missing Aaron Ramsdale, Granit Xhaka, Emile Smith Rowe, Nicolas Pepe, Alexandre Lacazette and Thomas Partey, were humbled.
Arsenal had gone on a five-game winning run prior to their 2-1 league loss to leaders Manchester City, thus the FA Cup exit came as a shock to Arteta.
“It’s really hurting,” Arteta told reporters. “It’s a competition that is related to our history and to go out of it today is a big bump.”
The Gunners have lifted the FA Cup on four of the past eight occasions it has been played.
“Im really disappointed with the performance, first of all - not with the attitude, but how much purpose we had and what determination we showed to change the game when its difficult to play against (them) and the way they play,” Arteta added.
“More drive, more hunger to win at any cost in every single action. We were not there at our level today.”
Elsewhere in the FA Cup on Sunday, Liverpool fielded an under-strength line-up but still eased past Shrewsbury 4-1, although they had to come from behind.
Wolverhampton defeated Sheffield United 3-0 and Norwich City edged past Charlton 1-0.