United took an early lead from Mason Greenwood but the Foxes hit back with strikes from Youri Tielemans and Caglar Soyuncu.
Marcus Rashford marked his return from injury with an equaliser in the 82nd minute before late Leicester strike from Jamie Vardy and Patson Daka.
The result leaves United fifth on the Premier League table, having suffered losses to Leicester and Aston Villa as well as a draw with Everton in their past three league games.
Solskjaer’s side had been billed as title contenders after a bright start to the season buoyed by Cristiano Ronaldo’s signing along with Raphael Varane and Jadon Sancho.
But the situation appears to be changing for United, with calls already for Solskjaer to be axed, while the Norwegian himself admitted he “made a couple of bad decisions” against City.
Sky Sports pundit Paul Merson said Solskjaer is in a “spot of bother” while ex-United defender Gary Neville labelled the side “shoddy” and “scrappy and scruffy to watch”.
The defeat also ended United’s 29-game unbeaten away record in all competitions.
“In the second half, it was end-to-end but they created more chances and conceding at set-pieces was disappointing,” Solskjaer said.
“Conceding seconds after the equaliser shouldn’t happen. I picked the team; Harry showed no reaction. I hold my hand up if it doesn’t work out and sometimes it doesn’t. When you concede four, I probably made a couple of bad decisions.”
Elsewhere, the top three flexed their muscle with Chelsea edging past Brentford 1-0 thanks to Edouard Mendy’s outstanding work in goal.
Liverpool thrashed Watford 5-0 with Mohamed Salah netting a stunning goal while Roberto Firmino scored a hat-trick.
Manchester City got past Burnley 2-0 to remain third behind the Blues and Reds.