Pellegrini attempted to ease his influential captain back into action as a second-half substitute after an eight-game absence as City cantered towards victory at the Etihad Stadium, but the Belgian lasted just nine minutes.
Kompany shook his head as he left the field and headed straight down the tunnel after suffering a recurrence of the same calf problem
The 29-year-old will now be assessed but City will fear another lengthy lay-off.
Pellegrini said: "Vincent is not the best news.
"He was ready to return
I did not want him to play the whole game because we have to play so many games, but he worked the whole week with normality
It is really difficult to understand what happened.
"We will see tomorrow with the doctor's report but it is the same injury, the same calf."
Otherwise it was a good day for City as they got back to winning ways in the Barclays Premier League after Monday's disappointing loss at title rivals Arsenal.
Raheem Sterling opened the scoring after just 12 minutes before Yaya Toure and Wilfried Bony added two more in quick succession.
The outstanding Kevin De Bruyne claimed a deserved fourth early in the second half before Fabio Borini netted a Black Cats consolation.
Pellegrini said: "It was a very important victory, not only because we won the three points but also in the way we played
We returned to our normal performance - playing attractive football, scoring goals.
"I said after we lost against Arsenal it was very difficult to talk about positive things but I think we had more than 60 per cent of possession of the ball
That gave me a sign we were returning to our style.
"It was very important today to score four goals and create at least four more clear chances."
The defeat was Sunderland's third in succession and left them adrift in the relegation zone.
Manager Sam Allardyce recognises the urgency of the situation and feels he must act in the January transfer window.
Allardyce, who made six changes for the game, said: "We have got players who are saying they deserve a chance in the team and, 'Why am I not playing when the team is struggling?'
"Everyone has had a chance now and obviously they have not done as well as they should have done
Now it is my time to make some big decisions about that squad, and think about the January window and ask 'who can we bring in?' and 'who am I going to move out?'."
Allardyce, who was without centre-back Younes Kaboul, felt his defence were opened up too easily by City.
He said: "Their three goals in nine minutes were three goals we could have done a lot better with, particularly the first and the third.
"In the last three games we have gifted teams goals and found it difficult to come back
As much as you change the side around, which is what I have been doing since I've been here, and think you are going in the right direction, clearly we are slipping back into the danger zone and it is hard work to think now what my best team is.
"Until we start getting better defensively we are going to struggle to get results
That is our big problem
As a team we do not defend as well as we should do."
Source : PA
Source: PA