Grayson's position has been the subject of debate in a season that has seen the Yorkshire club struggle to compete with the top teams in the Championship.
Leeds said in a statement that Grayson had been dismissed in order to give the club the best possible chance of forcing their way into contention for a place in the play-offs to reach the Premier League.
"Leeds United have today dispensed with the services of Simon Grayson and his coaching staff," read a club statement.
"Chief Executive Shaun Harvey said: 'We have 18 games to go this season and are still within touching distance of the Play-Offs, but felt with the transfer window now closed we needed to make the change at this time in the belief that a new managerial team will be able to get more out of the existing squad of players and make the difference.'
"'Simon and his staff have given the club over three good years service and we would like to place on record our thanks and wish them all the best for the future.'
The statement also said youth team manager Neil Redfearn would take temporary charge of the club, although media reports swiftly linked former Queens Park Rangers coach Neil Warnock to the vacancy.