One person was killed and 132 others wounded during the attack, Ethiopia's health minister said.
The explosion in packed Meskel Square in the capital, Addis Ababa, followed weeks of reforms that had shocked many in the East African nation after years of anti-government tensions, states of emergency, thousands of arrests and long internet shutdowns.
Witness Seyoum Teshome, a member of the organising committee, told local media: "It was a grenade".
"We ask the prime minister to now amend laws that restrict free speech and to end a system of surveillance that has always been used to intimidate journalists".
"Some whose heart is filled with hate attempted a grenade attack", the prime minister's chief of staff Fitsum Arega wrote on Twitter, vowing that the perpetrators will be brought to justice.
A witness near the scene of a deadly blast at a huge rally for Ethiopia's new prime minister says the man who tried to throw the grenade was wearing a police uniform.
Ethiopia's government has given few details of the incident, which Abiy blamed on "anti-peace forces".
President Kenyatta offered his deepest condolences and solidarity to Prime Minister Abiy Ahmed and the people of the Federal Democratic Republic of Ethiopia, saying Kenya stands with Ethiopia during this time. "To those who tried to divide us, I want to tell you that you have not succeeded", he said, adding: "Love will win. forgiveness will win".
Cleveland Cavaliers have no interest in trading Kevin Love
Which team Toronto could potentially make a deal with remains to be seen, and a trade might not develop until Thursday night. As the conversation came to an end, my source gave me information on LeBron's ultimate end game with the Cavaliers .
More than 100 people stormed the stage, hurling various objects at police, shouting: "Down, down Woyane", and "Woyane thief", using a pejorative term for the ruling party, an AFP correspondent said.
Mr Ahmed was taken away by security personnel immediately after his speech, which he gave in the Ethiopian capital's Meskel Square.
Shortly after Abiy wrapped up his speech, an explosion went off among the demonstrators.
The blast came just as the master of ceremonies was welcoming viewers from overseas and said in English "this is the day that Ethiopia has become proud".
The United States has been among other nations expressing support for the changes in the country, a key security ally.
Since assuming office, Abiy has introduced a number of reforms and has overseen the release of jailed dissidents and moves to liberalise the economy.
Ethiopian Prime Minister Hailemariam Desalegn resigned unexpectedly in February, saying he hoped to end years of unrest and political upheaval.
Political rallies of Saturday's scale are rare in Ethiopia, where the EPRDF controls all seats in parliament and opposition parties complain of harassment.