The bus was travelling to the capital, Nairobi, when it was stopped in Mandera county, not far from the Somali border.
Gunmen separated out non-Muslims by asking passengers to read from the Koran, officials and witnesses said. Those who failed were then shot in the head.
Al-Shabab has carried out a series of attacks in Kenya since 2011.
A statement on a website linked to the Islamist group carried a statement saying the attack was carried out in retaliation for security raids on mosques in the coastal city of Mombasa earlier this week.
Kenya's interior ministry said on its Twitter feed that a camp belonging to the attackers had been destroyed by Kenyan military helicopters and jets, with "many killed".
'Point blank'
One of the passengers on the bus, Ahmed Mahat, told the BBC that there were more than 60 passengers on board when it was attacked, before dawn on Saturday, about 30km (19 miles) from Mandera town.
The driver tried to accelerate away, but the vehicle became stuck in mud caused by recent heavy rains, he said.
About 10 heavily armed men talking Somali ordered the passengers off the bus.
"When we got down, passengers were separated according to Somali and non-Somalis," Mr Mahat said.
"The non-Somalis were ordered to read some verses of the holy Koran, and those who failed to read were ordered to lie down. One by one they were shot in the head at point blank range."
Some Somalis were shot after pleading with the gunmen to spare non-Somali passengers, Mr Mahat added.
Security agencies were "in pursuit of the criminal gang" that carried out the attack, the interior ministry said. It described the assailants as "bandits".
After the attack, a local official quoted by Kenyan media said the government had failed to answer their pleas for extra security.
"This place has been prone to attacks," county official Abdullahi Abdirahman told The Daily Nation.
"This is not the first time the government has totally ignored us, and you can now see the how many innocent precious lives have been lost."
Mr Mahat, a teacher from Mandera, said police never came to rescue people from attacks for fear of being ambushed themselves.
Mombasa raids
The attack comes after a week of heightened tension in Mombasa, which has suffered a series of al-Shabab attacks.
Security forces raided mosques in the city, saying they were being used to store weapons. The raids triggered apparent revenge attacks by Muslim youths.
Kenya has experience a series of al-Shabab attacks since it sent troops to Somalia three years ago to help fight the militant group.
Mandera, a remote area in Kenya's north-east that shares a long and porous border with Somalia, has been one of the regions worst-affected by the violence.
On the Somali side of the border, al-Shabab is said to have a base that was recently bombed by Kenyan warplanes. It was not immediately clear whether this was the same base targeted by Kenya following Saturday's attack.
There was unrest in Mandera in June after two clerics accused of belonging to al-Shabab were shot dead. Residents protested that the clerics had no links to the group.
Al-Shabab - a potent threat in East Africa
Are you, or do you know anyone, in the area? Email haveyoursay@bbc.co.uk
Or comment below.