The subsequent disaster spewed radiation over a wide area and forced the evacuation of more than 160,000 local people.
Five years on, most have not been able to return to their homes, despite extensive decontamination work.
Image copyrightReutersImage captionPeople in Tokyo stopped in the streets to observe the moment of silenceImage copyrightGetty ImagesImage captionPeople marked the anniversary in public gatherings and more personal actsImage copyrightGetty ImagesImage captionBuddhist monks gathered to pray in front of the former disaster prevention centre in Minamisanriku
The earthquake on 11 March 2011 was one of the most powerful ever recorded.
But it was the resulting tsunami that claimed the most lives, as a wall of seawater powered through coastal areas of Tohoku, flattening entire towns and villages.
At 14:46 Tokyo time (05:56 GMT), the exact moment the quake was detected, people across Japan bowed their heads as a mark of respect for the victims.
Bells rang, and in the capital the underground metro came to halt.
"Many of the people affected by the disaster are aging, and I worry that some of them may be suffering alone in places where our eyes and attention don't reach," Emperor Akihito said at the ceremony.
The day before the anniversary, Prime Minister Shinzo Abe reiterated his government's commitment to nuclear power, saying Japan "cannot do without it" in the long term.
Media captionRupert Wingfield-Hayes reports from tsunami-hit coast
180,000 people have not returned home, of which 100,000 are Fukushima evacuees.
As of 12 February 174,000 people were still living in temporary, rental or other housing as evacuees.
Nearly 800,000 tons of tainted water is stored in more than 1,000 tanks at the Fukushima plant. No firm plans have been made to dispose of the water.
The government-set time frame for intensive reconstruction will be over at the end of this month.
Source: Reconstruction agency
All of Japan's nuclear power plants were ordered offline amid safety concerns following the Fukushima disaster, forcing it to rely on expensive imported fossil fuels for power.
Only a few have since been restarted, despite opposition from residents who say not enough has been done to ensure such a disaster can never happen again.
Earlier this week, two plants were ordered to shut down again because of safety fears.