Yemeni security officials, who spoke on condition of anonymity because they weren't authorized to release the information, identified the slain embassy security official as Qassem Aqlani, in his 50s.
He was on his way to work when a gunman on a motorcycle opened fire at him and fled the scene, they said. The attack was near Aqlani's home in western Sanaa, while the embassy is located in the eastern half of the city.
Aqlani had been working for the U.S. Embassy for nearly 20 years, most recently as a lead investigator into last month's assault on the compound by Yemenis protesting the film that mocked the Prophet Muhammad, the officials said. Protesters stormed the embassy and set fire to a U.S. flag before government forces dispersed them with tear gas.
Al-Qaida's Yemen branch has called for attacks on U.S. embassies in a bid to take advantage of the anti-American sentiment that has swept the Middle East and other parts of the Muslim world in the past month over the film.
Initially, the film was linked to a Sept. 11 attack on the U.S. Consulate in the Libyan city of Benghazi that left four Americans dead, including U.S. Ambassador Chris Stevens. U.S. officials said later the attack was not linked to the video.
AQAP praised the killing of U.S. diplomats in Libya, describing it as "the best example" for those attacking embassies to follow.
The group had taken advantage of a security and political vacuum created by last year's uprising that led to the ouster of longtime leader Ali Abdullah Saleh and seized territories and cities in the south. The government-led offensive has pushed the militants out to mountainous areas from where they have been staging suicide attacks and assassinations inside cities.