In between, he was a member of the Maharashtra legislative assembly from Latur for two terms between 1972 and 1980. He was the Lok Sabha member from Latur constituency for seven consecutive terms, winning the elections in 1980, 1984, 1989, 1991, 1996, 1998 and 1999.

First inducted into the Indira Gandhi-led government as Minister of State for Defence (1980–82), Patil was given independent charge of the Commerce Ministry (1982–83), from where he was shifted to Science and Technology, Atomic Energy, Electronics, Space and Ocean Development (1983–84).

During the Rajiv Gandhi-led government, he also took charge as Minister for Personnel, Defence production, Civil Aviation and Tourism.

Patil was trusted with the Union Home Ministry in 2004, but resigned on November 30, 2008, taking moral responsibility for the security lapses during the 26/11 Mumbai terror attack.

In March 1979, there were reports that Gandhian leader Jayaprakash Narayan, admitted to a hospital in Mumbai, had passed away. Many legislatures and even the then Prime Minister Morarji Desai in Parliament offered their condolences based on these unverified reports.

However, Shivraj Patil, then Speaker of the Maharashtra assembly, insisted on official confirmation. He adjourned the House and verified the reports by contacting the doctors treating Narayan at the hospital. The doctors confirmed that Narayan was still alive.

Maharashtra assembly then passed a resolution wishing Narayan a long life. He died several months later, in October 1979.