The Chinese PC maker will pay $2.07 billion in cash and the rest in shares to be issued to IBM, according to a filing with the Hong Kong Stock Exchange on Thursday.
It will allow Lenovo to diversify revenues away from its PC business -- as global shipments fall in the face of competition from smartphones and tablets -- and try to establish itself as a key player in the mobile devices and storage server market.
IBM missed revenue forecasts when it reported earnings on Tuesday, suffering from slowing demand for its servers and storage products from growing markets like China.
The Lenovo deal will allow IBM to ditch its low-margin x86 business and focus on higher-margin services and software businesses, which it has been expanding for the last decade.
(Read more: Lenovo dominated worst-ever world PC market in 2013)
The Beijing-based PC maker will hope the deal could establish the company as a major player in the global enterprise server market, according to analysts.
While IBM's server business was the world's second-largest, with a 22.9 percent share of the $12.3 billion market in the third quarter of 2013, according to technology research firm Gartner, Lenovo did not appear in the top five.
The deal was a "hand-and-glove-type of transition" for Lenovo and could provide the growth needed for the company, according to Errol Rasit, research director at Gartner.
"Lenovo are extremely strong as a provider for China and has been trying to break in to the global market. This acquisition is really high-profile and will help Lenovo to gain global success and market share" Rasit told CNBC in a phone interview.
"It will see a lot of positive momentum from cross-selling IBM technology to Lenovo accounts."
(Read more: IBM earnings beat estimates, revenue comes in light)
As part of the deal, Lenovo plans to offer employment to about 7,500 of IBM's staff members. Lenovo will also be taking over related customer service and maintenance operations.
Negotiations between the two companies regarding the low-end server business broke down last year due to conflicts over pricing, according to media reports, which suggested at the time that IBM wanted $6 billion for the unit.
This is not the first time Lenovo and IBM have struck an acquisition deal. The Beijing-based firm bought IBM's ThinkPad PC business in 2005 for $1.75 billion, and is now the world's largest PC maker by shipments.
—By CNBC's Arjun Kharpal: Follow him on Twitter @ArjunKharpal