Oracle made a $6.66 billion unsolicited offer to buy BEA, which is under pressure from activist shareholder Carl Icahn to put itself up for sale.
Shares in BEA jumped some 33 percent, more than $1 above Oracle's offer price, suggesting investors think a rival bid will emerge.
"It is apparent to our board...that BEA is worth substantially more to Oracle, to others and, importantly, to our shareholders than the price indicated," William Klein, BEA's vice president of business planning and development wrote in the rejection letter.
Analysts have said that International Business Machines, Microsoft, CAand Hewlett-Packardare also potential suitors for BEA, which sells software that helps business computer systems communicate with each other.
Icahn, who first reported an ownership stake in BEA in mid-August, has acquired more than a 13 percent voting stake in BEA, saying its stock is undervalued and that the company would be better off as part of a larger business.
In an interview on CNBC, Icahn said he is glad Oracle made the bid, and he sees Hewlett-Packard and IBM as other potential bidders.
"I was a bit surprised, but this will be coming more and more," Icahn told CNBC in a telephone interview. "I do think there will be other synergistic bidders. I was very surprised, I really was, that there was going to be a bear hug in this one."