Salesforce said on Friday that a technical snafu resulted in customers losing access to some of the company's most popular online services.
Sales Cloud and Service Cloud, the two largest products for Salesforce by revenue, are both affected by the downtime, the company said on a status page. Parker Harris, Salesforce's chief technology officer and a co-founder, acknowledged the issue at 12:40 p.m. Eastern time, making it clear that that employees were working on the problem.
Salesforce's cloud-based software is used by salespeople, marketers and customer service staffers across the globe, and the company has picked up so many customers that CEO Marc Benioff routinely calls it the fastest-growing software business in history.
For Salesforce and other companies that provide online services, outages can have financial consequences, like having to provide credits to customers or pay out penalties. Interruptions could "cause customers to make warranty or other claims against us or to terminate their subscriptions and adversely affect our attrition rates and our ability to attract new customers, all of which would reduce our revenue," Salesforce said in its most recent annual report.
Salesforce's status page suggested the service disruption began at 12:56 p.m. Eastern time. At 1:33 p.m. Eastern time Salesforce provided an update on the issues:
"The Salesforce Technology team is investigating an issue impacting Salesforce customers who use Pardot, or have used Pardot in the past. The deployment of a database script resulted in granting users broader data access than intended. To protect our customers, we have blocked access to all instances that contain affected customers until we can complete the removal of the inadvertent permissions in the affected customer orgs. As a result, customers who were not impacted may experience service disruption. In parallel, we are working to restore the original permissions as quickly as possible. Customers should continue to check Trust for updates."
At 4:36 p.m. Eastern, Salesforce provided an update. It said customers depending on versions of the company's software operating in North America and Europe were seeing problems.
"We have started unblocking customers who were not affected by the permission issues," Salesforce said.
Another update came at 5:49 p.m. Eastern. By that point Salesforce had completed the unblocking work for those customers that hadn't been hit by the permission issues.
By 7:03 p.m. Eastern, Salesforce said that it was still working to restore access to the customers that the permission issues affected, and that "some affected customers may receive error messages related to account status, billing, or single sign-on functionality after logging into Salesforce."
The next morning, at 5:40 a.m. Eastern time, Salesforce announced on its status page that access had been restored for administrators of all organizations that had been affected by the permission issues.
"We are preparing a set of instructions for admins that may need guidance on how to manually restore those permissions. As soon as the instructions are final, we will inform admins via an email that will contain a link to the instructions," the company said.
People had posted on Twitter about losing access to Salesforce tools as early as 10 a.m. Eastern time on Friday.
The company had an outage affecting access for some customers in 2016. Salesforce shares were down as much as 3% on Friday.
WATCH: Cramer on Salesforce
