Holey S3 Buckets and Human Error: Why We Can’t Have Nice Things

The cloud has paved the way for large-scale business transformations that revolutionized customer and employee management. While organizations continue to rely on it, cloud data security is facing a paradox: the wholesale movement and access of data is necessary for any organization’s survival, but that data must be stopped short of falling into the hands of malicious actors.

How Cloud Data Became a Problem

Cloud storage revolutionized the way in which companies handle and care for their customer data. Amazon’s S3 provides one of the most popular cloud storage services, hosting tens of thousands of data lakes for both multinational and rapidly-scaling companies. S3 presents a hugely important solution for scaling and business insight discovery. 

However, human error has combined with Amazon’s less-than-ideal user experience to create one of today’s largest cloud security liabilities. Within cloud storage, an object describes the file in question and any metadata that describes that data. In S3, buckets are the containers within which objects are stored. This enables the multitenant architecture that forms the backbone of Amazon’s budget-friendly storage solution. 

However, these buckets are suffering from a chronic misconfiguration crisis. Swathes of buckets used by individual companies have mistakenly been set to public when they should – without a doubt – have been kept private. This issue was a cause of one recent major leak that affected swathes of airports around Colombia and Peru. The leaky bucket was owned and maintained by Sweden-based Securitas, a company that provides on-site guards, electronic security solutions, and enterprise risk management. Ironically, this unsecured server contained around 3 terabytes worth of employee and airport information. At least four airports were named in exposed files, with two main datasets relating both to employee staff and Securitas’ own teams. Among the exposed data were ID and passport photos, alongside Personally identifiable information (PII) that included full names, occupations, and signatures.

Alongside individual records, researchers found that the exposed database included photos of planes, employees, fuel lines for the planes, and even data surrounding the luggage handling process. The metadata of these pictures further provided the time and date that the photos were taken, alongside their GPS coordinates. The risk this presents to national security cannot be understated.

Securitas’ oversight marks only one of the many S3 leaks plaguing all industries. Researchers at Truffle Security recently developed an automated S3 secret sniffer, with which they stumbled across 4,000 completely exposed Amazon S3 buckets that included data that simply cannot be made public – information such as login credentials, security keys, and API keys. For every file in the analyzed dataset, they found an average of 2 and a half passwords. In some cases, more than 10 secrets were found in a single file.

The truffle team’s secret discovery process found unsecured S3 buckets throughout startups, fortune 500 companies, and NGOs alike. Amazon S3 insecurities are a major security risk that stubbornly continues to tag the heels of every major industry. Exposed secrets have the potential to drastically amplify an attack. Thanks to these totally public secrets, attackers can now get into more secure accounts and services.

Also, Check – A Quick Guide To Agile Product Management

Automated Secret Scanner

A major cause of this insecure storage could be found in the settings layout that Amazon offers. Whilst it’s totally unacceptable for companies to leak sensitive data through their cloud, and the onus is on them to store the data they profit off, the constant and unending mistake being made throughout industries suggests a deeper source to the problem. 

Amazon’s settings for each bucket include an access policy to allow “any authenticated AWS user”. This setting literally means that any AWS user – even outside your organization – can access that data. However, it’s easy to read this setting as requiring the user to be authenticated within the account itself. So, by checking this one setting, AWS then spills this data across the public cloud.

Given the potentially-confusing state of settings, and the ever-greater complexity of modern companies’ cloud layouts, it could be argued that the problem of leaky S3 buckets is not wholly user error. 

The potential for major data spills has now risen to such a degree that tools such as the open-source S3crets Scanner are now commonplace. This scanner automatically combs through the lists of public buckets, searching for text files. Once text files are found, they’re downloaded and the contents are scanned. In this manner, S3crets Scanner can automatically highlight secrets that lie within the public cloud. Any relevant secrets in text format are then forwarded to the organization’s security team. This process occurs rapidly and repeatedly, allowing for an always-on approach to cloud secret management. 

Don’t Just See Secrets – Stop Them

While open-source secret scanners certainly play a role in cloud data security, the next step for evolving maturity is an automated solution that fully catalogs and tracks the data hanging around in your cloud. With a quality third-party provider, even rapidly-scaling cloud databases can be protected. This tool automatically discovers and classifies the sensitive data pouring into an organization, then classifies it as such. Thanks to this right perimeter now surrounding secrets, it then becomes possible to automatically detect policy violations and monitor the data for any risky access behavior. Now you know who’s accessing the sensitive data and what they’re doing with it. 

This focus on automation extends even further, with reporting allowing for regular overviews, supporting an overall security stance that regularly reacts and adapts to the changing landscape of a company’s own database. This paves the way for easy compliance process adherence. Ultimately, automation delivers the range of scale and accuracy rapidly changing cloud environments now require – eliminating the need for people to keep pace.