ProPublica

Journalism in the Public Interest

Cancel

NIST to Review Standards After Cryptographers Cry Foul Over NSA Meddling

The federal institute that sets national standards for data encryption has announced it is reviewing all of its previous recommendations.

Credit: Quinn Norton, Flickr

The federal institute that sets national standards for how government, private citizens and business guard the privacy of their files and communications is reviewing all of its previous recommendations.

The move comes after ProPublica, The Guardian and The New York Times disclosed that the National Security Agency had worked to secretly weaken standards to make it easier for the government to eavesdrop.

The review, announced late Friday afternoon by the National Institute for Standards and Technology, will also include an assessment of how the institute creates encryption standards.

The institute sets national standards for everything from laboratory safety to high-precision timekeeping. NIST’s cryptographic standards are used by software developers around the world to protect confidential data. They are crucial ingredients for privacy on the Internet, and are designed to keep Internet users safe from being eavesdropped on when they make purchases online, pay bills or visit secure websites.

But as the investigation by ProPublica, The Guardian and The New York Times in September revealed, the National Security Agency spends $250 million a year on a project called “SIGINT Enabling” to secretly undermine encryption. One of the key goals, documents said, was to use the agency’s influence to weaken the encryption standards that NIST and other standards bodies publish.

“Trust is crucial to the adoption of strong cryptographic algorithms,” the institute said in a statement on their website. “We will be reviewing our existing body of cryptographic work, looking at both our documented process and the specific procedures used to develop each of these standards and guidelines.”

The NSA is no stranger to NIST’s standards-development process. Under current law, the institute is required to consult with the NSA when drafting standards. NIST also relies on the NSA for help with public standards because the institute doesn’t have as many cryptographers as the agency, which is reported to be the largest employer of mathematicians in the country.

“Unlike NSA, NIST doesn’t have a huge cryptography staff,” said Thomas Ptacek, the founder of Matasano Security, “NIST is not the direct author of many of most of its important standards.”

Matthew Scholl, the deputy chief at the Computer Security Division of the institute, echoed that statement, "As NIST Director Pat Gallagher has said in several public settings, NIST is designed to collaborate and the NSA has some of the world’s best minds in cryptography." He continued, "We also have parallel missions to protect federal IT systems, so we will continue to work with the NSA."

Some of these standards are products of public competitions among academic cryptography researchers, while others are the result of NSA recommendations. An important standard, known as SHA2, was designed by the NSA and is still trusted by independent cryptographers and software developers worldwide.

NIST withdrew one cryptographic standard, called Dual EC DRGB, after documents provided to news organizations by the former intelligence contractor Edward Snowden raised the possibility that the standard had been covertly weakened by the NSA.

Soon after, a leading cryptography company, RSA, told software writers to stop using the algorithm in a product it sells. The company promised to remove the algorithm in future releases.

Many cryptographers have expressed doubt about NIST standards since the initial revelations were published. One popular encryption library changed its webpage to boast that it did not include NIST-standard cryptography. Silent Circle, a company that makes encryption apps for smartphones, promised to replace the encryption routines in its products with algorithms not published by NIST.

If the NIST review prompts significant changes to existing encryption standards, consumers will not see the benefit immediately. “If the recommendations change, lots of code will need to change,” said Tanja Lange, a cryptographer at the University of Technology at Eindhoven, in the Netherlands. “I think that implementers will embrace such a new challenge, but I can also imagine that vendors will be reluctant to invest the extra time.”

In Friday’s announcement, NIST pointed to its long history of creating standards, including the role it had in creating the first national encryption standard in the 1970s — the Data Encryption Standard, known as DES. “NIST has a proud history in open cryptographic standards, beginning in the 1970s with the Data Encryption Standard,” the bulletin said. But even that early standard was influenced by the NSA.

During the development of DES, the agency insisted that the algorithm use weaker keys than originally intended — keys more susceptible to being broken by super computers. At the time, Whitfield Diffie, a digital cryptography pioneer, raised serious concerns about the keys. “The standard will have to be replaced in as few as five years,” he wrote.

The weakened keys in the standard were not changed. DES was formally withdrawn by the institute in 2005.

The announcement is the latest effort by NIST to restore the confidence of cryptographers. A representative from NIST announced in a public mailing list, also on Friday, that the institute would restore the original version of a new encryption standard, known as SHA3, that had won a recent design competition but altered by the institute after the competition ended. Cryptographers charged that NIST’s changes to the algorithm had weakened it.

The SHA3 announcement referred directly to cryptographers’ concerns. “We were and are comfortable with that version on technical grounds, but the feedback we’ve gotten indicates that a lot of the crypto community is not comfortable with it,” wrote John Kelsey, NIST’s representative. There is no evidence the NSA was involved in the decision to change the algorithm.

The reversal took Matthew Green, a cryptographer at Johns Hopkins University, by surprise. “NIST backed down! I’m not sure they would have done that a year ago,” he said.

Update: A NIST spokesperson responded on Monday afternoon (this story initially stated that NIST declined to comment).

Squarely Rooted

Nov. 4, 2013, 4:43 p.m.

This is total anecdotal hearsay, so weigh as you will, but it is fairly commonly tossed about in the DC area among folks involved in some way in this kind of scene that NIST is at least in some part a front for the NSA; specifically, that NIST is often a cover employer for the NSA and other security state agencies.

abinico warez

Nov. 4, 2013, 5:32 p.m.

If it is secure, it will not be allowed.

Its ironic that the same conservatives who worked to strengthen the NSA and to empower them to violate the privacy of average Americans will now make the case that the government they weakened can’t be trusted.  All part of making it small enough to strangle in the bathtub.  Freedom is drowning in rising tide of fear in the post 9-11 security state we call America.  Time to reign in the snoops.  .

“time to rei(g)n in the snoops..”? Really? How many deaths here on the US soil are tolerable so those who believe they have absolute privacy with their electronic communications? 3,000 per year? How much money?

The NYTs estimated 9/11 cost more than $3.3 Trillion and counting. Can we afford many more of those? And why let the terrorists of China in the ongoing cyberwar, win by failing to compete?

Most of the critics would be the first to complain the government hadn’t done enough, when the next terrorist attack hits our Homeland as it assuredly will.

Absolutely fascinating! And chilling! We all know when abuse is possible, abuse happens. Hope we get a heads up so all of us who’ve fully embraced the digital age can revert back to a more secure analogue financial existence - preferably before NSA-trained cyber criminals wipe out our bank accounts and send our life savings to untraceable offshore accounts.

On the plus side, the US post office will be regaining a lot more of their lost business, right? Gotta get me some more “forever” stamps pretty soon.

Not SeattleGuy

Nov. 5, 2013, 9:25 a.m.

SeattleGuy, you are psychologically projecting.

Not everyone feels as you claim to, that it’s worth having an agency with no oversight and no rules spy on everyone just to protect us from occasional terrorist threats. And no, when another terror attack happens, we will certainly not complain that the NSA didn’t spy on everyone enough and didn’t weaken our software security enough.

If you care so much about people’s deaths, that large NSA budget would be far better spent doing something more constructive, like dealing with the tens of thousands of automobile and tobacco and gun deaths every year. You’re far more likely to die from a car crash than a terror attack.

The terror threat is a transparent excuse for the US government to spy on its citizens, whom it sees as adversaries.

@SeattleGuy - Freedom is dangerous, but trading it for security is folly and (hopefully) downright unamerican.  A better anti-terrorist campaign might be to win hearts and minds by playing a constructive role in the world, rather than sponsoring dictators and pushing the rest of the world around.  Of course that would not be in the interests of our transnational capitalist elite, so it won’t happen, and support for the security state and the military-industrial complex will continue to be actively supported in the media and by brainwashed members of the public.  More and more, I see politicians and pundits tell us that black is white, that up is down, and that right is wrong, which increasingly suggests to me that this house of cards will collapse sooner rather than later.

@not- Where did I say I supported an “agency with no oversight?” That’s absolutely false. The NSA surveillance are probably the most regulated programs we have and they have been approved and reauthorized more than 30 times by politicians of both parties, president of both parties and several times by the SCOTUS.

Nothing could be more constructive than using modern data mining with supercomputers to identify new threats and stop them. Nothing. If there was, they’d do it. If you doubt this, please give me what you offer. I am absolutely certain of this as I was associated with the development of these technologies almost 20 years ago.

Your fears are unwarranted and illogical. The NSA accomplishes their first cut with nothing but numbers. No names or addresses or places. None! That’s remarkably clean. Those that continue to use words like “listen” or “read” or “spy” are willfully ignorant of the facts. They are not reading the targeting and minimization procedures. If they did, they would see how foolish they look to anyone paying attention.

The threat of terror is real and until something better comes along, we will continue to use all means to stop it. Turning the other cheek would be the quickest way to meet your maker.

@jimbo- There is no tradeoff. That is the beauty of the NSA’s programs. They do not need to compromise our freedom nor personal liberties to identify the bad guys. They do this with a traffic analysis with publicly-available numbers only. Now, I think that is worth continuing.

I’m all for winning hearts and minds, but just look at how effective right-wing propaganda is here and around the World. AQ sympathizers would have you believe that there are a lot of innocents killed in our drone strikes. Don’t believe them. They’re just trying to stop these very limited strikes because they are perfect for the asymmetric warfare we are now involved in in the tribal regions of Pakistan.

The same is true with the Israeli/Palestinian conflict. In the eyes of most of the World, Israel is the bad guy. They have been hoodwinked into missing the constant barrage of activities that on occasion, spur Israel to respond in a measured way. We all know that Israel could reduce their adversaries to dust in a few days if they wished, but you don’t seem to hear that in the false equivalence of the media.

I do not think we are pushing the other nations of the World around now. That should never have been our goal and, thankfully, we’ve set aside the Bush Doctrine. (I’m not sure if Palin got the memo, though.)We have moved in the direction of less adventurism as it was described under the old USSR.

I am fully behind reversing the Neocon policies that were unleashed under Bush/Cheney, but that will take time. We can’t just pull out of Iraq in a week. We just need to use our heads to do a better job of choosing real threats as Obama did when he chose to work with Russia to remove chemical weapons from the Syrian battlefield. That was a nice outcome, wasn’t it?

@SeattleGuy The cat is out of the bag now and no one is going to be convinced that spying on Angela Merkel is preventing terrorism. No, the work of the NSA is not performed with adequate oversite. Some NSA employees were caught spying on their girlfriends for heavens sake. What about corporate espionage? Why was the NSA and Canada spying on foreign energy ministries? I can’t believe this is the only case of corp. Espionage they engage in, and this kind of use of our public funds is beyond disturbing.

And your defense of drone strikes on innocents (that is doesn’t happen) needs no comment.

@chris- No one said spying on Merkel had anything to do with terrorism. Where did you get that from? As Hayden said over the weekend, the agency has always be chartered with garnering information about our allies thoughts for strategic purposes. He said they do likewise and laughed at their faux surprise.

Snowden and others have misused their access to information and should be prosecuted to the full extent of the law. I agree, but just because people break the law doesn’t mean the programs are not doing what they were intended to do. An IRS employee actually has your personal information or didn’t you know that?

RE: corporate espionage; what about China’s espionage? Shiite happens.

As I have posted here and elsewhere, the billions we spend disrupting new terrorist threats is chump change compared to what we would spend cleaning up the mess afterwards. $3.3 Trillion and counting for 9/11. How many lives are you willing to lose to satisfy your desire to have absolute privacy of your public information?

There are very few innocents killed in our drone strikes. We would kill far more with boots on the ground. Drones actually save lives. The reason there is so much noise coming from those we attack is they realize how effective these strikes are. Besides, if you are sleeping in the same room with known AQ terrorists, you are not innocent. Sorry.

@chris- RE: Corporate espionage:“Snowden couldn’t have played better into China’s strategy for protecting its cyber activities if he had been doing it on purpose,’’ one American intelligence official says.”
http://mag.newsweek.com/2013/11/01/edward-snowden-escalated-cyber-war.html

Are we really so certain Snowden was just an ideological misfit or might he have actually been working for the Chinese government? Time will tell.

@SeattleGuy, After reading your comments, I think that you are the problem. We live in a nation of sheep.

But you did support the NSA breaking the Japanese code during WWII, right? Why the big change, rick? Isn’t our nation’s sovereignty worth protecting or should we just turn the other cheek and let AQ set up shop in your town?

Who is the sheep now?

“NIST’s cryptographic standards are used by software developers around the world to protect confidential data. They are crucial ingredients for privacy on the Internet, and are designed to keep Internet users safe from being eavesdropped on when they make purchases online, pay bills or visit secure websites”


Crucial ingredients for the illusion of privacy. Because in a (globalized) post 9-11 world protecting sovereignty and freedom of speech (you know standing for what the terrorists do not) means that if you don’t have millions to turn into dark money and buy your privacy as well as influence what speech is heard and what is not, you could be helping Al Qaeda set up shop in America

NIST simply has no credibility IMO after their connections to the 911 event . Their opinion is worthless to those in the know.

This article is part of an ongoing investigation:
Surveillance

Surveillance

ProPublica investigates the threats to privacy in an era of cellphones, data mining and cyberwar.

Get Updates

Stay on top of what we’re working on by subscribing to our email digest.

optional

Our Hottest Stories

  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •