We’re excited to announce the latest feature enhancement to our award-winning network testing platform – Achilles. In an effort to help improve the security and robustness of emerging industrial wireless and smart grid infrastructure, we have expanded the testing capabilities of the Achilles Satellite to allow manufacturers of wireless networked devices communicating over IEEE 802.15.4 to identify cyber vulnerabilities before deploying them into our critical electric power infrastructures. Click here to read the press release. 

Wurldtech’s CEO, Tyler Williams, will be on the road for the next few weeks presenting at two high-profile industry conferences where Smart Grid security will be the primary focus.  

UTC Telecom 2009

At UTC, Mr. Williams will be delivering a presentation called “From FUD to Fact: Simple solutions to improve the security & robustness of Smart Grid infrastructure.” This will be held (6/3; 9:15 a.m. – 19:30 a.m.). With all the attention lately in the media surrounding cyber terrorism and the vulnerabilities found on the networks that control and manage global critical services such as power, energy, transportation and telecommunications, the need for more practical, data-driven solutions has never been so great. Mr. Williams will talk about how an independent group of large oil and gas companies worked together to improve the security of their industrial control systems and what key lessons the power industry can take to improve the resilience of emerging smart grid deployments.

For more information please visit: http://www.utctelecom2009.utc.org/

ConnectivityWeek

At ConnectivityWeek, Mr. Williams will participate in two panel sessions discussing cyber security issues and best practices from the control system domain and how they relate to emerging smart grid initiatives. The first session falls in the automation & IT convergence track and focuses on the cyber security issues from device integration. This will be held (6/9; 11 a.m. – 12:30 p.m.).  Mr. Williams will also participate in a panel in the cyber security track entitled, “Where should we focus our security efforts for the Smart Grid.” This will be held (6/10; 4 – 5:30 p.m.). This session will attempt to consolidate cyber security efforts from other industries, as well as current emerging smart grid initiatives, and set out recommendations as to how the community can best focus its limited resources to advance the goal of achieving assurance that security is being built into emerging smart systems.

For more information please visit: http://www.connectivityweek.com/2009/

Last week the first set of standards to be implemented by the Smart Grid were announced by NIST. Having defined standards to follow and work with is usually a good thing (unless your standard gets rejected), but standards come with more implications than most realize. In the past, electric utilities have relied on a combination of security by obscurity, and also the fact that fewer devices were networked to make cyber security a non-issue. The Smart Grid has changed all of that. Once there are clearly defined standards, the obscurity goes away. That’s not a bad thing, but it is important to recognize.  

The first step in hacking any system is always to get as much info on the system as possible. A set of standards is always a great place to start. The NIST standards include all the details on how communications should be done for each segment of the Smart Grid. They also include all the cyber security standards that will be used. 

For example: 

Say you wanted free electricity, or maybe you wanted your neighbor to pay 10 times as much, you could look at the OpenHAN, ZigBee, and ANSI C12.19 standards, and then the AMI-SEC standards to find the security that will be in place for data going back to the electric company. Or, if you wanted to target the transmission side of things, the IEEE C37.118, DNP3 and IEC 61850 standards will explain how to talk to the devices. To go with that, there’s the IEEE 1686-2007, IEC 62351, and NERC CIP 002-009 standards which each give details on the security required to be in place. Sure, there’s a lot more to hacking an electric utility than reading a standard, but there’s not much more that a hacker could ask for in terms of background information - full details how to talk to the system, and the security that you’re going to need to get past.

If there was no background information, finding a good starting point would be more difficult (though not impossible, which is why obscurity always fails). We now have a clear open list of standards (many on that list free to access, some, such as DNP3 costing only $300). Hopefully this should make it clear that obscurity can no longer even be an excuse. The IT world is well aware that if you do not do proper security testing on your applications and devices, sooner or later someone else will. The same will apply to the Smart Grid as the number of devices connected and its communication network grows.  

Including in the list of standards one that requires “cyber security” is not the complete solution either. If it was that easy, governments would’ve made the Internet a safe and secure place by now. Security is constantly mentioned as something needed for the Smart Grid, but rarely in anything more than generalities. At the same time, there is a huge push to roll out Smart Grid systems as soon as possible. Every week it seems that more networks are being deployed, more devices made. Yet we still only have generalities and wishful thinking for the security side of things. We now have the standards that will make up the smart grid, we should get to actually working on the security.

We’re excited to announce the next phase in our Achilles certification program with an agreement with MatrikonOPC to create the worlds first security certified industrial connectivity infrastructure. View the press release here.

With all the attention lately in the media surrounding cyber terrorism and the vulnerabilities found on the networks that control and manage global critical services such as power, energy, transportation and telecommunications, the need for more practical, data-driven solutions has never been so great. This announcement highlights the continued expansion and growth of our internationally-recognized certification program and further solidifies the “Achilles Certified” brand as the defacto standard for communicating industrial control infrastructure security & robustness.

With 14 systems now certified, many large end-users, such as BP and Shell, formally mandating Achilles certification on their suppliers, and an ever expanding program to meet the evolving needs of the industry, the Achilles certification program continues to be one of the most successful cyber security initiatives the industry has ever seen.

If you have any questions regarding this release please let me know.

I just returned from a meeting where I was invited to speak.  The event was called The Infrastructure Modernization Initiative: Homeland Security Implications and Challenges and was hosted by the Center for Homeland Defense and Security (CDHS https://www.chds.us/?home ) at the Naval Postgraduate School (NPS) in Monterey.  There was a multidisciplinary group representing critical infrastructure from police to fire fighters, physical security, and of course cyber security as well as control systems security.  

I would describe the overall feeling as upbeat but concerned.  They are upbeat because critical infrastructure is finally getting some badly needed attention.  The concern stems from the general lack of any provision in the stimulus package to specifically address security.  I am not suggesting that the packages in the stimulus bill are inherently insecure, but in the rush to get the economy moving we are once again relegating security to the “some day” category. 

Just yesterday the US Department of Energy announced that it would award $2 billion for the development of new battery technology to support electric and plug-in hybrid vehicles http://www.autoblog.com/2009/03/20/president-obama-announces-2-4-billion-for-electric-vehicles/ . This is just one example of the money flowing into critical infrastructure and it is obviously needed to upgrade the power grid (i.e., Smart Grid) if we all expect to drive electric vehicles in the next few years.

Will these efforts and all these billions of dollars produce the expected results? Using history as the best predictor of future events, let’s take a quick stroll down memory lane http://www.energy.gov/about/timeline1971-1980.htm .  30 years ago US President Carter dedicated millions of dollars to R&D in solar technology and oversaw the installation of solar cells at the White House.  In addition, President Carter also created an $88 billion dollar program to develop synthetic fuels.  So, where have those investments, programs, and entire departments gotten us?  Are we more energy independent or less?  Are we more secure or less?  Obviously, these are rhetorical questions.  We are MORE dependent on foreign oil than at any time in our history.  And, we are LESS secure.

When I stop to ponder these two issues I can see the relationship.  It is not owing to some cleverly crafted conspiracy.  It is a simple principle of business.  Profit.  Maximize profit.  This one single guiding principle leads us to import foreign oil when the price is low and then we have no capacity and no alternative when the price goes up.  This single principle forces us to outsource function after function to the lowest bidder, which is increasingly overseas.  And now with Internet connections available to even small town electric substations, it makes perfect business sense to connect everything to the Internet to reduce overhead costs. 

It’s time to come to grips with the notion that we simply cannot maintain our economy, our Nation, or even the continued functioning of the world without securing our critical infrastructure.

Regards,

Perry

Initially, I wondered about the value of the NERC-CIP standard (http://www.nerc.com/page.php?cid=2|20) that allowed me to opt-out by not defining any of my assets as critical cyber assets (CCA). Furthermore, defining the electronic security perimeter (ESP) is another challenge because if you touch the Internet anywhere you can be touched from anywhere. So, in some sense, you have no perimeter. In my simple way of thinking about these things it seems the only real perimeter is an air gap. However, the process is maturing and companies are developing clear, defensible, and documented processes for defining CCAs and ESPs.

Obviously, the rules changed when FERC stepped into the game (http://www.ferc.gov/news/news-releases/2008/2008-1/01-17-08-E-2.asp) and industry had no choice but to take NERC-CIP serious. Up to $1M per day is certainly a motivator and compliance is at least measurable through audits, but are the systems measurably more secure? What kind of security metrics have been collected to compare the state of the system before and after compliance? And what about the ROI of NERC-CIP? Has anyone made the argument that compliance with NERC-CIP will actually pay dividends in addition to avoiding hefty fines?

If I assume that one day even my refrigerator (along with all other “smart” appliances http://www.sciencentral.com/video/2008/10/17/smart-appliances/) will be compliant with NERC-CIP, where do we go from there to continuously improve security? The NERC-CIP is a good place to start, but for those of us in the security business, we know that security is a journey and not a destination.

- Perry

There has been a lot of traffic in the blogsphere about Smart Grid security.  In the real world, working groups are being formed, standards are being written, and there are many activities by the GridWise Architecture Council (http://www.gridwiseac.org/), NIST (http://www.nist.gov/smartgrid/) and a host of people that truly get it when it comes to security.  I think all of this work is absolutely needed and there are a lot of very smart people working on those groups/standards and they are doing a great job.

However…

We are not just talking about the designed-in functionality of all these devices and what they are “supposed” to do, we are talking about all of the ingenious things that someone with malicious intent can make them do.  In other words, many systems meet their design and functional specification, but what else are these devices capable of doing?

One of the biggest vulnerabilities discovered while I was at DHS (http://www.cnn.com/2007/US/09/26/power.at.risk/index.html#cnnSTCText) was based on a device operating within its design parameters.  There was no “bug” in the software and the device did only what it was designed to do.  The problem was nobody ever considered what bad things could be done, because the people who designed these systems are not bad guys.  Consequently, they simply didn’t think that way when the systems were designed.

So, after all the standards are met, after all the policies and regulations are complied with, somebody has to be willing to do something very unpopular (and sometimes expensive).  Take a device, a system, a process, or what have you, hand it over to a bunch of clever people and ask them one question and then get out of the way.  Ask them what if you wanted to inflict damage, do harm, or otherwise cause havoc with the system, how would you do it?

20 years ago devices were designed without much consideration for malicious hackers or criminals and here we are today doing our best to patch and mitigate vulnerabilities that in some cases were designed-in.  Smart Grid technology is already being pushed out the door and implemented in major rural and metropolitan areas across this country (http://coloradoenergynews.com/2008/11/boulders-smart-grid-project-gets-serious/) and around the world.  Whatever we install now will be with us for a very long time and we should at least be asking the right questions.

Regards,

Perry

The topic of 0Day Threat or 0Day vulnerabilities certainly get a lot of press.  And this is probably for good reason.  The common notion is that the 0Day vulnerabilities are the ones that can cause the most harm because they are the ones you’re not prepared for.  The industry has matured significantly from the days (and yes, I can actually remember those days) when hackers demonstrated their skill for bragging rights.  These days, we’ve got sophisticated markets where vulnerabilities are bought and sold and I am sure it does not surprise anybody reading this that there is also a lucrative black market for vulnerabilities every bit as pervasive as the legitimate market.

However, with all this attention on the 0Day, what about the so-called common vulnerabilities?  These are the vulnerabilities that have been captured and cataloged in the Common Vulnerabilities and Exposures (CVE) database (http://cve.mitre.org/) - as mentioned in my previous post.  The CVE database contains over 34K entries and the most interesting thing to note is that while all of these are published (in other words anybody has access to them), many devices, systems, and networks are still not protected.  So, we have many people scurrying around trying to patch the 0Day vulnerabilities, yet the vulnerabilities that are well known and published continue to cause problems.

Many of the major outbreaks in recent years were malicious code that took advantage of one or more of the vulnerabilities that should have been patched.  Many vendors are highly responsible in responding to vulnerabilities with patches, mitigation steps, or other work-arounds.  However, not every industrial control system (ICS) in use today has a vendor that’s still in business.  Sometimes the operators are on their own to implement mitigation measures until a patch or new software version becomes available maybe several months or even a year later.  This is a significant challenge for the owners and operators of ICS, managing the gap in the “find-to-fix” cycle.

What is the best way to manage the “find-to-fix” cycle?  What is the best way to handle patches?  These are very basic questions within the ICS industry that have not gone away; in fact they seem to have intensified along with the complexity and heterogeneousness of our networks. 

I can only hope that the various agencies and institutes that fund R&D in cyber security aren’t so totally fixated on the problem de jour they forget that yesterday’s problems aren’t completely solved yet. 

- Perry

As of today, the Common Vulnerabilities and Exposures (CVE) database, hosted by Mitre Corporation (http://cve.mitre.org/) for the Department of Homeland Security (DHS), contains 34,542 entries.  That may not seem like a large number, but any one of those entries can translate to multiple instances in the field.  While the contents of this database are very important in the IT world to help security practitioners ply their trade, build rule sets, etc., there is a glaring lack of information on industrial control systems (ICS).

A search of the CVE database using “SCADA” or “DCS” or “PLC” as a search term returns 9 entries, which represents about .02% of the total.  By comparison, researchers in the ICS industry suggest that a reasonable representation in CVE would be about 10%.  In other words, common ICS vulnerabilities may be underrepresented in CVE by 99.98%.  So, what does this mean to the ICS world?  It means that the current contents of the CVE are not nearly as useful as they could be to the ICS community and the critical infrastructure we are trying to protect.

So, what are the roadblocks to making this happen?  The tools are available and the standards exist to build a process to feed the CVE with ICS vulnerability information, so that isn’t the problem.  Is it the IT centric language used to describe vulnerabilities in the CVE?  Perhaps what we need is an extension to the Common Vulnerability Scoring System (CVSS - an open public standard for scoring vulnerabilities - http://www.first.org/cvss/) to accommodate the ICS specific vulnerability descriptors.  Perhaps if there were ICS appropriate CVSS extensions the CVE content would more relevant and useful.

It may be difficult to quantify the value to the ICS community and our society in expanding the content of the CVE to include ICS data, but it is easy to qualify.  Just imagine the IT world trying to provide a reasonable measure of protection without the CVE.  The economic damage that would result from the loss of the CVE would be huge and represents a common interest of everyone in society.

The value of having the CVE populated with the appropriate level of ICS vulnerability data will be immediately obvious to the owners and operators of critical infrastructure.  Likewise, it will be immediately obvious to the general public that we should somehow figure out a way to get this done and share that information.

- Perry

There seems to be a lot of buzz in the industry about the Smart Grid and the expectation is high that a boatload of money will find its way to the electrical sector for a couple of reasons.  First, the investment is sorely needed to support future growth and update an aging infrastructure; and second, such investment will provide an economic stimulus and create Green jobs that can’t be outsourced.  The logic is sound and, for all intents and purposes, seems like a good idea.

The basic notion behind the Smart Grid is simple - the Smart Grid is made up of smart appliances that can communicate their current status and their needs in addition to managing the balance between the load (consumers) and the generation (power plants).   While this traditional model is straightforward to manage, the lines between load and generation begin to blur when individuals equip their homes with wind generators or solar panels and then elect to sell electricity back to the utility companies.

Why am I concerned?  Just imagine you are the North American Electric Reliability Corporation (NERC) and you’re charged with securing the grid.  Until recently you only had to worry about the big players - the big power plants and big transmission towers and lines.  But now, instead of managing a few hundred or a few thousand points of power generation (about 1,800 registrants of the bulk power system according to a recent NERC report), somebody will have to manage many times that.

So, let’s say that within the next 10 years, 10% of the homes in the US will generate some form of electricity either from solar or wind (and we’re all hoping the number is much higher).  Based on a recent census there are about 160 million homes in the US and if 10% install solar or wind, that will translate to about 16 million discrete points on the grid capable of generating electricity.

As we continue this thought experiment we come to the real challenge.  Not that managing 16 million electrical generation sites is easy, but it is an engineering challenge that I believe can be solved, except for one little twist.  Security.

There are indeed forces out there that will be looking for any weakness, any design flaw, and any opportunity to hack into the system.  So, whatever vulnerabilities exist within our current systems, consider the fact that you have now just increased the attack surface (i.e., opportunities for hackers) by several orders of magnitude.

Can I with certainty tell you there are security problems or vulnerabilities in the so-called Smart Grid technologies?  No.  But, what I can tell you is that Wurldtech has evaluated one of the Smart Grid technologies and found issues.  Does this imply that all Smart Grid systems have problems?  Again, no.  But, as you might guess, I believe that if I run one test and find a problem, I feel confident that I didn’t stumble upon the only problem.

What is being done about it?  Scant little I’m afraid.  There is a huge push for new Smart Grid technology and governments are investing millions of dollars in R&D, and now with the US stimulus package the number could reach into the billions.  But, where is the security?  When will we find out that our security measures are too little and too late?  Why aren’t security professionals at the table when these systems are designed?  Why does it seem that security is always an afterthought?  Why aren’t the agencies that are promoting Smart Grid technology demanding security?  Sorry for all the questions, maybe you can help me with the answers.

I can only hope that we’ll find a way to thwart the bad guys before they get hold of our economic jugular.

Perry