Why monitor your data centre environment?

Before environment monitors are introduced in detail it makes sense to understand what motivations you may have for monitoring your environment in the first place. Plainly, your requirements direct the equipment you will need.

Alert on heat emergency

Probably the biggest motivation for monitoring the data centre environment is the anxiety many sys admins feel that a heat emergency will happen. I'm sure you dread entering your data centre after a pleasent weekend at home only to find it in the midst of a severe heat emergency.

Plainly, in order to detect a heat emergency you need to be actively monitoring your data centre. The primary cause of heat emergencies is the operation of the air conditioner. If one or more air conditioners fail then the heat generated by the servers will not be removed from the data centre. Data centres are sealed environments so when cold air is not introduced into the cold aisles and hot air is not expelled from the hot aisles, the servers will be recycling air that is already hot.

The quickest way to detect air conditioner failure is by monitoring the air conditioning system itself. However, if you can't monitor the air conditioning system directly, then monitoring the temperature is a good proxy.

Monitoring air conditioner health

It is arguable that the air conditioner is the single most important piece of equipment in your data centre. Without modern air conditioning systems the data centre would not exist in its current form.

Air conditioning plays a central role in the successful operation of the data centre, consequently, air conditioning systems require careful monitoring. Unfortunately, given the amount of heat generated by modern servers, you won't have very long to react to an air conditioner failure. So the earlier you are able to detect the failure the more time you will have to mitigate the effects.

The fastest way to detect air conditioning system failure is to monitor the unit's operational status dry contact point. The main benefit of using the operational status of the air conditioner is that you are informed immediately of system failure rather than waiting for the temperature to rise above your alert threshold.

Ensure you aren't wasting energy

Whilst a major concern with the data centre environment is the air being too hot, equally you could be wasting money by running the data centre too cold. If you place a bunch of data centre admins in a room each one will have their own idea as to what the ideal temperature should be. ASHRAE, the American Society of Heating, Refrigeration and Air-Conditioning Engineers, in their booklet Thermal Guidelines for Data Processing Environments recommends a temperature between 18-27°C.

If the temperature inside your data centre is lower than the ASHRAE recommendation, ie below 18°C, then you may well be over cooling and consequently consuming more electricity than is necessary. Of course, you can only find out precisely what the temperature is by actively monitoring the environment.

Find hotspots

In (very) small server room environments, without hot/cold aisle configuration, the variability of air temperature is likely to be minimal. In more structured environments the temperature variability is likely to be much higher. In addition, the likelihood of hotspots developing is much higher too. A hotspot is a location where the air being fed into one or more devices is too hot.

One of the reasons to monitor the environment is to detect hotspots within the data centre. Hotspots may be very localised and could only affect a few servers. Consequently they can be hard to locate. One tell tale sign may be one or more servers manifesting component failure. Hard disks are especially prone to heat issues. If you begin to see patterns of component failure that may be indicative of a hotspot. If you find servers are shutting themselves down for no apparent reason you may find that the servers thermal shutdown is being invoked.

Hotspots are most likely to be created during data centre re-organisations when equipment is re-arranged. Consequently, it can be helpful to provide high sensor density monitoring after the move to ensure that no hotspots have been created. When you have established that the move has not adversely affected the environment, move back to a more normal level of sensor density.

ISO 9001 Mandate

Many organisations have implemented the ISO 9001 family of standards in order to improve their product and service delivery. The operation of I.T. systems is of critical importance to product or service delivery in many organisations. It would be sensible to place the operation of the I.T. systems within the scope of the quality system.

ISO 9001 mandates that the performance of processes critical to your ability to implement a high quality product and service delivery should be specified, monitored and verified. Consequently, you need to specify in your policy document what are acceptable environmental conditions for your data centre. In addition, you need to maintain a continuous log of the environmental conditions so you can demonstrate that you have maintained the environment as per your specification in your policy document during a quality system audit.

Unfortunately, things don't stop at providing an auditable log. You also need to verify that the equipment measuring the environment is working as expected. Typically you will need to calibrate your measuring equipment every year to ensure that it is working within the levels of accuracy you have specified in your quality document. Calibration can usually be performed on site in order minimise the downtime of your monitoring system.

Your calibration provider is usually certified by an acreditation body such as UKAS in the United Kingdom and NIST in the U.S.A.

Conclusion

Whilst there are many reasons for performing environmental monitoring in your data centre there is a lot of overlap. If you have a mandate for monitoring then your data centre then you may as well do the job right and find hotspots and ensure that you're not wasting energy at the same time. None of the different motivations outlined above are mutually exclusive, you can do most of the above quite easily and at relatively modest cost.

When choosing a solution it is helpful to understand what your requirements so that you don't find yourself spending more money than you need to.

Environment Monitor Introduction

Environment monitors are available with widely differing capabilities. This section outlines the main features and provides a summary table matching the features to the specific make and model of monitor.

Integrated website

All network enabled environment monitors have an integrated website. The capabilities of the website tend to reflect the underlying features of the environment monitor.

Integrated websites are particularly helpful for managing remote locations. You can place the monitor anywhere on your network and still be able to access it as if it was in the same room.

Environment monitors with integrated websites also mean that you don’t need to install third party applications, you only need a standard web browser.

SNMP

The Simple Network Management Protocol (SNMP) is a standard for managing network enabled devices.

Any environment monitor supporting SNMP can be integrated into a wide range of network management software like HP OpenView, SolarWinds Orion or Nagios.

Log readings

Whilst the primary purpose of most environment monitors is to alert you to unusual environmental conditions, retaining a log of your environmental conditions can be very useful for future planning.

Once the data has been logged you can export the data from the environment monitor using CSV. You can then visualise the data using a tool such as Microsoft Excel or Open Office.

A number of environment monitors can log the environment data as well as graph the information in a highly accessible way. Whilst a data log is very useful, nothing beats a graph for quick and easy comprehension.

Graph

A number of environment monitors can log the environment data as well as graph the information in a highly accessible way. Whilst a data log is very useful, nothing beats a graph for quick and easy comprehension.

Server Shutdown

Whilst a number of environment monitors support sensor controlled relays, able to switch on and off a power supply to a device, few are able to safely shut down servers.

The main drawback of a sensor controlled relay when shutting down a none-abended server is that the server may not be in a state to safely shut down without data loss. By using a safe shutdown the server goes through its normal shutdown procedure that ensures that data is properly written to disk.

A relay can be very useful when a server abend has occurred. An abended server is unlikely to respond to a normal shutdown request.

Models

Sensatronics E4/E16

The Sensatronics E series devices are able to measure temperature only. The Sensatronics E4 can monitor up to 4 temperature probes. The Sensatronics E16 can monitor up to 16 temperature probes.

The temperature probes are available in regular and heavy duty varieties and in a variety of lengths up to a maximum of 2500 feet.

The regular temperature probes are ideal for data centre use in environments with a minimum of electrical interference.

The heavy duty probes have a stainless steel tip with shielded cable and can be used in more electrically noisy environments and outdoors. In fact, one customer used a heavy duty probe down a bore hole 500 metres into the earth!

The Sensatronics E series does not record a temperature log so it is usually combined with third party software, like Intellipool, to provide logging, graphing and alerting functionality.

The unit supports SNMP and is able to generate SNMP traps when a temperature threshold is exceeded.

Sensatronics EM1

The Sensatronics EM1 is an easy to use, fully scaleable environmental monitor that connects directly to your organisation's I.T. network. Probes collect temperature, humidity and/or wetness data and report back to the monitor.

The EM1 unit can support up to 4 temperature or temperature/humidity probes and 4 wetness probes.

The Sensatronics EM1 does not record an environmental log so it is usually combined with third party software, like Intellipool, to provide logging, graphing and alerting functionality.

The unit supports SNMP and is able to generate SNMP traps when an environmental threshold is exceeded.

Sensatronics Senturion

The Senturion is a plug and play 19” rack mount environment monitor. Supplied with built-in temperature, humidity and light sensors. The Senturion can take up to 8 additional probes.

The unit supports SNMP and is able to generate SNMP traps when an environmental threshold is exceeded.

AKCP sensorProbe2/8

The AKCP sensorProbe range provide high quality, fully featured environment monitors with a vast range of probes.

The sensorProbe2 supports up to two external probes. The sensorProbe8 supports up to eight external probes.

The sensorProbe range are highly functional devices that are able to log and graph environmental readings as well as alarm when things go wrong.

All probes are auto-sensed by the environment monitor and use CAT5 cable so that you can use your existing structured cabling to extend the range of your monitoring.

The sensorProbe8 is also available with either 20 or 60 extra dry contact ports.

AKCP securityProbe8

The AKCP securityProbe is a high quality, fully featured environment monitors with a vast range of probes.

The securityProbe8 supports up to eight external probes.

The securityProbe8 range are highly functional devices that are able to log and graph environmental readings as well as alarm when things go wrong.

The securityProbe8 supports the addition of a dedicated SMS modem for integrated SMS and MMS alerting.

The securityProbe8 also supports up to four additional cameras so can function as a security solution for your data centre.

All probes are auto-sensed by the environment monitor and use CAT5 cable so that you can use your existing structured cabling to extend the range of your monitoring.

The securityProbe8 is also available with either 20 or 60 extra dry contact ports.

Summary

For a simple and easy way to choose an environment monitor that fits your requirements, please go to the Environment Monitor Product Finder.

[1] The AKCP sensorProbe2 logs probe readings but does not store them in permanent memory. A device restart will lose all logged readings. The sensorProbe8 does log  the readings into permanent memory

Rack ’n Room: What’s the difference?

A data centre is usually made up of a number of cabinets.

Typically the cabinets are aligned in hot and cold aisles. The cabinet inlet air is taken from the cold aisle and the hot air is expelled into the hot aisle. Air is then taken from the hot aisle and cooled by the air conditioner. The cool air is then injected back into the cold aisle again.

Any problems with the air conditioning are first going to manifest in the cold aisle. You should start your monitoring efforts by recording the ambient temperature in your cold aisle(s).

Effectively, the only difference between a rack and room environment monitor is the form factor.

Rack-mount monitors are very convenient for cabinets. The AKCP sensorProbe8 or securityProbe8 and Sensatronics Senturion can often be fitted into a rack “zero U” because of their thin design, it can fit behind other equipment taking up space that is otherwise unusable.

A room monitor, like the AKCP sensorProbe2, EM1 or E4, can be placed into a rack though it is more difficult to effect a tidy install.

Both rack and room monitors support external probes. So, they are very flexible in where you can place your probes.

A rack monitor can monitor a room and a room monitor can monitor a rack. Precisely how you lay out the probes then becomes a matter of your own convenience.

Is your air-conditioner the single most critical piece of kit in your data centre & you don’t control it?

You are not alone. It is quite common for IT staff to have very little control over their data centre or server room air conditioning systems.

Facilities people often don’t feel they need to know the day to day operation of their air conditioning systems. Or if they do, they aren’t guaranteed to communicate system failures to IT people in a timely manner.

This can cause considerable problems.

If the temperature in the server room is going to go very high then you’re going to need to take steps to mitigate any possible damage to your equipment.

At worst, you may have to switch off some or all of your equipment in order to prevent possible long term damage.

Data centre environments can be quite worker hostile places. We often hear horror stories from companies when people switch off the air-conditioning in order to work in the data centre. Unfortunately, workers sometimes forget to switch the air-conditioning system back on again. The server room environment then invariably starts to warm up, sometimes to very high temperatures.

How can environment monitoring help?

By installing your own monitoring equipment you don’t need to rely on anybody else informing you of air-con failures. You find out first!

You can inform the duty IT staff of the problem and they can take action to resolve the problem.

But the benefits don’t stop at that. By monitoring the environment in the data centre you can record a historical trend of your data 24/7/365.

You can tell whether the environment, even when the air-conditioning is working properly, is within the range you expect. You may be keeping the environment too cold, in which case you can save money on your electricity bills. If the environment is too hot you can take steps to cool things down.

Historical information allows you to examine your data for signs of longer term trends.

A gradual rise in temperature may indicate that your air conditioning system is unable to cope with a rising temperature load. Typically, air conditioner upgrade cycles are longer than server upgrade cycles.

As many data centre managers will attest, servers aren’t getting any cooler.

How to monitor your air-conditioning system

There are two basic strategies you can use. You can either monitor your air conditioners themselves, or monitor the environment the air conditioners condition or preferably, both.

Both options are discussed below in more detail.

Monitor the air-conditioner itself

Monitoring the air conditioner is the obvious thing to do. The air conditioner knows exactly what’s going on, so it should be able to tell you when things are going wrong.

One of the problems with monitoring the air conditioner is that there isn’t a single simple way of doing it.

Air conditioners in use in your data centre can go from the very simple all the way up to very sophisticated network enabled devices.

Where your air conditioner is on the scale of sophistication outlined above will indicate how you should go about monitoring.

Many air conditioners have a simple relay switch. When the air conditioner unit has a fault, or is switched off the relay changes state. The relay can be monitored using an environment monitor to detect when the relay changes state.

You can then notify appropriate people of the air conditioner failure.

More sophisticated (read expensive) air conditioners are network enabled and support SNMP.

SNMP is a widely used network management protocol supported by a vast array of network management software.

If you already have a network management system in operation you can integrate your air conditioning system alerting into your existing SNMP polling and alarming architecture.

Your SNMP enabled air conditioning system may well be able to pro-actively send you SNMP based alerts. The alerts come in the form of SNMP traps.

A normal SNMP monitoring system polls a device for information. If the data is found to be out of a normal range the network management system can then raise an alarm.

With SNMP traps, the device itself decides (according to your configuration) when an alarm condition has been reached. It then sends a trap to the configured trap server. The trap server then receives the trap and raises an alarm.

Trap servers can be very sophisticated event correlation engines or they can be very simple. Most network management or network monitoring tools have a built in trap server.

Monitor the environment

Even if you can’t monitor your air conditioning unit directly, you can always monitor the effect the air conditioner has on your server room environment.

In fact, monitoring your server room environment is likely to be something you want to do anyway.

Monitoring your server room environment won’t detect air conditioner failure as quickly as a direct air conditioner monitor.

The environment in your data centre is going to take at least a few minutes to warm up sufficiently to set off your environmental alarm system.

Why? Two reasons.

Firstly, the air making up the environment in your data centre has to be heated up. Heating the volume of air in your data centre is going to take time.

Secondly, you are going to wish to avoid false alarms. So, you need to set the upper temperature band above the normal every day fluctuating level in your data centre. In order for an alarm to be generated, the temperature needs to go above your upper temperature threshold.

The trick is to place the upper threshold low enough so that you have plenty of time to respond to a problem, yet high enough to avoid false alarms. Each data centre is different so you need to find the level that is right for you.

You can find the appropriate higher temperature threshold through experimentation and reviewing your historical data for trending information.

Unfortunately, trending information takes a while to collect. I would suggest at least a week. So you may need to tweak your upper threshold from time to time as your trending information slowly builds up.

Open protocols set you free

One of the sneakier tricks manufacturers sometimes try to pull off is keeping your data proprietary.

That way, if you want to access your information you need to buy software that is available exclusively from the manufacturer. The software usually comes with a hefty price tag and usually doesn’t have the features you want.

Make sure that any environment monitoring equipment you buy has a set of publicly documented standard interfaces from which you can access your data.

The data needs to be in a format that can be converted into a form that can be easily manipulated, visualised and reported. Formats like Microsoft Excel, CSV and the like should be supported.

The ultimate interoperable protocol in network management is, of course, SNMP. More about SNMP in the next section.

What is SNMP and why do I need it?

An introduction to SNMP

Most networks of any size and age have lots and lots of equipment from many different manufacturers.

If each manufacturer implemented their own management systems, managing the network would quickly become a nightmare. Each manufacturer’s equipment would need its own management system. None of the different manufacturer’s management systems would talk to each other.

All told an expensive, inflexible mess.

To that end the manufacturers agreed to create a single management protocol called Simple Network Management Protocol or SNMP that they could all implement on their devices.

You would then only need a single SNMP based management system to manage your network. The management system would then be able to talk to all of the devices on a network from any compliant manufacturer.

What has SNMP got to do with environment monitoring?

If you wish to create a point tool for your environment monitoring without any integration into your existing network management system then you don’t need to worry about SNMP.

However, if you wish to integrate your environmental monitoring efforts into your existing network management system then SNMP support is essential.

SNMP is a great integration and interoperability protocol.

SNMP enables your environmental monitoring devices to integrate into your existing network management system.

Your network management system can be anything from a Big 4 enterprise solution like HP OpenView or IBM Tivoli, a mid market solution like Ipswitch What’s Up Gold and Solar Winds Orion or open source tools like MRTG and Nagios.

All of the above mentioned tools use SNMP to fetch performance information. The systems then, to various levels of sophistication, log, trend and correlate the information.

Environment monitor alarms

Network management systems typically have highly functional alarm sub-systems. At the very least you should be able to create alarms that respond differently at different times of day.

During the working week when you have IT people on site, you will probably want a different set of actions than at night and weekends.

Many larger data centres have security people on site at all times. When problems crop up out of hours you are likely to wish to inform them of a problem so that they can then take appropriate steps, like ringing the duty IT person for instance. During the day it is unlikely you’d want the security people to be informed. It is more likely that an email or text to your IT staff would be more appropriate.

One of the problems with integrating your environment monitor with a network management system is that you are required to do the integration yourself.

Some network management systems have plug-ins which greatly speed up the process. But where a plug-in is not available, you are likely to require an understanding of SNMP. Most SNMP devices do come with a MIB which does make integration much easier.

Don’t forget the data

Most of you are motivated principally by a desire to be informed of extreme environmental events in your data centre.

One area you shouldn’t forget about is logging your environmental data.

Fortunately, it is very easy to do. In fact, a system set up for performing alarms will give you the logging for free. Nor is the logging functionality restricted to a high end monitoring system, all of our solutions provide both alarming and logging.

Why is having a record of your environmental readings important?

Data centres are dynamic places. Equipment is being installed on a regular basis to cope with changing business requirements.

By monitoring your server room environment you can see how changes are affecting your environment.

By spotting trends early, you can plan future infrastructure upgrades in good time, before the server room environment becomes a problem.

It’s the servers, stupid

Data centres are designed for one purpose: to house, in a safe environment, servers of various kinds.

Whilst monitoring your ambient data centre environment will cover you against air conditioner failure events, localised problems may not be detected.

A number of solutions are available to help minimise the chances of you missing hotspots in your data centre.

You can locate a number of temperature probes around your data centre, paying particular attention to the cold air inlets to your cabinets. If the air going into your cabinet is warm it will only become warmer the further up the cabinet it travels. Equipment higher up the cabinet may well be drawing much hotter air than the ambient temperature of the data centre as a whole.

An alternative is to monitor the CPU temperature of your data centre equipment. Most modern servers, switches and routers have sophisticated instrumentation built in.

Not only can you monitor CPU temperature, fan speed is often available too.

The bad news is that information isn’t always easy to access. Typically, you need to integrate the information into your network monitoring tool.

The information can be accessed using either SNMP and, on Microsoft Windows, via WMI. See your equipment vendor for further information.

Appendix A: Terminology

You are going to come up against a bit of terminology in this Buyer’s Guide so you may as well know exactly what it all means. If you are experienced in the field, you may want to skip forward, I doubt there’ll be anything new for you here.

Environment monitor

A system for monitoring the environment in a building. Typically the monitoring system runs 24/7/365 in areas of a building that are sensitive to heat, like a server room.

An environment monitor can be very simple, measuring temperature in one location or very complex, measuring multiple environmental values in many locations.

Temperature monitor

An environment monitor restricted solely to monitoring temperature.

Probe

Probes are the mechanism by which the monitor measures its environment.

Probes of various types are available, including temperature, relative humidity, wetness, door contact and dry contact.

Some probes are built  into the environment monitor, in the case of the Senturion. Most probes are external and attach to the environment monitor.

Probes are supplied in a variety of lengths to give you maximum flexibility in where you place your monitor.

Sensor

See environment monitor.

SNMP

Taken from the SNMP entry at Wikipedia:

“The Simple Network Management Protocol (SNMP) forms part of the internet protocol suite as defined by the Internet Engineering Task Force (IETF). SNMP is used in network management systems to monitor network-attached devices for conditions that warrant administrative attention. It consists of a set of standardsApplication Layer protocol, a database schema, and a set of data objects.”

SNMP provides a mechanism for reading network related instrumentation from any supporting device irrespective of manufacturer.

Network management

Taken from the network management entry at Wikipedia:

“Network management refers to the activities, methods, procedures, and tools that pertain to the operation, administration, maintenance, and provisioning of networked systems.”

The term network management has become a little corrupted. It is often used as short hand to refer to network & system monitoring.

A network management system is a tool to help you manage your network. It can range from a very simple router link trending point tool all the way up to Big 4 enterprise level management platform.

Data centre

Taken from the data center entry at Wikipedia:

“A data center is a facility used to house computer systems and associated components, such as telecommunications and storage systems.”

Server room

Taken from the server room entry at Wikipedia:

“Server room is the name of a room, where mostly computer servers are housed. In Information Technology circles the term server room, is generally used for smaller setups. As the number of computer servers grow, they may be housed in data centers.”

Computer room

See data centre.