The Data Logging Cycle


Teaching: 20 min
Exercises: 0 min
  • Why is the ModularSensors library necessary, and how does it make it easier to program a data logger?

  • Learn the basic steps of a data logging cycle.

Episode 8: The Data Logging Cycle

DIYers generally find rapid success at reading data from simple sensors to an Arduino board. However, it is much more challenging to program an Arduino to perform all required functions of a solar-powered station that collects data from several research-grade environmental sensors, saves to an SD card, transmits to a public server with web services, and puts the sensors to sleep to conserve energy between logging intervals. The EnviroDIY ModularSensors library was designed coordinate these tasks.

Let’s dig in study the various steps of a data logging cycle, along with additional things a data logger needs to do.

Timing, Sleep and Wake

A datalogger needs to carefully keep track of time for a wide variety of purposes:

Take home on Timing

  • Selecting a Logging Interval is a balance between choosing a small-enough interval to capture the environmental dynamics of interest versus selecting a longer interval that minimizes your radio communication costs and preserves the solar charging capacity of your station.
  • Typical logging intervals range from 5 to 30 minutes. A 15 minute interval is most common.
  • Specifying sensor warmup, stabilization, and measurement times typically requires studying the sensor manufacturer’s specification sheet, but sometimes requires experimenting with the sensor.
  • Deciding on an optimal number of readings and/or averaging time for a sensor typically requires experimenting with the sensor.

Collecting Data from Various Sensors

A key requirement of any datalogger is to mediate all of the communications with different kinds of sensors. Data loggers are designed to potentially interface with hundreds of different sensors from dozens of manufacturers. Fortunately, there are only about a dozen communication protocols used by environmental sensors, such as:

Some of these are native to the Arduino microcontroller hardware (e.g. ADC; TTL Serial) and standard libraries (e.g. I2C), while others are enabled by widely-used community libraries (e.g. RS232 Serial, OneWire, AT). Several specialized communication protocols also exist for environmental and industrial applications (e.g. NMEA, SDI12, Modbus).

Some of these communication protocols are “smart”, in that the sensor might have an ID, or address, so that many sensors of a similar type can be connected to the same set of wires. These smart sensors often expect to receive a command before making a measurement and returning data, many of these can also be asked to return additional information, such as a serial number or calibration coefficients. Other sensors are passive, and just start return values shortly after they receive power.

Another key requirement of any data logger is to provide the appropriate power to each sensor. Most sensors have specified:

Writing Data to an SD Card

Once a datalogger had retreived data values from each of the connected sensors, it needs to write those values from short-term memory (i.e. which is erased when the microcontroller is turned off) to a form of long-term data storage (i.e. which can persist for decades even with no power). To do this, the data values need to be organized into a comma-separated string of numbers and characters, along with the date-time at which the measurements were made. For the user to make sense of the data later, the datalogger also needs to record “headers”, which are one or more special rows that contain information about the entire file and also what is found in each column, such as the sensor name, variable name and units

Figure from Jeff Horsburgh’s lecture 7 in his 2016 HydroInformatics (USU CEE-6110) class.

Transmitting Data via Radio

A wireless datalogger needs to:


Few users can setup a programable datalogger without encountering errors. The key to fixing these errors is a datalogger program that provides meaningful output describing where those errors might have occurred. Debugging information might answer such questions as:

Key Points

  • All dataloggers manage a common sequence of steps, each of which requires coordination of many individual and often complex tasks.