How to create graphs with RRDTool



This article shall explain, how to use RRDTool to log values and build nice graphs to show the change of (KNX)-values over time.
I mainly use RRDTool to show weather-data, send by an ABB (Busch-Jaeger) Weather-station for
temperature [°C], wind-speed [m/s], light [lx] and rain-sensor [yes/no].
I collect these data every 5 minutes and build graphs for last 12 hours, last 24 hours, (last 3 days,) last 1 week, last month and last year.
Depending on the data, maximum, minimum and average values are aggregated. (Logging of minimum light-values makes little sense, because this is 0 lux :-)
In the picture below you can see the change of temperature over the last 24 hours. The red band shows the minimum & maximum range during 1 hour. Round about 16:00 a rain-shower started - that's the reason for the temperature change.



Needed Tools & Setup

The following components are needed:

  1. eibd

  2. linknx

  3. rrdtool

  4. Perl

  5. (apache) webserver

For me the setup of the tools was quite simple. Setup of eibd and linknx are not in scope of this article. I use eibd 0.0.4 and linknx 0.0.1.26, which I have compiled on a Buffalo Linkstation HD-HGLAN with Openlink. The sources for 0.0.1.26 I have slightly changed to catch every message (and not just the ones, for which a value changes). See this topic - but this is not relevant for the RRDTool.
My Buffalo Linkstation supports Optware. There was no need to compile RRDtool on my own.
Therefore the installation for RRDTool was just:

apt-get install rrdtool

Perl Module installation

For Perl use CPAN or another package manager to install the RRDs package. I Installed the CPAN Bundle first and installed afterwards the RRDs.
RRDs documentation can be found on the developers page.

perl -MCPAN -e shell
install Bundle::CPAN
reload cpan
install RRDs

Configuration of linknx and RRDTool

RRDtool is a "Round-Robin Database", which reads values in specific intervals and aggregates them with help of different aggregation functions. (minimum, maximum, average...)
A good (german) introduction can be found in [2].
The trick is to make the values permanently available to RRDTool for import. This is quite easy with linknx, if persistence is used.
I.e. for the object-definition, the Init Attribute is set to "persist"

<config>
    <services>
    ...
    <persistence type="file" path="/opt/var/linknx/persist" logpath="/opt/var/linknx/log"/>
    ...
    </services>
<objects>
...
<object id="Weather_Temperature" gad="10/0/86" type="9.xxx" flags="cwu" init="persist">Weather_Temperature</object>
...
</objects>
</config>

(I use my excel tool to keep track of my ~ 250 objects definitions,)

This setup write every change in a value persistently into a file. You can check this with:

cat /opt/var/linknx/persist/Weather_Temperature

Working with RRDTool

The main work consists of following steps:

  1. Define / Setup a "database"

  2. import data periodically

  3. define a layout for the graph and periodically create a new graph.

Definition of a database

RRD stands for Round-Robin-Database. In difference to a "regular" SQL-database, data is written in defined intervalls and after a configurable timeframe previous values are overwritten. In most cases, data is not just overwritten but an aggregation function is used to combine newest value with the previous ones. Aggregation functions are: MAX - maximum value is kept, MIN..., AVERAGE - average value is calculated.

To define a database you need
- a filename
- start-time (the time of the first value)
- a base intervall in seconds (with which data will be fed into the DB)
- one or multiple DATASOURCES, with name and Type and optional "heartbeat" intervall and optional min / max values.
- one or multiple RRAs (Round Robin Archives), which define, which data value are kept and when statistics are calculated.
The detailed definition of the parameters (options) can be found in [1]

in PERL, the definition of DB for the temperature would look like this: (I adapted the sample code in [2])

my $DB = "/opt/var/rrdtool/temp.rrd";
if(! -f $DB) {
RRDs::create($DB, "--step=300", # create DB, data intervall is 5 minutes
"DS:temp:GAUGE:600:U:U", # store values (and not deltas), at max 10min time-intervalls, calulate lower an upper boundaries
"RRA:LAST:0.5:1:288", # store "last" value (i.e overwrite), every 5 minutes (i.e 1 intervall) and remember 288 * 5 min = 24h,
"RRA:LAST:0.5:1:2016", # same for 1 week (used this for tests)
"RRA:MAX:0.5:12:168", # calculate a MAXIMUM for the 5min * 12 (=1h) and remember values for 5min * 12 * 168 = 7 days
"RRA:MAX:0.5:288:365", # calculate a MAXIMUM for the 5min * 288 (=24h) and remember values for 5min * 288 * 365 = 1 year
"RRA:AVERAGE:0.5:12:168", # calculate an AVERGAE for the 5min * 12 (=1h) and remember values for 5min * 12 * 168 = 7 days
"RRA:AVERAGE:0.5:288:365", # calculate an AVERAGE for the 5min * 288 (=24h) and remember values for 5min * 288 * 365 = 1 year
"RRA:MIN:0.5:12:168", # same for MINIMUM
"RRA:MIN:0.5:288:365" # same for MINIMUM
) or die "Create error: ($RRDs::error)";
}

what the 0.5 - xfiles factor does - please check the documentation ;-). This is used for interpolation - but is not relevant for our case.

Filling the DB with data

I use a cronjob to execute the perl script every 5 minutes.
In this intervall, data is read from the "persistent" file and fed into the RRA.
in perl this looks like this:

my $INFILE = "cat /opt/var/linknx/persist/Weather_Temperature";

my $value = `$INFILE`;
my ($temp) =($value);
# my ($temp) =($value=~ /(\d+.\d+)/); # could be used for format conversion
RRDs::update($DB, time() . ":$temp") or
die "Update error: ($RRDs::error)";

the only task left is creating the Graph(s)

Create Graphs

RRD has a HUGE amount of options for graphs. Please see [1] for the details and the samples on the page.
The graph above was created with the following script:

my $PIC = "/opt/www/rrd";

RRDs::graph("$PIC/temp-d.png", # create a png in an apaches www-directory
"--vertical-label=Temperatur", # vertical label
"--start","now-1d", # start-date for the graph is 1 day ago
"--end", "now", # end now
"--width","720","--height","200", # size of the graph
"--x-grid", "MINUTE:10:HOUR:1:HOUR:2:0:%H:00", # show line every 10minutes, major line every hour, name every 2hours in the middle
"DEF:mytemp1=$DB:temp:MIN", # minimum temperature
"DEF:mytemp2=$DB:temp:LAST", # actual
"DEF:mytemp3=$DB:temp:MAX", # maximum
"VDEF:tempmax=mytemp3,MAXIMUM", # calculated maximum for the whole period
"VDEF:tempavg=mytemp2,AVERAGE", # calculated average
"VDEF:tempmin=mytemp1,MINIMUM", # calculated minimum
"VDEF:templast=mytemp2,LAST", # last value
"AREA:mytemp3#FF000044:Maximum\\t", # red area for the maximum in the background
"GPRINT:tempmax:%2.1lf C\\n", # show value in the
"AREA:mytemp1#FFFFFF", # overlay with the minimum values (this creates the min-max-bands)
"LINE1:tempmin#0000FF:Minimum\\t", # show blue minimum line
"GPRINT:tempmin:%2.1lf C\\n",
"LINE1:tempavg#00FF00:Average\\t", # show green average line
"GPRINT:tempavg:%2.1lf C\\n",
"LINE1:mytemp2#000000:Actual\\t", # show black values line
"GPRINT:templast:%2.1lf C\\n",
"LINE1:tempmax#FF0000",
"LINE1:tempavg#00FF00",
) or
die "graph failed ($RRDs::error)";

All the code is in one script, which can be called with 2 parameters: "u" - for "update" and "g" for "graph".
I have it entered in a crontab as follows:

*/5 * * * * /opt/share/rrdtool/rrdtemp -ug

The overall creation and update scripts for the temperature would look like this:

#!/usr/bin/perl
###########################################
# JH, rrdtemp script to create eib/knx-log graphs
# based on script by Mike Schilli, 2004 ([[mailto:m@perlmeister.com|m@perlmeister.com]])
###########################################
use warnings;
use strict;
use lib "/opt/lib/perl5/site_perl/5.8.8/arm-linux";
use RRDs;
use Getopt::Std;
my %opt;

getopts('ugf', \%opt);

my $DB = "/opt/var/rrdtool/temp.rrd";
my $PIC = "/opt/www/rrd";

my $INFILE = "cat /opt/var/linknx/persist/Weather_Temperature";
if(! -f $DB) {
RRDs::create($DB, "--step=300",
"DS:temp:GAUGE:600:U:U",
"RRA:LAST:0.5:1:288",
"RRA:LAST:0.5:1:2016",
"RRA:MAX:0.5:12:168",
"RRA:MAX:0.5:288:365",
"RRA:AVERAGE:0.5:12:168",
"RRA:AVERAGE:0.5:288:365",
"RRA:MIN:0.5:12:168",
"RRA:MIN:0.5:288:365"
) or die "Create error: ($RRDs::error)";
}

if(exists $opt{u}) {
my $value = `$INFILE`;
my ($temp) =($value);
print $temp;
RRDs::update($DB, time() . ":$temp") or
die "Update error: ($RRDs::error)";
}

if(exists $opt{g}) {
RRDs::graph("$PIC/temp-d.png",
"--vertical-label=Temperatur",
"--start","now-1d",
"--end", "now",
"--width","720","--height","200",
"--x-grid", "MINUTE:10:HOUR:1:HOUR:2:0:%H:00",
"DEF:mytemp1=$DB:temp:MIN",
"DEF:mytemp2=$DB:temp:LAST",
"DEF:mytemp3=$DB:temp:MAX",
"VDEF:tempmax=mytemp3,MAXIMUM",
"VDEF:tempavg=mytemp2,AVERAGE",
"VDEF:tempmin=mytemp1,MINIMUM",
"VDEF:templast=mytemp2,LAST",
"AREA:mytemp3#FF000044:Maximum\\t",
"GPRINT:tempmax:%2.1lf C\\n",
"AREA:mytemp1#FFFFFF",
"LINE1:tempmin#0000FF:Minimum\\t",
"GPRINT:tempmin:%2.1lf C\\n",
"LINE1:tempavg#00FF00:Average\\t",
"GPRINT:tempavg:%2.1lf C\\n",
"LINE1:mytemp2#000000:Actual\\t",
"GPRINT:templast:%2.1lf C\\n",
"LINE1:tempmax#FF0000",
"LINE1:tempavg#00FF00",
) or
die "graph failed ($RRDs::error)";
}

Quelle: 
http://sourceforge.net/apps/mediawiki/linknx/index.php?title=How_to_create_graphs_with_RRDTool