NetAtmo Programming Synology

Export NetAtmo weather station data to CSV / Excel

I’m a proud owner of a NetAtmo weather station for a couple of months and I think it’s the best wheater station with a great user experience you can get:

NetAtmo_Dashboard

Some days ago I noticed the option in the web dashboard to export all my weather data into a comma-separated (CSV) or Microsoft Excel file. That’s a great feature for archiving your measured data….but wait – how could this be done in an automated way?

NetAtmo_Export

The short answer is: yes – but unfortunately not out-of-the-box. NetAtmo offers a great RESTful API for accessing the weather station data, so I started to do some coding with their web API. But I had no luck – there isn’t a method available for exporting the data. So I did a manual download and created some http/https traces to see what’s going on there. I simply put together the required http-calls and made a shell script that will do the job for me:

#!/bin/bash

getmeasurecsv() {
	# ------------------------------------------------------
	# Help
	# ------------------------------------------------------
	# usage: getmeasurecsv <USER> <PASSWORD> <DEVICE_ID> <MODULE_ID> <TYPE> <STARTDATE> <ENDDATE> <FORMAT>
	#
	# USER + PASSWORD -> your NetAtmo Website login
	# DEVICE_ID -> Base Station ID
	# MODULE_ID -> Module ID
	# TYPE -> Comma-separated list of sensors (Temperature,Humidity,etc.)
	# STARTDATE -> Begin export date in format YYYY-mm-dd HH:MM
	# ENDDATE -> End export date in format YYYY-mm-dd HH:MM
	# FORMAT -> csv or xls

	# ------------------------------------------------------
	# Parsing Arguments
	# ------------------------------------------------------
	USER=$1
	PASS=$2

	DEVICE_ID=$3
	MODULE_ID=$4
	TYPE=$5
	DATETIMEBEGIN=$6
	DATETIMEEND=$7
	FORMAT=$8

	# ------------------------------------------------------
	# Define some constants
	# ------------------------------------------------------
	URL_LOGIN="https://auth.netatmo.com/de-DE/access/login"
	API_GETMEASURECSV="https://my.netatmo.com/api/getmeasurecsv"
	SESSION_COOKIE="cookie_sess.txt"
	AUTH_COOKIE="cookie_auth.txt"


	# ------------------------------------------------------
	# Convert start and end date to timestamp
	# ------------------------------------------------------
	DATEBEGIN="$(date --date="$DATETIMEBEGIN" "+%d.%m.%Y")"
	TIMEBEGIN="$(date --date="$DATETIMEBEGIN" "+%H:%M")"
	DATE_BEGIN="$(date --date="$DATETIMEBEGIN" "+%s")"
	DATEEND="$(date --date="$DATETIMEEND" "+%d.%m.%Y")"
	TIMEEND="$(date --date="$DATETIMEEND" "+%H:%M")"
	DATE_END="$(date --date="$DATETIMEEND" "+%s")"


	# ------------------------------------------------------
	# URL encode the user entered parameters
	# ------------------------------------------------------
	USER="$(urlencode $USER)"
	PASS="$(urlencode $PASS)"
	DEVICE_ID="$(urlencode $DEVICE_ID)"
	MODULE_ID="$(urlencode $MODULE_ID)"
	TYPE="$(urlencode $TYPE)"
	DATEBEGIN="$(urlencode $DATEBEGIN)"
	TIMEBEGIN="$(urlencode $TIMEBEGIN)"
	DATEEND="$(urlencode $DATEEND)"
	TIMEEND="$(urlencode $TIMEEND)"
	FORMAT="$(urlencode $FORMAT)"


	# ------------------------------------------------------
	# Now let's fetch the data
	# ------------------------------------------------------

	# first we need to get a valid session cookie
	curl --silent -c $AUTH_COOKIE $URL_LOGIN  > /dev/null

	# then extract the ID from the authentication cookie
	SESS_ID="$(cat $AUTH_COOKIE | grep netatmocomci_csrf_cookie_na | cut -f7)"

	# and now we can login using cookie, id, user and password
	curl --silent -d "ci_csrf_netatmo=$SESS_ID&mail=$USER&pass=$PASS&log_submit=LOGIN" -b $AUTH_COOKIE -c $SESSION_COOKIE  $URL_LOGIN > /dev/null

	# next we extract the access_token from the session cookie
	ACCESS_TOKEN="$(cat $SESSION_COOKIE | grep netatmocomaccess_token | cut -f7)"

	# build the POST data
	PARAM="access_token=$ACCESS_TOKEN&device_id=$DEVICE_ID&type=$TYPE&module_id=$MODULE_ID&scale=max&format=$FORMAT&datebegin=$DATEBEGIN&timebegin=$TIMEBEGIN&dateend=$DATEEND&timeend=$TIMEEND&date_begin=$DATE_BEGIN&date_end=$DATE_END"

	# now download data as csv
	curl -d $PARAM $API_GETMEASURECSV

	# clean up
	rm $SESSION_COOKIE
	rm $AUTH_COOKIE
}

#____________________________________________________________________________________________________________________________________

urlencode() {
	# ------------------------------------------------------
	# urlencode function from mrubin
	# https://gist.github.com/mrubin
	#
	# usage: urlencode <string>
	# ------------------------------------------------------
	local length="${#1}"

	for (( i = 0; i < length; i++ )); do
		local c="${1:i:1}"

		case $c in [a-zA-Z0-9.~_-])
			printf "$c" ;;
			*) printf '%%%02X' "'$c"
			esac
	done
}

#____________________________________________________________________________________________________________________________________

getmeasurecsv "user@email.com" "mySecretPassword" "12:23:45:56:78:33" "02:00:00:12:23:45" "Temperature,Humidity" "2015-05-17 10:00:00" "2015-05-18 12:00:00" "csv"

You just need to modify the last line to fit your environment. Here are some detailed information regarding the DEVICE_ID and the MODULE_ID. Both IDs can be found in your NetAtmo Dashboard settings. The DEVICE_ID will be the MAC-Address of your NetAtmo Base Station. The MODULE_ID is based on the serial number of your additional NetAtmo Modules:

  • Serial numbers starting with iXXXXXX are indoor modules – the MODULE_ID will be: 03:00:00:XX:XX:XX (e.g. i1245E4 – 03:00:00:12:45:E4)
  • Serial numbers starting with hXXXXXX are outdoor modules – the MODULE_ID will be: 02:00:00:XX:XX:XX (e.g. h76A123 – 02:00:00:76:A1:23)
  • When querying the base station the DEVICE_ID and MODULE_ID value must be the same – the MAC-Address of the base station. The Output in the CSV file will look like this:

    NetAtmo_shell

    I duplicated the last line for each of my NetAtmo modules and configured the script to run on my Synology DiskStation NAS for an automatic backup.

      UPDATE 2017:

    See my recent blog post Netatmo Weather Station Shell Script

    You may also like
    Read NetAtmo weather station data via Script
    Synology DS Photo+ DLNA Mirroring Reverse Engineering Part 1
    Changing a Template or Templatepath in Word Documents via Script
    24 Comments
    • Eivind
      2017-04-29T10:10:36+00:000000003630201704 at 10:10
      Reply

      Hello, and thank you for a great script! I’ve used it to fetch data from my newly bought Netatmo, and it’s working great.

      I’m trying to expand a little on your work here, but I don’t have much experience with bash. What I’m planning to do is to have a perl script execute your bash script, and have the bash script store the exported data in csv files. Then, the perl script will parse those csv files, and put the data into a mysql database. Then I’ll make a separate web interface that outputs the data from mysql.

      That might sound silly, given that Netatmo already provides a web interface. I do like to keep my own backup of the data, though. Also, I miss several features on the Netatmo interface – for instance being able to compare data (e.g. indoor and outdoor temperature in the same chart, or the outdoor temperature of this year compared to last year on the same chart).

      So, my question is: Is there any way to set the parametres of the bash script via command line? That way I would be able to have the perl script give the date range when running the script.

      Thanks again for the great work in making the script!

      • Eivind
        2017-04-29T10:39:04+00:000000000430201704 at 10:39
        Reply

        Sorry to bother you – I figured it out.

        In case you or anyone else would like to do the same, here’s what I’ve done:

        After the end of getmeasurecsv() {}, I’ve added this:


        while [[ $# -gt 1 ]]
        do
        key="$1"

        case $key in
        -s|--startdate)
        STARTDATE="$2"
        DATETIMEBEGIN="$2"
        shift # past argument
        ;;
        -e|--enddate)
        ENDDATE="$2"
        DATETIMEEND="$2"
        shift # past argument
        ;;
        --default)
        DEFAULT=YES
        ;;
        *)
        # unknown option
        ;;
        esac
        shift # past argument or value
        done

        echo START DATE = "${STARTDATE}"
        echo END DATE = "${ENDDATE}"

        Then I’ve added several lines calling the getmeasurecsv(), specifying the different modules and the date range:


        #Inndoors
        getmeasurecsv "my@email.com" "my_password" "70:ee:50:00:00:00" "70:ee:50:00:00:04" "Temperature,Humidity,CO2,Pressure,Noise" "${STARTDATE}" "${ENDDATE}" "csv" > inndoors.csv

        #Rain
        getmeasurecsv "my@email.com" "my_password" "70:ee:50:00:00:00" "70:ee:50:00:00:04" "Rain" "${STARTDATE}" "${ENDDATE}" "csv" > rain.csv

        Then I call the script by command line like this:


        bash getmeasurecsv.sh -s "2017-04-25 00:00:00" -e "2017-04-29 10:15:00"

        Next up is writing the perl script to parse the data and add it to mysql. Thanks again for the great script! 🙂

    • sunitha
      2017-04-10T18:39:13+00:000000001330201704 at 18:39
      Reply

      Hello,
      I would like to retrieve the public data from netatmo. I know only python. Could you help me to retrieve to csv file. Thank you.

    • Jiri Dobsovic
      2017-04-07T23:03:03+00:000000000330201704 at 23:03
      Reply

      Hello,
      I´m sorry for my stupid question (I’m a beginner), but how can I run this script? Can I run it in Windows computer or how can I run it in Synology, as you wrote…
      Thank you for your help.

      • Frank
        2017-04-10T20:07:40+00:000000004030201704 at 20:07
        Reply

        Hi Jiri,
        this is a bash script (bash – born again shell) . Windows will not work, bash is running on unix systems like ubuntu. also unix based raspberry os will work.

    • Lode
      2016-12-28T00:26:16+00:000000001631201612 at 00:26
      Reply

      Great job !!!!

    • 2016-07-22T07:58:27+00:000000002731201607 at 07:58
      Reply

      […] two really cool bash scripts using curl to query base information about one’s devices and download the weather data. I modified them, especially for postprocessing (=grouping) the rain data, so I attach them to this […]

    • 2016-06-09T12:28:49+00:000000004930201606 at 12:28
      Reply

      Thanks a lot for this script and the script to get the device data. With it getting my netatmo data was a piece of cake.

    • Sandro
      2016-05-15T13:36:21+00:000000002131201605 at 13:36
      Reply

      Hallo Michael,
      Kennst Du dich mit VBA auch aus?
      Da ich kein Linux habe, würde ich die Abfrage gerne direkt mit Excel machen.
      Gruss
      Sandro

      • 2016-06-01T20:27:54+00:000000005430201606 at 20:27
        Reply

        Hallo,

        in VBA nur ein bisschen, aber in VB. Das ganze nennt sich ReST API – im Internet gibts viele Beispiele wie man über VBA Daten von einer ReST Schnittstelle abfragen kann. Ich vermute aber über VBA könnte es etwas umständlich werden, da VBA ja schon etwas betagt ist – aber machbar ist es sicherlich.
        Gruss
        Michael

    • Niklaus Schneeberger
      2016-04-14T21:26:36+00:000000003630201604 at 21:26
      Reply

      Hi Michael,

      This looks exactly like what I need but I don’t get it to work. First I get en email from Netatmo everytime I run the script and second I get the following error: date: illegal option — –
      Any ideas?

      • 2016-04-28T09:22:10+00:000000001030201604 at 09:22
        Reply

        Hello Niklaus,

        you can change your notification settings from the NetAtmo Website and disable the login notification.

    • Fiaskogaul
      2016-02-12T13:38:46+00:000000004629201602 at 13:38
      Reply

      Ich will die Daten via Statusboard darstellen. Da kommt mir das CSV sehr gelegen.

      Ich bekomm aber beim Ausführen des Script einige Rechtefehler:

      Returncode: 126
      Ausgabe des Scripts:
      Array
      (
      [0] => netatmo.sh: line 73: /bin/grep: Permission denied
      [1] => netatmo.sh: line 73: /bin/cat: Permission denied
      [2] => netatmo.sh: line 79: /bin/grep: Permission denied
      [3] => netatmo.sh: line 79: /bin/cat: Permission denied
      [4] => % Total % Received % Xferd Average Speed Time Time Time Current
      [5] => Dload Upload Total Spent Left Speed
      [6] =>
      0 0 0 0 0 0 0 0 –:–:– –:–:– –:–:– 0
      100 257 100 3 100 254 11 946 –:–:– –:–:– –:–:– 1909
      [7] => 9;
      [8] => netatmo.sh: line 88: /bin/rm: Permission denied
      [9] => netatmo.sh: line 89: /bin/rm: Permission denied
      [10] => netatmo.sh: line 73: /bin/grep: Permission denied
      [11] => netatmo.sh: line 73: /bin/cat: Permission denied
      [12] => netatmo.sh: line 79: /bin/grep: Permission denied
      [13] => netatmo.sh: line 79: /bin/cat: Permission denied
      [14] => % Total % Received % Xferd Average Speed Time Time Time Current
      [15] => Dload Upload Total Spent Left Speed
      [16] =>
      0 0 0 0 0 0 0 0 –:–:– –:–:– –:–:– 0
      100 257 100 3 100 254 20 1713 –:–:– –:–:– –:–:– 2134
      [17] => 9;
      [18] => netatmo.sh: line 88: /bin/rm: Permission denied
      [19] => netatmo.sh: line 89: /bin/rm: Permission denied
      )

      Ich versteh nicht so recht was das Script da jetzt nicht darf.
      Freue mich über jegliche Hilfe!

      Vielen Dank
      Grüße
      Kai

    • Kai
      2016-02-05T15:16:17+00:000000001729201602 at 15:16
      Reply

      Hallo Michael,
      eine andere Frage: Wäre es möglich das Skript so zu modifizieren, dass immer nur ein Zeitraum von z.B. 24h als CSV Datei exportiert würde? Aktuell ist es ja so, dass die Startzeit/Datum stets definiert werden MUSS. Das Enddatum kann auch weiter in die Zukunft gesetzt werden – das ist kein Problem, ergibt lediglich NaN am Ende der CSV Datei. Ich nutze dein Skript nicht nur als Datenbackup, sondern lasse jede Stunde damit die Werte ab dem 1.1. in eine CSV Datei exportieren. Somit kommen jede Stunde neue Daten hinzu (der Scale Wert wurde dafür bereits angepasst, so dass nur eine Zeile / Wert(e) pro Stunde hinzukommt).
      Mit einem fest definiertem Zeitraum von stets 24h könnte dann jede Stunde ein neuer, aktueller Zeitraum von 24h (jetzt minus 24h) exportiert werden und wie bei mir mit Charts visualisiert werden.
      Grüße
      kai

    • dagrun schuler
      2016-01-26T08:53:02+00:000000000231201601 at 08:53
      Reply

      I wonder if you have any idea if it is possible to download data from other netatmo weather stations which distribute their data on the netatmo world maps?
      Dagrun

      • 2016-01-26T11:55:12+00:000000001231201601 at 11:55
        Reply

        Hello Dagrun,

        you can access the weather map data by using the public API method GETPUBLICDATA. NetAtmo offers a great example on their website:
        https://dev.netatmo.com/doc/methods/getpublicdata
        You can pass an area specified by longitude and latitude to the API and get the measures from all weather stations within the specified area.

        Michael

      • 2017-03-23T11:54:08+00:000000000831201703 at 11:54
        Reply

        You are so awesome for helping me solve this mysetry.

    • Kai
      2016-01-17T22:34:34+00:000000003431201601 at 22:34
      Reply

      Hallo Michael,
      ersteinmal vielen Dank für Deine tolle Anleitung zur Ausgabe der Netatmo Daten im CSV Format. Das ist genau das was ich gesucht habe 🙂
      Aber: Wie kann ich die ausgegebenen Daten auch als dateiname.csv in einem bestimmten Pfad abspeichern? Oder wird hier schon irgendwo das Ergebnis der Abfrage abgelegt? Wenn ja, wo? Das Skript läuft in der Konsole bei mir einwandfrei und spuckt alle Werte der definierten Zeitspanne aus – aber halt nur in der Konsole, oder?
      Grüße
      Kai

      • 2016-01-18T06:20:56+00:000000005631201601 at 06:20
        Reply

        Hallo Kai,

        damit die Ausgabe in eine Datei geschrieben wird, musst du am Ende der letzten Zeile “getmeasurecsv…..” einfach “> /pfad/zu/meiner/datei.csv” anhängen. Somit wird die gesamte Ausgabe in die angegebene Datei umgeleitet /geschrieben.

        Grüße

        Michael

        • Kai
          2016-01-18T10:44:40+00:000000004031201601 at 10:44
          Reply

          Hallo Michael,
          prima – vielen Dank für die schnelle Hilfe – funktioniert!
          Grüße
          Kai

    • 2015-08-31T21:47:17+00:000000001731201508 at 21:47
      Reply

      The script works great for the base module. But I’ve discovered some problems with the outside- and rain module. Somehow there is a little mismatch in the adress that I can’t figure out the examples.
      “12:23:45:56:78:33” “02:00:00:12:23:45”
      vs
      “03:00:00:XX:XX:XX” “03:00:00:12:45:E4”
      I tried both combinations but it only downloads “9;”
      Somebody else having the same?

      • 2015-09-01T07:00:26+00:000000002630201509 at 07:00
        Reply

        Unfortunately, I don’t have a rain module. But the outdoor module is working for me. What is the first character of your outdoor module serial number shown on the netatmo page? First character should be something between A-Z?

        • 2016-01-26T14:04:21+00:000000002131201601 at 14:04
          Reply

          Hi there,
          my rain module has a serial number starting with ‘k’, so the module id starts with ’05’. The only possible filter ($TYPE) is “Rain”.
          My wind module starts with a ‘l’, module id: ’06:..:.’. filter: “WindAngle,WindStrength,GustAngle,GustStrength”

          Viele Grüße
          Frank

          @Michael: Schönes bash-script, sehr verständlich geschrieben. Sieht man leider selten.

      • 2015-09-01T14:15:30+00:000000003030201509 at 14:15
        Reply

        Please have a look on my new blog post how to get all module data, including base_id and module_id:

        Read NetAtmo weather station data via Script

    Leave Your Comment

    Your Comment*

    Your Name*
    Your Webpage