Update (2019-03-14): Netatmo changed their login process on Mar 7th 2019 – the script below is the updated version also using the new getstationsdata API endpoint.
Based on the feedback for my previous blog post Export NetAtmo weather station data to CSV / Excel I created a small script to get the module, device ID, temperature, humidity and also all sensor data from your NetAtmo weather station and modules.
This scripts also uses curl to login to the web page and then download the device data using the NetAtmo Web API. This script will return all the information in JSON format.
#!/bin/bash listDevices() { # ------------------------------------------------------ # Help # ------------------------------------------------------ # usage: listdevices <USER> <PASSWORD> # # USER + PASSWORD -> your NetAtmo Website login # ------------------------------------------------------ # Parsing Arguments # ------------------------------------------------------ USER=$1 PASS=$2 # ------------------------------------------------------ # Define some constants # ------------------------------------------------------ URL_LOGIN="https://auth.netatmo.com/en-us/access/login" URL_POSTLOGIN="https://auth.netatmo.com/access/postlogin" API_GETMEASURECSV="https://api.netatmo.com/api/getstationsdata" SESSION_COOKIE="cookie_sess.txt" # ------------------------------------------------------ # URL encode the user entered parameters # ------------------------------------------------------ USER="$(urlencode $USER)" PASS="$(urlencode $PASS)" # ------------------------------------------------------ # Now let's fetch the data # ------------------------------------------------------ # get token from hidden <input> field TOKEN="$(curl --silent -c $SESSION_COOKIE $URL_LOGIN | sed -n '/token/s/.*name="_token"\s\+value="\([^"]\+\).*/\1/p')" # and now we can login using cookie, id, user and password curl --silent -d "_token=$TOKEN&email=$USER&password=$PASS" -b $SESSION_COOKIE -c $SESSION_COOKIE $URL_POSTLOGIN > /dev/null # next we extract the access_token from the session cookie ACCESS_TOKEN="$(cat $SESSION_COOKIE | grep netatmocomaccess_token | cut -f7)" # build the POST data PARAM="access_token=$ACCESS_TOKEN" # now download json data curl -d $PARAM $API_GETMEASURECSV # clean up rm $SESSION_COOKIE } #____________________________________________________________________________________________________________________________________ urlencode() { # ------------------------------------------------------ # urlencode function from mrubin # https://gist.github.com/mrubin # # usage: urlencode <string> # ------------------------------------------------------ local length="${#1}" for (( i = 0; i < length; i++ )); do local c="${1:i:1}" case $c in [a-zA-Z0-9.~_-]) printf "$c" ;; *) printf '%%%02X' "'$c" esac done } #____________________________________________________________________________________________________________________________________ listDevices "user@email.com" "mySecretPassword"
When you execute the script above you should see all your NetAtmo module details in a JSON format – unfortunately this is not easy to read:
Therefore I used JQ – which is a great JSON parser to make it human-readable. I just piped the output of my script straight into JQ:
./ListDevices.sh | ./jq
… and viola I got a proper formatted JSON file containing all data:
This JSON output contains the module_id (referred as _ID), and all other sensor data, such as temperature, humidity, battery level, etc.
I just updated the blog post containing a new version of the script
unfortunately the same as CSV getmeasurecsv()
still (with just copy&paste with my netatmo credentials)
{“error”:{“code”:2,”message”:”Invalid access token”}}
Hi
can it be, that you use a MacOS when getting the error: “Invalid access token”? I get this on my Mac as well, but when using the same script on my RaspberryPi it works like a charm.
On my Mac running macOS High Sierra (V 10.13.6) I had to modify the `sed` command for receiving the initial `TOKEN`:
TOKEN=”$(curl –silent -c $SESSION_COOKIE $URL_LOGIN | tee login.html | sed -n ‘/_token/s/.*value=”(.*)”.*/1/p’)”
Probably netatmo have modified the login date.
{“error”:{“code”:2,”message”:”Invalid access token”}
Netatmo modified the login process. If i find some time i will update the script
I am getting the same error message, would be really great if you could update the script!
Thanks in advance
Hi can you tell us what to change or when you can change it ? Maybe we can help to build the script. Netatmo always changing things in the backend 🙁
Thanks in advance
Hello,
Netatmo changed the login process. I already fixed it in a python script that I’m using:
https://github.com/michaelmiklis/loxberry-plugin-netatmo-weather/blob/master/data/netatmo.py
The key changes are as follows:
– changed the url to /en-us/access/login
– grab the HTML output and search for a element
– extract the value string
– put the value, email and password in the new post request (names of the values changed)
– send the data to https://auth.netatmo.com/access/postlogin
after that you are successfully logged in. Hardest part would be to extract the hidden field value using grep or awk. Any help appreciated.
Michael
Durty but works:
POST_LOGIN_URL=”https://auth.netatmo.com/access/postlogin”
—–
# first we need to get a valid session cookie
curl –silent -o $AUTH_COOKIE $URL_LOGIN > /dev/null
# then extract the ID from the authentication cookie
SESS_ID=”$(cat $AUTH_COOKIE | grep -oP ‘_token” value=”K[^”]+’)”
# and now we can login using cookie, id, user and password
curl –silent -d “email=$USER&password=$PASS&_token=$SESS_ID” -c $SESSION_COOKIE $POST_LOGIN_URL > /dev/null
If I put the information below in the script it errors me:
Could not resolve: –silent (Domain name not found)
URL_LOGIN=”https://auth.netatmo.com/en-us/access/login”
POST_LOGIN_URL=”https://auth.netatmo.com/access/postlogin”
SESS_ID=”$(cat $AUTH_COOKIE | grep -oP ‘_token” value=”K[^”]+’)”
curl –silent -d “email=$USER&password=$PASS&_token=$SESS_ID” -c $SESSION_COOKIE $POST_LOGIN_URL > /dev/null
What’s wrong (damn Netatmo as if I hadn’t enough IT the whole day….)
You need to replace API_GETMEASURECSV=”https://my.netatmo.com/api/devicelist”
to
API_GETMEASURECSV=”https://api.netatmo.com/api/devicelist”
Thank you for your reply – I updated the script.
Same applies to reading the CSV file:
https://api.netatmo.com/api/getmeasurecsv
instead of
https://my.netatmo.com/api/getmeasurecsv
Thanks, I was struggling with the same problem – this fixed it! 🙂
The script not work anymore… return
An Error Was Encountered
The action you have requested is not allowed.
Does this approach still work with HttpOnly cookies?
[…] it should be used for that task. I was lucky to find two really cool bash scripts using curl to query base information about one’s devices and download the weather data. I modified them, especially for postprocessing (=grouping) the rain […]
Very nice and helpful script. Thanks a lot, with this I was able to establish with little effort my data download for all my netatmo devices!
Hallo Michael,
ich bin auf deine Webseite gestoßen, da ich eine Verbindung von Netatmo zu einer RRD Datenbank suche. Hast du hiermit auch Erfahrung? Ich würde gerne die Raumtemperaturen auslesen und in eine bestehende RRDChart einfügen, nur leider weiß ich gar nicht wo und wie ich anfangen soll an die Temperaturdaten zukommen.
Als Server habe ich einen Raspberry pi, wo auch schon die rrd Datenbank drauf liegt und die RRDTool Chart erstellt wird.
Ich hoffe du kannst mir weiterhelfen.
Vielen Dank,
Markus
Hallo Markus,
zu RRD kann ich dir leider nichts sagen – aber ich denke du solltest bei Google sicher fündig werden, wie vorhandene CSV Dateien mit dem RRDTool in ein entsprechendes Chart gewandelt werden. Dann könntest du zuerst mit dem Script von meinem Blog die Daten in einer CSV Datei herunterladen und dann eine entsprechende RRD Datei erstellen lassen.
Grüße
Michael
Hallo Michael,
mit deinem Script (dieser Eintrag) brauch ich dein erstes Script auch noch dazu oder löst dieses das erste ab?
Das konnte ich noch nicht nachvollziehen. Weil bei diesem Script muss ich keine Adressen der Module und Zeiträume der Daten angeben.
Oder muss ich beide kombinieren?
Danke,
Markus
Dieses Script liest die Daten im JSON Format – das ist in einem entsprechenden Script besser zu verwenden. Aber hierzu wirst du das Script entsprechend anpassen müssen, dass die Daten aus dem JSON Format irgendwie in eine RRD-Datei gewandelt / geschrieben werden.
Bei meinem anderen Script kannst du alle Daten einer bestimmten Station / eines bestimmten Moduls als CSV Datei downloaden – ich nutze dies um die Daten auf meinem NAS zu archivieren.