Sitemap / Advertise

Introduction

Collates weather data on Google Sheets and interprets it with a neural network built in TensorFlow to make predictions on the rainfall intensity.


Tags

Share

IoT Weather Station Predicting Rainfall Intensity w/ TensorFlow

Advertisement:


read_later

Read Later



read_later

Read Later

Introduction

Collates weather data on Google Sheets and interprets it with a neural network built in TensorFlow to make predictions on the rainfall intensity.

Tags

Share





Advertisement

Advertisement




    Components :
  • [1]DFRobot Weather Station Kit with Anemometer / Wind Vane / Rain Bucket
  • [1]NodeMCU ESP-12E (ESP8266)
  • [1]DFRobot SIM808 GPS/GPRS/GSM Shield
  • [1]Arduino Uno
  • [1]Raspberry Pi 3B+ or 4

Description

Especially in 2020, instant heavy and violent rainfalls have caused detrimental effects on greenhouses and crop yields in my country. Unfortunately, these days unexpected and excessive rainfalls occur more frequently and became relatively common due to climate change in some areas. Meanwhile, other regions suffer from drought. As 2020 passes, more than thirty-five incidents that caused the loss of crops and livestock due to heavy and violent rainfalls are recorded in my region. To contribute to the solution to forfend and avert excessive rainfall instigated problems like agricultural productivity loss, I contemplated this project as a starting point.

Primarily, I focused on excessive rainfall effects on agriculture in this project. However, excessive rainfalls can lead to numerous hazards, for example [1]:

As shown in the latest studies, excessive rainfall can affect crop productivity in various ways, including direct physical damage, delayed planting and harvesting, restricted root growth, oxygen deficiency, and nutrient loss. Thus, too much rain can be as dangerous as drought for agriculture: "Data from the past three decades suggest that excessive rainfall can affect crop yield as much as excessive heat and drought. In a new study, an interdisciplinary team from the University of Illinois linked crop insurance, climate, soil, and corn yield data from 1981 through 2016. The study found that, during some years, excessive rainfall reduced U.S. corn yield by as much as 34% relative to the expected yield. Data suggest that drought and excessive heat caused a yield loss of up to 37% during some years [2]".

Unfortunately, early warning systems are not enough to entirely prevent exacerbating effects of excessive rainfall on agricultural products and processes without adequate and preliminary local weather data for each greenhouse or farm. Therefore, in the hope of averting the detrimental effects of excessive rainfall on agriculture, I decided to create this project as a budget-friendly remote weather station to collect and interpret local weather information to make more accurate predictions on the rainfall intensity.

After doing some research about deep learning and neural networks, I decided to build an artificial neural network (ANN) based on the classification model to interpret the accumulated local weather data to predict the rainfall intensity according to the given classes. As the following study indicates, researchers have applied different neural network models and types to forecast not only rainfalls but also weather phenomena: "Many experiments have used neural networks to predict quantitative rainfall amounts at various locations and look-ahead ranges. For instance, researchers in Thailand were able to obtain highly accurate forecasts using feed-forward neural networks to predict quantitative rainfall amounts in the one to three-hour look-ahead range in order to predict possible flooding dangers. Neural networks have also been used to predict weather phenomena besides the traditional forecast values, such as probability/amount of rainfall, wind speed, barometric pressure, etc. They have been used very successfully to predict tornadoes. Additionally, researchers in Australia successfully used a neural network to identify fog at various forecast ranges ranging from 3 hours to 18 hours around Canberra International Airport [3]."

Before building and testing my neural network model, I had collected weather data at my balcony and backyard in three kilometers radius for more than three months to create a local weather data set showing weather information in my neighborhood. To be able to elicit weather information even in extreme weather conditions, I used a Weather Station Kit with Anemometer, Wind Vane, and Rain Bucket by DFRobot with two different development boards:

Thus, I could transfer weather data via WiFi at my balcony or GPRS if the distance between my router and the weather station is too far away.

I decided to use Google Sheets, a cloud-based spreadsheet program, to collate weather data generated by the weather station because creating big data sets and appending new information are simple with the Google Sheets API. Hence, I developed a web application in PHP to obtain weather data transferred by the development boards and append it to a spreadsheet on Google Sheets.

After completing collecting weather data for more than three months every five minutes, I built an artificial neural network (ANN) model with TensorFlow to make predictions on the rainfall intensity. Theoretically, I assigned a rainfall intensity class for each reading after scaling and normalizing the data set. Then, I trained the neural network model to classify these classes:

So, this is my project in a nutshell 😃

In the following steps, you can find more detailed information on coding, local weather data collection, and building an artificial neural network (ANN) model with TensorFlow.

Huge thanks to DFRobot for sponsoring this project.

Sponsored products by DFRobot:

⭐ Weather Station Kit with Anemometer/Wind Vane/Rain Bucket | Inspect

⭐ SIM808 GPS/GPRS/GSM Shield For Arduino | Inspect

project-image
Figure - 58.1


project-image
Figure - 58.2


project-image
Figure - 58.3

Step 1: Assembling the weather station kit with an anemometer, wind vane, and rain bucket

This weather station kit includes an anemometer, wind vine, rain bucket, and a sensor board to send weather data via the serial communication protocol. Thus, it is easy-to-use with development boards to collect local weather data.

It outputs 37 bytes per second, including the end CR/LF.


c000s000g000t086r000p000h53b10020

Data Parser:

⭐ First of all, attach the anemometer and the wind vane to the bridge by using screws and nuts.

⭐ Connect the wind vane to the anemometer by using the RJ11 phone cord and tighten it under the bridge.

project-image
Figure - 58.4


project-image
Figure - 58.5

⭐ Then, mount the stainless steel stick to the bridge with screws.

project-image
Figure - 58.6

⭐ Install the rain bucket joint under the rain bucket with screws.

project-image
Figure - 58.7

⭐ Lastly, place the rain bucket on the stainless steel stick by using screws and nuts.

project-image
Figure - 58.8

⭐ Now, it is ready to collect weather data even in extreme weather conditions. However, the sensor (converter) board is not weather-proof, so I added a plastic box to the weather station to protect the sensor board and the SIM808 shield outdoors.

Step 2: Collecting weather data on Google Sheets

To be able to send information to a spreadsheet on Google Sheets, you need to create a new application in the Google Developer Console and get verification credentials to utilize the Google Sheets API. If you are a novice in creating applications with the Google Developer Console, follow the instructions below.

⭐ Go to Google Developer Console and click the 'NEW PROJECT' button. Enter the project name as Remote Weather Station.

project-image
Figure - 58.9

⭐ Go to the APIs & Services and click to the API library to enable the Google Sheets API.

project-image
Figure - 58.10


project-image
Figure - 58.11

Step 2.1: Getting credentials to use the Google Sheets API

⭐ Open the project dashboard and click the Credentials button. Then, fill the form as depicted below to create the service account and key.

project-image
Figure - 58.12


project-image
Figure - 58.13


project-image
Figure - 58.14

⭐ Click the CREATE CREDENTIALS button and configure the consent screen to be able to create an OAuth client ID.

project-image
Figure - 58.15


project-image
Figure - 58.16

⭐ Then, create a new OAuth client ID named Client under Desktop app (application type).

project-image
Figure - 58.17

⭐ After creating the OAuth client ID (Client), download the credentials (ID and Secret) and name the file as credentials.json.

project-image
Figure - 58.18


project-image
Figure - 58.19

Step 2.2: Creating a spreadsheet on Google Sheets

⭐ Go to Google Sheets and select Personal.

project-image
Figure - 58.20

⭐ Create a Blank spreadsheet and name it Remote Weather Station.

⭐ Then, define the column names as the first row:

⭐ If needed, modify a chart to inspect the collected data, as did I.

project-image
Figure - 58.21

Step 3: Developing a web application in PHP to append weather data to the spreadsheet

This PHP web application is for getting weather data transferred by the development boards (ESP8266 and SIM808) via an HTTP Get request and appending that data to a given spreadsheet (by ID) on Google Sheets with verified credentials and token.

The web application requires the Google Client Library for PHP to append weather data to a given spreadsheet on Google Sheets.

⭐ So, first of all, create a folder named remote_weather_station in your server and install the Google Client Library for PHP in it.

The remote_weather_station folder should include these four files in addition to the Google Client Library to work:

In this step, I will discuss the first two files - index.php and account_verification_token.php:

💻 index.php

⭐ Get local weather data from the weather station via a Get request:


// Wind Direction (wd), Average Wind Speed (a_ws), Max Wind Speed (m_ws), 1hr Rainfall (1_rf), 24hr Rainfall (24_rf), Temperature (tem), Humidity (hum), Barometric Pressure (b_pr).
$variables_from_module;
if(isset($_GET['wd']) && isset($_GET['a_ws']) && isset($_GET['m_ws']) && isset($_GET['1_rf']) && isset($_GET['24_rf']) && isset($_GET['tem']) && isset($_GET['hum']) && isset($_GET['b_pr'])){
	$variables_from_module = [
	  "wd" => (int)$_GET['wd'],
	  "a_ws" => (float)$_GET['a_ws'],
	  "m_ws" => (float)$_GET['m_ws'],
	  "1_rf" => (float)$_GET['1_rf'],
	  "24_rf" => (float)$_GET['24_rf'],
	  "tem" => (float)$_GET['tem'],
	  "hum" => (int)$_GET['hum'],
	  "b_pr" => (float)$_GET['b_pr']
    ];
}else{
    $variables_from_module = [
	  "wd" => "err",
	  "a_ws" => "err",
	  "m_ws" => "err",
	  "1_rf" => "err",
	  "24_rf" => "err",
	  "tem" => "err",
	  "hum" => "err",
	  "b_pr" => "err"
    ];
}

⭐ Connect to the Google Developer Console application named Remote Weather Station by using the credentials.json file (Step 2.1).


    $client = new Google_Client();
    $client->setApplicationName('Remote Weather Station'); // Enter your application name.
    $client->setScopes('https://www.googleapis.com/auth/spreadsheets');
    $client->setAuthConfig('credentials.json');
    $client->setAccessType('offline');
    $client->setPrompt('select_account consent');

⭐ Load previously authorized token from a file (token.json), if it exists.


    $tokenPath = 'token.json';
    if (file_exists($tokenPath)) {
        $accessToken = json_decode(file_get_contents($tokenPath), true);
        $client->setAccessToken($accessToken);
		print("Token Found!");
    }

⭐ If there is no previous token or expired, request authorization from the user and set the verification code to create the token.json file with the file_put_contents() function.


    if ($client->isAccessTokenExpired()) {
        // Refresh the token if possible, else fetch a new one.
        if ($client->getRefreshToken()) {
            $client->fetchAccessTokenWithRefreshToken($client->getRefreshToken());
        } else {
            // Request authorization from the user.
            $authUrl = $client->createAuthUrl();
			
			...

            // Set the verification code to create the token.json.
			$authCode = trim($GLOBALS['account_verification_token']);
			
			...

            // Check to see if there was an error and the account_verification_token is entered.
            if (array_key_exists('error', $accessToken)) {
                throw new Exception(join(', ', $accessToken));
            }else{
				print("Successful! Refresh the page.");
			}
        }
        // Save the token to a file.
        if (!file_exists(dirname($tokenPath))) {
            mkdir(dirname($tokenPath), 0700, true);
        }
        file_put_contents($tokenPath, json_encode($client->getAccessToken()));
    }

⭐ Enter the spreadsheet ID and range to append recent findings from the weather station to the spreadsheet.

https://docs.google.com/spreadsheets/d/spreadsheetId/edit#gid=0


// Enter your spreadsheetId:
$spreadsheetId = '<spreadsheet_ID>';
// Enter the range (the first row) under which new values will be appended (8 rows):
$range = 'A1:H1';
// Append recent findings from the weather station to the spreadsheet.
$values = [
    [$variables_from_module["wd"], $variables_from_module["a_ws"], $variables_from_module["m_ws"], $variables_from_module["1_rf"], $variables_from_module["24_rf"], $variables_from_module["tem"], $variables_from_module["hum"], $variables_from_module["b_pr"]]
];
$body = new Google_Service_Sheets_ValueRange([
    'values' => $values
]);
$params = [
    'valueInputOption' => "RAW"
];

...

    $result = $service->spreadsheets_values->append($spreadsheetId, $range, $body, $params);
    printf("<br><br>%d cells appended.", $result->getUpdates()->getUpdatedCells());

💻 account_verification_token.php

⭐ After authorizing the web application by the created AuthUrl, enter the given verification code to set an access token.


$account_verification_token = "<Enter Token>"; // Enter the verification code after authorization.

project-image
Figure - 58.22


project-image
Figure - 58.23

Step 3.1: Setting up the web application on Raspberry Pi (or any server)

After creating the PHP web application, I decided to run it on my Raspberry Pi, but you can run the application on any server as long as it is a PHP server.

If you want to use Raspberry Pi but do not know how to set up a LAMP web server on Raspberry Pi, you can inspect this tutorial.

⭐ First of all, move the application folder (remote_weather_station) to the apache server (/var/www/html) by using the terminal since the apache server is a protected location.

sudo mv /home/pi/Downloads/remote_weather_station /var/www/html/

project-image
Figure - 58.24

⭐ The web application throws an error when opened for the first time due to the missing verification code. To obtain the verification code, go to the generated authorization link by the application.

project-image
Figure - 58.25

⭐ Then, click to Go to Remote Weather Station (unsafe).

project-image
Figure - 58.26

⭐ Grant the application with the required permissions.

project-image
Figure - 58.27

⭐ Now, copy the verification code and paste it into account_verification_token.php.

project-image
Figure - 58.28

⭐ Return to the web application. It should display the confirmation message: Successful! Refresh the page.

project-image
Figure - 58.29

⭐ After refreshing the page, the web application uses the verification code to get an access token and saves the access token in a file named token.json. Thus, it does not throw an error again.

project-image
Figure - 58.30

⭐ To test whether the PHP web application is appending new data to the given spreadsheet accurately:

http://localhost/remote_weather_station/?wd=12&a_ws=8&m_ws=11&1_rf=14&24_rf=84&tem=24&hum=32&b_pr=57

project-image
Figure - 58.31


project-image
Figure - 58.32

Step 4: Reading and sending weather data with ESP8266 (WiFi)

To collect weather data at my balcony, I used a NodeMCU ESP-12E (ESP8266) development board with the weather station.

I programmed it to send weather data to the PHP web application every five minutes.

project-image
Figure - 58.33

⭐ Include required libraries and define the WiFi settings.


#include <ESP8266WiFi.h>
#include <WiFiClient.h> 
#include <ESP8266WebServer.h>
#include <ESP8266HTTPClient.h>
#include <SoftwareSerial.h>

// Define your WiFi settings.
const char *ssid = "<SSID>";
const char *password = "<PASSWORD>";

⭐ Define weather station settings and the serial connection pins - D6, D5.


// Define weather station settings:
char databuffer[35];
double temp;
int transferring = 0;

// Define the serial connection pins - RX and TX.
SoftwareSerial Serial_1(D6, D5); // (Rx, Tx)

⭐ In the getBuffer() function, get data from the weather station.


void getBuffer(){
  int index;
  for (index = 0;index < 35;index ++){
    if(Serial_1.available()){
      databuffer[index] = Serial_1.read();
      if (databuffer[0] != 'c'){
        index = -1;
      }
    }
    else{
      index --;
    }
  }
}

⭐ Debug the weather data and create the link.


String weather_data = "wd=" + String(WindDirection()) + "&a_ws=" + String(WindSpeedAverage()) + "&m_ws=" + String(WindSpeedMax()) + "&1_rf=" + String(RainfallOneHour()) + "&24_rf=" + String(RainfallOneDay()) + "&tem=" + String(Temperature()) + "&hum=" + String(Humidity()) + "&b_pr=" + String(BarPressure());
String server = "http://192.168.1.24/remote_weather_station/?";

...

int WindDirection(){ return transCharToInt(databuffer,1,3); } // Wind Direction (deg)

float WindSpeedAverage(){ temp = 0.44704 * transCharToInt(databuffer,5,7); return temp; } // Average Air Speed (1 minute)

float WindSpeedMax(){ temp = 0.44704 * transCharToInt(databuffer,9,11); return temp; } //Max Air Speed (5 minutes)

float Temperature(){ temp = (transCharToInt(databuffer,13,15) - 32.00) * 5.00 / 9.00; return temp; } // Temperature ("C")

float RainfallOneHour(){ temp = transCharToInt(databuffer,17,19) * 25.40 * 0.01; return temp; } // Rainfall (1 hour)

float RainfallOneDay(){ temp = transCharToInt(databuffer,21,23) * 25.40 * 0.01; return temp; } // Rainfall (24 hours)

int Humidity(){ return transCharToInt(databuffer,25,26); } // Humidity (%)

float BarPressure(){ temp = transCharToInt(databuffer,28,32); return temp / 10.00; } // Barometric Pressure (hPA)

⭐ Send data packets every 5 minutes to Raspberry Pi (or any server).


  transferring++; Serial.println("Time => " + String(transferring) + "s / " + String(int(5*60)) + "s\n\n");
  if(transferring == 5*60){
   // Create the HTTP object to make a request to the server. 
   HTTPClient http; 
   http.begin(server + weather_data);
   int httpCode = http.GET();           
   String payload = http.getString();    
   Serial.println("Data Send...\nHTTP Code => " + String(httpCode) + "\nServer Response => " + payload + "\n\n"); 
   http.end();
   transferring = 0;  
  }
  // Wait 1 second...
  delay(1000);

project-image
Figure - 58.34

⭐ After uploading the code to the NodeMCU ESP-12E (ESP8266) development board, it displays weather data every second on the serial monitor and sends data packets every five minutes (300 seconds) to the PHP web application.


Connection is successful!

...

Weather Data => wd=0&a_ws=0.00&m_ws=0.00&1_rf=0.00&24_rf=0.00&tem=21.67&hum=29&b_pr=1016.70
Buffer => c000s000g000t071r000p000h29b10167*3
Time => 299s / 300s


Weather Data => wd=0&a_ws=0.00&m_ws=0.00&1_rf=0.00&24_rf=0.00&tem=21.67&hum=29&b_pr=1016.70
Buffer => c000s000g000t071r000p000h29b10167*3
Time => 300s / 300s

...


⭐ Then, it shows the response from the server.


...

Data Send...
HTTP Code => 200
Server Response => Token Found!

8 cells appended. ...
project-image
Figure - 58.35


project-image
Figure - 58.36

Step 4.1: Reading and sending weather data with SIM808 (GPRS)

To collect weather data in my backyard, I used a SIM808 shield for Arduino Uno if the distance between the weather station and my router is too far away.

Most of the code is the same and covered in the previous step aside from the parts below.

project-image
Figure - 58.37

⭐ Include required libraries and define the sim808 object.

For SIM808 GPS/GPRS/GSM Shield | Download


#include <DFRobot_sim808.h>
#include <SoftwareSerial.h>

// Define the sim808.
DFRobot_SIM808 sim808(&Serial);

⭐ Initialize the SIM808 module and continue if it is working accurately.


  //******** Initialize sim808 module *************
  while(!sim808.init()) {
     delay(1000);
     Serial.print("Sim808 init error\r\n");
  }
  delay(2000);
  // Continue if the SIM808 Module is working accurately.
  Serial.println("Sim808 init success");
  delay(5000);

⭐ Send data packets every 5 minutes to the server by establishing a TCP connection to make a GET request.


  transferring++; Serial.println("Time => " + String(transferring) + "s / " + String(int(5*60)) + "s\n\n");
  if(transferring == 5*60){
    //*********** Attempt DHCP *******************
    while(!sim808.join(F("cmnet"))) {
      Serial.println("Sim808 join network error!");
      delay(2000);
    }
    //************ Successful DHCP ****************
    delay(5000);
    Serial.println("Successful DHCP");
    //*********** Establish a TCP connection ************
    if(!sim808.connect(TCP,"192.168.1.24", 80)) { // Change it with your server.
      Serial.println("Connection Error");
    }else{
      Serial.println("Connection OK");
    }
    delay(2000);
	
	...
	
	

⭐ After creating the line string, convert it from string to char array to make an HTTP Get Request with the SIM808.


    ...
	
    String line = "GET /remote_weather_station/?" + weather_data_1 + weather_data_2 + weather_data_3 + " HTTP/1.0\r\n\r\n";
    Serial.println(line);
    char buffer[512];
    // Convert the line from string to char array to make an HTTP Get Request with the SIM808.
    char web_hook[110];
    String_to_Char(line, 110, web_hook);
    sim808.send(web_hook, sizeof(web_hook)-1);
    while (true) {
      int ret = sim808.recv(buffer, sizeof(buffer)-1);
      if (ret <= 0){
          Serial.println("Fetch Over...");
          break; 
      }
	  
	  ...
	  

project-image
Figure - 58.38

⭐ After uploading the code to the SIM808 shield, it displays weather data every second on the serial monitor and sends data packets every five minutes (300 seconds) to the PHP web application.

project-image
Figure - 58.39


project-image
Figure - 58.40

Step 4.2: Creating the weather data set for more than three months

After finishing coding, I started to collate weather data on Google Sheets every five minutes for more than three months to build a neural network model and make predictions on the rainfall intensity.

Collected Weather Data:

Then, I downloaded the spreadsheet as Remote Weather Station.csv, consisting of 32219 rows as my preliminary local weather data set. I am still collating weather data to improve my data set and model :)

project-image
Figure - 58.41


project-image
Figure - 58.42

Step 5: Building an Artificial Neural Network (ANN) with TensorFlow

When I completed collating my preliminary local weather data set, I started to work on my artificial neural network (ANN) model to make predictions on the rainfall intensity.

I decided to create my neural network model with TensorFlow in Python. So, first of all, I followed the steps below to grasp a better understanding of the weather data:

After applying these steps, I decided to use my neural network model to classify different rainfall intensity classes theoretically assigned as labels (outputs) for each input (row). I created my classes according to the rate of precipitation, which depends on the considered time.

The following categories are used to classify rainfall intensity by the rate of precipitation (rainfall):

According to the precipitation (rainfall) rates, I preprocessed the weather data to assign one of these five classes for each input as its label:

After scaling (normalizing) and preprocessing the weather data, I elicited seven input variables and one label for each reading, classified with the five mentioned classes. Then, I built an artificial neural network model with TensorFlow to obtain the best possible results and predictions with my preliminary data set.

Layers:

project-image
Figure - 58.43


project-image
Figure - 58.44

To execute all steps above, I created a class named Weather_Station in Python after including the required libraries:


import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

Subsequently, I will discuss coding in Python for each step I mentioned above.

Also, you can download IoT_weather_station_neural_network.py to inspect coding.

project-image
Figure - 58.45


project-image
Figure - 58.46

Step 5.1: Visualizing and scaling (normalizing) the weather data

Before diving in to build a model, it is important to understand the weather data to pass the model appropriately formatted data.

So, in this step, I will show you how to visualize weather data and scale (normalize) in Python.

⭐ First of all, read the weather data from Remote Weather Station.csv.


csv_path = "E:\PYTHON\Remote Weather Station.csv"
df = pd.read_csv(csv_path)

⭐ In the graphics() function, visualize the requested columns from the weather data set by using the Matplotlib library.


    def graphics(self, column_1, column_2, xlabel, ylabel):
        # Show requested columns from the data set:
        plt.style.use("dark_background")
        plt.gcf().canvas.set_window_title('IoT Weather Station')
        plt.hist2d(self.df[column_1], self.df[column_2])
        plt.colorbar()
        plt.xlabel(xlabel)
        plt.ylabel(ylabel)
        plt.title(xlabel)
        plt.show()

⭐ In the data_visualization() function, inspect all columns before scaling weather data to build a model with appropriately formatted data.


    def data_visualization(self):
        # Inspect requested columns to build a model with appropriately formatted data:
        self.graphics('WD', '1h_RF', 'Wind Direction (deg)', 'One-Hour Rainfall (mm)')
        self.graphics('Av_WS', '1h_RF', 'Average Wind Speed (m/s)', 'One-Hour Rainfall (mm)')
        self.graphics('Mx_WS', '1h_RF', 'Maximum Wind Speed (m/s)', 'One-Hour Rainfall (mm)')
        self.graphics('24h_RF', '1h_RF', '24-Hour Rainfall (mm)', 'One-Hour Rainfall (mm)')
        self.graphics('Tem', '1h_RF', 'Temperature (°C)', 'One-Hour Rainfall (mm)')
        self.graphics('Hum', '1h_RF', 'Humidity (%)', 'One-Hour Rainfall (mm)')
        self.graphics('b_PR', '1h_RF', 'Barometric Pressure (hPA)', 'One-Hour Rainfall (mm)')  

project-image
Figure - 58.47


project-image
Figure - 58.48


project-image
Figure - 58.49


project-image
Figure - 58.50

After visualizing weather data, I scaled (normalized) each column to format it properly.

Normally, each row (reading) looked like this before scaling:

⭐ In the scale_data() function:

⭐ The wind direction in units of degrees and angles do not make good model inputs. 360° and 0° should be close to each other and wrap around smoothly. The direction should not matter if the wind is not blowing. Thus, convert the wind direction and velocity columns to a wind vector to interpret them easily with a neural network model.


    def scale_data(self):
        # Wind Direction and Speed:
        wv = self.df.pop('Av_WS')
        max_wv = self.df.pop('Mx_WS')
        # Convert to radians.
        wd_rad = self.df.pop('WD')*np.pi / 180
        # Calculate the wind x and y components.
        self.df['scaled_WX'] = wv*np.cos(wd_rad)
        self.df['scaled_WY'] = wv*np.sin(wd_rad)
        # Calculate the max wind x and y components.
        self.df['scaled_max_WX'] = max_wv*np.cos(wd_rad)
        self.df['scaled_max_WY'] = max_wv*np.sin(wd_rad)
		
		...
		

⭐ For the remaining columns except for 1h_RF, divide them into average reading values to scale (normalize) and format.



        ...

        # Temperature:
        tem = self.df.pop('Tem')
        self.df['scaled_Tem'] = tem / 25
        # Humidity:
        hum = self.df.pop('Hum')
        self.df['scaled_Hum'] = hum / 70 
        # Barometric Pressure:
        bPR = self.df.pop('b_PR')
        self.df["scaled_bPR"] = bPR / 1013     
        # 24 Hour Rainfall (Approx.)
        rain_24 = self.df.pop('24h_RF')
        self.df['scaled_24h_RF'] = rain_24 / 24

After completing scaling (normalizing), I extracted these new columns from the weather data set:

Step 5.2: Training the model (ANN) on the rainfall intensity classes

Before building and training a neural network model in TensorFlow, I needed to create the input array and the label array by preprocessing the scaled and normalized weather data set.

⭐ In the create_input_and_label() function:

⭐ Firstly, append each input element as a NumPy array to the input array and convert it to a NumPy array by using the asarray() function.

⭐ Each input element includes seven variables [shape=(7,)]:


    def create_input_and_label(self):
        n = len(self.df)
        # Create the input array using the scaled variables:
        for i in range(n):
            self.input.append(np.array([self.df['scaled_WX'][i], self.df['scaled_WY'][i], self.df['scaled_max_WX'][i], self.df['scaled_max_WY'][i], self.df['scaled_Tem'][i], self.df['scaled_Hum'][i], self.df['scaled_bPR'][i]]))
        self.input = np.asarray(self.input)
		
		...
		

⭐ Then, evaluate the approximate rainfall (precipitation) rate for each reading (row).

approx_RF_rate = (1h_RF + scaled_24h_RF) * 100

⭐ According to the rainfall rate, assign a class [0 - 4] for each input element and append them to the label array.


        
		...
		
        for i in range(n):
            _class = 0
            # Evaluate the approximate rainfall rate:
            approx_RF_rate = (self.df['1h_RF'][i] + self.df['scaled_24h_RF'][i]) * 100
            # As labels, assign classes of rainfall intensity according to the approximate rainfall rate (mm):
            if approx_RF_rate == 0:
                _class = 0
            elif approx_RF_rate < 2.5:
                _class = 1
            elif 2.5 < approx_RF_rate and approx_RF_rate < 7.6:
                _class = 2
            elif 7.6 < approx_RF_rate and approx_RF_rate < 50:
                _class = 3
            else:
                _class = 4
            self.label.append(_class)
        self.label = np.asarray(self.label)

After preprocessing the scaled weather data to create input and label arrays, I split them as training (60%) and test (40%) data sets:


    def split_data(self):
        n = len(self.df)
        # (60%, 40%) - (training, test)
        self.train_input = self.input[0:int(n*0.6)]
        self.test_input = self.input[int(n*0.6):]
        self.train_label = self.label[0:int(n*0.6)]
        self.test_label = self.label[int(n*0.6):]

Then, I built my artificial neural network (ANN) model by using Keras and trained it with the training data set for nineteen epochs.

You can inspect these tutorials to learn about activation functions, loss functions, epochs, etc.


    def build_and_train_model(self):
        # Build the neural network:
        self.model = keras.Sequential([
            keras.Input(shape=(7,)),
            keras.layers.Dense(16, activation='relu'),
            keras.layers.Dense(32, activation='relu'),
            keras.layers.Dense(64, activation='relu'),
            keras.layers.Dense(128, activation='relu'),
            keras.layers.Dense(5, activation='softmax')
        ])
        # Compile:
        self.model.compile(optimizer='adam', loss="sparse_categorical_crossentropy", metrics=['accuracy'])
        # Train:
        self.model.fit(self.train_input, self.train_label, epochs=19)
		
		...
		

After training with the preliminary training data set, the accuracy of the model is between 0.83 and 0.85.

project-image
Figure - 58.51

Step 5.3: Evaluating the model and making predictions on the rainfall intensity

After building and training my artificial neural network model, I tested its accuracy by using the preliminary testing data set.

For now, the evaluated accuracy of the model is between 0.72 and 0.73 due to inadequate testing data set overfitting the model. However, I am still collecting weather data to improve the model accuracy.


        
		...
		
		# Test the accuracy:
        print("\n\nModel Evaluation:")
        test_loss, test_acc = self.model.evaluate(self.test_input, self.test_label) 
        print("Evaluated Accuracy: ", test_acc)

Then, I used my neural network model to make predictions on the rainfall intensity with a given prediction array consisting of readings from the weather station after the training of the model. As a starting point, the model works fine :)

The model predicts possibilities of labels for each input element as an array of 5 numbers. They represent the model's "confidence" that the given input element corresponds to each of the five different classes of rainfall intensity [0 - 4].

⭐ In the make_prediction() function, make a prediction for each input element in a given array and get the most accurate label [0 - 4] by using the argmax() function to display its class name.


    def make_prediction(self, pre_array):
        print("\n\nModel Predictions:\n")
        prediction = self.model.predict(pre_array)
        for i in range(len(pre_array)):
            print("Prediction => ", self.class_names[np.argmax(prediction[i])])

Prediction Inputs:

Prediction Outputs:

project-image
Figure - 58.52

Connections and Adjustments (ESP8266)


// Connections
// NodeMCU ESP-12E (ESP8266) :           
//                                Weather Station
// VV  --------------------------- 5V
// D5  --------------------------- RX
// D6  --------------------------- TX
// G   --------------------------- GND

I connected the sensor (converter) board to the NodeMCU ESP-12E (ESP8266) development board and fastened them to a plastic box while collecting weather data at my balcony.

project-image
Figure - 58.53

Connections and Adjustments (SIM808)


// Connections
// Arduino Uno:           
//                                SIM808 GPS/GPRS/GSM Shield For Arduino
// D0  --------------------------- RX
// D1  --------------------------- TX
// D12 --------------------------- POWER          
//                                Weather Station
// 5V  --------------------------- 5V
// D5  --------------------------- RX
// D6  --------------------------- TX
// GND --------------------------- GND

⭐ Note: D0, D1, D12 pins are occupied by the SIM808 GPS/GPRS/GSM Shield.

⭐ Connect an external battery (7-23V) for the SIM808 module to work properly.

⭐ Attach the GPS antenna and the GSM antenna to the SIM808 shield.

⭐ Insert a SIM card into the SIM slot on the SIM808 shield.

⭐ Before uploading the code, set the function switch on the shield to None (1).

⭐ Upload the code.

⭐ Then, set the function switch to Arduino (3).

⭐ Press the Boot button on the shield until seeing the Net indicator LED flashing every 1 second and wait for the SIM card to register the network - the Net indicator LED will slowly flash every 3 seconds.

⭐ Click here to get more information about the SIM808 GSM/GPS/GPRS Shield.

After setting up the SIM808 shield with Arduino Uno, I connected the sensor (converter) board to it.

project-image
Figure - 58.54

I placed all components into a plastic box to collect weather data outside, even in extreme weather conditions, since the sensor board is not weather-proof.

project-image
Figure - 58.55

Then, I fastened the plastic box to the weather station.

project-image
Figure - 58.56

Videos and Conclusion



After completing coding and assembling the weather station, I collated local weather data for more than three months every five minutes to build my neural network model successfully.

I am still collecting weather data at my balcony and backyard to improve my neural network model and its accuracy :)

project-image
Figure - 58.57


project-image
Figure - 58.58


project-image
Figure - 58.59

Further Discussions

☔ 💧 Since we need local weather data sets to get more accurate predictions on weather phenomena, budget-friendly NN-enabled weather stations like this can be placed on farms and greenhouses to avert the detrimental effects of excessive rainfall on agriculture.

☔ 💧 As early warning systems working with neural networks, we can use them simultaneously to create a swarm of weather stations communicating and feeding each other to forecast rainfall intensity precisely in local areas.

☔ 💧 Also, we can utilize that information for assessing:

project-image
Figure - 58.60


project-image
Figure - 58.61

References

[1]Extreme weather - heavy rainfall, NIWA, https://niwa.co.nz/natural-hazards/extreme-weather-heavy-rainfall

[2]University of Illinois at Urbana-Champaign, News Bureau. "Excessive rainfall as damaging to corn yield as extreme heat, drought." ScienceDaily. ScienceDaily, 30 April 2019. www.sciencedaily.com/releases/2019/04/190430121744.htm.

[3]Andrew Culclasure, Using Neural Networks to Provide Local Weather Forecasts, Georgia Southern University, Spring 2013, 29https://digitalcommons.georgiasouthern.edu/cgi/viewcontent.cgi?article=1031&context=etd.

Code

IoT_weather_station_neural_network.py

Download



# IoT Weather Station Predicting Rainfall Intensity w/ TensorFlow

# Windows, Linux, or Ubuntu

# By Kutluhan Aktar

# Collates weather data on Google Sheets and interprets it with a neural network built in TensorFlow to make predictions on the rainfall intensity. 

# For more information:
# https://www.theamplituhedron.com/projects/IoT_Weather_Station_Predicting_Rainfall_Intensity_with_TensorFlow

import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Create a class to build a neural network after getting, visualizing, and scaling (normalizing) weather data.
class Weather_Station:
    def __init__(self, data):
        self.df = data
        self.input = []
        self.label = []
        # Define class names for different rainfall intensity predictions and values.
        self.class_names = ['None', 'Light Rain', 'Moderate Rain', 'Heavy Rain', 'Violent Rain']
    # Create graphics for requested columns.
    def graphics(self, column_1, column_2, xlabel, ylabel):
        # Show requested columns from the data set:
        plt.style.use("dark_background")
        plt.gcf().canvas.set_window_title('IoT Weather Station')
        plt.hist2d(self.df[column_1], self.df[column_2])
        plt.colorbar()
        plt.xlabel(xlabel)
        plt.ylabel(ylabel)
        plt.title(xlabel)
        plt.show()
    # Visualize data before creating and feeding the neural network model.
    def data_visualization(self):
        # Inspect requested columns to build a model with appropriately formatted data:
        self.graphics('WD', '1h_RF', 'Wind Direction (deg)', 'One-Hour Rainfall (mm)')
        self.graphics('Av_WS', '1h_RF', 'Average Wind Speed (m/s)', 'One-Hour Rainfall (mm)')
        self.graphics('Mx_WS', '1h_RF', 'Maximum Wind Speed (m/s)', 'One-Hour Rainfall (mm)')
        self.graphics('24h_RF', '1h_RF', '24-Hour Rainfall (mm)', 'One-Hour Rainfall (mm)')
        self.graphics('Tem', '1h_RF', 'Temperature (°C)', 'One-Hour Rainfall (mm)')
        self.graphics('Hum', '1h_RF', 'Humidity (%)', 'One-Hour Rainfall (mm)')
        self.graphics('b_PR', '1h_RF', 'Barometric Pressure (hPA)', 'One-Hour Rainfall (mm)')    
    # Scale (normalize) data depending on the neural network model.
    def scale_data(self):
        # Wind Direction and Speed:
        wv = self.df.pop('Av_WS')
        max_wv = self.df.pop('Mx_WS')
        # Convert to radians.
        wd_rad = self.df.pop('WD')*np.pi / 180
        # Calculate the wind x and y components.
        self.df['scaled_WX'] = wv*np.cos(wd_rad)
        self.df['scaled_WY'] = wv*np.sin(wd_rad)
        # Calculate the max wind x and y components.
        self.df['scaled_max_WX'] = max_wv*np.cos(wd_rad)
        self.df['scaled_max_WY'] = max_wv*np.sin(wd_rad)
        # Temperature:
        tem = self.df.pop('Tem')
        self.df['scaled_Tem'] = tem / 25
        # Humidity:
        hum = self.df.pop('Hum')
        self.df['scaled_Hum'] = hum / 70 
        # Barometric Pressure:
        bPR = self.df.pop('b_PR')
        self.df["scaled_bPR"] = bPR / 1013     
        # 24 Hour Rainfall (Approx.)
        rain_24 = self.df.pop('24h_RF')
        self.df['scaled_24h_RF'] = rain_24 / 24
    # Define the input and label arrays.
    def create_input_and_label(self):
        n = len(self.df)
        # Create the input array using the scaled variables:
        for i in range(n):
            self.input.append(np.array([self.df['scaled_WX'][i], self.df['scaled_WY'][i], self.df['scaled_max_WX'][i], self.df['scaled_max_WY'][i], self.df['scaled_Tem'][i], self.df['scaled_Hum'][i], self.df['scaled_bPR'][i]]))
        self.input = np.asarray(self.input)
        # Create the label array using the one-hour and 24-hour rainfall variables:
        for i in range(n):
            _class = 0
            # Evaluate the approximate rainfall rate:
            approx_RF_rate = (self.df['1h_RF'][i] + self.df['scaled_24h_RF'][i]) * 100
            # As labels, assign classes of rainfall intensity according to the approximate rainfall rate (mm):
            if approx_RF_rate == 0:
                _class = 0
            elif approx_RF_rate < 2.5:
                _class = 1
            elif 2.5 < approx_RF_rate and approx_RF_rate < 7.6:
                _class = 2
            elif 7.6 < approx_RF_rate and approx_RF_rate < 50:
                _class = 3
            else:
                _class = 4
            self.label.append(_class)
        self.label = np.asarray(self.label)
    # Split the data for the training and test sets.
    def split_data(self):
        n = len(self.df)
        # (60%, 40%) - (training, test)
        self.train_input = self.input[0:int(n*0.6)]
        self.test_input = self.input[int(n*0.6):]
        self.train_label = self.label[0:int(n*0.6)]
        self.test_label = self.label[int(n*0.6):]
    # Build and train the artificial neural network (ANN) to make predictions on the rainfall intensity with classes.
    def build_and_train_model(self):
        # Build the neural network:
        self.model = keras.Sequential([
            keras.Input(shape=(7,)),
            keras.layers.Dense(16, activation='relu'),
            keras.layers.Dense(32, activation='relu'),
            keras.layers.Dense(64, activation='relu'),
            keras.layers.Dense(128, activation='relu'),
            keras.layers.Dense(5, activation='softmax')
        ])
        # Compile:
        self.model.compile(optimizer='adam', loss="sparse_categorical_crossentropy", metrics=['accuracy'])
        # Train:
        self.model.fit(self.train_input, self.train_label, epochs=19)
        # Test the accuracy:
        print("\n\nModel Evaluation:")
        test_loss, test_acc = self.model.evaluate(self.test_input, self.test_label) 
        print("Evaluated Accuracy: ", test_acc)
    # Make rainfall intensity class [0 - 4] predictions using different input arrays.
    def make_prediction(self, pre_array):
        print("\n\nModel Predictions:\n")
        prediction = self.model.predict(pre_array)
        for i in range(len(pre_array)):
            print("Prediction => ", self.class_names[np.argmax(prediction[i])])
    # Save the model for further usage without training steps:
    def save_model(self):
        self.model.save("E:\PYTHON\weather_station.h5")
    # Run Artificial Neural Network (ANN):
    def Neural_Network(self, save):
        self.scale_data()
        self.create_input_and_label()
        self.split_data()
        self.build_and_train_model()
        if save == True:
            self.save_model()
        # Example Input and Layer:
        print("\nScaled Input [EXP]:\n")
        print(self.train_input[0])
        print("\nScaled Label [EXP]:\n")
        print(self.train_label[0])
        
    
# Read data (Remote Weather Station.csv):
csv_path = "E:\PYTHON\Remote Weather Station.csv"
df = pd.read_csv(csv_path)

# Define a new class object named 'station':
station = Weather_Station(df)

# Visualize data:
#station.data_visualization()

# Artificial Neural Network (ANN):
station.Neural_Network(False)

# Enter inputs for making predictions:
prediction_array = np.array([
    [0, 0, 0.31819805, 0.31819805, 0.6988, 0.81498571, 0.99349753],
    [0, -0, 0, -0, 0.8444, 1, 0.96835143],
    [0, 0, 0.45, 0, 0.87577, 0.95857143, 1.00128332],
    [-0, -0, -0, -0, 0.8224, 1.05714286, 0.99279368]
])
# Prediction Results:
station.make_prediction(prediction_array)


ESP12E_remote_weather_station.ino

Download



         /////////////////////////////////////////////  
        //    IoT Weather Station Predicting       //
       //    Rainfall Intensity w/ TensorFlow     //
      //         ------------------------        //
     //           NodeMCU (ESP-12E)             //           
    //           by Kutluhan Aktar             // 
   //                                         //
  /////////////////////////////////////////////

//
// Collates weather data on Google Sheets and interprets it with a neural network built in TensorFlow to make predictions on the rainfall intensity. 
//
// For more information:
// https://www.theamplituhedron.com/projects/IoT_Weather_Station_Predicting_Rainfall_Intensity_with_TensorFlow
//
// Connections
// NodeMCU (ESP-12E) :           
//                                Weather Station
// VV  --------------------------- 5V
// D5  --------------------------- RX
// D6  --------------------------- TX
// G   --------------------------- GND


// Include required libraries:
#include <ESP8266WiFi.h>
#include <WiFiClient.h> 
#include <ESP8266WebServer.h>
#include <ESP8266HTTPClient.h>
#include <SoftwareSerial.h>

// Define your WiFi settings.
const char *ssid = "<SSID>";
const char *password = "<PASSWORD>";

// Define weather station settings:
char databuffer[35];
double temp;
int transferring = 0;

// Define the serial connection pins - RX and TX.
SoftwareSerial Serial_1(D6, D5); // (Rx, Tx)

void setup() {
  // Wait until connected.
  delay(1000);
  // Initiate serial ports:
  Serial.begin(115200);
  Serial_1.begin(9600);
  // It is just for assuring if connection is alive.
  WiFi.mode(WIFI_OFF);
  delay(1000);
  // This mode allows NodeMCU to connect any WiFi directly.
  WiFi.mode(WIFI_STA);        
  // Connect NodeMCU to your WiFi.
  WiFi.begin(ssid, password);
  
  Serial.print("\n\n");
  Serial.print("Try to connect to WiFi. Please wait! ");
  Serial.print("\n\n");
  // Halt the code until connected to WiFi.
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print("*");
  }

  // If connection is successful:
  Serial.print("\n\n");
  Serial.print("-------------------------------------");
  Serial.print("\n\n");
  Serial.print("Connection is successful!");
  Serial.print("\n\n");
  Serial.print("Connected WiFi SSID : ");
  Serial.print(ssid);
  Serial.print("\n\n");
  Serial.println("Connected IPAddress : ");
  Serial.println(WiFi.localIP());
  Serial.print("\n\n");

}

void loop() {
  // Get data from the remote weather station:
  getBuffer(); 
  // Debug the information and create the link:
  String weather_data = "wd=" + String(WindDirection()) + "&a_ws=" + String(WindSpeedAverage()) + "&m_ws=" + String(WindSpeedMax()) + "&1_rf=" + String(RainfallOneHour()) + "&24_rf=" + String(RainfallOneDay()) + "&tem=" + String(Temperature()) + "&hum=" + String(Humidity()) + "&b_pr=" + String(BarPressure());
  String server = "http://192.168.1.24/remote_weather_station/?";
  Serial.println("Weather Data => " + weather_data);
  Serial.println("Buffer => " + String(databuffer));
  // Send data packets every 5 minutes to Raspberry Pi (or any server).
  transferring++; Serial.println("Time => " + String(transferring) + "s / " + String(int(5*60)) + "s\n\n");
  if(transferring == 5*60){
   // Create the HTTP object to make a request to the server. 
   HTTPClient http; 
   http.begin(server + weather_data);
   int httpCode = http.GET();           
   String payload = http.getString();    
   Serial.println("Data Send...\nHTTP Code => " + String(httpCode) + "\nServer Response => " + payload + "\n\n"); 
   http.end();
   transferring = 0;  
  }
  // Wait 1 second...
  delay(1000);
}

// WEATHER STATION
void getBuffer(){
  int index;
  for (index = 0;index < 35;index ++){
    if(Serial_1.available()){
      databuffer[index] = Serial_1.read();
      if (databuffer[0] != 'c'){
        index = -1;
      }
    }
    else{
      index --;
    }
  }
}

int transCharToInt(char *_buffer, int _start, int _stop){
  int _index;
  int result = 0;
  int num = _stop - _start + 1;
  int _temp[num];
  for (_index = _start;_index <= _stop;_index ++){
    _temp[_index - _start] = _buffer[_index] - '0';
    result = 10*result + _temp[_index - _start];
  }
  return result;
}

int WindDirection(){ return transCharToInt(databuffer,1,3); } // Wind Direction (deg)

float WindSpeedAverage(){ temp = 0.44704 * transCharToInt(databuffer,5,7); return temp; } // Average Air Speed (1 minute)

float WindSpeedMax(){ temp = 0.44704 * transCharToInt(databuffer,9,11); return temp; } //Max Air Speed (5 minutes)

float Temperature(){ temp = (transCharToInt(databuffer,13,15) - 32.00) * 5.00 / 9.00; return temp; } // Temperature ("C")

float RainfallOneHour(){ temp = transCharToInt(databuffer,17,19) * 25.40 * 0.01; return temp; } // Rainfall (1 hour)

float RainfallOneDay(){ temp = transCharToInt(databuffer,21,23) * 25.40 * 0.01; return temp; } // Rainfall (24 hours)

int Humidity(){ return transCharToInt(databuffer,25,26); } // Humidity (%)

float BarPressure(){ temp = transCharToInt(databuffer,28,32); return temp / 10.00; } // Barometric Pressure (hPA)


SIM808_remote_weather_station.ino

Download



         /////////////////////////////////////////////  
        //    IoT Weather Station Predicting       //
       //    Rainfall Intensity w/ TensorFlow     //
      //         ------------------------        //
     //           Arduino Uno (SIM808)          //           
    //             by Kutluhan Aktar           // 
   //                                         //
  /////////////////////////////////////////////

//
// Collates weather data on Google Sheets and interprets it with a neural network built in TensorFlow to make predictions on the rainfall intensity. 
//
// For more information:
// https://www.theamplituhedron.com/projects/IoT_Weather_Station_Predicting_Rainfall_Intensity_with_TensorFlow
//
// Connections
// Arduino Uno:           
//                                SIM808 GPS/GPRS/GSM Shield For Arduino
// D0  --------------------------- RX
// D1  --------------------------- TX
// D12 --------------------------- POWER          
//                                Weather Station
// 5V  --------------------------- 5V
// D5  --------------------------- RX
// D6  --------------------------- TX
// GND --------------------------- GND


// Include required libraries:
#include <DFRobot_sim808.h>
#include <SoftwareSerial.h>

// Define the sim808.
DFRobot_SIM808 sim808(&Serial);

// Define weather station settings:
char databuffer[35];
double temp;
int transferring = 0;

// Define the serial connection pins - RX and TX.
SoftwareSerial Serial_1(6, 5); // (Rx, Tx)

void setup() {
  // Wait until connected.
  delay(1000);
  // Initiate serial ports:
  Serial.begin(9600);
  Serial_1.begin(9600);

  //******** Initialize sim808 module *************
  while(!sim808.init()) {
     delay(1000);
     Serial.print("Sim808 init error\r\n");
  }
  delay(2000);
  // Continue if the SIM808 Module is working accurately.
  Serial.println("Sim808 init success");
  delay(5000);
}

void loop() {
  // Get data from the remote weather station:
  getBuffer(); 
  // Debug the information and create the link:
  String weather_data_1 = "wd=" + String(WindDirection()) + "&a_ws=" + String(WindSpeedAverage()) + "&m_ws=" + String(WindSpeedMax());
  String weather_data_2 = "&1_rf=" + String(RainfallOneHour()) + "&24_rf=" + String(RainfallOneDay()) + "&tem=" + String(Temperature());
  String weather_data_3 = "&hum=" + String(Humidity()) + "&b_pr=" + String(BarPressure());
  Serial.print("Weather Data => " + weather_data_1);
  Serial.print(weather_data_2);
  Serial.println(weather_data_3);
  Serial.println("Buffer => " + String(databuffer));
  // Send data packets every 5 minutes to the server.
  transferring++; Serial.println("Time => " + String(transferring) + "s / " + String(int(5*60)) + "s\n\n");
  if(transferring == 5*60){
    //*********** Attempt DHCP *******************
    while(!sim808.join(F("cmnet"))) {
      Serial.println("Sim808 join network error!");
      delay(2000);
    }
    //************ Successful DHCP ****************
    delay(5000);
    Serial.println("Successful DHCP");
    //*********** Establish a TCP connection ************
    if(!sim808.connect(TCP,"192.168.1.24", 80)) { // Change it with your server.
      Serial.println("Connection Error");
    }else{
      Serial.println("Connection OK");
    }
    delay(2000);
    //*********** Send a GET request *****************
    String line = "GET /remote_weather_station/?" + weather_data_1 + weather_data_2 + weather_data_3 + " HTTP/1.0\r\n\r\n";
    Serial.println(line);
    char buffer[512];
    // Convert the line from string to char array to make an HTTP Get Request with the SIM808.
    char web_hook[110];
    String_to_Char(line, 110, web_hook);
    sim808.send(web_hook, sizeof(web_hook)-1);
    while (true) {
      int ret = sim808.recv(buffer, sizeof(buffer)-1);
      if (ret <= 0){
          Serial.println("Fetch Over...");
          break; 
      }
      // Uncomment to view the response in the serial monitor.
      /*
      buffer[ret] = '\0';
      Serial.print("Recv: ");
      Serial.print(ret);
      Serial.print(" bytes: ");
      Serial.println(buffer);
      */
      Serial.println("\nData Send");
      break;
    }
    //************* Close TCP or UDP connections **********
    sim808.close();
    //*** Disconnect wireless connection, Close Moving Scene *******
    sim808.disconnect();
    // Exit.
    transferring = 0;  
  }
  // Wait 1 second...
  delay(1000);
}

// WEATHER STATION
void getBuffer(){
  int index;
  for (index = 0;index < 35;index ++){
    if(Serial_1.available()){
      databuffer[index] = Serial_1.read();
      if (databuffer[0] != 'c'){
        index = -1;
      }
    }
    else{
      index --;
    }
  }
}

int transCharToInt(char *_buffer, int _start, int _stop){
  int _index;
  int result = 0;
  int num = _stop - _start + 1;
  int _temp[num];
  for (_index = _start;_index <= _stop;_index ++){
    _temp[_index - _start] = _buffer[_index] - '0';
    result = 10*result + _temp[_index - _start];
  }
  return result;
}

int WindDirection(){ return transCharToInt(databuffer,1,3); } // Wind Direction (deg)

float WindSpeedAverage(){ temp = 0.44704 * transCharToInt(databuffer,5,7); return temp; } // Average Air Speed (1 minute)

float WindSpeedMax(){ temp = 0.44704 * transCharToInt(databuffer,9,11); return temp; } //Max Air Speed (5 minutes)

float Temperature(){ temp = (transCharToInt(databuffer,13,15) - 32.00) * 5.00 / 9.00; return temp; } // Temperature ("C")

float RainfallOneHour(){ temp = transCharToInt(databuffer,17,19) * 25.40 * 0.01; return temp; } // Rainfall (1 hour)

float RainfallOneDay(){ temp = transCharToInt(databuffer,21,23) * 25.40 * 0.01; return temp; } // Rainfall (24 hours)

int Humidity(){ return transCharToInt(databuffer,25,26); } // Humidity (%)

float BarPressure(){ temp = transCharToInt(databuffer,28,32); return temp / 10.00; } // Barometric Pressure (hPA)

void String_to_Char(String _String, int _size, char _convert[]){
  for(int i=0;i<_size;i++){
    _convert[i] = _String[i];
  }
}


index.php (web application)

Download



<?php
require_once $_SERVER['DOCUMENT_ROOT']."google-api-php-client-2.4.1/vendor/autoload.php"; // Google Client API v2.0
require_once "account_verification_token.php"; // Get the verification code.

// Get the variables from the weather station:
// Wind Direction (wd), Average Wind Speed (a_ws), Max Wind Speed (m_ws), 1hr Rainfall (1_rf), 24hr Rainfall (24_rf), Temperature (tem), Humidity (hum), Barometric Pressure (b_pr).
$variables_from_module;
if(isset($_GET['wd']) && isset($_GET['a_ws']) && isset($_GET['m_ws']) && isset($_GET['1_rf']) && isset($_GET['24_rf']) && isset($_GET['tem']) && isset($_GET['hum']) && isset($_GET['b_pr'])){
	$variables_from_module = [
	  "wd" => (int)$_GET['wd'],
	  "a_ws" => (float)$_GET['a_ws'],
	  "m_ws" => (float)$_GET['m_ws'],
	  "1_rf" => (float)$_GET['1_rf'],
	  "24_rf" => (float)$_GET['24_rf'],
	  "tem" => (float)$_GET['tem'],
	  "hum" => (int)$_GET['hum'],
	  "b_pr" => (float)$_GET['b_pr']
    ];
}else{
    $variables_from_module = [
	  "wd" => "err",
	  "a_ws" => "err",
	  "m_ws" => "err",
	  "1_rf" => "err",
	  "24_rf" => "err",
	  "tem" => "err",
	  "hum" => "err",
	  "b_pr" => "err"
    ];
}


/**
 * Returns an authorized API client.
 * @return Google_Client the authorized client object
 */
function getClient()
{
    $client = new Google_Client();
    $client->setApplicationName('Remote Weather Station'); // Enter your application name.
    $client->setScopes('https://www.googleapis.com/auth/spreadsheets');
    $client->setAuthConfig('credentials.json');
    $client->setAccessType('offline');
    $client->setPrompt('select_account consent');

    // Load previously authorized token from a file, if it exists.
    // The file token.json stores the user's access and refresh tokens, and is
    // created automatically when the authorization flow completes for the first
    // time.
    $tokenPath = 'token.json';
    if (file_exists($tokenPath)) {
        $accessToken = json_decode(file_get_contents($tokenPath), true);
        $client->setAccessToken($accessToken);
		print("Token Found!");
    }

    // If there is no previous token or it's expired.
    if ($client->isAccessTokenExpired()) {
        // Refresh the token if possible, else fetch a new one.
        if ($client->getRefreshToken()) {
            $client->fetchAccessTokenWithRefreshToken($client->getRefreshToken());
        } else {
            // Request authorization from the user.
            $authUrl = $client->createAuthUrl();
			// Do not forget to refresh the page after getting the verification code and entering it to the account_verification_token.php.
            printf("Open the following link in your browser:<br><br>%s<br><br>", $authUrl); // <= Comment
            print 'Do not forget to refresh the page after getting the verification code and entering it to the account_verification_token.php.<br><br>Set the verification code in the account_verification_token.php file.'; // <= Comment
            // Set the verification code to create the token.json.
			$authCode = trim($GLOBALS['account_verification_token']);

            // Exchange authorization code for an access token.
            $accessToken = $client->fetchAccessTokenWithAuthCode($authCode);
            $client->setAccessToken($accessToken);

            // Check to see if there was an error and the account_verification_token is entered.
            if (array_key_exists('error', $accessToken)) {
                throw new Exception(join(', ', $accessToken));
            }else{
				print("Successful! Refresh the page.");
			}
        }
        // Save the token to a file.
        if (!file_exists(dirname($tokenPath))) {
            mkdir(dirname($tokenPath), 0700, true);
        }
        file_put_contents($tokenPath, json_encode($client->getAccessToken()));
    }
    return $client;
}


// Get the API client and construct the service object.
$client = getClient();
$service = new Google_Service_Sheets($client);

// Enter your spreadsheetId:
$spreadsheetId = '<spreadsheet_ID>';
// Enter the range (the first row) under which new values will be appended (8 rows):
$range = 'A1:H1';
// Append recent findings from the weather station to the spreadsheet.
$values = [
    [$variables_from_module["wd"], $variables_from_module["a_ws"], $variables_from_module["m_ws"], $variables_from_module["1_rf"], $variables_from_module["24_rf"], $variables_from_module["tem"], $variables_from_module["hum"], $variables_from_module["b_pr"]]
];
$body = new Google_Service_Sheets_ValueRange([
    'values' => $values
]);
$params = [
    'valueInputOption' => "RAW"
];
// Append if only requested!
if(isset($_GET['wd']) && isset($_GET['a_ws']) && isset($_GET['m_ws']) && isset($_GET['1_rf']) && isset($_GET['24_rf']) && isset($_GET['tem']) && isset($_GET['hum']) && isset($_GET['b_pr'])){
    $result = $service->spreadsheets_values->append($spreadsheetId, $range, $body, $params);
    printf("<br><br>%d cells appended.", $result->getUpdates()->getUpdatedCells());
}else{
	print ("<br>Missing Data!");
}


account_verification_token.php (web application)

Download



<?php $account_verification_token = "<Enter Token>"; // Enter the verification code after authorization. ?>


Schematics

project-image
Schematic - 58.1


project-image
Schematic - 58.2

Downloads

Remote Weather Station.csv

Download