by Manoj Rupareliya on December 19, 2019
With cloud computing and other computing methods on the rise, businesses today are taking their Data Analytics operations to the cloud servers to reduce the dependency over local servers and improve latency issues.
The growth of sensor-based devices and smart gadgets have exponentially increased data troubles. BigData needs several intelligent solutions for its operations like data mining, analytics, and reduction in the edge of the fog devices in the cloud.
Before, we jump into the realm of Fog Data Analytics, let us first discover, what is Fog computing?
Fog computing is considered as the decentralized infrastructure of BigData. FC(Fog Computing) possesses a considerable processing power of edge nodes, that allows these nodes to process all the computation of large amounts of data on their own without parsing it to distant servers.
Image source: memecenter.com
The fog has these “Cloudlets”- small scale data centers that can support data-intensive IoT devices and smart gadgets that require low latency during the data exchange. FC offloads the cloud servers to enable large data storage and increase the processing power of the entire infrastructure.
FDA- Fog Data Analytics
Putting it simply, Fog Data Analytics is nothing but an amalgamation of the Cloud Computing model and Fog Computing. The basic function of Fog for which it is recommended is its ability to decrease latency and improve data distribution.
A typical FDA cycle consists of several functions like BigData storage, distribution, and visualization. These data are collected from several IoT devices through the sensors on the devices and are stored at the edge devices on the cloud.
After the required analytics is carried out successfully, all the relevant information pertaining to the data is sent to the cloud to handle further issues of BIgData. Every single data collected from the sensors of IoT devices of end-users are first sent to a Fog Layer(FL).
Each device communicates with the Fog Network(FN) through the Fog Gateways. FL combines different FNs and keeps a check on the priority requests. The Fog to Cloud communication is facilitated through the Cloud gateways and this part of the FDA architecture is known as Cloud Layer.
Attributes of The FDA for IoT-Based Applications
Heterogeneity in the FDA architecture mainly consists of hierarchal components that work as building blocks of a distributed network. The Fog Computing structure to the core of the FDA facilitates data storage, computation and networking functions in between the Cloud and end-user devices in a virtualized dimension.
It achieves the cognitive approach of client objectives and user needs through data access and data analytics. The functional abilities of the FDA structure to store, retrieve, distribute and transmit data from Cloud to end-devices make it more efficient for the cognitive approach.
The Fog Computing technologies manage to provide QoS(Quality of Services)- Technologies used for reducing packet loss, jitter and network loss in a network, for dynamic and static IoT-based devices.
The FN consists of geographically distributed nodes and sensors across environments such as healthcare monitoring, smart devices, smart homes, and weather monitoring services.
FDA can provide real-time interaction among the devices and Cloud. The capacity to facilitate real-time communication in different areas of applications makes it fit for smart applications.
With the development and recent innovations in smart devices, there has been some disappointment due to a decrease in the proximity of these devices. So, the weight of QoS falls on the network structures and the need for low latency has been on the rise at the edge of a network.
FDA facilitates many services depending on the interoperability of the environments like predictive analysis, data streaming and real-time analytics.
The FDA Process:
In this process, the data from several heterogeneous fog devices is collected and these aggregated data are Multi-model heterogeneous models. The already processed data is passed on to the analytics machine for real-time analysis and is further processed or stored according to the need.
After the processing part, data intelligence is developed and retrieval of intelligent information takes place. Further, these data are sent to the cloud or Fog network for more definite analysis.
Scheme of Classification for Fog Data Analytics
Data Collection and Storage
There are several data-collection systems and components in the data analytics ecosystem today. Some of them are:
IoT(Internet of Things)
The digital revolution that transformed digital devices from smart to smarter. IoT is basically a network of devices that can record, share and even compute some amount of data through sensors, digital apparatus, software, and embedded chips.
Every IoT devices have their own unique internet protocol (IP) address, that can establish valid connections between these smart devices. This communication does not require pre-defined interactions such as Human-to-Human(H2H), Human to Computer(H2C) or Computer to Human(C2H).
IoT as a data collection network has been really efficient, with its presence in different and diverse sections of industries. As a data storage facility. Though, IoT has several limitations and that are now mitigated through approaches like FDA.
In the world of smartwatches and health trackers, sensory devices have taken center stage on data collection, when it comes to Cloud and Fog applications.
These data generated through several sources are categorized into weather, temperature, vibration, voice, current, pressure, vehicles, etc. These data are transferred mainly through a Local Area Network(LAN) network or a wireless network for collection, storage, and processing.
All these data interacts through an API of the apps on devices. These APIs are specifically designed for IoT applications by a mobile app development company and are optimized to establish a link with the network through the unique IP address.
But, there is a human side to these sensory devices and that is defined through the Social Media data. With the emergence of Social Media, data aggregated from several Social Media platforms too work as the potential data source.
Data storage is classified into three major categories- Indexing, Clustering, and Replication.
A bunch of data is collected and stored into the fog storage devices.
In the indexing process, the data has relative indexations for fast retrieval and access. With a real-time indexing approach, indexing the complex data that is streamed with the restrictions. This approach combines a sequential approach for new data and an indexing approach to access the old database.
In replication, the same data is replicated over other machines to be duplicated for fault tolerance.
As we all know that Fogging is a distributed structure of data exchange and processing. In this structure, many of the systems are managed remotely and others are handled on the edge.
Fog Computing tends to reduce the data volume that is passed on to Cloud computing for storage, analysis, and processing. The data processing in a Fog Computing environment occurs in the smart devices at the edge of the network. Fog Computing also ensures data protection and data security.
The most vital attribute of Fog Data Analytics is its capability of filtering the data that is to be processed at the cloud layer. There are three basic components used in the Fog Computing environment.
- IoT Verticals
- Orchestration Layer
- Abstraction Layer
IoT verticals occupant applications or products- Smart Devices. Smart devices like external interface, communicators, controllers, automated systems, transmitters, sensors, etc. They support multi-tenancy to accommodate several clients to host an application on a single Fog server, which provides higher efficiency and interoperability.
This layer has pre-defined functions like data sharing, data migration, decision making, decision making, and policy management. It helps communication through a distributed network and secures the entire network communication.
This layer delivers a uniform interface to the client- just like a Cloud-based model. This layer uses virtualization to provide generic APIs, one that can be hosted by the devices for data exchange and data collection.
The services offered in IoT comprises of three major functions of data:
As we already discussed how the data is sensed through the sensors on smart devices, transmitted through transmitters, monitored through controllers, collected through smart devices and after due filtration through the FL layer is sent for further analysis to the cloud layer.
The analysis part is where the Fog Computing sets itself apart from all the conventional computing methods. In the IoT revolution, industries are moving forward with higher volumes of data to be processed and analyzed in real-time to achieve better QoS.
This brings us to the importance of real-time analysis in present data-intensive industrial operations. Analysis of real-time data can be facilitated through the Fog Computing model. While industries leverage the deep BigData analysis for prediction of failures, power consumption, and demand forecasting for higher productivity and QoS(Quality of Services).
Challenges to FDA approach
- The Fog Network is located at the edge of the network and this heterogeneity is the root cause of many challenges. For this, problem, Software Defined Network(SDN) is used which allows the Fog Nodes to become routers for nearby FNs and maintain the network. But, yet the challenges of wireless link failures and FN mobility persist.
- There are possibilities of network failures that lead to low reliability, though this scenario can be changed through a replication method. But, it needs a thorough application and experimentation.
- Fog-based applications need real-time data processing, which is hampered by the delay in complex data processing systems.
- Storage capacity and network bandwidth are a huge hindrance in Fog-based applications, which can be improved through further innovations.
- Regular monitoring and evaluation of the pre-defined attributes for secure data distribution need to be done.
- Lack of a Fog-based business model for billing, accounting, and monitoring such processes.
- Mobility of end nodes, storage, bandwidth latency, etc. creates challenges for application-awareness provisioning.
- There are resource detection and sharing challenges for Fog-based applications that rise due to serviceability, power consumption and revenues involved.
In a data-driven world, trailing on innovations and leaving the business in the hands of conventional technology, does not help the business needs. So, more enterprises and companies should invest in the research and development of Fog-based solutions and technologies like Fog Data Analytics.
Further, there is an urgent need for processing systems that can facilitate real-time data processing. With the increasing popularity of IoT verticals and its related usage, the need for a more reliable computing model with better network capabilities and data storage capacity needs to be developed. As the data analytics applications are increasingly laden with BigData!