Tag: data

The Silent Lifeline of IoT: Why Compression is the Future of Air Quality Monitoring

Sotirios PapathanasiouLeave a comment

For years, the Internet of Things (IoT) conversation has been dominated by sensors, cloud platforms, and the flashier world of AI analytics. While AI remains the current “buzzword,” the reality is that millions of IoT devices are already quietly working in the background, providing the essential sensor data that powers our modern world. However, as these deployments mature and scale, a critical bottleneck has emerged: the cost and physical limits of data transmission.

In my experience with Air Quality Monitoring (AQM) solutions, I’ve seen this play out repeatedly. Projects often aim to transmit high-frequency, continuous air quality measurements over long distances, only to hit a wall. Whether it’s the strict payload size limits of LoRaWAN or the spiraling costs of high-frequency transmissions over LTE/NB-IoT, the “raw data” approach is no longer sustainable.

The Problem with “Raw” Transmission

Most IoT data, especially from air quality sensors, is highly structured and repetitive. Devices often transmit variations of the same environmental measurements over and over. Sending this information raw ignores a simple reality: transmission is expensive, not just in terms of data plans, but in battery life, maintenance, and long-term operational costs.

As the industry matures, we are seeing a shift in mindset. Compression is no longer just a low-level technical detail; it is becoming a foundational technology because it makes large-scale deployments sustainable.

Three Pillars of IoT Compression

Integrating lossless compression directly onto the device, rather than relying on the cloud, transforms it into a perpetual efficiency engine. This creates several vital second-order effects:

  • System Resilience: Fewer transmissions lead to less network congestion and fewer collision points. This reduces “chatter” and makes systems like city-wide AQM grids significantly more reliable.
  • Extended Battery Life: Radio transmissions are the primary power drain for most IoT devices. By reducing how often a device needs to “speak,” we can extend battery life dramatically, reducing the need for expensive “truck rolls” to replace batteries in the field.
  • Enhanced Data Quality: Paradoxically, compression allows you to collect more data. By transmitting more intelligently, devices can sample at higher frequencies to capture micro-events and short-lived anomalies that would otherwise be lost due to bandwidth constraints.

Real-World Efficiency: The Up-to 87% Reduction

The potential for this technology is best illustrated by modern encoders capable of high-ratio reduction. For instance, testing with 50 timeseries payloads (32 bytes each) shows a raw size of 1600 bytes being compressed down to just 202 bytes—an 87.4% reduction (Source: Zetako Lab Demo Tool). This level of efficiency allows for high-granularity monitoring even on restricted protocols like LoRaWAN.

MetricRaw DataCompressed Data
Payload Size1600 Bytes202 Bytes
Reduction0%87.4%
IntegrityN/ASHA256 Verified

Conclusion

The future of IoT won’t be defined by who collects the most raw data, but by who uses fewer resources to learn more. In critical infrastructure like healthcare, transportation, and air quality monitoring, these efficiency choices compound.

Compression is no longer just a “feature”, it is a lifeline. Without it, IoT cannot scale sustainably to meet the demands of our data-driven future.

Unlock the Power of Data Narratives in Our “Storytelling with Data” Webinar

Sotirios PapathanasiouLeave a comment

Are you ready to transform complex data into compelling stories that resonate and drive impact? Join us for our insightful webinar, “Storytelling with Data,” on April 10th at 11:30 AM EDT on LinkedIn Live.

In today’s information-rich world, simply presenting data isn’t enough. True understanding and engagement come from weaving data into compelling narratives. This webinar delves into the art and science of Information Design, demonstrating that it’s far more than just creating charts and graphs. It’s about strategically transforming raw data into meaningful stories that captivate audiences and inspire action.

Our upcoming session brings together a panel of global experts (Gabrielle Merite, Florent Lavergne, Sotirios Papathanasiou, Nicole Lachenmeier, & Maggie Shi ) at the forefront of information design. We’ll explore how mission-driven marketers and environmentally conscious data visualization professionals can leverage the power of storytelling to amplify their message and create lasting impact.

Read More »

Air Quality Data & Ownership

Sotirios PapathanasiouLeave a comment

In an age where information is power, the question of who owns the data generated by air quality monitors and sensors has become increasingly important. This is especially true for air quality monitors that provide crucial insights into the air we breathe. While these devices offer valuable information, users should be aware of potential issues related to data ownership and accessibility.   

The Risks of Changing Terms and Closed Systems

In some cases, companies have sold air quality monitors with “unlimited” data storage, only to later change their terms of service and require users to pay for continued access to their own data. This bait-and-switch tactic leaves consumers feeling betrayed and exploited, as they are forced to pay for something they thought they already owned.

Read More »

MasterClass: Air Quality Data Visualization with R Studio & Packages

Sotirios Papathanasiou6 Comments

R Studio and its packages are used by hundreds of thousands of people to make millions of plots. I use it to compare air sensor data from different air quality monitors/sensors or to visualize air pollution levels.

In this article we will explore both how we can visualize air quality data from publicly available sources and how you can create statistical correlations between different pollutants or different sensors to find the correlation coefficient or correlation of determination.

First: Get the Right Packages

Packages are collections of functions, data, and compiled code in a well-defined format, created to add specific functionality. Here are some of the packages that we will install inside RStudio and use.

#You can either get ggplot2 by installing the whole tidyverse library
install.packages(tidyverse)

#Alternatively, install just ggplot2
install.packages(ggplot2)

#saqgetr is a package to import European air quality monitoring data in a fast and easy way
install.packages(saqgetr)

#worldmet provides an easy way to access data from the NOAA Integrated Surface Database
install.packages(worldmet)

#Date-time data can be frustrating to work with in R and lubridate can help us fix possible issues
install.packages(lubridate)

#Openair is a package developed for the purpose of analysing air quality data
linstall.packages(openair)
Read More »

From Boom to Bust: The Great IoT Air Quality Recession

Sotirios Papathanasiou2 Comments

The once booming Internet of Things (IoT) air quality monitoring market is facing a harsh reality check. Fueled by a surge in AI startups attracting investments and a subsequent saturation of low-cost air quality monitors, the industry is experiencing a period of upheaval. This downturn, dubbed “The Great IoT Air Quality Recession,” is forcing companies to adapt or face extinction. I see many high-profile executives leaving previously thought innovative startups in the realm of air quality in search of a more “stable” future.

A Wave of Investment and Sensor Saturation

AI startups like ChatGTP and similar, promising to leverage the power of machine learning to generate content or analyze data, became investor darlings. This new influx of cash is fueling the decline of IoT low-cost air quality solutions.

After the COVID-19 pandemic, the market quickly became saturated with low-cost monitors that promised that will fix indoor and outdoor environments. Buildings were filled with cheap monitors, but actionable insights remained scarce. The promised AI-powered analysis, in many cases, failed to materialize. Consumers were left with a plethora of data points with no clear understanding of what it all meant or what to do.

Read More »