#8 - The magic of data
Best real-world examples of how sensors, drones and big data processing technology have made the world a better place
Hi there data leaders,
The holiday is around the corner, and that means a long quiet time to pause, think and reflect.
This edition of Data & Beyond Dispatch, wrapping up the year, is going to be a bit lengthier than our usual updates. But trust me, it's worth every moment for the insights and contemplations it'll bring your way.
Despite its length, there is one crucial message I want to convey: Data can bring tremendous value to the world, and as data leaders, you have the special power to make it a better place, bit by bit. (Anyone see the pun?)
Personally, I've been both humbled and blown away by the incredible data-driven innovations of recent years. From a system combating terrorism, interconnected breath monitoring devices to futuristic cashierless grocery stores - these are just a few examples of the remarkable solutions that have left me in awe.
But before diving into that, let's kick things off with some on-theme wisdom of this week.
This Week’s Wisdom
“We don’t get a chance to do that many things, and every one should be really excellent. Because this is our life. Life is brief, and then you die, you know? And we’ve all chosen to do this with our lives. So, it better be damn good. It better be worth it.” - Steve Job
This Week’s Top Pick
Kyiv outflanks analogue Russia with ammunition from Big Tech
Ukrainian soldiers have revolutionised the way battles will be fought in the 21st century by waging an “algorithmic war” that enables Kyiv to outgun invading forces with far fewer troops.
Artificial intelligence developed by companies in the West has given Ukraine a technological edge over Russia, military experts said, turning the tide of the war - The Times
The company behind such signification technological edge is Palantir, a US tech firm well-known for its extensive work for goverment agencies. Palantir has turned data on satellite constellations into useful insights to power Ukraine military - via the flagship software called MetaConstellation.
How exactly is MetaConstellation used to help ?
Intelligence gathered on enemy troop positions is collected from commercial satellites, heat sensors and drones, as well as spies.
AI model is used to transformed those data into a map highlighting the probable locations of Russian artillery, tanks and troops.
The pin-pointed locations of Russian troops are given to a Ukrainian soldier using a tablet device to direct their fire.
Remarkably, the technology can also “learns” from previous strikes and is constantly getting better at identifying and locating materiel.
The result is remarkable, the number of targets identified a day per command centre increases from 10 (as of twenty years ago) to 300 nowadays, giving Kyiv the ability to overwhelm invading forces despite a significant troop disadvantage.
An interconnected data-powered ecosystem to help asthma and COPD patients stay healthier
More than 140,000 people with ashthma and COPD across 27 countries have been able to control their breathing better with Propeller - a digital therapeutic platform that leverage sensor data, ML algorithms and integration to deliver the best care services for patients.
The anatomy of Propeller:
The core component of the platform is a system of sensors capturing signals that passively record events such as medication usage or respiration, and transmit data to Propeller via Bluetooth or hub connection. It also includes a hub plugging into a standard wall outlet, collecting sensor data and transmitting them to Propeller via WiFi and/or cellular connection.
The next component is a mobile app that empowers patients to understand popular conditions of their triggers, and gain useful personalised information to inform their habits.
Patent’s sensor data can also be shared and integrated with other EHR (Electrical Health Record) systems, or summarised in native analytics dashboards to help doctors gain insights and better support their patients.
Finally, support and coaching is given from a network of connected caregivers and clinicians to help patients cope with their condition.
From a technology standpoint, Propeller utilised Amazon SageMaker and Amazon Redshift to handle most workloads in production.
The solution has resulted in 57% reduction in emergency room visits and hospitalisations among asthma patients using Properller. This also means hospital resources are saved as a result of decreased emergency demand, the intangible impacts are tremendous:
Propeller’s platform enables more efficient remote monitoring and quality care beyond the walls of a clinic to help reduce physical barriers to care and improve access to patients in rural and/or medically underserved communities and patients with unmet social needs. - Propeller Health’s press release
How the Amazon Go Store’s AI Works
Amazon has taken retail shopping to the next level with its no-check-out experiences which, when introduced to the public, felt like magic. Unsurprisingly, the cloud giant has galvanised their best power in big data processing, cloud computing and machine learning to bring this once futuristic solution to reality via its Just Walk Out solution:
The heart of the Go Store is the Computer Vision based Machine Learning that is used to seamlessly track and estimate the intention of everyone in the store. - Ryan Gross
The infrastructure behind Amazon Go was designed to solve 3 key problems:
Person Identification: Custom RGB cameras were used to segment the images into blob of person/ non-person. Labels for a particular person are preserved across frames in the video for tracking their positions continuously while they are in store.
Item Identification: Products are identified using residual neural networks, using combinations of images from camera and weight sensors from the cells.
Who-bought-what Identification: A deep work model was trained on both camera-generated and simulated data to decide which is a valid “add-to-cart” action. It detects joint from the human’s body and estimates the pose closest to “add-to-cart” action. To account for customers who put the item back on the shelves, the algorithm also factors in the total count of items.
Additionally, the streaming services in the infrastructure are crucial to get the solution run in production. They are made up of compute devices, streamers, servers and storage in S3 and Dynamo.
After incorporating the technology in Whole Food, Amazon saw a rise in customers’ traffic to these locations. On the other hand, cashierless checkout technology has proven to be beneficial, based on a Verizon’s study:
78% increase in transaction count
80% increase in basket size
50% reduction in customer transaction time
139% increase in revenue
For data practitioners, it's easy to get caught up in the technicalities of data models, systems, algorithms, and lose sight of real-world impact of what we do. These stories serve as a reminder that data is incredibly powerful, and it's the data practitioners who are bringing this power to life.
So, what tangible impacts can your work bring to end users? Let’s dive into that and uncover the magic your data can bring to the table.
That’s it for this week - wish everyone a great holiday season, with lots of insights!
If you enjoy the newsletter, subscribe and stay tuned for more.