ROOK's data processing is a critical step that ensures the health data delivered to clients is high-quality, consistent, and usable. ROOK employs a multi-stage process to transform raw data from various sources into a standardized format. This process occurs after data extraction and before delivery via webhooks or the ROOK API. The main steps include:
Data Harmonization:
This stage ensures consistency across different data formats, units, and definitions.
For example, distance values are converted into a unified unit, such as kilometers.
Timestamps are adjusted to align with the user's local time zone.
This results in a uniform data representation across all health data sources.
Data Standardization:
This step applies recognized industry standards to the collected data.
For instance, sleep stages from various providers are mapped to a common standard.
Heart rate intervals are aligned for consistent reporting.
This ensures that data is compatible and reliable across different health data providers.
Data Cleaning:
This stage focuses on eliminating inconsistencies and resolving duplicates. ROOK’s Duplicity Feature is used to manage data from multiple sources.
For event processing, events from multiple sources within a ±10-minute window are merged or prioritized based on source ranking. Redundant events are discarded.
For summary processing, summaries are generated using the highest-priority data source. Updates to summaries incorporate complementary data from secondary sources. Updated summaries are versioned for traceability using the document_version key.
Summaries are retained for 15 minutes before delivery to incorporate delayed updates from connected sources.
All data is ultimately reported in UTC for consistency.
This step ensures clean, accurate, and enriched data by eliminating redundancies and preserving the best information.
Data Normalization:
This stage adjusts data to a uniform scale and format.
For example, calorie data is converted to a standardized unit (e.g., kilocalories).
Step counts are aligned into consistent time intervals.
This makes the data comparable across diverse data sources.