BC Children’s Hospital Validation Study
To measure the accuracy of the Diabits algorithm for predicting future blood glucose levels in diabetic patients, Diabits performed validation testing in 4 studies, 2 of them using in-silico virtual patient data, and 2 with live patients.
Prior to distribution on the app store, the Diabits app was tested in a blind observational study in association with BC Children’s Hospital (BCCH). During the study, 10 patients are asked to use a Continuous Glucose Monitor (CGM) and a wrist-worn activity and heart rate monitor (Fitbit) to enable real-time monitoring of their heart rate, activity, and location. Patients were provided with CGM devices and had the app loaded onto their phone. However, the users were unable to access the display of the app or see blood sugar predictions for safety reasons. The patient was run through 2 phases:
a) Model Building (Learning Period): During the first 30 days of the trial the software accumulated information on patient’s glucose fluctuations. After the learning period was complete, the algorithm was able to make hour-by-hour predictions and refine those predictions using ML techniques.
b) Model Refinement (Personalization): During the next 30 days in the second phase of the trial, the software adjusted the weight of the variables used for prediction to achieve a higher rate of accuracy and a more precise patient-specific model using machine learning techniques.
Although the pathophysiology of diabetes may differ between children and adults, our data shows that the general laws governing glucose metabolism are similar. The BCCH study was performed using pediatric subjects. The estimation method employed in the Diabits app does not rely on any features that are specific to a particular age group. We have found that, generally, the Diabits app’s personalized models perform better on all metrics with increasing age.
During this study 172,800 data points were evaluated to confirm accuracy, meaning these many CGM data points were compared to the Diabits estimation model’s values. The result of this blind trial was an accuracy of 94.9% of predicted values in the A and B regions of the Parkes error grid. The study also determined that using heart rate and activity data for patients who do not engage in prolonged continuous exercise does not result in noticeable improvement of blood glucose estimate accuracy. To fidnout more please review our 2018 American Diabetes Association poster below.
