Thursday, September 15, 2016

Calibration Basics - or why I stopped worrying about others and ran my calibration algorithm.

While I love the various diabetes open-source projects, the way they moved the industry forward and how they made what was either non-existent or wildly expensive, I have mostly stopped using them as such. I either run my own stuff, use modified freeware versions or, am generally happy with what the G4 505 AP offers in terms of accuracy (speed not so much).

I remember vividly one incident, almost a couple of years ago, where an open source solution displayed negative or impossibly low glucose values when the sensor was pulled out. The root cause of that issue was an intercept set to zero in what was supposed to be a slope + intercept calibration model. At the time, I tried to argue that this was a bad idea and I was kindly mobbed as people who dare to criticize aspects of open source projects often are.

I confess that I was partly to blame for my conciliatory surrender as I did not want to get into endless discussions on the issue. But, now and then, I relaunch one of those solutions and use them for a while. For some reasons, I was hit by even fancier errors caused by forced bad slopes and intercepts.

In this post, I will explain why I think of you need to use a slope intercept model, with a positive intercept. OK, I admit I use "think you should use" as a polite version "you definitely must use"...

Driving a fitted straight line to zero assumes the observed signal is directly proportional to the glucose concentration. That is totally inaccurate, at least until manufacturers decide otherwise.

Sensors are driven (I will get back to this in a bigger 'sensor' post) by a potential difference between the electrodes. That is why sensors become unreliable when their batteries go low and why people who replace batteries in old transmitters check the voltage. The consequence of that voltage is that current always flows between the electrodes of the sensor. That flow is not perfect and is subject to noise. The next figure shows what it looks like.
If the current generated by the sensing process was directly proportional to the glucose concentrations (we assume here that all the linearity issues have been solved by the different membranes around the electrodes), we would get something like this, again with some noise.
On that type of measurement, a slope intercept model with a nil intercept would be roughly valid (you would still have to add a bias so you don't get negative values now and then or if you want to have some room to characterize the noise, but anyway...)

From the above, you should see that the actual physical signal, reflected in the raw count, looks like this (with both the noise of the driving current and the noise of the glucose measurement playing a role)

From the above, it is obvious that no linear estimator forced through a zero intercept or allowed to go through a negative intercept will ever be satisfactory.

The "divide by 10" approach that I observed in some Libre freeware a few months ago was, by definition, heading to a zero intercept: it was doomed to be inaccurate, even if it had somehow used the correct slope. But that basic mistake (sorry) is also present, or potentially present in some situations, in other solutions.

Here is such an idealized (no noise) slope, from a Dexcom patent but, if you still need to convince yourself, there are dozens of similar examples on the net, thousands if one include other sensors, electrods and methods.

Last not least: this is of course valid for as long as manufacturers deliver a raw signal that is actually connected to the real RAW. The Libre signal can currently be considered as raw, coming directly from the TI FAL152. The Dexcom signal of the G4 is partly processed and denoised on transmitter (a noise estimation is provided along with the counts).

But nothing will prevent manufacturers from doing full on transmitter or on sensor processing in the future and deliver some value that has already been fully denoised and from which driving current will already have been subtracted. When that time comes, some will certainly complain that "raw" isn't available anymore, but the truth is that there would be very little to with raw values at that point, unless you can do a better low-level denoising job than the manufacturer.

Note: picoAmps are realistic values, noise is present but idealized for easy visualization. Real noise can be proportionally lower or higher, depending on situations (muscle triboelectric activity for example)

Wednesday, September 7, 2016

Back to the Libre, well... kind of...

Libre now fully covered by social security in Belgium

That's the big news. Since this Summer (2016), the Abbott Freestyle Libre is now fully covered for Type 1 diabetics in Belgium. To be honest, given the complete lack of CGM/FGM awareness that I encountered a couple of years ago and after reading the opinion (in 2015) that CGMs weren't working well in the magazine of the Belgian diabetic association, I was a bit surprised to see the rapid evolution.

There is a catch though - totally understandable - but still a catch. When you declare your interest in the Libre, you sign an new paper that modifies your "Convention of Diabetes Co-Management" and switch from free finger testing strips to free Libre sensors. Instead of the truckload of strips you would have received for the 3-4 months until your next endocrinologist visit, you get sensors to cover that period and a box of Abbott test strips to test your sensor now and then and, eventually, contact Abbott for a replacement. Essentially, if you choose the Libre, it is intended to fully replace your blood testing routine, which is of course in line with Abbott's marketing.

Note: we did try to use the Abbott strips a couple of time and that proved to be a challenge! Those things are blood vampires compared to the Menarini Glucomen LX plus we have been using for the last 3 years, to our complete satisfaction (well within the latest ISO requirements). The Freestyle Libre BGMeter could be slightly more accurate than our current one, but I don't believe we will find out any time soon.

Back to the Libre in Belgium. I managed to talk myself out of the "training session" - I think I can fairly say that our family knows enough about the Libre already. I confess that I had the somewhat perverse idea to take advantage of that training session to ask a few questions about that annoying dual thermistor approach to temperature compensation but I doubt I would have received meaningful answers from the Abbott representative.

More interesting was the reaction I got when I told our team that we intended to enter the "Libre" part of our care convention: "are you really sure?" From what I understood, the average patient is not too enthusiastic yet. The usual issues (running low post insertion, adhesion problem, skin irritations, inaccurate results or confusion about accurate results, teens or kids not wanting to put something in their body...) are apparently slowing adoption.

Back on the Libre but...

The long term readers of this blog are well aware that I liked the Libre a lot when it was released, to the point that we went through the hoops, with the help of the French part of our family, to obtain sensors. The Libre sensors we had back then proved to be more accurate than the Dexcom G4 non AP (14 days comparison). They were also much faster, which was essential for sports (speed matters, tennis and the Libre). We were a bit lucky back then (an informal poll in the Libre user groups reported roughly a 20% failure rate at the time) and enjoyed several complete runs with minimal BGMeter testing. In fact, we enjoyed the Libre so much that we forgave it for stealing our data covertly (the license terms have been updated, your data is now lifted officially) . I dug a bit into its mode of operation and used some of what I discovered/read about in the process to improve our G4's behavior. At that point, the tedium of ordering in France, the lack of convenient remote monitoring (more about this later) and the improvements I made to the processing of the G4 non AP raw data led us to abandon the Libre.

On top of that, several generous readers of this blog offered to send us a G4 AP from the US and, I finally accepted one in November 2015. You can see my G4 non AP vs G4  AP comparison here. In short, the Dexcom G4 AP (and G5) remains slower at picking up trends than the Libre, is a bit faster than the old G4 and, more importantly is now much better at dealing with sub optimal situations, noisy startup sequences and erroneous / non optimal calibrations. While I have taken a long hard look at the G4 AP algorithm, I haven't found anything (given the frequency of the signal) that I really wanted to improve upon or that I felt I could improve.

Finally, and that is subjective, based on a single patient and a year of sensor use, we do feel that the Dexcom sensors have improved a lot: we have not had a single sensor failure in our last 30 Dexcom sensors and, more importantly, they mostly all seem to run acceptably with the same calibration parameters for longer time (this implies that Dexcom has improved both the consistency of what they deliver and the stability of their sensors) 

That means that the Libre is now facing, as far as we are concerned, much stronger competition.

And of course... 

Nonetheless, we decided to give the Libre another shot and inserted a sensor as soon as we came back from our endo's office. As it was to be expected, that sensor was a dud. An interesting dud, but a dud.

Here is an (approximate) snapshot of its behaviour.
In practice, such a dud behaves in a way that may be surprising to an average user
  • around 100 mg/dl, it will appear to display the correct value. 
  • below that value, it will under estimate the correct value.
  • above that value, it will over estimate the correct value.
  • that "bad" slope will increase the apparent variability of your interstitial glucose tremendously.
In fairness, that is probably a sensor that I should have exchanged but we decided to keep it in order to investigate it a bit more. That I could understand quickly the sensor behavior and that Max understood the above representation and its impact on his displayed BG values made the situation tolerable. But, after 2-3 days, when it became clear that the sensor would stay stuck on that bad slope, we inserted a Dexcom...

There is that old saying in French that roughly: "you never get the second chance to make a first impression". In our case, the first Libre impression was extremely positive. We have been running CGMs for several years and understand bad sensors can happen. But what about a first time user, who has just gone through basic training and has never tried a CGM/FGM? I really don't know...

Now, putting my tinkerer's hat back on, the interesting thing about that bad behavior is that it is exactly the kind of situation you can calibrate your way out. Should you? That possibility is already available through third party software. It is a valid question, but the answer is not easy: there are other potentially confusing situations and other Libre malfunctions (temporary or permanent) that can not be handled or should not be handled through calibration.

I'll try to explore the issue in more details in a follow up post.

Last thing before closing this post: Sandra Kessler has launched a crowd funded project for a Libre add-on that would bridge periodic NFC reads to BT and possibly transform the Libre in a full fledged CGM. There are many potential issues and risks involved in supporting such a project but I have decided to support it. I am well aware of other hacks (in the positive sense of the term) to achieve the same goal and have, in the past, expressed my disappointment about the algorithms those projects used. As far as I can tell, that profound algorithm issue has now been mostly solved. That does not mean these projects, Sandra's and the others, don't face additional hurdles. Time will tell.

I will also try to devote a blog post to the blueReader project in the future and, if we are still on the Libre, possibly contribute a bit more.