Accessing the NightScout Mongo database in Python.

I've already mentioned the NighScout and xDrip projects many times on these pages. Unless you have been living under a rock for the last 18 months, you probably know that these free and open source projects have had a major impact on the quality of life of many Type 1 diabetics and caregivers.

Paradise Lost

In late 2015, MongoLab upgraded their database system and subsequently their Mongo-db as a service web site to a new version (version 3.x). The NightScout developers and support team, once again, delivered: thanks to an upgrade, their users were able to continue to use the system with minimal inconvenience.

Unfortunately, for a non standard user such as your servitor, the new system had several drawbacks:

• data could not be posted directly to the database anymore, it had to be sent to the web site, then to the database. 
• the web site itself had started to evolve into something much bigger than a simple remote viewer.
• for most users, this meant that setting up the full new portal on Azure was the only practical solution.

My own requirements were

• the database is all that matters.
• simpler architecture that I can fire up in VMs, local computers, Raspberry PI, any hosting service.
• no need for the current additional features but the necessity to be compatible with my analytical interests.
• no desire to depend on the availability of chained elements (uploaders, azure, database, azure again, github for updates)

This is why I decided to leave the NightScout paradise for a while and revert to my private purgatory which includes, I shamefully admit, SQL and SQLITE databases... 

Born again? 

But then, right in time for the Holiday season, good news showed up. xDrip will support direct database uploads. Thank you guys, you made my day! And that brings me to the point of this short post: if MongoDB was again a viable option, how hard was it to access it outside the normal setup? I've always been a bit suspicious of MongoDB, as a memory and resource hog... Plus, if I had had the final say on product naming, I would have called it LISCB (lots of insane stupid curly braces). I was a bit reluctant and I was WRONG!. It turns out that, in Python at least, the task was extremely easy and took a few minutes!

Assuming you are running Python 3.4+, here's how to do it.

Start by installing the pymongo package

python -m pip install pymongo

and here is some simple sample code, with no error checking included, I just want the smallest demo possible. The prints aren't needed, they are just there to show the output. If you want to test it on your own installation, replace the user, password, 00000s db and collections with your own identifiers of course.

# import the required package
from pymongo import MongoClient
# database connection setup
uri = "mongodb://testuser:testpassword@ds000000.mongolab.com:00000/testdb?authMechanism=SCRAM-SHA-1"
client = MongoClient(uri)
print(client)
# set the db we need, we could of course query what's available
db = client['testdb']
# get the list of available collections, call them tables for fun
tables = db.collection_names(include_system_collections=False)
print(tables)
# set the collection we need
testcoll = db['testcoll']
# see if we can get a record
result = testcoll.find_one()
print('Result\n______\n', result)
# find out how many record we have
count = testcoll.count()
print('Number of Documents in collection :', count)
# let's look for values above 270 mg/dl
cursor = db.testcoll.find({"sgv": {"$gt": 270}})
# and display some information about them
for value in cursor:
    print(value['sgv'], 'provided by', value['device'], 'on', ['dateString'] )

And here is the output

/mongodb/minimal.py

getting there

MongoClient(host=['ds000000.mongolab.com:00000'], document_class=dict, tz_aware=False, connect=True, authmechanism='SCRAM-SHA-1')

collections

['devicestatus', 'objectlabs-system', 'objectlabs-system.admin.collections', 'profile', 'testcoll', 'treatments']

first record in that test db

Result
______
 {'dateString': 'Fri Oct 24 14:14:00 CEST 2014', 'direction': 'SingleUp', 'date': 1414152840000, 'device': 'dexcom', '_id': ObjectId('544c6816a885b50eecc2571e'), 'sgv': 101}

number of records in the testcoll collection

Number of Documents in collection : 37192

result of the conditional query

271 provided by dexcom
271 provided by dexcom
271 provided by dexcom
271 provided by dexcom
271 provided by xDrip-BluetoothWixel
271 provided by xDrip-BluetoothWixel
271 provided by xDrip-BluetoothWixel
271 provided by xDrip-BluetoothWixel
271 provided by xDrip-BluetoothWixel
273 provided by xDrip-BluetoothWixel
275 provided by dexcom
276 provided by xDrip-BluetoothWixel
277 provided by dexcom
277 provided by xDrip-BluetoothWixel
277 provided by xDrip-BluetoothWixel
278 provided by dexcom
278 provided by xDrip-BluetoothWixel
278 provided by xDrip-BluetoothWixel
278 provided by xDrip-BluetoothWixel
279 provided by dexcom
279 provided by xDrip-BluetoothWixel
279 provided by xDrip-BluetoothWixel
280 provided by xDrip-BluetoothWixel
281 provided by dexcom
282 provided by xDrip-BluetoothWixel
282 provided by xDrip-BluetoothWixel
283 provided by xDrip-BluetoothWixel
286 provided by xDrip-BluetoothWixel
287 provided by xDrip-BluetoothWixel
289 provided by xDrip-BluetoothWixel
290 provided by xDrip-BluetoothWixel
292 provided by xDrip-BluetoothWixel
292 provided by xDrip-BluetoothWixel
294 provided by xDrip-BluetoothWixel
296 provided by xDrip-BluetoothWixel
299 provided by xDrip-BluetoothWixel
303 provided by xDrip-BluetoothWixel
310 provided by xDrip-BluetoothWixel

That's all folks, back to my obscure useless stuff.